Having difficulty with threads and input()

[sawtooth500]

I may have stumbled upon a devlishly simply solution... the standard exit() function wasn't doing it but...

handle_input(ws, ask4input = False)
os._exit(0)

Even though handle_input is running on a thread waiting for me to type "x" and then enter... when I reach a condition in the script where I want to end it I can just call handle_input with ask4input = False, bypassing the input() in the function, then my essential cleanup activities of cancelling open orders and closing out of open positions will occur, then os._exit(0) will hard kill the entire script no questions asked. It seems to work, but I can't robustly test until market hours tomorrow.

[deanhystad]

Carl+c can be graceful. You catch the keyboard interrupt exception using try/ except and run your shutdown code. It is the same as waiting for ‘x’

[Gribouillis]

This I am fascinated by... how can I connect to my script? If I can send an external signal to it to start termination activities that could work instead of using an input().

You could use any kind of server such as a http server or whatever. Here is an example using a rpyc server (remote procedure calls). The main program does some work but it also starts a thread running a server on a port that waits for one connection

# rpycserver.py
from threading import Thread, Event
import rpyc
from rpyc.utils.server import OneShotServer
from time import sleep

PORT = 18861
STOPPED = Event()


class StopService(rpyc.Service):
    def exposed_stop(self):
        WORKER.stop()


class ServerThread(Thread):
    def __init__(self):
        Thread.__init__(self)
        self.server = OneShotServer(StopService, port=PORT)

    def run(self):
        self.server.start()

    def stop(self):
        STOPPED.set()
        self.server.close()


if __name__ == "__main__":
    WORKER = ServerThread()
    WORKER.start()
    while True:
        # This is the program's main loop
        # It checks now and then if the STOPPED event is set
        # indicating that the stop_service was invoked from
        # the outside.
        # It could also stop programmatically by calling
        # WORKER.stop()
        if STOPPED.is_set():
            break
        sleep(2)
    WORKER.join()
    print("Running cleanup code")

In order to stop the program, I run another program

# stoprpycserver.py
import rpyc

PORT = 18861

if __name__ == "__main__":
    c = rpyc.connect("localhost", PORT)
    try:
        c.root.stop()
    except EOFError:
        pass

Output:
λ python paillasse/pf/rpycserver.py &
[1] 5091
λ python paillasse/pf/stoprpycserver.py
λ Running cleanup code

[1]+  Done                    python paillasse/pf/rpycserver.py

You could add features to the server, such as authentication etc.

[Gribouillis]

In this new version, the stoprpycserver.py scripts waits until the rpycserver.py process has exited, so that one can be sure that the program exited when the other program invoked to stop it exits.

This is achieved by letting the stopping program ask the main program's PID through the server and using the psutil module to wait for this pid. All of this is multiplatform.

# rpycserver.py
import os
import rpyc
from rpyc.utils.server import ThreadedServer
from threading import Thread, Event
from time import sleep

PORT = 18861
STOPPED = Event()


class StopService(rpyc.Service):
    def exposed_pid(self):
        return os.getpid()

    def exposed_stop(self):
        stop_service.stop()


class ServerThread(Thread):
    def __init__(self):
        Thread.__init__(self)
        self.server = ThreadedServer(StopService, port=PORT)

    def run(self):
        self.server.start()

    def stop(self):
        STOPPED.set()
        self.server.close()


if __name__ == "__main__":
    stop_service = ServerThread()
    stop_service.start()
    while True:
        # This is the program's main loop
        # It checks now and then if the STOPPED event is set
        # indicating that the stop_service was invoked from
        # the outside.
        # It could also stop programmatically by calling
        # stop_service.stop()
        if STOPPED.is_set():
            break
        sleep(2)
    stop_service.join()
    print("Running cleanup code")

# stoprpycserver.py
import psutil
import rpyc

PORT = 18861

if __name__ == "__main__":
    c = rpyc.connect("localhost", PORT)
    proc = psutil.Process(c.root.pid())
    try:
        c.root.stop()
    except EOFError:
        pass
    proc.wait()

Output:
λ python paillasse/pf/rpycserver.py &
[1] 4801
λ python paillasse/pf/stoprpycserver.py
Running cleanup code
[1]+  Done                    python paillasse/pf/rpycserver.py
λ 

Note: if you don't want to use rpyc module, you can achieve the same with the standard library's xmlrpc module.