starting multi threads

[MuntyScruntfundle]

Hi folks.

I have need to merge two bits of code, one is a very simple socket server and one is a reporting module that writes bumf to a mysql server at regular intervals. Both applications run independently perfectly well. So I thought I'd put both apps into functions and call them as threads.

Psudo code would be:

------------------------------------------
import blah, blah

def Reporter():
while True:
-DO MYSQL STUFF STUFF-

def SocketServer():
while True:
-ACCEPT AND PROCESS SOCKET DATA-

## Main
thread.start_new_thread(Reporter,(1,))
thread.start_new_thread(SocketServer,(1,))

------------------------------------------

The app spits out text here and there so I can see what's going on, and when I run the app both threads appear to get started, but then I get nothing. There's no pgrep id of the app, but n errors either. So, where does my app go?

Then if I run the app again I start getting random errors from lines of code that can't error, as if variables are completely messed up.

So what's happening with these threads that I wasn't expecting?


Many thanks.

[nilamo]

If you don't call a thread's .join() method, then the main program will continue on, and exit at the end. If you want the program to wait for them to finish, you need to explicitly wait for them to finish.

[MuntyScruntfundle]

For join I nee to:

th=threading.Thread(name, target)
th.start
th.join()

?

[nilamo]

You'd want to call the start method, not just refer to it. But otherwise yes.

[MuntyScruntfundle]

I could't get join working at all, all sorts of problems.

However
While True:
running=True

after setting off the threads works. Is there any reason I shouldn't use this approach?

Thanks again.

[nilamo]

I could't get join working at all, all sorts of problems.

Please show some code.

Is there any reason I shouldn't use this approach?

It could keep your processor busy looping forever. If you're comfortable going -1 core on your processor, I guess it's fine lol.

Aside from that, it also doesn't make sense. while True: pass means you're intentionally doing nothing at all, forever, with no way of ever stopping. It's sloppy programming. It makes more sense, from an objective point of view, to say "I'm done doing stuff, but want to wait until those other things are done doing their thing."

Here's a simple example of how .join() works:

>>> def spam():
...   time.sleep(5)
...   for i in range(10):
...     time.sleep(0.1)
...     print(f"spam! {i}")
...
>>> def foo():
...   time.sleep(5)
...   for i in range(10):
...     time.sleep(0.25)
...     print(f"foo! {i}")
...
>>> threads = [threading.Thread(target=func) for func in [spam, foo]]
>>> [thread.start() for thread in threads]
[None, None]
>>> [thread.join() for thread in threads]
spam! 0
spam! 1
foo! 0
spam! 2
spam! 3
foo! 1
spam! 4
spam! 5
spam! 6
foo! 2
spam! 7
spam! 8
foo! 3
spam! 9
foo! 4
foo! 5
foo! 6
foo! 7
foo! 8
foo! 9
[None, None]
>>> 

[MuntyScruntfundle]

Yeah, I found that putting a sleep in made the machine responsive again!!

So, changing nilamo's code to suit I've got it all running lovely. I must learn more about python compound scripting like this, you've turned 12 lines of code into 3.

One other question: When I'm setting off 2 non ending threads and joining like this, there's nowhere to put a CTRL+C escape. Yes, while it's meant to be non ending, its always good to have an emergency exit button! How would this be done?

Many many thanks.

[nilamo]

In that case, you should use the thread's timeout parameter, so .join() doesn't wait forever for the thread to finish, and use a Queue (or other synchronization mechanism, such as a Lock around a bool) to send a message to the various threads that they should stop what they're doing.

Also, KeyboardInterrupt is the error that's thrown when you type ctrl-c.

Here's an example of that:

import queue
import threading
import time

def worker(q):
    iterations = 0
    running = True
    while running:
        iterations += 1
        print(f"I've run {iterations} times!")
        if not q.empty():
            item = q.get()
            if item == "stop":
                # put the message back in, so other threads can see it
                q.put(item)
                print("Got the message to stop.")
                running = False
        time.sleep(1)

messager = queue.Queue()
thread = threading.Thread(target=worker, args=(messager, ))
thread.start()

try:
    while True:
        # use a timeout, so it doesn't wait forever for the thread
        thread.join(timeout=1)
except KeyboardInterrupt:
    messager.put("stop")
finally:
    # don't use a timeout here, so we can make sure it gets the message and stops
    thread.join()

Sample output:

Output:
E:\Projects\etc>python spam.py
I've run 1 times!
I've run 2 times!
I've run 3 times!
I've run 4 times!
Got the message to stop.