Python Forum
Get return value from a threaded function
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Get return value from a threaded function
#1
I have the following function that builds a file list.

    import os
    import fnmatch
    import threading

    def getFiles(root='.', pattern='*.*', recurse=False):

        results = []
        if recurse:
            for base, dirs, files in os.walk(root):
                match = fnmatch.filter(files, pattern)
                results.extend(os.path.join(base, f) for f in match)
        else:
            results = fnmatch.filter(os.listdir(root),pattern)
            for i in range(0, len(results)): results[i] = os.path.join(root,results[i])

        return results
I want to run this function in a thread so that it can build the file list while the script is busy doing other things. How can I get the returned list from the main script? Here is a stripped down version.

    #Create the thread and start parallel execution

    t = threading.Thread(target=getFiles, args=['.', '*', False], daemon=False)
    t.start()

    #Here's where I do the "other things"

    # Done other things. Wait for the file list thread to complete

    t.join()
How do I get the returned list? I have found several examples but none that are not obfuscated by irrelevant bells and whistles.
Reply
#2
With a little more effort I found the solution although it didn't involve adding to my existing code. The full solution is below

import os
import fnmatch
from   multiprocessing.pool import ThreadPool

def getFiles(root='.', pattern='*.*', recurse=False):
    """
    Return a list of all files in the given root that match the
    given pattern. If recurse=True the also do all sub-folders.
    """
    results = []
    
    if recurse:
        for base, dirs, files in os.walk(root):
            match = fnmatch.filter(files, pattern)
            results.extend(os.path.join(base, f) for f in match)
    else:
        results = fnmatch.filter(os.listdir(root),pattern)
        for i in range(0, len(results)): results[i] = os.path.join(root,results[i])

    return results
             
#Create the thread and start parallel execution

pool = ThreadPool(processes=1)
thread = pool.apply_async(getFiles, ('.','*.*',False))

#The code below this will run in parallel with the above thread

print("continue on with other code in parallel")

#This will pause until the thread completes.

files = thread.get()

for file in files:
    print(file)
Reply
#3
Key here is that you have the wrong idea. You do not "return" a value from a thread. The thread runs, and eventually, terminates. There is no concept of "returning" anything.

However, you can ideally send a message between two threads, or have a shared queue into which you place results. Ultimately, the "main thread" can decide to examine this queue and find whatever elements you have put in it. I am not familiar enough with Python to say how this is done, but this is how I do it in other languages. Threads never "return" values because there is no place that they can return them to. But you can use shared resources, which you have to make sure are properly locked and synchronized, to pass information between threads. In some systems and libraries these capabilities are built in. I have not tried multithreading in Python, although I have depended on it for dozens of projects done in C/C++ and other languages (I've been using multithreading since about 1969).
Reply
#4
If that is the case then how do you explain that the line "files = thread.get()" gets me the value that is returned by the threaded function?
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  nested function return MHGhonaim 2 562 Oct-02-2023, 09:21 AM
Last Post: deanhystad
  return next item each time a function is executed User3000 19 2,175 Aug-06-2023, 02:29 PM
Last Post: deanhystad
  function return boolean based on GPIO pin reading caslor 2 1,131 Feb-04-2023, 12:30 PM
Last Post: caslor
  return vs. print in nested function example Mark17 4 1,674 Jan-04-2022, 06:02 PM
Last Post: jefsummers
  Improves performance on socket multi-threaded servers for object detection pennant 1 1,856 Aug-31-2021, 08:43 AM
Last Post: Larz60+
  How to invoke a function with return statement in list comprehension? maiya 4 2,752 Jul-17-2021, 04:30 PM
Last Post: maiya
  Function - Return multiple values tester_V 10 4,320 Jun-02-2021, 05:34 AM
Last Post: tester_V
  Return not exiting function?? rudihammad 3 5,192 Dec-01-2020, 07:11 PM
Last Post: bowlofred
  Why does my function return None? vg100h 3 2,157 Oct-29-2020, 06:17 AM
Last Post: vg100h
  how to keep a Popen instance existant in a function return? Skaperen 7 3,080 Sep-17-2020, 07:10 PM
Last Post: Skaperen

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020