Python Forum
How to run multiple threads properly
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How to run multiple threads properly
#4
(Dec-22-2017, 08:20 AM)cyberion1985 Wrote: Thanks for your reply. My functions run like this

def PR01()
    # process file

PR01()
PR02()
PR03()
PR04()
The reason why I am using this approach is because each function is somewhat unique and on each function I receive feedback on how long it took to run. The largest file takes 11 minutes to generate for example. To clarify ; in each function :

1.) file might be be renamed
2.) file might be converted to XLSX
3.) file might be receive a header
4.) file might be autofit

So each file gets processed with PR01 then passed on to PR02, then to PR03, etc.? If so, you have yourself a pipeline. (https://www.cise.ufl.edu/research/Parall...peline.htm)

If you're not doing a pipeline and your processing doesn't rely on global state of the program, you might consider using the subprocess module and do the work in individual processes instead of threads. You might also consider the multiprocessing module (https://docs.python.org/3/library/multiprocessing.html). I have never used it and have no idea if it would suit your needs.
Reply


Messages In This Thread
RE: How to run multiple threads properly - by wavic - Dec-22-2017, 07:38 AM
RE: How to run multiple threads properly - by mpd - Dec-22-2017, 12:41 PM
RE: How to run multiple threads properly - by wavic - Dec-22-2017, 02:42 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Trying to separate a loop across multiple threads stylingpat 0 1,711 May-05-2021, 05:21 PM
Last Post: stylingpat
  Get average of multiple threads Contra_Boy 1 18,398 May-05-2020, 04:51 PM
Last Post: deanhystad
  stop multiple threads jeuvrey 5 3,461 Nov-15-2018, 01:34 PM
Last Post: jeuvrey
  Quitting multiple threads MuntyScruntfundle 3 2,715 Oct-17-2018, 05:14 AM
Last Post: volcano63

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020