Python Forum
How to to tie the execution of one process to another inside a loop in Python
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How to to tie the execution of one process to another inside a loop in Python
#1
How can I keep a loop going, while having some processes that are waiting for others inside the loop? (see the code below for clarification, it makes more sense to explain in code). PS: I'm using Popen from the multithreading module

for i in range(0, len(my_list), batch_size):
    current_batch = my_list[i:i+batch_size]
    download_processes = [Popen('do stuff')] # NOT ACTUAL CODE. This downloads files.
    _ = [p.wait() for p in download_processes] # wait until all files above download before executing the code below 

    more_processes = [Popen('do stuff')] # NOT ACTUAL CODE. This zips the downloaded files
    # execute yet more processes (that process the zips files) when "more_processes" end, but keep the loop going to start downloading more files
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  kill python execution program lebossejames 0 234 Mar-16-2024, 11:16 AM
Last Post: lebossejames
  Variable definitions inside loop / could be better? gugarciap 2 430 Jan-09-2024, 11:11 PM
Last Post: deanhystad
  How to create a variable only for use inside the scope of a while loop? Radical 10 1,677 Nov-07-2023, 09:49 AM
Last Post: buran
  Question about Creating an Automated Process in Python Supratik1234 0 741 Jan-13-2023, 08:29 PM
Last Post: Supratik1234
  Help adding a loop inside a loop Extra 31 4,512 Oct-23-2022, 12:16 AM
Last Post: Extra
  Confused about python execution jpezz 4 1,369 Oct-09-2022, 06:56 PM
Last Post: Gribouillis
  Python multiprocessing Pool apply async wait for process to complete sunny9495 6 6,388 Apr-02-2022, 06:31 AM
Last Post: sunny9495
  Process the image on the Python HTTP server Aleks 0 3,195 Dec-02-2021, 11:43 PM
Last Post: Aleks
  How to measure execution time of a multithread loop spacedog 2 2,860 Apr-24-2021, 07:52 AM
Last Post: spacedog
  Adding to the dictionary inside the for-loop - weird behaviour InputOutput007 5 2,691 Jan-21-2021, 02:21 PM
Last Post: InputOutput007

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020