Jul-11-2020, 03:44 AM
(This post was last modified: Jul-11-2020, 03:44 AM by ignorant_wanderer.)
How can I keep a loop going, while having some processes that are waiting for others inside the loop? (see the code below for clarification, it makes more sense to explain in code). PS: I'm using Popen from the multithreading module
for i in range(0, len(my_list), batch_size): current_batch = my_list[i:i+batch_size] download_processes = [Popen('do stuff')] # NOT ACTUAL CODE. This downloads files. _ = [p.wait() for p in download_processes] # wait until all files above download before executing the code below more_processes = [Popen('do stuff')] # NOT ACTUAL CODE. This zips the downloaded files # execute yet more processes (that process the zips files) when "more_processes" end, but keep the loop going to start downloading more files