Python Forum

Full Version: waiting for many processes in parallel
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
i have started about 20 processes using subprocess.Popen(command,stdout=PIPE) and have gotten EOF on all their pipes. now i want the parent script to wait for all of them to exit. for various reasons some could take several minutes to exit. i'd like to allow up to 5 minutes for the whole bunch. is there a way to wait for them all as a group with just a single timeout? normally i would just for loop through the list of objects i got from Popen and call each.join(). but putting the 300 second timeout in there could end up being a long time if several are slow. is there a better way to wait for them all while giving up after 5 minutes?
i'm not using .communicate(). each process has a pipe its stdout goes to.

there are many processes. i need to exit quickly/immediately if ALL of the processes exit in a timely manner (the usual case).

the issue is how to wait on them ALL in a manner that when they ALL exit it wakes up. the timer is good enough for the timeout event if one or more stays running.