Python Forum
Run some subrocesses at a time
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Run some subrocesses at a time
#1
Hello.

Lets say we need to run 100-1000 subprocesses to do some work. Each subprocess is some kind of app, for example zip.

it is not good to spawn all process at a time. We need to limit them. But the question is how can we monitor that a subpocess has finished and it is time to start a new one?

procs_list = [Popen(arg) for arg in args]
for proc in procs_list:
	proc.wait()
This doesn't work because it waits all processes to finish but we need to spawn a new one as soon as any subprocess has finished.

I think it comes down to a question if it possible to determine if any process from a list has finished.
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020