Python Forum
How to parallel executing a command using subprocess?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How to parallel executing a command using subprocess?
#1
./PoreDy is the command I want to execute, and jobname and 1001 is the argument it required.
My code is as follows:

TotalThread=24
CheckInterval=2
for i in range(CasesCount):
		jobname=Name(FolderName+'_', i)
		cmd='cp -f'+SettingMatch+' '+jobname
		subprocess.call(cmd, shell=True) # Pay Attention
		print('executed ', cmd)

		cmd='./PoreDy '+jobname+' 1001'
		job=subprocess.Popen(cmd, shell=True) # Pay Attention
		Runs.append(int(job.pid)+1)
		print('executing ', cmd, ' Job ID: ', int(job.pid)+1)
		print(Runs)
		time.sleep(1)

		while len(Runs)>=TotalThread:
			for run in Runs:
				if Running(run)==False:
					print('PID ', run, ' Finished!')
					Runs.remove(run)
			time.sleep(CheckInterval)
I got two problems:
1: the command has many outputs, I don't want to see its stdout which usually shows on screen when I manually run it.
2: looks this code doesn't fully use my all threads, actually only 4 or 5 of them are used simultaneously.

Please help out, this stuck me for a long time.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  subprocess for executing over CLI flash77 5 682 Jul-09-2023, 04:51 AM
Last Post: flash77
  Using subprocess to execute complex command with many arguments medatib531 5 1,897 Apr-27-2023, 02:23 PM
Last Post: medatib531
  use subprocess on linux\pi wwith a "grep " command korenron 2 8,121 Oct-19-2021, 10:52 AM
Last Post: DeaD_EyE
  How to run parallel command (same command -ping) korenron 3 2,836 Dec-12-2020, 11:04 AM
Last Post: palladium
  Error when running mktorrent subprocess command pythonnewbie138 4 3,880 Sep-16-2020, 01:55 AM
Last Post: pythonnewbie138
  Select correct item from list for subprocess command pythonnewbie138 6 3,331 Jul-24-2020, 09:09 PM
Last Post: pythonnewbie138
  Executing a command which fails ebolisa 0 1,503 May-01-2020, 01:51 PM
Last Post: ebolisa
  Subprocess command working for one cmd and for cmd one not wrking PythonBeginner_2020 0 4,147 Mar-25-2020, 01:52 PM
Last Post: PythonBeginner_2020
  A question about subprocess taking input from command line and returning output! Aurimas 8 5,225 May-15-2019, 04:02 PM
Last Post: Aurimas
  Error executing Jupyter command 'notebook': [Errno 'jupyter-notebook' not found] 2 Newtopython123 10 31,348 Apr-25-2019, 07:30 AM
Last Post: banu0395

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020