Python Forum
multi-threading error in minimal script
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
multi-threading error in minimal script
#1
i am getting the error shown below in a minimal little script that runs just 2 threads:
Output:
lt2a/forums /home/forums 1> cat -n trytwocommands.py 1 #!/usr/bin/env python3 2 import subprocess,threading 3 one = threading.Thread(target=subprocess.call,args=['sleep','10001']) 4 two = threading.Thread(target=subprocess.call,args=['sleep','10002']) 5 one.start() 6 two.start() 7 print('started 2 threads, now joining them',flush=1) 8 one.join() 9 two.join() 10 print('joined 2 threads, now done',flush=1) lt2a/forums /home/forums 2> py3 trytwocommands.py started 2 threads, now joining them Exception in thread Thread-2: Traceback (most recent call last): File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/usr/lib/python3.6/subprocess.py", line 287, in call with Popen(*popenargs, **kwargs) as p: File "/usr/lib/python3.6/subprocess.py", line 629, in __init__ raise TypeError("bufsize must be an integer") TypeError: bufsize must be an integer Exception in thread Thread-1: Traceback (most recent call last): File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/usr/lib/python3.6/subprocess.py", line 287, in call with Popen(*popenargs, **kwargs) as p: File "/usr/lib/python3.6/subprocess.py", line 629, in __init__ raise TypeError("bufsize must be an integer") TypeError: bufsize must be an integer joined 2 threads, now done lt2a/forums /home/forums 3>
i presume the 2 errors happen in the 2 threads, especially considering the output from the print calls before and after.
what is bufsize and what set it to a non-integer? is there a way i can set it? or is subprocess.call() not thread-safe?
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#2
It seems to me that you should pass ,args=[['sleep','10001']] otherwise python "thinks" that the args to Popen is 'sleep' and that '10001' is the bufsize argument of Popen.
Reply
#3
i was making sure subprocess.call (via Popen) can run commands in separate processes from subthreads so i can, at least, consider threads as a way to avoid having 2 processes per concurrent command. it still won't let me leave the last bunch of commands running, but for now this gets this project going (faster backups to AWS S3).

#!/usr/bin/env python3
import subprocess,threading
m = 64
threads = [threading.Thread(target=subprocess.call,args=(['sleep',str(10000+x)],)) for x in range(1,m+1)]
[threads[x].start() for x in range(m)]
print(f'started {m} threads',flush=1)
[threads[x].join() for x in range(m)]
print(f'joined {m} threads',flush=1)
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Concurrent futures threading running at same speed as non-threading billykid999 13 1,716 May-03-2023, 08:22 AM
Last Post: billykid999
  Trouble with threading and reading variable from a different script Lembas 14 2,841 Apr-26-2023, 11:21 PM
Last Post: Lembas
  Tutorials on sockets, threading and multi-threading? muzikman 2 2,076 Oct-01-2021, 08:32 PM
Last Post: muzikman
  Need help multi-threading scraping spacedog 2 2,434 Apr-28-2021, 03:48 PM
Last Post: spacedog
  Embedding python cause crash when use boost::asio multi threading udvatt108 0 1,695 Oct-04-2020, 03:15 PM
Last Post: udvatt108
  Multi-threading Evil_Patrick 2 21,470 Jul-15-2020, 09:55 AM
Last Post: snippsat
  Minimal python install for thonny ide DanielRossinsky 0 1,487 Jun-11-2020, 02:31 PM
Last Post: DanielRossinsky
  object base multi threading maboobelahi93 0 1,411 Jan-29-2020, 11:21 AM
Last Post: maboobelahi93
  Problem with Python, MySQL and Multi-threading queries zagk 1 11,812 Jul-01-2017, 12:15 AM
Last Post: zagk
  SSH Multi Threading nizami22247 0 9,544 Mar-31-2017, 06:04 PM
Last Post: nizami22247

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020