Python Forum
no coding= option fo subprocess.Popen()
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
no coding= option fo subprocess.Popen()
#1
i want to have utf8 decoding on the stdout pipe with subprocess.Popen() so when it is read i get str instead of bytes, but i see no option to set that. any suggestions?
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#2
Have you looked at doc?
See parameter encoding=None setting to encoding='utf-8' stdout will by string(Unicode default) Python 3.

Work in run() to which should be used in all cases it can handle.
subprocess.run() Wrote:The recommended approach to invoking subprocesses is to use the run() function for all use cases it can handle.
For more advanced use cases, the underlying Popen interface can be used directly.
import subprocess
 
output = subprocess.run(['ping', '-n', '4', 'google.com'], encoding='utf-8', capture_output=True)
print(output.stdout)
Output:
Pinging google.com [2a00:1450:400f:80b::200e] with 32 bytes of data: Reply from 2a00:1450:400f:80b::200e: time=42ms Reply from 2a00:1450:400f:80b::200e: time=33ms Reply from 2a00:1450:400f:80b::200e: time=46ms Reply from 2a00:1450:400f:80b::200e: time=50ms Ping statistics for 2a00:1450:400f:80b::200e: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 33ms, Maximum = 50ms, Average = 42ms
Reply
#3
i need to read the data as it arrives, hence the need for the pipe. it had previously written the data to a temporary file and read the file and that got str from read, but now i need to read from a pipe, and now i get bytes. i will look at another possible design, but i do want to solve this with Popen(). on your ping command test, try doing it without the option -n 4 on the command and get your pings out as they arrive.

i just did try run() and it looks like (from the trace dump) that it calls Popen() itself, passing all the keyword options. my documentation does not show an encoding= option for either run() or Popen().

Output:
Traceback (most recent call last): File "test1.py", line 10, in <module> out = run(['aws','s3','ls'],encoding='utf-8',capture_output=True) File "/usr/lib/python3.5/subprocess.py", line 693, in run with Popen(*popenargs, **kwargs) as process: TypeError: __init__() got an unexpected keyword argument 'encoding'

from the source code:
Output:
lt2a/forums /home/forums 8> cat -n /usr/lib/python3.5/subprocess.py|lines 830 874 830 class Popen(object): 831 832 _child_created = False # Set here since __del__ checks it 833 834 def __init__(self, args, bufsize=-1, executable=None, 835 stdin=None, stdout=None, stderr=None, 836 preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, 837 shell=False, cwd=None, env=None, universal_newlines=False, 838 startupinfo=None, creationflags=0, 839 restore_signals=True, start_new_session=False, 840 pass_fds=()): 841 """Create new Popen instance.""" 842 _cleanup() 843 # Held while anything is calling waitpid before returncode has been 844 # updated to prevent clobbering returncode if wait() or poll() are 845 # called from multiple threads at once. After acquiring the lock, 846 # code must re-check self.returncode to see if another thread just 847 # finished a waitpid() call. 848 self._waitpid_lock = threading.Lock() 849 850 self._input = None 851 self._communication_started = False 852 if bufsize is None: 853 bufsize = -1 # Restore default 854 if not isinstance(bufsize, int): 855 raise TypeError("bufsize must be an integer") 856 857 if _mswindows: 858 if preexec_fn is not None: 859 raise ValueError("preexec_fn is not supported on Windows " 860 "platforms") 861 any_stdio_set = (stdin is not None or stdout is not None or 862 stderr is not None) 863 if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: 864 if any_stdio_set: 865 close_fds = False 866 else: 867 close_fds = True 868 elif close_fds and any_stdio_set: 869 raise ValueError( 870 "close_fds is not supported on Windows platforms" 871 " if you redirect stdin/stdout/stderr") 872 else: 873 # POSIX lt2a/forums /home/forums 9>
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#4
(Jun-21-2019, 08:07 PM)Skaperen Wrote: my documentation does not show an encoding= option for either run() or Popen().
Upgrade your Python version,what show here don't work for Python 3.5.
In Python 3.5 has to use decode() on stdout output.
Reply
#5
i found that run() can capture stdout when given stdout=PIPE but that capture is bytes and i still need str.

i'm still trying to find a way to upgrade it. pyenv doesn't do it.

a function to convert bytes to str would solve this. oh ... bytes.decode().
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#6
i got a new design working that could handle the output all at once.

another coming program will need to do that ... it can handle all the output all at once ... from 16 concurrent processes. it needs to make sure the outputs don't get mixed up, although it can take the 16 outputs in any order.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
Information subprocess.Popen() suddenly giving me grief? davecotter 3 528 Dec-13-2023, 10:49 PM
Last Post: davecotter
  Use subprocess.Popen and time.sleep chucky831 2 1,890 Aug-11-2022, 07:53 PM
Last Post: carecavoador
  Subprocess.Popen() not working when reading file path from csv file herwin 13 14,614 May-07-2021, 03:26 PM
Last Post: herwin
  Did subprocess.Popen() causes main routine to pause stdout? liudr 4 3,551 May-04-2021, 08:58 PM
Last Post: liudr
  disable subprocess.popen prompt echo paul18fr 1 1,966 Feb-04-2021, 02:50 AM
Last Post: Larz60+
  how to pass the interactive string to Popen subprocess maiya 1 1,845 Sep-18-2020, 09:36 PM
Last Post: Larz60+
  How to get program output from subprocess.Popen? glestwid 1 2,331 Aug-19-2020, 05:44 AM
Last Post: buran
  subprocess.Popen() and encodings voltron 0 5,680 Feb-20-2020, 04:57 PM
Last Post: voltron
  Mock call to subprocess.Popen failing with StopIteration tharpa 7 5,814 Nov-08-2019, 05:00 PM
Last Post: Gribouillis
  subprocess.Popen jenya56 3 3,329 Apr-03-2019, 07:02 AM
Last Post: Gribouillis

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020