Controlling what get outputted to stdout when running external commands

Controlling what get outputted to stdout when running external commands - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Controlling what get outputted to stdout when running external commands (/thread-36197.html)

Controlling what get outputted to stdout when running external commands - Daring_T - Jan-27-2022

I am using Python to compress video files via Handbrake and subprocess. When I run my current code the HandbrakeCli output still outputs to stdout. Is there a better way to contain and control the output of HandbrakeCli in Python?

import subprocess
from tqdm import tqdm

handbrake_cli = "HandBrakeCLI.exe"
input_path = ""
json_preset = ""
output_path = ""


command = [handbrake_cli, "-i", input_path, "--json", json_preset, "-o", output_path]
p = subprocess.Popen(command, stdout=subprocess.PIPE, text=True)

with tqdm(total=100, dynamic_ncols=True, desc="Compressing file... ") as prbar:
	while (line := p.stdout.readline()) != "":
		line = line.rstrip()
		if '"Progress"' in line:
			progress = line.split(":")[1].lstrip(".").split(".")[0]
			# print(f"Progress: {progress}")
			prbar.update(progress)

print(f"End of output.  Return code: {p.wait()}")

RE: Controlling what get outputted to stdout when running external commands - Daring_T - Jan-29-2022

Here is the output of the parsing code above and the control as well. I am trying to get the output to should look like this so I can manipulate it further.

Output:Progress: 0.60
Progress: 0.12
Progress: 0.22
Progress: 0.38
Progress: 0.42
Progress: 0.52
Progress: 0.60
Progress: 0.82
Progress: 0.94

RE: Controlling what get outputted to stdout when running external commands - bowlofred - Jan-29-2022

I wonder if some of this is because the console is getting both stdin and stderr.

If you run handbrake [options....] > /dev/null do you get the output you're trying to ignore? If so, (and if you don't need the stderr information), then try adding stderr=subprocess.DEVNULL to have that stream ignored.

If you do need to parse the stderr stream, then setting it to stderr=subprocess.STDOUT should work to combine them.

RE: Controlling what get outputted to stdout when running external commands - Daring_T - Jan-30-2022

(Jan-29-2022, 07:21 PM)bowlofred Wrote: I wonder if some of this is because the console is getting both stdin and stderr.

If you run handbrake [options....] > /dev/null do you get the output you're trying to ignore? If so, (and if you don't need the stderr information), then try adding stderr=subprocess.DEVNULL to have that stream ignored.

If you do need to parse the stderr stream, then setting it to stderr=subprocess.STDOUT should work to combine them.

I put the stderr stream inline with the stdout stream like shown above and that woked.

p = subprocess.Popen(command, stdout=subprocess.PIPE, text=True, stderr=subprocess.STDOUT)

Is there any reason as to why its is hard to capture two or three streams? If you have any good resorces on the topic I would a greatly aprecate it if you could send them my way.

RE: Controlling what get outputted to stdout when running external commands - bowlofred - Jan-30-2022

(Jan-30-2022, 04:06 PM)Daring_T Wrote: Is there any reason as to why its is hard to capture two or three streams? If you have any good resorces on the topic I would a greatly aprecate it if you could send them my way.

What streams do you want to capture? You can read from stdout and stderr directly. Combining them is usually easier because you don't have to do any complicated poll or non-blocking reads. If both streams are independent and you do a blocking read on one, you won't see any updates on the other one until the first one sends more data.