Save console output to text file (like a log)

[koticphreak]

I have a python script that runs and it outputs things along the way. It's important that it outputs them in the command line, but at the end, I want to save whatever it output to a txt file, like a log. How would I do this? Thanks in advance!

[buran]

check logging module. you can specify both console and file handler(s)

[koticphreak]

check logging module. you can specify both console and file handler(s)

Thanks, this is helpful! If I'm reading this correctly, I have to both print things and send them to the log, is that right? So like this:

print('Data Ingestion Successful')
logging.info('Data Ingestion Successful')

There's no way for it to just capture everything that gets output? As part of the script, it runs a 3rd party program and half of the output is from that program - trying to capture everything, and not seeing a simple way to do that? Thanks again!

[buran]

As part of the script, it runs a 3rd party program and half of the output is from that program - trying to capture everything, and not seeing a simple way to do that?

how do you run this 3-rd party program?
you can capture the output when run it using subprocess module

[buran]

this is helpful! If I'm reading this correctly, I have to both print things and send them to the log, is that right?

you can specify console handler and it will output to console. then if you specify also file handler/rotating file handler then you can have the output also in the log, e..

#set console handler
#set file handler
logging.info('Data Ingestion Successful') # this will go in both file and console

there are tutorials how to set the logging handlers

[koticphreak]

As part of the script, it runs a 3rd party program and half of the output is from that program - trying to capture everything, and not seeing a simple way to do that?

how do you run this 3-rd party program?
you can capture the output when run it using subprocess module

Yes, that's exactly how I do it, using subprocess. Struggling to understand how to have it log it all. Having the logging module setup, but can only seem to pass things to the log using "logging.info/debug/warning", rather than having it just capture everything.

[buran]

https://stackoverflow.com/questions/4675...-in-python

[snippsat]

An example using logging and catch subprocess output.
Using logzero simplify logging,as don't have to write all that boilerplate logging code.

import subprocess
import logzero
from logzero import logger

# 3 rotations, each with a maximum filesize of 1MB:
logzero.logfile("logfile.log", maxBytes=1e6, backupCount=3, disableStderrLogger=True)

out = subprocess.run(['ping', 'google.com'], capture_output=True)
output = out.stdout.decode()
logger.info(output)

So now output only goes to logfile,disableStderrLogger=True False if want output also in console.

Output:
[I 190602 20:45:39 log_zero:10]
    Pinging google.com [2a00:1450:400f:809::200e] with 32 bytes of data:
    Request timed out.
    Request timed out.
    Reply from 2a00:1450:400f:809::200e: time=94ms 
    Reply from 2a00:1450:400f:809::200e: time=84ms 
    
    Ping statistics for 2a00:1450:400f:809::200e:
        Packets: Sent = 4, Received = 2, Lost = 2 (50% loss),
    Approximate round trip times in milli-seconds:
        Minimum = 84ms, Maximum = 94ms, Average = 89ms

[koticphreak]

Using logzero simplify logging,as don't have to write all that boilerplate logging code.

Thanks for the help! Getting this error:

Error:
Traceback (most recent call last):
  File "C:\script.py", line 83, in <module>
    output = out.stdout.decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa9 in position 44: invalid start byte

[snippsat]

Take away .decode(),will save in bytes this may be okay as you can work with bytes to.
Your output can have different encoding than utf-8.
Try .decode('utf-8', errors="ignore")
Can guess eg .decode('latin-1'),a module that can guess on encoding chardet.