Save console output to text file (like a log)

Save console output to text file (like a log) - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Save console output to text file (like a log) (/thread-18824.html)

Save console output to text file (like a log) - koticphreak - Jun-02-2019

I have a python script that runs and it outputs things along the way. It's important that it outputs them in the command line, but at the end, I want to save whatever it output to a txt file, like a log. How would I do this? Thanks in advance!

RE: Save console output to text file (like a log) - buran - Jun-02-2019

check logging module. you can specify both console and file handler(s)

RE: Save console output to text file (like a log) - koticphreak - Jun-02-2019

(Jun-02-2019, 04:57 PM)buran Wrote: check logging module. you can specify both console and file handler(s)

Thanks, this is helpful! If I'm reading this correctly, I have to both print things and send them to the log, is that right? So like this:

print('Data Ingestion Successful')
logging.info('Data Ingestion Successful')

There's no way for it to just capture everything that gets output? As part of the script, it runs a 3rd party program and half of the output is from that program - trying to capture everything, and not seeing a simple way to do that? Thanks again!

RE: Save console output to text file (like a log) - buran - Jun-02-2019

(Jun-02-2019, 05:15 PM)koticphreak Wrote: As part of the script, it runs a 3rd party program and half of the output is from that program - trying to capture everything, and not seeing a simple way to do that?

how do you run this 3-rd party program?
you can capture the output when run it using subprocess module

RE: Save console output to text file (like a log) - buran - Jun-02-2019

(Jun-02-2019, 05:15 PM)koticphreak Wrote: this is helpful! If I'm reading this correctly, I have to both print things and send them to the log, is that right?

you can specify console handler and it will output to console. then if you specify also file handler/rotating file handler then you can have the output also in the log, e..

#set console handler
#set file handler
logging.info('Data Ingestion Successful') # this will go in both file and console

there are tutorials how to set the logging handlers

RE: Save console output to text file (like a log) - koticphreak - Jun-02-2019

(Jun-02-2019, 05:31 PM)buran Wrote:
(Jun-02-2019, 05:15 PM)koticphreak Wrote: As part of the script, it runs a 3rd party program and half of the output is from that program - trying to capture everything, and not seeing a simple way to do that?
how do you run this 3-rd party program?
you can capture the output when run it using subprocess module

Yes, that's exactly how I do it, using subprocess. Struggling to understand how to have it log it all. Having the logging module setup, but can only seem to pass things to the log using "logging.info/debug/warning", rather than having it just capture everything.

RE: Save console output to text file (like a log) - buran - Jun-02-2019

https://stackoverflow.com/questions/4675728/redirect-stdout-to-a-file-in-python

RE: Save console output to text file (like a log) - snippsat - Jun-02-2019

An example using logging and catch subprocess output.
Using logzero simplify logging,as don't have to write all that boilerplate logging code.

import subprocess
import logzero
from logzero import logger

# 3 rotations, each with a maximum filesize of 1MB:
logzero.logfile("logfile.log", maxBytes=1e6, backupCount=3, disableStderrLogger=True)

out = subprocess.run(['ping', 'google.com'], capture_output=True)
output = out.stdout.decode()
logger.info(output)

So now output only goes to logfile,disableStderrLogger=True False if want output also in console.

Output:[I 190602 20:45:39 log_zero:10]
    Pinging google.com [2a00:1450:400f:809::200e] with 32 bytes of data:
    Request timed out.
    Request timed out.
    Reply from 2a00:1450:400f:809::200e: time=94ms 
    Reply from 2a00:1450:400f:809::200e: time=84ms 
    
    Ping statistics for 2a00:1450:400f:809::200e:
        Packets: Sent = 4, Received = 2, Lost = 2 (50% loss),
    Approximate round trip times in milli-seconds:
        Minimum = 84ms, Maximum = 94ms, Average = 89ms

RE: Save console output to text file (like a log) - koticphreak - Jun-02-2019

(Jun-02-2019, 06:48 PM)snippsat Wrote: Using logzero simplify logging,as don't have to write all that boilerplate logging code.

Thanks for the help! Getting this error:

Error:Traceback (most recent call last):
  File "C:\script.py", line 83, in <module>
    output = out.stdout.decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa9 in position 44: invalid start byte

RE: Save console output to text file (like a log) - snippsat - Jun-02-2019

Take away .decode(),will save in bytes this may be okay as you can work with bytes to.
Your output can have different encoding than utf-8.
Try .decode('utf-8', errors="ignore")
Can guess eg .decode('latin-1'),a module that can guess on encoding chardet.