Python Forum
What does .flush do? How can I change this to write to the file? - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: What does .flush do? How can I change this to write to the file? (/thread-42005.html)

What does .flush do? How can I change this to write to the file? - Pedroski55 - Apr-22-2024

Looking at using yield, I have this simulated log file: logsim.py

logsim.py has 2 very long lists, ips and docs

ips like this: ips = ['129.194.8.73', '198.37.27.153', '217.127.12.71', ...
docs like this: docs = ['"GET /ply/ply-2.0.tar.gz HTTP/1.0" 304 -',
'"GET /python/tutorial/beazley_advanced_python/Slides/SLIDE066.HTM HTTP/1.0" 200 1538', ....

The tutorial is here.

logsim.py code, apart from the lists ips and docs:

import time, random
# a fake log programme

f = open("/home/pedro/temp/access-log.log","w")

while True:
    time.sleep(random.random())
    n = random.randint(0,len(ips)-1)
    m = random.randint(0,len(docs)-1)
    t = time.time()
    date = time.strftime("[%d/%b/%Y:%H:%M:%S -0600]",time.localtime(t))
    #print(f,"%s - - %s %s" % (ips[n],date,docs[m]))
    print(f'{f} {ips[n]},{date},{docs[m]}')      
    f.flush()
For the tutorial it says, leave this running in the background, so I run it in bash.
But nothing is written to /home/pedro/temp/access-log.log when I run this in bash

Should I change f.flush() for f.write(ips[n] + ',' + date + ',' + docs[m])

Often, the file does not change until f.close() is called, but I can't do that, because logsim.py is still writing to f!

How to get the data actually written to /home/pedro/temp/access-log.log without closing the file?
How to dynamically update the file, without closing it?

RE: What does .flush do? How can I change this to write to the file? - Gribouillis - Apr-22-2024

Try
print(f'{ips[n]},{date},{docs[m]}', file=f)
f.flush()


RE: What does .flush do? How can I change this to write to the file? - snippsat - Apr-22-2024

Can also write it like this,updatet for Python 3.
log_sim.py:
import time
import random

ips = [
    '99.167.100.246', '74.6.23.48', '77.91.224.3', '67.176.147.11',
    '84.237.120.134', '64.92.161.138', '87.244.30.18', '84.110.229.30',
    '201.213.27.208', '80.99.119.201', '74.6.28.156', '213.217.40.133',
    '62.224.58.211', '74.6.28.151', '65.166.139.21', '192.25.206.10',
    '66.79.171.46', '80.91.229.6', '201.63.117.142', '144.51.43.161',
    '200.13.243.76', '213.67.242.79', '159.149.89.40', '41.222.70.189',
    '201.18.38.242', '220.239.245.127', '74.6.26.203', '216.160.75.249',
    '203.101.103.2', '74.6.24.228'
]

docs = [
    '"GET /ply/ply-2.0.tar.gz HTTP/1.0" 304 -',
    '"GET /python/tutorial/beazley_advanced_python/Slides/SLIDE066.HTM HTTP/1.0" 200 1538',
]

with open("access-log", "w") as f:
    while True:
        time.sleep(random.random())
        n = random.randint(0, len(ips) - 1)
        m = random.randint(0, len(docs) - 1)
        t = time.time()
        date = time.strftime("[%d/%b/%Y:%H:%M:%S -0600]", time.localtime(t))
        f.write(f"{ips[n]} - - {date} {docs[m]}\n")
        f.flush()
follow.py:
import time

def follow(thefile):
    thefile.seek(0, 2)
    while True:
        line = thefile.readline()
        if not line:
            time.sleep(0.1)
            continue
        yield line

if __name__ == '__main__':
    with open("access-log", "r") as logfile:
        for line in follow(logfile):
            # print function in Python 3
            print(line, end='')
Test make sure that start log_sim.py first.
Output:
G:\div_code\reader_env
λ python follow.py
203.101.103.2 - - [22/Apr/2024:13:31:37 -0600] "GET /python/tutorial/beazley_advanced_python/Slides/SLIDE066.HTM HTTP/1.0" 200 1538
144.51.43.161 - - [22/Apr/2024:13:31:38 -0600] "GET /ply/ply-2.0.tar.gz HTTP/1.0" 304 -
84.237.120.134 - - [22/Apr/2024:13:31:38 -0600] "GET /python/tutorial/beazley_advanced_python/Slides/SLIDE066.HTM HTTP/1.0" 200 1538
67.176.147.11 - - [22/Apr/2024:13:31:38 -0600] "GET /python/tutorial/beazley_advanced_python/Slides/SLIDE066.HTM HTTP/1.0" 200 1538
159.149.89.40 - - [22/Apr/2024:13:31:39 -0600] "GET /ply/ply-2.0.tar.gz HTTP/1.0" 304 -
213.217.40.133 - - [22/Apr/2024:13:31:39 -0600] "GET /ply/ply-2.0.tar.gz HTTP/1.0" 304 -
220.239.245.127 - - [22/Apr/2024:13:31:40 -0600] "GET /ply/ply-2.0.tar.gz HTTP/1.0" 304 -
66.79.171.46 - - [22/Apr/2024:13:31:40 -0600] "GET /python/tutorial/beazley_advanced_python/Slides/SLIDE066.HTM HTTP/1.0" 200 1538
201.18.38.242 - - [22/Apr/2024:13:31:40 -0600] "GET /ply/ply-2.0.tar.gz HTTP/1.0" 304 -
99.167.100.246 - - [22/Apr/2024:13:31:41 -0600] "GET /python/tutorial/beazley_advanced_python/Slides/SLIDE066.HTM HTTP/1.0" 200 1538
74.6.24.228 - - [22/Apr/2024:13:31:41 -0600] "GET /python/tutorial/beazley_advanced_python/Slides/SLIDE066.HTM HTTP/1.0" 200 1538
87.244.30.18 - - [22/Apr/2024:13:31:42 -0600] "GET /python/tutorial/beazley_advanced_python/Slides/SLIDE066.HTM HTTP/1.0" 200 1538
77.91.224.3 - - [22/Apr/2024:13:31:42 -0600] "GET /python/tutorial/beazley_advanced_python/Slides/SLIDE066.HTM HTTP/1.0" 200 1538
192.25.206.10 - - [22/Apr/2024:13:31:42 -0600] "GET /python/tutorial/beazley_advanced_python/Slides/SLIDE066.HTM HTTP/1.0" 200 1538
Traceback (most recent call last):
  File "G:\div_code\reader_env\follow.py", line 14, in <module>
    for line in follow(logfile):
  File "G:\div_code\reader_env\follow.py", line 8, in follow
    time.sleep(0.1)
KeyboardInterrupt
^C
G:\div_code\reader_env


RE: What does .flush do? How can I change this to write to the file? - snippsat - Apr-22-2024

Also for fun as this is written to be like tail -f which is command line tool.
Then can write to be a command line tool,eg i think Typer is cool.
So Typer is tool like eg argparse in standard library. 
# py_tail.py
import typer
from collections import deque
import time

app = typer.Typer()

def follow_file(filename: str) -> None:
    with open(filename, 'r') as fh:
        # Go to the end of the file
        fh.seek(0, 2)
        while True:
            line = fh.readline()
            if not line:
                time.sleep(1)  # Wait a bit for new content
                continue
            print(line.rstrip('\n'))

@app.command()
def tail(
    filename: str,
    n: int = typer.Option(10, "--number", "-n", help="The last n lines of file"),
    follow: bool = typer.Option(False, "--follow", "-f", help="Show data as the file grows.")
    ):
    """
    Print the last n lines from a file. Optionally follow the file for new lines.
    """
    lines = deque(maxlen=n)
    try:
        with open(filename) as fh:
            for line in fh:
                lines.append(line.rstrip('\n'))
        for line in lines:
            print(line)
    except FileNotFoundError:
        typer.echo(f"File not found: {filename}")
        raise typer.Exit(code=1)
    if follow:
        follow_file(filename)

if __name__ == "__main__":
    app()
Now it's a command line tool and have also added eg tail -n 5 show the n lines of the file.
See that color is added automatic(it use Rich) which make it a good looking tool.
[Image: qUXPoG.png]