Python Forum

i want to have my script read a file that another process may or may not have written more to that file extending its size. in unix, the "tail -f" command does this once it has reached EOF. i want my script to read this way. imagine the script is implementing that "tail -f" command. my script will be doing something different but still needs to continuously read a status file up to the end. what i am worried about is if there is something i need to do for this in the Python I/O logic. once a file being read reaches EOF could Python save the EOF state and give EOF each time the script tries to read that file, even if the underlying file in the system has grown? is there anything in particular my script code needs to do if i use the Python I/O interface or should i just read that file via the unix system I/O interface? this project will only be running on Linux.

I suggest to read David Beazley's presentation Generator Tricks for Systems Programmers.

On slides 74, 75, 76 there is example of tail -f implementation in Python. Following is excerpt from these slides:

A Python version of 'tail -f'

import time
import os

def follow(thefile):
    thefile.seek(0, os.SEEK_END) # End-of-file
    while True:
         line = thefile.readline()
         if not line:
             time.sleep(0.1)    # Sleep briefly
             continue
         yield line

Idea : Seek to the end of the file and repeatedly try to read new lines. If new data is written to the file, we'll pick it up.

Using our follow function

logfile  = open("access-log")
loglines = follow(logfile)

for line in loglines:
    print(line, end='')

This produces the same output as 'tail -f'

i think that excerpt tells me all i need to know ... that Python doesn't get in the way of doing this, such as setting a flag when reading get EOF that prevents further reading. well, maybe seek to SEEK_END resets it