how to change the range of read CSV file every time python file runs

how to change the range of read CSV file every time python file runs - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: how to change the range of read CSV file every time python file runs (/thread-31390.html)

how to change the range of read CSV file every time python file runs - greenpine - Dec-08-2020

Hello;

I have a function that uses data from CSV file row by row, until the range has been used.

[Edited]
however I want to use the rest of rows of data automatically, each time it runs.

when it runs for the second time, I want it starts from where it was left off,
in this example that I have, for the second run it should read from row 2 to 4, and when it runs again for the 3rd time, it should read data from row 4 to 6.
so every time the file runs, it print 2 rows of data ( not the same row) until there is no data on CSV file.
is this doable?
your help is appreciated.

here is the part of read CSV file in range that works:

with open('file1.csv', 'r') as f:
                        for i, line in enumerate(f, start=0):
                            if i in range(0, 2):       
                                print(line.rstrip()))

Out put:

220445566
334587993

RE: how to change the range of read CSV file every time python file runs - bowlofred - Dec-08-2020

Line 2 loops over the entire CSV (consuming it) until complete, or unless you break out of the loop. The main 2 ways to do this would be:

* Loop over a range and consume one line each time. Similar to:

with open("my.csv") as f:
    for index in range(2):
        line = f.readline().rstrip()
        print(f"Got line {index} {line}")

    ### Can read more lines from f here
    ### If you read past the end, f.readline() will start returning empty strings

* Loop over the file, but break out when your limit is reached.

with open("my.csv" as f:
    for index, line in enumerate(f):
        print(f"Got line {index} {line.rstrip()}")
        if index >= (2 - 1):
            break
    
    ### Can read more lines from f here
    ### If you read past the end of the file, the loop exits and the break is never run

RE: how to change the range of read CSV file every time python file runs - greenpine - Dec-08-2020

thanks for the reply;
I tried both codes, it returns the same result (first 2 row),every time I run the file,
I don't understand how this could go over the next line on the second run,
what I was thinking, I have to somehow record the ranges when it runs, perhaps in a text file, the next run it reads from the recorded text file and add one or two to the digit, and change the range to a new one, and so on. I might be wrong, or something easier is the solution, but I am not in that level to do this.

RE: how to change the range of read CSV file every time python file runs - bowlofred - Dec-08-2020

I thought you meant that you wanted to read the rest of the CSV later in the same script.

I see now that you just want to read the "new" stuff each time it runs. Yes, you'd have to keep state somewhere like the file offset or the number of lines previously read.

This partial solution can have problems if it crashes or is run simultaneously (no locking, no error checking, etc.)

csv_filename = "my.csv"
csv_read_counter_file = "csv_last_read"
with open(csv_read_counter_file, "r") as f:
    try:
        lines_read = int(f.readline().rstrip())
    except:
        # On any error, just reset back to start of file.
        lines_read = 0

with open(csv_filename, "r") as f:
    for index, line in enumerate(f):
        if index < lines_read:
            continue
        print(line.rstrip())

with open(csv_read_counter_file, "w") as f:
    f.write(str(index))

RE: how to change the range of read CSV file every time python file runs - greenpine - Dec-08-2020

thanks for the code,
it prints only the first row from CSV file, I run it number of times,
and inside the "csv_last_read.csv" file there is recorded digit of 9,
is there a way to add a range to it?

RE: how to change the range of read CSV file every time python file runs - bowlofred - Dec-08-2020

I meant for the open attempt on the read file to be inside the "try" section. This is a minor change:

csv_filename = "my.csv"
csv_read_counter_file = "csv_last_read"
try:
    with open(csv_read_counter_file, "r") as f:
        lines_read = int(f.readline().rstrip())
except:
    # On any error, just reset back to start of file.
    lines_read = 0

with open(csv_filename, "r") as f:
    for index, line in enumerate(f):
        if index <= lines_read:
            continue
        print(line.rstrip())

with open(csv_read_counter_file, "w") as f:
    f.write(str(index))

I don't know why you would only be getting the first line. It should skip until the lines after that point are read.

I'm sure you could incorporate a range check instead of a simple "index <= lines_read" check. You would just need to determine how you want to populate the range.

RE: how to change the range of read CSV file every time python file runs - greenpine - Dec-08-2020

thanks for the new code,
I run the new code:

csv_filename = "my.csv"
csv_read_counter_file = "csv_last_read.csv"
try:
    with open(csv_read_counter_file, "r") as f:
        lines_read = int(f.readline().rstrip())
except:
    # On any error, just reset back to start of file.
    lines_read = 0
 
with open(csv_filename, "r") as f:
    for index, line in enumerate(f):
        if index <= lines_read:
            continue
        print(line.rstrip())
 
with open(csv_read_counter_file, "w") as f:
    f.write(str(index))

now it prints all the data at once, I have now 20 rows of numbers in CSV file instead of 10 to test.
if I write a number in "csv_last_read.csv" and save the file, let's say 3, when I run the file, it prints all rows after row 3.