Python Forum
how to change the range of read CSV file every time python file runs
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
how to change the range of read CSV file every time python file runs
#1
Hello;

I have a function that uses data from CSV file row by row, until the range has been used.

[Edited]
however I want to use the rest of rows of data automatically, each time it runs.

when it runs for the second time, I want it starts from where it was left off,
in this example that I have, for the second run it should read from row 2 to 4, and when it runs again for the 3rd time, it should read data from row 4 to 6.
so every time the file runs, it print 2 rows of data ( not the same row) until there is no data on CSV file.
is this doable?
your help is appreciated.

here is the part of read CSV file in range that works:

with open('file1.csv', 'r') as f:
                        for i, line in enumerate(f, start=0):
                            if i in range(0, 2):       
                                print(line.rstrip()))
Out put:
220445566
334587993
Reply
#2
Line 2 loops over the entire CSV (consuming it) until complete, or unless you break out of the loop. The main 2 ways to do this would be:

* Loop over a range and consume one line each time. Similar to:
with open("my.csv") as f:
    for index in range(2):
        line = f.readline().rstrip()
        print(f"Got line {index} {line}")

    ### Can read more lines from f here
    ### If you read past the end, f.readline() will start returning empty strings
* Loop over the file, but break out when your limit is reached.
with open("my.csv" as f:
    for index, line in enumerate(f):
        print(f"Got line {index} {line.rstrip()}")
        if index >= (2 - 1):
            break
    
    ### Can read more lines from f here
    ### If you read past the end of the file, the loop exits and the break is never run
Reply
#3
thanks for the reply;
I tried both codes, it returns the same result (first 2 row),every time I run the file,
I don't understand how this could go over the next line on the second run,
what I was thinking, I have to somehow record the ranges when it runs, perhaps in a text file, the next run it reads from the recorded text file and add one or two to the digit, and change the range to a new one, and so on. I might be wrong, or something easier is the solution, but I am not in that level to do this.
Reply
#4
I thought you meant that you wanted to read the rest of the CSV later in the same script.

I see now that you just want to read the "new" stuff each time it runs. Yes, you'd have to keep state somewhere like the file offset or the number of lines previously read.

This partial solution can have problems if it crashes or is run simultaneously (no locking, no error checking, etc.)

csv_filename = "my.csv"
csv_read_counter_file = "csv_last_read"
with open(csv_read_counter_file, "r") as f:
    try:
        lines_read = int(f.readline().rstrip())
    except:
        # On any error, just reset back to start of file.
        lines_read = 0

with open(csv_filename, "r") as f:
    for index, line in enumerate(f):
        if index < lines_read:
            continue
        print(line.rstrip())

with open(csv_read_counter_file, "w") as f:
    f.write(str(index))
Reply
#5
thanks for the code,
it prints only the first row from CSV file, I run it number of times,
and inside the "csv_last_read.csv" file there is recorded digit of 9,
is there a way to add a range to it?
Reply
#6
I meant for the open attempt on the read file to be inside the "try" section. This is a minor change:

csv_filename = "my.csv"
csv_read_counter_file = "csv_last_read"
try:
    with open(csv_read_counter_file, "r") as f:
        lines_read = int(f.readline().rstrip())
except:
    # On any error, just reset back to start of file.
    lines_read = 0

with open(csv_filename, "r") as f:
    for index, line in enumerate(f):
        if index <= lines_read:
            continue
        print(line.rstrip())

with open(csv_read_counter_file, "w") as f:
    f.write(str(index))
I don't know why you would only be getting the first line. It should skip until the lines after that point are read.

I'm sure you could incorporate a range check instead of a simple "index <= lines_read" check. You would just need to determine how you want to populate the range.
Reply
#7
thanks for the new code,
I run the new code:

csv_filename = "my.csv"
csv_read_counter_file = "csv_last_read.csv"
try:
    with open(csv_read_counter_file, "r") as f:
        lines_read = int(f.readline().rstrip())
except:
    # On any error, just reset back to start of file.
    lines_read = 0
 
with open(csv_filename, "r") as f:
    for index, line in enumerate(f):
        if index <= lines_read:
            continue
        print(line.rstrip())
 
with open(csv_read_counter_file, "w") as f:
    f.write(str(index))
now it prints all the data at once, I have now 20 rows of numbers in CSV file instead of 10 to test.
if I write a number in "csv_last_read.csv" and save the file, let's say 3, when I run the file, it prints all rows after row 3.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Read csv file through PyCharm kimx0961 3 119 Yesterday, 07:05 PM
Last Post: perfringo
Sad SyntaxError: from simple python example file from mind-monitor code (muse 2) warmcupoftea 4 210 Jul-16-2021, 02:51 PM
Last Post: warmcupoftea
  passing php variable to python file jerald 1 134 Jul-07-2021, 11:46 AM
Last Post: Larz60+
  Importing a function from another file runs the old lines also dedesssse 6 230 Jul-06-2021, 07:04 PM
Last Post: deanhystad
  Why it does not print(file.read()) Rejaul84 1 246 Jul-01-2021, 10:37 PM
Last Post: bowlofred
  my python file closing rwahdan 1 270 Jun-30-2021, 08:25 AM
Last Post: Gribouillis
  Read and write active Excel file euras 4 370 Jun-29-2021, 11:16 PM
Last Post: Pedroski55
  [Solved] Trying to read specific lines from a file Laplace12 7 521 Jun-21-2021, 11:15 AM
Last Post: Laplace12
  [Solved] Using readlines to read data file and sum columns Laplace12 4 313 Jun-16-2021, 12:46 PM
Last Post: Laplace12
Question Embedding a python file online Dreary35 0 203 Jun-10-2021, 05:05 PM
Last Post: Dreary35

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020