Python Forum
how to change the range of read CSV file every time python file runs
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
how to change the range of read CSV file every time python file runs
#1
Hello;

I have a function that uses data from CSV file row by row, until the range has been used.

[Edited]
however I want to use the rest of rows of data automatically, each time it runs.

when it runs for the second time, I want it starts from where it was left off,
in this example that I have, for the second run it should read from row 2 to 4, and when it runs again for the 3rd time, it should read data from row 4 to 6.
so every time the file runs, it print 2 rows of data ( not the same row) until there is no data on CSV file.
is this doable?
your help is appreciated.

here is the part of read CSV file in range that works:

with open('file1.csv', 'r') as f:
                        for i, line in enumerate(f, start=0):
                            if i in range(0, 2):       
                                print(line.rstrip()))
Out put:
220445566
334587993
Reply
#2
Line 2 loops over the entire CSV (consuming it) until complete, or unless you break out of the loop. The main 2 ways to do this would be:

* Loop over a range and consume one line each time. Similar to:
with open("my.csv") as f:
    for index in range(2):
        line = f.readline().rstrip()
        print(f"Got line {index} {line}")

    ### Can read more lines from f here
    ### If you read past the end, f.readline() will start returning empty strings
* Loop over the file, but break out when your limit is reached.
with open("my.csv" as f:
    for index, line in enumerate(f):
        print(f"Got line {index} {line.rstrip()}")
        if index >= (2 - 1):
            break
    
    ### Can read more lines from f here
    ### If you read past the end of the file, the loop exits and the break is never run
Reply
#3
thanks for the reply;
I tried both codes, it returns the same result (first 2 row),every time I run the file,
I don't understand how this could go over the next line on the second run,
what I was thinking, I have to somehow record the ranges when it runs, perhaps in a text file, the next run it reads from the recorded text file and add one or two to the digit, and change the range to a new one, and so on. I might be wrong, or something easier is the solution, but I am not in that level to do this.
Reply
#4
I thought you meant that you wanted to read the rest of the CSV later in the same script.

I see now that you just want to read the "new" stuff each time it runs. Yes, you'd have to keep state somewhere like the file offset or the number of lines previously read.

This partial solution can have problems if it crashes or is run simultaneously (no locking, no error checking, etc.)

csv_filename = "my.csv"
csv_read_counter_file = "csv_last_read"
with open(csv_read_counter_file, "r") as f:
    try:
        lines_read = int(f.readline().rstrip())
    except:
        # On any error, just reset back to start of file.
        lines_read = 0

with open(csv_filename, "r") as f:
    for index, line in enumerate(f):
        if index < lines_read:
            continue
        print(line.rstrip())

with open(csv_read_counter_file, "w") as f:
    f.write(str(index))
Reply
#5
thanks for the code,
it prints only the first row from CSV file, I run it number of times,
and inside the "csv_last_read.csv" file there is recorded digit of 9,
is there a way to add a range to it?
Reply
#6
I meant for the open attempt on the read file to be inside the "try" section. This is a minor change:

csv_filename = "my.csv"
csv_read_counter_file = "csv_last_read"
try:
    with open(csv_read_counter_file, "r") as f:
        lines_read = int(f.readline().rstrip())
except:
    # On any error, just reset back to start of file.
    lines_read = 0

with open(csv_filename, "r") as f:
    for index, line in enumerate(f):
        if index <= lines_read:
            continue
        print(line.rstrip())

with open(csv_read_counter_file, "w") as f:
    f.write(str(index))
I don't know why you would only be getting the first line. It should skip until the lines after that point are read.

I'm sure you could incorporate a range check instead of a simple "index <= lines_read" check. You would just need to determine how you want to populate the range.
Reply
#7
thanks for the new code,
I run the new code:

csv_filename = "my.csv"
csv_read_counter_file = "csv_last_read.csv"
try:
    with open(csv_read_counter_file, "r") as f:
        lines_read = int(f.readline().rstrip())
except:
    # On any error, just reset back to start of file.
    lines_read = 0
 
with open(csv_filename, "r") as f:
    for index, line in enumerate(f):
        if index <= lines_read:
            continue
        print(line.rstrip())
 
with open(csv_read_counter_file, "w") as f:
    f.write(str(index))
now it prints all the data at once, I have now 20 rows of numbers in CSV file instead of 10 to test.
if I write a number in "csv_last_read.csv" and save the file, let's say 3, when I run the file, it prints all rows after row 3.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Python openyxl not updating Excel file MrBean12 1 250 Mar-03-2024, 12:16 AM
Last Post: MrBean12
  Python logging RotatingFileHandler writes to random file after the first log rotation rawatg 0 341 Feb-15-2024, 11:15 AM
Last Post: rawatg
  connect sql by python using txt. file dawid294 2 381 Jan-12-2024, 08:54 PM
Last Post: deanhystad
  file open "file not found error" shanoger 8 946 Dec-14-2023, 08:03 AM
Last Post: shanoger
  Recommended way to read/create PDF file? Winfried 3 2,784 Nov-26-2023, 07:51 AM
Last Post: Pedroski55
  python Read each xlsx file and write it into csv with pipe delimiter mg24 4 1,310 Nov-09-2023, 10:56 AM
Last Post: mg24
  Replace a text/word in docx file using Python Devan 4 2,854 Oct-17-2023, 06:03 PM
Last Post: Devan
  Help creating shell scrip for python file marciokoko 10 1,257 Sep-16-2023, 09:46 PM
Last Post: snippsat
  How to do 100 runs simulation based on the current codes? dududada 6 902 Sep-03-2023, 01:43 PM
Last Post: deanhystad
  Need to replace a string with a file (HTML file) tester_V 1 699 Aug-30-2023, 03:42 AM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020