Python Forum

I have a working Python program (nested loop).
When the file runs:

1- Starts from first CSV file, reads first row (within a given range, e.g. 2 rows), use the data to run the function.

2- After some delay, it reads the next row and runs the function.

3- When it is done with ranges of rows on the first CSV file, after another delay, it uses the next CSV file to read the rows (within the same given range), do the same thing, when done it goes to the next CSV file, and so on until the last CSV file.

Here is the program that works fine:

from abc.zzz  import xyz
path_id_map = [
    {'path':'file1.csv', 'id': '12345678'},
    {'path':'file2.csv', 'id': '44556677'}
    {'path':'file3.csv', 'id': '33377799'}
    {'path':'file4.csv', 'id': '66221144'}]
s_id = None

for pair in path_id_map:
    with open(pair['path'], 'r') as f:
        next(f)  # skip first header line
        for _ in range(1, 3):      
            line = next(f)
            img_url, title_1, desc_1, link_1 = map(str.strip, line.split(';'))
            zzz.func1(img_url=img_url, title_1=title_1, desc_1=desc_1, 
                      link_1=link_1, B_id=B_id=pair['id'], s_id=s_id)
            time.sleep(25)

sample CSV file:

Quote:img_url,desc_1 title_1,link_1
site.com/image22.jpg;someTitle;description1;site1.com
site.com/image32.jpg;someTitle;description2;site2.com
site.com/image44.jpg;someTitle;description3;site3.com

** EDITED **
The code above works fine, but the problem is:
when you run the file again, it reads the same data from CSV files, (in my example the given range is 2 rows, so every time I run it, it reads row 1 and row 2).

I want the code to use other rows as well when it runs again, within the same given range of course.
So I want to add another loop to it, which each time file runs, it starts from the next set of rows that have not been used.

as an example, if the ranges of rows are set to 2, it means it should read only 2 rows of data from each CSV file that are in

path_id_map = []

.

1- file runs (first time), reads row 1, uses the data, some delay then read row 2 uses the data, some delay, goes to next CSV file, the same process until the last CSV file.

2- file runs for the second time, it should start to use from row 3, uses the data, some delay then it should read row 4, uses the data, some delay goes to the next CSV file, the same process again until the last CSV file.
the functionality of the code remains the same.
The whole process is to read data from CSV files and run the function, no writing to CSV files.
Appreciate your help to make this work.

Maybe this could help:

from abc.zzz  import xyz
path_id_map = [
    {'path':'file1.csv', 'id': '12345678'},
    {'path':'file2.csv', 'id': '44556677'}
    {'path':'file3.csv', 'id': '33377799'}
    {'path':'file4.csv', 'id': '66221144'}]
s_id = None

from os.path import dirname as fdir
from os.path import join as fjoin
from os.path import exists as fexist
scriptdir = fdir(__file__)
currentinfo = fjoin(scriptdir, 'currentinfo.py')
if fexist(currentinfo):
    with open(currentinfo, 'r', encoding='utf-8') as fpt:
        script = fpt.read()
    exec(script, globals())
else:
    foff = 0
    rfrom = 1
    rto = 3
for pair in path_id_map:
    with open(pair['path'], 'r') as f:
        if foff == 0:
            next(f)  # skip first header line
        else:
            f.seek(foff)
        for _ in range(rfrom, rto):
            line = next(f)
            foff = f.tell()
            img_url, title_1, desc_1, link_1 = map(str.strip, line.split(';'))
            zzz.func1(img_url=img_url, title_1=title_1, desc_1=desc_1, 
                      link_1=link_1, B_id=B_id=pair['id'], s_id=s_id)
            time.sleep(25)
with open(currentinfo, 'w', encoding='utf-8') as fpt:
    fpt.writelines('foff =', foff)
    fpt.writelines('rfrom =', rto)
    fpt.writelines('fto =', rto + 2)

Edited
thanks for the reply and code,
I made an empty "currentinfo.py" file in the same folder that codes are.
I tired the code I get an error:

 line 41, in <module>
    if foff == 0:
NameError: name 'foff' is not defined

but is see on upper line there is:

foff = 0

Well, foff is only declared in the else branch. What happens if execution goes into the if branch?

I have question about business logic. Why not read all the lines at once? Or additional lines will be written during delay to files?

Thanks for your suggestions,
perfringo, I am not sure what do you mean, as I am new to Python, and as ndc85430 suggested, I am not sure about that either, as you mentioned variable foff is only defined in else
perhaps if I define the foff, then I get errors to define the rto and rfrom variables.

If anyone can help to make it work, I appreciate it.

anyone can help make this working, please?

(Dec-27-2020, 07:36 PM)greenpine Wrote: [ -> ]perfringo, I am not sure what do you mean, as I am new to Python,

I don't understand why you need to read in three runs:

Quote:I want to add another loop to it, that each time it runs, it goes to next set of rows.

first time run, use: row 1,2
second time run, use: row 3,4
third time run, use: row 5,6

Why not read in one run all 6 rows in a way that you can access specific rows as needed and do something with them?

thanks for the reply,
sorry, if it is confusing.

3 runs are just an example, maybe I should not put it that way.
That is to explain what the file should read when it is time for the next run.

the file is going to be scheduled to run once a day.
so on day one, it should only read and use rows 1 and 2, (of all CSV files), assuming the given range is only 2 rows.
the next day it should read and use rows 3 and 4 ( of all CSV files), and so on for the next run to come, that is what I mean.

there could be hundreds of rows in CSV files, so on each run new data should be read and used.

thanks.

Yes, this is confusing. You haven't answered my question - will additional data written to files between runs? This can be the reason not read all rows at once.

It's hard to advise if it is not clear what the objective is and what are the limiting terms and conditions. Unfortunately there are no clues for me in the code - i have no idea what it suppose to do and why this way.

As the sidenote - if I see exec (or eval) I tend to think: 'somebody is trying to shoot his foot'