Aug-22-2022, 09:23 AM
Hi,
After an OCR-session, I have very large files for people to search for data (prayer cards).
I "dump" them both in text format (.txt) and in binary, using pickle.
So far , so good.
Now I need to read the data:
With pickle it is file.load(...) and I get the whole thing as a list, I can go through, record by record, that is ok.
My traditional way of reading the .txt file is:
The pickle file is much smaller that my txt file, but:
Question: is the pickle load(...) method more taxing on the computer's memory that the enumerate(...) method.
Even if I can empty the list after using it?
Any pros or cons?
thx,
Paul
After an OCR-session, I have very large files for people to search for data (prayer cards).
I "dump" them both in text format (.txt) and in binary, using pickle.
So far , so good.
Now I need to read the data:
With pickle it is file.load(...) and I get the whole thing as a list, I can go through, record by record, that is ok.
My traditional way of reading the .txt file is:
with open('sourcefile', 'r') as source: for idx, line in enumerate(source): ...code ...In both cases, after scanning through all data , i just close the file, and continue with the results.
The pickle file is much smaller that my txt file, but:
Question: is the pickle load(...) method more taxing on the computer's memory that the enumerate(...) method.
Even if I can empty the list after using it?
Any pros or cons?
thx,
Paul