Python Forum
filtering files using 'any()"
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
filtering files using 'any()"
#12
(May-05-2021, 07:25 PM)tester_V Wrote: The files are big and many, I do not want to do 'readlines' and if I'll do open each file then it is like doing 'for' loop, and I do not need 'any()'

I thought I can test each file as a whole file without checking each line.

Thank you.

If the files aren't large, you can just read the entire file into a string and then check if you have a string match. There is no loop necessary.

with open(filename, "r") as f:
    if matchstring in f.read():
        ...
But if the files are large, you don't want to load in the entire file into a single string. That implies that you will need to loop over chunks of the file. If it's a "sane" text file, you can assume newlines every so often and let the file iterator hand you one line at a time. Loop over each and exit when you get your first match. any() would be nice about doing that.

with open(filename, "r") as f:
    # f is the file iterator.  The expression inside the parens is a generator comprehension, so
    # the entire file is not read into memory, just each line is matched.  any() will exit after any
    # match is successful and doesn't have to read the rest of the file.
    if any(matchstring in line for line in f):
        ...
tester_V and snippsat like this post
Reply


Messages In This Thread
filtering files using 'any()" - by tester_V - May-05-2021, 04:55 AM
RE: filtering files using 'any()" - by bowlofred - May-05-2021, 05:18 AM
RE: filtering files using 'any()" - by perfringo - May-05-2021, 05:20 AM
RE: filtering files using 'any()" - by Gribouillis - May-05-2021, 09:36 AM
RE: filtering files using 'any()" - by tester_V - May-05-2021, 04:54 PM
RE: filtering files using 'any()" - by bowlofred - May-05-2021, 05:03 PM
RE: filtering files using 'any()" - by snippsat - May-05-2021, 05:05 PM
RE: filtering files using 'any()" - by tester_V - May-05-2021, 05:07 PM
RE: filtering files using 'any()" - by bowlofred - May-05-2021, 07:22 PM
RE: filtering files using 'any()" - by tester_V - May-05-2021, 07:25 PM
RE: filtering files using 'any()" - by bowlofred - May-05-2021, 07:58 PM
RE: filtering files using 'any()" - by DeaD_EyE - May-06-2021, 10:42 AM
RE: filtering files using 'any()" - by tester_V - May-05-2021, 07:34 PM
RE: filtering files using 'any()" - by Gribouillis - May-05-2021, 08:48 PM
RE: filtering files using 'any()" - by tester_V - May-06-2021, 02:46 AM
RE: filtering files using 'any()" - by perfringo - May-06-2021, 09:29 AM
RE: filtering files using 'any()" - by snippsat - May-06-2021, 10:32 AM
RE: filtering files using 'any()" - by tester_V - May-08-2021, 04:23 AM
RE: filtering files using 'any()" - by Gribouillis - May-08-2021, 06:32 AM
RE: filtering files using 'any()" - by snippsat - May-08-2021, 12:01 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Merge all json files in folder after filtering deneme2 10 2,415 Sep-18-2022, 10:32 AM
Last Post: deneme2
  Filtering files, for current year files tester_V 8 4,022 Aug-07-2021, 03:58 AM
Last Post: tester_V

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020