Python Forum
search file extention in sub directories
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
search file extention in sub directories
#1
Hi All,

I am making a script to find a file with certain extension.
here is my code
import glob
searchCriteria = myDir + r'\**\*.csv'
print(searchCriteria)
istpFiles = glob.glob(searchCriteria,recursive=True)
I have two major problems
1- it is very slow as reference mention :

Note: Using the “**” pattern in large directory trees may consume an inordinate amount of time.

2- it sometimes fail to chat files if they start with "." , as reference : If the directory contains files starting with . they won’t be matched by default. For example, consider a directory containing card.gif and .card.gif:
[inline]>>> import glob
>>> glob.glob('*.gif')
['card.gif']
>>> glob.glob('.c*')
['.card.gif'][/inline]

ref for the issues : https://docs.python.org/3.7/library/glob...#glob.glob


I am asking for a better code that can fit my needs. any suggestions?
Reply
#2
You can try this:

def find(path, ext):
    for root, dirs, files in os.walk(path):
        for file in files:
            if file.endswith(ext):
                fp = os.path.join(root, file)
                yield fp
I guess, it takes the same time. Maybe you repeat the search twice or more. Try it more then one time.
os.walk and glob is using os.scandir, which is not collecting the stat information, which is faster.
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Organization of project directories wotoko 3 443 Mar-02-2024, 03:34 PM
Last Post: Larz60+
  Search Excel File with a list of values huzzug 4 1,269 Nov-03-2023, 05:35 PM
Last Post: huzzug
  Navigating file directories and paths inside Jupyter Notebook Mark17 5 730 Oct-29-2023, 12:40 PM
Last Post: Mark17
  Search for multiple unknown 3 (2) Byte combinations in a file. lastyle 7 1,390 Aug-14-2023, 02:28 AM
Last Post: deanhystad
  search file by regex SamLiu 1 926 Feb-23-2023, 01:19 PM
Last Post: deanhystad
  Listing directories (as a text file) kiwi99 1 848 Feb-17-2023, 12:58 PM
Last Post: Larz60+
  rename same file names in different directories elnk 0 720 Nov-04-2022, 05:23 PM
Last Post: elnk
  If function is false search next file mattbatt84 2 1,163 Sep-04-2022, 01:56 PM
Last Post: deanhystad
  rename and add desire "_date" to end of file name before extention RolanRoll 1 1,252 Jun-13-2022, 11:16 AM
Last Post: gruntfutuk
  I need to copy all the directories that do not match the pattern tester_V 7 2,463 Feb-04-2022, 06:26 PM
Last Post: tester_V

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020