Python Forum

Hi All,

I am making a script to find a file with certain extension.
here is my code

import glob
searchCriteria = myDir + r'\**\*.csv'
print(searchCriteria)
istpFiles = glob.glob(searchCriteria,recursive=True)

I have two major problems
1- it is very slow as reference mention :

Note: Using the “**” pattern in large directory trees may consume an inordinate amount of time.

2- it sometimes fail to chat files if they start with "." , as reference : If the directory contains files starting with . they won’t be matched by default. For example, consider a directory containing card.gif and .card.gif:
[inline]>>> import glob
>>> glob.glob('*.gif')
['card.gif']
>>> glob.glob('.c*')
['.card.gif'][/inline]

ref for the issues : https://docs.python.org/3.7/library/glob...#glob.glob


I am asking for a better code that can fit my needs. any suggestions?

You can try this:

def find(path, ext):
    for root, dirs, files in os.walk(path):
        for file in files:
            if file.endswith(ext):
                fp = os.path.join(root, file)
                yield fp

I guess, it takes the same time. Maybe you repeat the search twice or more. Try it more then one time.
os.walk and glob is using os.scandir, which is not collecting the stat information, which is faster.