Python Forum
search file extention in sub directories
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
search file extention in sub directories
#1
Hi All,

I am making a script to find a file with certain extension.
here is my code
import glob
searchCriteria = myDir + r'\**\*.csv'
print(searchCriteria)
istpFiles = glob.glob(searchCriteria,recursive=True)
I have two major problems
1- it is very slow as reference mention :

Note: Using the “**” pattern in large directory trees may consume an inordinate amount of time.

2- it sometimes fail to chat files if they start with "." , as reference : If the directory contains files starting with . they won’t be matched by default. For example, consider a directory containing card.gif and .card.gif:
[inline]>>> import glob
>>> glob.glob('*.gif')
['card.gif']
>>> glob.glob('.c*')
['.card.gif'][/inline]

ref for the issues : https://docs.python.org/3.7/library/glob...#glob.glob


I am asking for a better code that can fit my needs. any suggestions?
Reply
#2
You can try this:

def find(path, ext):
    for root, dirs, files in os.walk(path):
        for file in files:
            if file.endswith(ext):
                fp = os.path.join(root, file)
                yield fp
I guess, it takes the same time. Maybe you repeat the search twice or more. Try it more then one time.
os.walk and glob is using os.scandir, which is not collecting the stat information, which is faster.
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Organization of project directories wotoko 3 360 Mar-02-2024, 03:34 PM
Last Post: Larz60+
  Search Excel File with a list of values huzzug 4 1,147 Nov-03-2023, 05:35 PM
Last Post: huzzug
  Navigating file directories and paths inside Jupyter Notebook Mark17 5 626 Oct-29-2023, 12:40 PM
Last Post: Mark17
  Search for multiple unknown 3 (2) Byte combinations in a file. lastyle 7 1,256 Aug-14-2023, 02:28 AM
Last Post: deanhystad
  search file by regex SamLiu 1 860 Feb-23-2023, 01:19 PM
Last Post: deanhystad
  Listing directories (as a text file) kiwi99 1 802 Feb-17-2023, 12:58 PM
Last Post: Larz60+
  Find duplicate files in multiple directories Pavel_47 9 2,924 Dec-27-2022, 04:47 PM
Last Post: deanhystad
  rename same file names in different directories elnk 0 680 Nov-04-2022, 05:23 PM
Last Post: elnk
  If function is false search next file mattbatt84 2 1,111 Sep-04-2022, 01:56 PM
Last Post: deanhystad
  rename and add desire "_date" to end of file name before extention RolanRoll 1 1,208 Jun-13-2022, 11:16 AM
Last Post: gruntfutuk

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020