Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
glob for dir listing
#1
Hi 
I am attempting to list all directories using a glob pattern.
In each case below I cannot obtain a definitive list.

for example:
>>> import glob
>>> dirPattern = '/data/part[0-9]'
>>> for d in glob.glob(dirPattern):
...   print(d)
... 
/data/part8
/data/part2
/data/part4
/data/part7
/data/part3
/data/part1
/data/part5
/data/part9
/data/part6
one of the directories is however missing - "/data/part10"

I have tried this as well:
>>> dirPattern='/data/part[0-9]{2}'
>>> for d in glob.glob(dirPattern):
...   print(d)
... 
>>> 
>>> dirPattern='/data/part[0-9][0-9]'
>>> for d in glob.glob(dirPattern):
...   print(d)
... 
/data/part10
But as you can see, either nothing appears or only part10 is listed.
Can anybody suggest a pattern that will match part1 to part10 ?

Thanks
Reply
#2
import glob
dirPattern = './part*'
for d in glob.glob(dirPattern):
   print(d)
This works for me
Reply
#3
that would also include other directories, for example "part_xyz".
Only directories with a numeric ending (2 digits only) should be listed.

Glob might not be the answer.
Reply
#4
(Mar-07-2017, 03:08 PM)bluefrog Wrote: Glob might not be the answer.
Yes glob can only do simple regex stuff.
Can use os.listdir() or newer Python version os.scandir().
Then can write own regex.
Eg:
import os
import re

for f_name in os.listdir():
   if re.match(r'^[A-Za-z]+\d{1,2}$', f_name):
       print(f_name)
Reply
#5
You can still use glob combined with re matching:

for d in glob.glob("/data/part[0-9]*"):
    if re.match("/data/part\d{1,2}$", d):
       print(d)
Compared to os.listdir(), it will iterate only on "prefiltered" list, practically it should be same.

And if you dont have mixed names like part2x, then even glob("/data/part[0-9]*") would work ...
Reply
#6
great, thanks!
I should've used a regex from the start.

The only thing for us to consider however is that often we have to query hadoop filesystems, so although your suggestion will work for data on shared linux file systems, I don't think it will do for hadoop.

I'll have to experiment with prefix's
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Listing directories (as a text file) kiwi99 1 802 Feb-17-2023, 12:58 PM
Last Post: Larz60+
  Read directory listing of files and parse out the highest number? cubangt 5 2,253 Sep-28-2022, 10:15 PM
Last Post: Larz60+
  |SOLVED] Glob JPGs, read EXIF, update file timestamp? Winfried 5 2,411 Oct-21-2021, 03:29 AM
Last Post: buran
  [SOLVED] Input parameter: Single file or glob? Winfried 0 1,539 Sep-10-2021, 11:54 AM
Last Post: Winfried
  q re glob.iglob iterator and close jimr 2 2,179 Aug-23-2021, 10:14 PM
Last Post: perfringo
  Listing All Methods Of Associated With A Class JoeDainton123 3 2,296 May-10-2021, 01:46 AM
Last Post: deanhystad
  Listing files with glob. MathCommander 9 4,814 Oct-26-2020, 02:04 AM
Last Post: MathCommander
  Listing data from a list ebolisa 1 1,707 Sep-29-2020, 02:24 PM
Last Post: DeaD_EyE
  Listing Attributes of Objects & Classes JoeDainton123 4 2,318 Aug-28-2020, 05:27 AM
Last Post: ndc85430
  Listing groups tharpa 2 2,541 Nov-26-2019, 07:25 AM
Last Post: DeaD_EyE

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020