Python Forum
Find duplicate files in multiple directories
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Find duplicate files in multiple directories
#2
Hello,

possible blueprint:

Make files a Dictionary, create for each file a key with a list as the value and and the file's corresponding path to the list.
Once the dict is complete, you can iterate over the dict looking for values with a length > 1.

To iterate over a directory, you may want to use the newer / nicer pathlib module. To iterate over files in a directory non-recursively:

>>> from pathlib import Path
>>> p = Path('/path/to/dir')
>>> for child in p.iterdir():
...     print(child)
To iterate recursively:

>>> from pathlib import Path
>>> p = Path('path/ot/dir')
>>> for file in p.glob('*.*'):
...    print(file)
Regards, noisefloor
Reply


Messages In This Thread
RE: Find duplicate files in multiple directories - by noisefloor - Dec-25-2022, 07:38 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Trying to generating multiple json files using python script dzgn989 4 432 May-10-2024, 03:09 PM
Last Post: deanhystad
  [SOLVED] Loop through directories and files one level down? Winfried 3 501 Apr-28-2024, 02:31 PM
Last Post: Gribouillis
  Organization of project directories wotoko 3 655 Mar-02-2024, 03:34 PM
Last Post: Larz60+
  python convert multiple files to multiple lists MCL169 6 1,839 Nov-25-2023, 05:31 AM
Last Post: Iqratech
  splitting file into multiple files by searching for string AlphaInc 2 1,126 Jul-01-2023, 10:35 PM
Last Post: Pedroski55
  Merging multiple csv files with same X,Y,Z in each Auz_Pete 3 1,394 Feb-21-2023, 04:21 AM
Last Post: Auz_Pete
  Listing directories (as a text file) kiwi99 1 952 Feb-17-2023, 12:58 PM
Last Post: Larz60+
  unittest generates multiple files for each of my test case, how do I change to 1 file zsousa 0 1,070 Feb-15-2023, 05:34 PM
Last Post: zsousa
  rename same file names in different directories elnk 0 802 Nov-04-2022, 05:23 PM
Last Post: elnk
  Python: re.findall to find multiple instances don't work but search worked Secret 1 1,332 Aug-30-2022, 08:40 PM
Last Post: deanhystad

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020