Python Forum
Compare two large CSV files for a match
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Compare two large CSV files for a match
#1
Hello, I am very new to python. trying to solve below issue.

We have two .csv files.

For example:
File Master: Column_A Column_B Column_C ..... Column Z
123 XYZ Z 1X
234 PQR Y 2X

File New: Column_C Column_A Colum_B
X 001 PQR
Y 123 XYZ
Y 234 PQR

Each file has similar data but not in the same order in terms of columns or rows. When there is a match between Master file and New file, Master file needs an update by adding new column and populate with Match or No Match. And also add weights, for example if ALL columns & values are matching then 1, partial match then 0.5 else 0

These files are large running into several GBs.

Please help!
Reply


Messages In This Thread
Compare two large CSV files for a match - by Python_Newbie9 - Apr-22-2019, 05:05 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Compare folder A and subfolder B and display files that are in folder A but not in su Melcu54 3 466 Jan-05-2024, 05:16 PM
Last Post: Pedroski55
  Move Files based on partial Match mohamedsalih12 2 744 Sep-20-2023, 07:38 PM
Last Post: snippsat
  Compare 2 files tslavov 2 911 Feb-12-2023, 10:53 AM
Last Post: ibreeden
  Compare fields from two csv files georgebijum 3 1,335 Apr-25-2022, 11:16 PM
Last Post: Pedroski55
  Compare filename with folder name and copy matching files into a particular folder shantanu97 2 4,390 Dec-18-2021, 09:32 PM
Last Post: Larz60+
  Open and read multiple text files and match words kozaizsvemira 3 6,672 Jul-07-2021, 11:27 AM
Last Post: Larz60+
  Iterate 2 large text files across lines and replace lines in second file medatib531 13 5,706 Aug-10-2020, 11:01 PM
Last Post: medatib531
  Iterating Large Files Robotguy 10 5,057 Jul-22-2020, 09:13 PM
Last Post: Gribouillis
  Look for match in two files and print out in the first file Batistuta 0 1,558 Mar-03-2020, 02:27 PM
Last Post: Batistuta
  Handling Large XML Files (>10GB) in Python onlydibs 1 4,140 Dec-22-2019, 05:46 AM
Last Post: Clunk_Head

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020