Python Forum
best option for comparing two csv files
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
best option for comparing two csv files
#1
Hi All,

I have the code below which compares 2 csv files and appends the differences found in the new_data file with file_with_all_data.
The aim of the code is to append new data to a historic file with all data.

I want to change the code to only compare data in the first column in both files and if there are differences, write the entire row to the difference file. Is this the best way to do this? Or would you recommend use the diff function?

import csv


with open('file_with_all_data.csv', 'r') as t1, open('new_data.csv', 'r') as t2:
    fileone = t1.readlines()
    filetwo = t2.readlines()

matches = []

with open('additions.csv', 'w') as outFile:
    for line in filetwo:
        if line not in fileone:
            matches.append(line)
            outFile.write(line)    

with open('file_with_all_data.csv', 'w' ) as outFile:
    outFile.write(''.join(fileone).strip() + '\n' + ''.join(matches))
Reply
#2
on Linux, use diff from command line.
If you need to do it programmatically, look through: https://pypi.org/search/?q=%27file+diff%27
This lists all file diff packages in 'last updated' order.
You will have to examine to see which packages may be applicable.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Comparing columns of Matrix stored in .txt files JoelFooCJ 2 2,226 Dec-11-2019, 07:21 AM
Last Post: JoelFooCJ
  Comparing values in large txt files StevenVF 2 2,693 Feb-28-2019, 09:07 AM
Last Post: StevenVF
  Comparing two files data while one got hex and other binary A_Embedded 3 3,661 Apr-07-2018, 09:58 PM
Last Post: A_Embedded
  Comparing two files and word frequency nix 2 4,573 Feb-02-2017, 08:29 PM
Last Post: nix

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020