Python Forum
Pulling and comparing CSV data
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Pulling and comparing CSV data
#1
Need a script that will take a CSV file inputted by user (in the form of filepath with filename), another directory path containing files inputted by the user, find the CSV files within that directory, compare the data between the two files, and if there is a match, print the filepath\filename (if no match found, print no matches found). Script runs but no data appears, and I know there are matches. I'm close but I can't put it all together.

##Import all of the required modules
import os, re, sys, csv, glob

##Ask user to specify the directory that contains various CSV files
searchPath = raw_input('Specify directory containing CSV Files: ')

##Ask user to specify the location of the csv file reference information
referencePath = raw_input('Specify the full path to the Reference CSV file. This file should only contain the data you are searching for: ')

##Create an empty list to hold values from Excel CSV used for search.
##These are the CSV values we do not have info for.
xUnmatchedValues = []

##Open and read the CSV file line by line
with open(referencePath) as exceldata:

   for e in exceldata:
       ##As you read each line, strip out any extra white spaces or new lines        
       e = e.strip()
       ##Append each cleaned up line (in this case each line contains a single value) into the list declared above   
       xUnmatchedValues.append(e)

##xUnmatchedValues contains the list of reference data from reference csv

##Walk through directory to find csv files and read their data
for root, dirs, files in os.walk(searchPath):
   for file in files:
      if file.endswith(".csv"):
          f=open(file, 'r')
          for m in exceldata:
              m = m.strip()
              xPossibleMatches.append(m)
              file_name_path = os.path.join(root, file)
          ##Check to see if data from xUmatchedValues is in data from xPossibleMatches
          if any(i in xUnmatchedValues for i in xPossibleMatches):
              Print ("Match found! File Name:",file)
              Print ("Match found! File Path:",file_name_path)

          if not any(i in xUnmatchedValues for i in xPossibleMatches):
              Print ("No Match")
          f.close()

##Indicate to the user that the directory search has finished.
print "Directory Search Complete"
Reply
#2
Debugging 101: Add print statements at strategic places to check the values of your variables... And where are you defining Print (with uppercase "p")?
Unless noted otherwise, code in my posts should be understood as "coding suggestions", and its use may require more neurones than the two necessary for Ctrl-C/Ctrl-V.
Your one-stop place for all your GIMP needs: gimp-forum.net
Reply
#3
Corrected the uppercase p for Print - getting error:
Traceback (most recent call last):
File "C:\Users\Generic\Downloads\PythonProjects\Matchv2.1.py", line 27, in <module>
f=open(file, 'r')
IOError: [Errno 2] No such file or directory: 'MatchRoot.csv'
Reply
#4
you need to specify the full path for file to open - use os.path.join(root,file)
also it's better to use context manager - with statement when open the file. it will close it for you when not needed anymore
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Pulling Specifics Words/Numbers from String bigpapa 2 750 May-01-2023, 07:22 PM
Last Post: bigpapa
  Having trouble installing scikit-learn via VSC and pulling my hair out pythonturtle 1 755 Feb-07-2023, 02:23 AM
Last Post: Larz60+
  (Python) Pulling data from UA Google Analytics with more than 100k rows into csv. Stockers 0 1,217 Dec-19-2022, 11:11 PM
Last Post: Stockers
  Pulling username from Tuple pajd 21 3,344 Oct-07-2022, 01:33 PM
Last Post: pajd
  pulling multiple lines from a txt IceJJFish69 3 2,570 Apr-26-2021, 05:56 PM
Last Post: snippsat
  Pulling Information Out of Dictionary Griever 4 2,890 Aug-12-2020, 02:34 PM
Last Post: Griever
  Comparing Items Different Data Frames With a WHILE Loop JoeDainton123 1 1,944 Jul-30-2020, 04:11 AM
Last Post: scidam
  hi guys, I got data reader error while pulling the data from yahoo gokulrajkmv 1 1,994 May-15-2020, 11:08 AM
Last Post: snippsat
  Comparing data from two Excel sheets Stuart_Houghton 0 1,682 Jan-19-2020, 02:49 PM
Last Post: Stuart_Houghton
  Pulling & Reading Date from UDF that Compare it to Live Data firebird 4 2,762 Jul-20-2019, 09:30 AM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020