Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Pandas merge csv files
#1
Hi,

I was trying to merge two csv files and it worked BUT the first column of the beginning of the merged file starts with a "," (see image). Any ideas?
Thanks

code

from os import chdir
from glob import glob
import pandas as pdlib

def merge_csv(list_raw_files, file_out):
   # Consolidate all CSV files into one object
   result_obj = pdlib.concat([pdlib.read_csv(file) for file in list_raw_files])
   # Convert the above object into a csv file and export
   result_obj.to_csv(file_out, index=False, encoding="utf-8")
    
# Move to the path that holds our CSV files
file_folder = 'my_path'
chdir(file_folder)

# List all CSV files in the working dir
list_raw_files = [f for f in listdir(file_folder) if isfile(join(file_folder, f))]
#print(list_raw_files)

file_out = "merged_file.csv"
merge_csv(list_raw_files, file_out)
Reply
#2
Are the separators commas?

Whatever.
You could put them together as strings.

file1 = "/path/file1.csv"
file2 = "/path/file2.csv"
outfile = "/path/all_together.csv"
all_together = ""

with open(file1, 'r') as f:
    all_together = f.read()
    
with open(file2, 'r') as f:
    all_together = f"{all_together}{f.read()}"

with open(outfile, 'w') as f:
    f.write(all_together)
Reply
#3
(Dec-16-2019, 10:01 AM)Axel_Erfurt Wrote: Are the separators commas?

Whatever.
You could put them together as strings.

file1 = "/path/file1.csv"
file2 = "/path/file2.csv"
outfile = "/path/all_together.csv"
all_together = ""

with open(file1, 'r') as f:
    all_together = f.read()
    
with open(file2, 'r') as f:
    all_together = f"{all_together}{f.read()}"

with open(outfile, 'w') as f:
    f.write(all_together)

yes they are commas! ok I would try your solution.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Import multiple CSV files into pandas Krayna 0 1,693 May-20-2021, 04:56 PM
Last Post: Krayna
  Python - Pandas writing blank files to file tdunphy 0 1,972 Jan-14-2021, 12:11 AM
Last Post: tdunphy
  Creating many csv files from Pandas EMA 0 1,609 Jul-26-2020, 06:39 PM
Last Post: EMA
  Pandas merge question smw10c 2 5,674 Jul-02-2020, 06:56 PM
Last Post: hussainmujtaba
  Python pandas merge with or conditional Lafayette 0 2,147 May-07-2020, 07:34 PM
Last Post: Lafayette
  Pandas dataframe merge snmmat 1 2,093 Mar-09-2020, 06:56 PM
Last Post: jefsummers
  Need Help With Filtering Data For Excel Files Using Pandas eddywinch82 9 5,982 Aug-06-2019, 03:44 PM
Last Post: eddywinch82
  Merge JSON files prioritizing the updated values from most recent file nebulae 0 2,519 Apr-17-2019, 10:15 AM
Last Post: nebulae
  Why can't I merge pandas dataframes learnpython2018 2 7,620 Sep-23-2018, 05:53 PM
Last Post: learnpython2018
  comparing two columns two different files in pandas nuncio 0 2,370 Jun-06-2018, 01:04 PM
Last Post: nuncio

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020