Python Forum
Add a row to a dataframe or append whole dataframe.
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Add a row to a dataframe or append whole dataframe.
#1
Hello to all,
Happy new year and good health!!


I am sure this is super simple, but I can't find the right way to do it.
I have a folder with some csv files.
All csv files have the same structure, same number of columns, similar data but the number of rows are different.
I want to regroup all that data in one dataframe to make a single CSV.

I have the following code.

import pandas as pd
import glob
path = r"path\to\my\folder\*.csv"
df2 = pd.DataFrame(columns=['Name','Reading', 'Office','Phone','address' ])

csvfiles = []
a=0
for file in glob.glob(path):
    csvfiles.append(file)
   # print(file)

for csvnub in csvfiles:
   df=pd.read_csv(csvnub)
   count_row = df.shape[0]

   print(count_row)
   df2.iloc[a] = df.iloc[count_row]
   a=a+1

print(df2)
df2.to_csv("InfosTotal.csv", index=True, encoding="utf_8_sig")
I tried append()...

for csvnub in csvfiles:
   df=pd.read_csv(csvnub)
   count_row = df.shape[0]
   print(count_row)
   for row in count_row:
       df2.append(row) 
or

df2.append(df, ignore_index = False) 
I tried concat() also...

Help will be extremely appreciated.
Reply
#2
OK,
I found the solution myself, I don't know if this is clean, proper but it works.
Never touch a working script...
hahaha
ok,
I am sure this will help others.

import pandas as pd
import glob
path = r"path\to\my\folder\*.csv"
df2 = pd.DataFrame(columns=['Name','Reading', 'Office','Phone','address' ])

csvfiles = []
for file in glob.glob(path):
    csvfiles.append(file)
   # print(file)

for csvnub in csvfiles:
   df=pd.read_csv(csvnub)
   count_row = df.shape[0]
   print(count_row)
   df2=pd.concat([df, df2])
   

df2.sort_values(by=['Name'])
df2.drop_duplicates(keep='first', inplace=True)
df2.to_csv("InfosTotal.csv", index=True, encoding="utf_8_sig")
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Elegant way to apply each element of an array to a dataframe? sawtooth500 5 178 3 hours ago
Last Post: deanhystad
  Dataframe copy warning sawtooth500 4 184 Mar-25-2024, 11:38 PM
Last Post: sawtooth500
  FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries i sawtooth500 3 391 Mar-22-2024, 03:08 AM
Last Post: deanhystad
  Adding PD DataFrame column bsben 2 245 Mar-08-2024, 10:46 PM
Last Post: deanhystad
  Python Alteryx QS-Passing pandas dataframe column inside SQL query where condition sanky1990 0 690 Dec-04-2023, 09:48 PM
Last Post: sanky1990
  Comparing Dataframe to String? RockBlok 2 363 Nov-24-2023, 04:55 PM
Last Post: RockBlok
  Filter data into new dataframe as main dataframe is being populated cubangt 8 922 Oct-23-2023, 12:43 AM
Last Post: cubangt
  DataFRame.concat() nafshar 3 731 Jul-14-2023, 04:41 PM
Last Post: nafshar
  Convert dataframe from str back to datafarme Creepy 1 584 Jul-07-2023, 02:13 PM
Last Post: snippsat
  Question on pandas.dataframe merging two colums shomikc 4 783 Jun-29-2023, 11:30 AM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020