Python Forum
Add a row to a dataframe or append whole dataframe.
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Add a row to a dataframe or append whole dataframe.
#1
Hello to all,
Happy new year and good health!!


I am sure this is super simple, but I can't find the right way to do it.
I have a folder with some csv files.
All csv files have the same structure, same number of columns, similar data but the number of rows are different.
I want to regroup all that data in one dataframe to make a single CSV.

I have the following code.

import pandas as pd
import glob
path = r"path\to\my\folder\*.csv"
df2 = pd.DataFrame(columns=['Name','Reading', 'Office','Phone','address' ])

csvfiles = []
a=0
for file in glob.glob(path):
    csvfiles.append(file)
   # print(file)

for csvnub in csvfiles:
   df=pd.read_csv(csvnub)
   count_row = df.shape[0]

   print(count_row)
   df2.iloc[a] = df.iloc[count_row]
   a=a+1

print(df2)
df2.to_csv("InfosTotal.csv", index=True, encoding="utf_8_sig")
I tried append()...

for csvnub in csvfiles:
   df=pd.read_csv(csvnub)
   count_row = df.shape[0]
   print(count_row)
   for row in count_row:
       df2.append(row) 
or

df2.append(df, ignore_index = False) 
I tried concat() also...

Help will be extremely appreciated.
Reply
#2
OK,
I found the solution myself, I don't know if this is clean, proper but it works.
Never touch a working script...
hahaha
ok,
I am sure this will help others.

import pandas as pd
import glob
path = r"path\to\my\folder\*.csv"
df2 = pd.DataFrame(columns=['Name','Reading', 'Office','Phone','address' ])

csvfiles = []
for file in glob.glob(path):
    csvfiles.append(file)
   # print(file)

for csvnub in csvfiles:
   df=pd.read_csv(csvnub)
   count_row = df.shape[0]
   print(count_row)
   df2=pd.concat([df, df2])
   

df2.sort_values(by=['Name'])
df2.drop_duplicates(keep='first', inplace=True)
df2.to_csv("InfosTotal.csv", index=True, encoding="utf_8_sig")
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Pandas DataFrame combine rows by column value, where Date Rows are NULL rhat398 0 81 May-04-2021, 10:51 PM
Last Post: rhat398
  How to get value in Dataframe given row & column values? moonlight 1 106 Apr-26-2021, 09:30 PM
Last Post: Larz60+
  dataframe cell conditional format by row SriRajesh 0 250 Apr-24-2021, 11:51 AM
Last Post: SriRajesh
Star Split and organize my Pandas Dataframe brunolelli 4 255 Apr-18-2021, 03:00 AM
Last Post: brunolelli
  How to save json data in a dataframe shantanu97 1 187 Apr-15-2021, 02:44 PM
Last Post: klllmmm
  Convert MultiLayer XML to DataFrame using Pandas vsingh17 0 221 Apr-14-2021, 03:50 PM
Last Post: vsingh17
  2 Different Sorts in a Dataframe stylingpat 1 175 Apr-14-2021, 03:02 PM
Last Post: stylingpat
  Exporting a huge dataFrame stylingpat 5 346 Mar-23-2021, 12:13 AM
Last Post: stylingpat
  Dataframe problem stylingpat 3 468 Mar-22-2021, 10:36 AM
Last Post: stylingpat
  Combining outputs into a dataframe rybina 0 194 Mar-15-2021, 02:43 PM
Last Post: rybina

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020