Python Forum
Dataframe not appending correctly
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Dataframe not appending correctly
#1
I have a dictionary of files in this format:

{'filea': ['test/folder2/filea', 'test/folder3/filea', 'test/folder1/filea'],
'fileb': ['test/folder2/fileb', 'test/folder3/fileb', 'test/folder1/fileb'],
'filec': ['test/folder2/filec', 'test/folder3/filec', 'test/folder1/filec']}

and I have created a for loop to go through each filename and create a dataframe that combines the files corresponding to each key in the dictionary above but when I run my loop, the next fileb in this case is getting appended into the dataframe created for file a. I am not sure how to fix this as I spent a few hours to no avail at solving this problem probably also because I have a very long code in between to understand where my mistake is with indentation. My code is as below:

Lets say the dictionary above is called file_list.
for key,files in file_list.items():
    #dataset = pd.Dataframe()
    for i in files: #loop over the files in each key
       #do something....

    df = pd.DataFrame({'A':B,'C':D,'E':F})
    print('This dataframe has the shape:',df.shape)

    #save dataframe
    df.to_hdf('xxx.hdf'.format(key[0:-4]),mode='w', key='df')
I still can't really see where my mistake is as when the loop works on the files in fileb, it is getting appended into the dataframe that has the data from filea instead of creating a whole new dataframe for fileb.
Any help on this is much appreciated!
Reply


Messages In This Thread
Dataframe not appending correctly - by python_newbie09 - Jul-09-2019, 07:53 PM
RE: Dataframe not appending correctly - by scidam - Jul-10-2019, 12:17 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Comparing results within a list and appending to pandas dataframe Aryagm 1 2,321 Dec-17-2020, 01:08 PM
Last Post: palladium

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020