Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Dropna not working
#1
Hi guys,

I am extracting tables using Pandas and want to get rid of empty rows in the dataframe. One of the tables looks like this:

Output:
Instrument Price Order Date Type Lots Gap Duration Note 0 nnNZD/USDnn nnSelln0.73250 Sep 27, 2016 7:17pm Limit 41 189 weeks 1 2 3 nnGBP/CHFnn nnSelln1.32000 Sep 1, 2016 11:33am Limit 595 193 weeks 4 5
so to delete empty rows I use in code:

df.dropna(axis = 0, how = 'all', inplace=True)
Unfortunately does not work!

Any ideas please?
Reply
#2
Your data frame probably filled with empty strings not nan values.
Reply
#3
Probably your data frame is filled with empty strings. Here is a way to remove them.
df[df['Price'].astype(bool)]
So basically, what we are doing here is that
df['Price'].astype(bool)
returns True wherever there is a value in Price columns, and False wherever there is empty string.So when we use
df[df['Price'].astype(bool)]
,it returns just the rows that have True value for Price column.You can use any column in place of price
Reply
#4
This does work thanks.

Is there a solution 'non column' specific?

So I really only want to remove rows where the whole row is empty (or with empty strings). This solution is based on if the rows of 'Column name' are empty as opposed to the complete row i.e across all columns.

I have used this before but is there a more elegant way?

for index, row in df.iterrows():                                                            #loop through rows of df for further df cleaning
            if row.any() == '':
                df.drop(index,inplace=True)
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020