Python Forum
Apply fillna to multiple columns in dataframe
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Apply fillna to multiple columns in dataframe
#1
Hi all,

I have just started with python and in particular pandas and the following question has come up in my mind a few times and I am wondering, if I am doing anything wrong?

In this particular example, I am trying to fill NAs using fillna and values in a pre-populated column of the data frame. I am currently using this logic to cycle through the 12 columns of the current year (=cy) and apply fillna column by column.

for cy_month in data_cols_cy:
    data.loc[:, cy_month] = data.loc[:, cy_month].fillna(value=data.loc[:, 'FC Month'], axis=0)
I have read numerous times that for loops should be avoided so I was playing with apply instead. I have come up with the following and it at least doesn't throw an error (it did though when I tried to reference the content of FC_Month within the lambda function and I am not sure why).

FC_Month = data.loc[:, 'FC Month']
data.loc[:, data_cols_cy].apply(lambda x: x.fillna(value=FC_Month, axis=0), axis=1)
The result is no error but also no NAs have been replaced. Would appreciate some feedback as to what I am not yet getting.

Thanks,

Rene
Reply
#2
Try adding;
inplace=True
Reply
#3
(Aug-05-2021, 01:30 AM)klllmmm Wrote: Try adding;
inplace=True

Thank you. I tried that and it didn't work (and the above was output to my Jupyter notebook so I could see the results) but I think, I have found a solution.

data.apply(lambda x: x.loc[data_cols_cy].fillna(value=x.loc['FC Month']), axis=1)
I think, my mistake was to restrict the data frame when calling apply. By just using the method on the entire data frame, I now have access to the other column I need.

Thanks again for the reply,

Rene
klllmmm likes this post
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Elegant way to apply each element of an array to a dataframe? sawtooth500 3 115 5 hours ago
Last Post: deanhystad
  How to check multiple columns value within range SamLiu 2 1,100 Mar-13-2023, 09:32 AM
Last Post: SamLiu
  Converting a json file to a dataframe with rows and columns eyavuz21 13 4,160 Jan-29-2023, 03:59 PM
Last Post: eyavuz21
  Nested for loops: Iterating over columns of a DataFrame to plot on subplots dm222 0 1,636 Aug-19-2022, 11:07 AM
Last Post: dm222
  How to move multiple columns to initial position SriRajesh 4 1,373 Jul-02-2022, 10:34 AM
Last Post: deanhystad
  df column aggregate and group by multiple columns SriRajesh 0 976 May-06-2022, 02:26 PM
Last Post: SriRajesh
  Extract parts of multiple log-files and put it in a dataframe hasiro 4 2,018 Apr-27-2022, 12:44 PM
Last Post: hasiro
  Split single column to multiple columns SriRajesh 1 1,290 Jan-07-2022, 06:43 PM
Last Post: jefsummers
  How to rename dataframe columns based on the content in an index? ar_mahdavi 2 2,430 Jun-07-2021, 06:09 AM
Last Post: ricslato
  IF statement to apply at each date illmattic 2 2,598 Apr-08-2021, 12:31 PM
Last Post: illmattic

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020