Python Forum
Apply fillna to multiple columns in dataframe
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Apply fillna to multiple columns in dataframe
Hi all,

I have just started with python and in particular pandas and the following question has come up in my mind a few times and I am wondering, if I am doing anything wrong?

In this particular example, I am trying to fill NAs using fillna and values in a pre-populated column of the data frame. I am currently using this logic to cycle through the 12 columns of the current year (=cy) and apply fillna column by column.

for cy_month in data_cols_cy:
    data.loc[:, cy_month] = data.loc[:, cy_month].fillna(value=data.loc[:, 'FC Month'], axis=0)
I have read numerous times that for loops should be avoided so I was playing with apply instead. I have come up with the following and it at least doesn't throw an error (it did though when I tried to reference the content of FC_Month within the lambda function and I am not sure why).

FC_Month = data.loc[:, 'FC Month']
data.loc[:, data_cols_cy].apply(lambda x: x.fillna(value=FC_Month, axis=0), axis=1)
The result is no error but also no NAs have been replaced. Would appreciate some feedback as to what I am not yet getting.


Try adding;
(Aug-05-2021, 01:30 AM)klllmmm Wrote: Try adding;

Thank you. I tried that and it didn't work (and the above was output to my Jupyter notebook so I could see the results) but I think, I have found a solution.

data.apply(lambda x: x.loc[data_cols_cy].fillna(value=x.loc['FC Month']), axis=1)
I think, my mistake was to restrict the data frame when calling apply. By just using the method on the entire data frame, I now have access to the other column I need.

Thanks again for the reply,

klllmmm likes this post

Possibly Related Threads…
Thread Author Replies Views Last Post
  How to rename dataframe columns based on the content in an index? ar_mahdavi 2 874 Jun-07-2021, 06:09 AM
Last Post: ricslato
  IF statement to apply at each date illmattic 2 1,123 Apr-08-2021, 12:31 PM
Last Post: illmattic
  Xlsxwriter: Create Multiple Sheets Based on Dataframe's Sorted Values KMV 2 1,340 Mar-09-2021, 12:24 PM
Last Post: KMV
  Pandas: how to split one row of data to multiple rows and columns in Python GerardMoussendo 4 2,368 Feb-22-2021, 06:51 PM
Last Post: eddywinch82
  How to sum across variable columns in a dataframe rennerom 2 1,094 Jan-31-2021, 05:44 PM
Last Post: rennerom
  Function won't apply dynamically in timeseries illmattic 1 635 Jan-08-2021, 03:15 PM
Last Post: stullis
  How to apply a class method to an entire dataframe column tirtha9 1 1,163 Jan-03-2021, 04:44 AM
Last Post: klllmmm
  How to fill parameter with data from multiple columns CSV file greenpine 1 618 Dec-21-2020, 06:50 PM
Last Post: Larz60+
  How to split dataframe object rows to columns Mekala 1 896 Nov-12-2020, 04:18 PM
Last Post: michael1789
  convert list to five columns dataframe in sequence tonycat 2 1,084 Sep-29-2020, 06:47 AM
Last Post: tonycat

Forum Jump:

User Panel Messages

Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020