Python Forum
Apply fillna to multiple columns in dataframe
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Apply fillna to multiple columns in dataframe
Hi all,

I have just started with python and in particular pandas and the following question has come up in my mind a few times and I am wondering, if I am doing anything wrong?

In this particular example, I am trying to fill NAs using fillna and values in a pre-populated column of the data frame. I am currently using this logic to cycle through the 12 columns of the current year (=cy) and apply fillna column by column.

for cy_month in data_cols_cy:
    data.loc[:, cy_month] = data.loc[:, cy_month].fillna(value=data.loc[:, 'FC Month'], axis=0)
I have read numerous times that for loops should be avoided so I was playing with apply instead. I have come up with the following and it at least doesn't throw an error (it did though when I tried to reference the content of FC_Month within the lambda function and I am not sure why).

FC_Month = data.loc[:, 'FC Month']
data.loc[:, data_cols_cy].apply(lambda x: x.fillna(value=FC_Month, axis=0), axis=1)
The result is no error but also no NAs have been replaced. Would appreciate some feedback as to what I am not yet getting.


Try adding;
(Aug-05-2021, 01:30 AM)klllmmm Wrote: Try adding;

Thank you. I tried that and it didn't work (and the above was output to my Jupyter notebook so I could see the results) but I think, I have found a solution.

data.apply(lambda x: x.loc[data_cols_cy].fillna(value=x.loc['FC Month']), axis=1)
I think, my mistake was to restrict the data frame when calling apply. By just using the method on the entire data frame, I now have access to the other column I need.

Thanks again for the reply,

klllmmm likes this post

Possibly Related Threads…
Thread Author Replies Views Last Post
  Split single column to multiple columns SriRajesh 1 116 Jan-07-2022, 06:43 PM
Last Post: jefsummers
  How to rename dataframe columns based on the content in an index? ar_mahdavi 2 1,019 Jun-07-2021, 06:09 AM
Last Post: ricslato
  IF statement to apply at each date illmattic 2 1,219 Apr-08-2021, 12:31 PM
Last Post: illmattic
  Xlsxwriter: Create Multiple Sheets Based on Dataframe's Sorted Values KMV 2 1,536 Mar-09-2021, 12:24 PM
Last Post: KMV
  Pandas: how to split one row of data to multiple rows and columns in Python GerardMoussendo 4 2,648 Feb-22-2021, 06:51 PM
Last Post: eddywinch82
  How to sum across variable columns in a dataframe rennerom 2 1,249 Jan-31-2021, 05:44 PM
Last Post: rennerom
  Function won't apply dynamically in timeseries illmattic 1 711 Jan-08-2021, 03:15 PM
Last Post: stullis
  How to apply a class method to an entire dataframe column tirtha9 1 1,420 Jan-03-2021, 04:44 AM
Last Post: klllmmm
  How to fill parameter with data from multiple columns CSV file greenpine 1 685 Dec-21-2020, 06:50 PM
Last Post: Larz60+
  How to split dataframe object rows to columns Mekala 1 1,004 Nov-12-2020, 04:18 PM
Last Post: michael1789

Forum Jump:

User Panel Messages

Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020