Apply fillna to multiple columns in dataframe

Apply fillna to multiple columns in dataframe - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Apply fillna to multiple columns in dataframe (/thread-34498.html)

Apply fillna to multiple columns in dataframe - rraillon - Aug-04-2021

Hi all,

I have just started with python and in particular pandas and the following question has come up in my mind a few times and I am wondering, if I am doing anything wrong?

In this particular example, I am trying to fill NAs using fillna and values in a pre-populated column of the data frame. I am currently using this logic to cycle through the 12 columns of the current year (=cy) and apply fillna column by column.

for cy_month in data_cols_cy:
    data.loc[:, cy_month] = data.loc[:, cy_month].fillna(value=data.loc[:, 'FC Month'], axis=0)

I have read numerous times that for loops should be avoided so I was playing with apply instead. I have come up with the following and it at least doesn't throw an error (it did though when I tried to reference the content of FC_Month within the lambda function and I am not sure why).

FC_Month = data.loc[:, 'FC Month']
data.loc[:, data_cols_cy].apply(lambda x: x.fillna(value=FC_Month, axis=0), axis=1)

The result is no error but also no NAs have been replaced. Would appreciate some feedback as to what I am not yet getting.

Thanks,

Rene

RE: Apply fillna to multiple columns in dataframe - klllmmm - Aug-05-2021

Try adding;
inplace=True

RE: Apply fillna to multiple columns in dataframe - rraillon - Aug-05-2021

(Aug-05-2021, 01:30 AM)klllmmm Wrote: Try adding;
inplace=True

Thank you. I tried that and it didn't work (and the above was output to my Jupyter notebook so I could see the results) but I think, I have found a solution.

data.apply(lambda x: x.loc[data_cols_cy].fillna(value=x.loc['FC Month']), axis=1)

I think, my mistake was to restrict the data frame when calling apply. By just using the method on the entire data frame, I now have access to the other column I need.

Thanks again for the reply,

Rene