How can I sort my column so that NaN is at the bottom?

[soft]

I tried sort_values to put na as last position. But it is not working. I do not wish to remove the NaN fields as it will affect my data analysis later.

I need to sort smokers by population before mortality rate without Nan.

def mortality_rate(row):
    if row['Population 2020 (in thousands)'] >= 50000:
        return row['Total Deaths'] / row['Total Infected']
   
df1['Mortality Rate'] = df1.apply(lambda row: mortality_rate(row), axis = 1)

df1 = df1.sort_values(['Smokers in Population (%)'],ascending=[False])

df1.sort_values(by = ['Mortality Rate'] , axis=0, ascending=False, inplace=False, kind='quicksort', na_position='last')

print(df1[['Country','Smokers in Population (%)', 'Mortality Rate']].head(10))

Current output

>                   Country  Smokers in Population (%)  Mortality Rate
>60              Montenegro                       45.9             NaN
>34                  Greece                       43.4             NaN
>39               Indonesia                       39.4        0.066014
>78                  Serbia                       38.9             NaN
>11  Bosnia and Herzegovina                       38.9             NaN

Desired output

>        Country  Smokers in Population (%)  Mortality Rate
>39    Indonesia                       39.4        0.066014
>30       France                       32.7        0.154892
>32      Germany                       30.6        0.045456
>89       Turkey                       27.2        0.027695
>17        China                       25.6        0.055173

[perfringo]

I see that you want to print result. If this is the case why not just use dropna for displaying required data, something like:

df1.sort_values('Smokers in Population (%)').dropna()

This will not change df1 but will display rows sorted by smokers and drop any row which has NaN.

Just to note: on row #9 you set inplace=False and therefore sorting has no effect on df1 (you don't need to set that anyway as this is default).

[soft]

I see that you want to print result. If this is the case why not just use dropna for displaying required data, something like:

df1.sort_values('Smokers in Population (%)').dropna()

This will not change df1 but will display rows sorted by smokers and drop any row which has NaN.

Just to note: on row #9 you set inplace=False and therefore sorting has no effect on df1 (you don't need to set that anyway as this is default).

Hi thanks for your suggestion. However I require the rows to be intact for other parts of the question. I did dropna before and it messes up with my results later. Is there anyway that I can sort ('Smokers in Population (%)') first then only display mortality rate for population with more than 50000?

[ibreeden]

I have no experience with pandas, so I am not sure. But in line 9 you do df1.sort_values(... inplace=False ...) but you do not assign the result of this method to anyting. Could that be the problem?