Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 replace nan values by mean group by date.year, date.month
#1
Hi for all

i have read a CSV file with tow series columns as follow:
Dateobs TMIN
2006-01-01 NAN
2006-01-02 12.3
2006-01-03 11.3
..
2006-02-01 15.2
2006-02-02 Nan
2006-03-03 11.3
..
2016-04-06 15.8
2016-04-07 11.6
2016-04-08 Nan
..
etc..

Error:
my question is how to replace NaN values for the daily TMIN column by mean group by (year month)
the table shown above was issued when i execute this pseudo code
    data_pd = pd.read_csv('beirut.csv')

    # overwriting data after changing format
    data_pd["Dateobs"] = pd.to_datetime(data_pd["Dateobs"])
    Dateobs_typedatetime64=  data_pd["Dateobs"]
    print(Dateobs_typedatetime64)

    TMIN_typeFloat64=  data_pd["TMIN"]
    print(TMIN_typeFloat64)
any help or suggestions to overpass this complexity will be appreciate.
thank you
Quote
#2
Your code should be something like this:

data_pd.fillna(data_pd.groupby([data_pd.Dateobs.dt.year, df.Dateobs.dt.month]).transform('mean'), inplace=True)
Quote
#3
(Jul-06-2019, 01:10 AM)scidam Wrote: Your code should be something like this:

data_pd.fillna(data_pd.groupby([data_pd.Dateobs.dt.year, df.Dateobs.dt.month]).transform('mean'), inplace=True)

hi Mr first thank you for your reply, i appreciate it
i tried this
result=data_pd.fillna(data_pd.groupby([data_pd.Dateobs.dt.year, data_pd.Dateobs.dt.month]).transform('mean'), inplace=True)
print (result)
and i am getting the following result:
Output:
None Process finished with exit code 0
still the same problem its not replace null value by mean
is there anything else i should try
thank you
Quote
#4
If you are using inplace=True you don't need to assign the result. Remove inplace=True in your code, or print(data_pd). result variable is None, because when inplace=true .fillna method returns None.

This is minimal working example:
import pandas as pd
df = pd.DataFrame({'date': ['2001-09-01', '2001-09-02', '2001-10-01', '2001-10-03', '2001-09-03'],
                   'value': [1, 2, 0, None, None]})
df.date = pd.to_datetime(df.date)
df.fillna(df.groupby([df.date.dt.year, df.date.dt.month]).transform('mean'), inplace=True)
Quote
#5
(Jul-06-2019, 01:49 AM)scidam Wrote: If you are using inplace=True you don't need to assign the result. Remove inplace=True in your code, or print(data_pd). result variable is None, because when inplace=true .fillna method returns None.

This is minimal working example:
import pandas as pd
df = pd.DataFrame({'date': ['2001-09-01', '2001-09-02', '2001-10-01', '2001-10-03', '2001-09-03'],
                   'value': [1, 2, 0, None, None]})
df.date = pd.to_datetime(df.date)
df.fillna(df.groupby([df.date.dt.year, df.date.dt.month]).transform('mean'), inplace=True)

Thank you a lot Mr you have helped me solve my problem Smile
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  Problem with date type (object to datetime) karlito 6 235 Oct-16-2019, 08:07 AM
Last Post: karlito
  Trying to Pass date to pandas search from input prompt curranjohn46 1 125 Oct-10-2019, 10:01 AM
Last Post: curranjohn46
  Need help passing date to pandas query curranjohn46 1 148 Oct-10-2019, 09:59 AM
Last Post: curranjohn46
  Pandas converting date to epoch randor 2 321 Jul-16-2019, 02:41 AM
Last Post: scidam
  Pandas and Date: problem with operator.How to resolve frame 6 1,179 May-13-2019, 05:50 PM
Last Post: frame
  Working with date indexes (pandas) dervast 0 371 Apr-05-2019, 01:29 PM
Last Post: dervast
  Help: how to deal with a invalid Matplotlib date value when use {gca()} pythopen 2 13,704 Oct-18-2018, 08:37 PM
Last Post: pythopen
  Error: Slicing a single date from Pandas cheerful 0 623 Jul-22-2018, 08:09 PM
Last Post: cheerful
  Finding date count from a list of date range in pandas trillerducas72 0 665 May-24-2018, 02:30 AM
Last Post: trillerducas72
  How to group variables & check correlation of group variables wrt single variable SriRajesh 2 685 May-23-2018, 03:01 PM
Last Post: SriRajesh

Forum Jump:


Users browsing this thread: 1 Guest(s)