Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 Pandas and Date: problem with operator.How to resolve
#1
Hi.

I have a file csv (es10.csv) like this:
Output:
Conc. Date I II 8926 18/11/2014 1 22 8927 20/11/2014 2 33 8928 30/11/2014 3 15 8929 18/12/2014 4 80 8930 28/12/2014 8 60 8931 31/12/2014 9 12
Note: Date = day**month**Year

Now, i would like to get the following output:
Output:
Conc. Date I II 8929 18/12/2014 4 1 8930 28/12/2014 8 0 8931 31/12/2014 9 1
Note: the value of the date must be greater than 30/11/2014

My code is:

df=pd.read_csv("esv10.csv",encoding='windows-1252',sep="\t",decimal=',',keep_default_na=False, na_values=[''], converters={"Conc.":str},)
pd.options.display.float_format = '{:,.0f}'.format
myfilter=df["Date"]>"30/11/2014"
df= df.where(myfilter ,axis=1).dropna()
print(df)

If i use the operator ==, output = 8928 30/11/2014 3 15
Is it possible to use operator: > or < or =!?

Thanks
Quote
#2
I changed the sep="\t" to sep=" "
and added df['Date'] = pd.to_datetime(df['Date']) to turn the date column into a datetime type
import pandas as pd

df=pd.read_csv("some_data.csv",encoding='windows-1252',sep=" ",decimal=',',keep_default_na=False, na_values=[''], converters={"Conc.":str},)
pd.options.display.float_format = '{:,.0f}'.format
df['Date'] = pd.to_datetime(df['Date'])
myfilter=df["Date"]>"30/11/2014"
df= df.where(myfilter ,axis=1).dropna()
print(df)
Output:
Conc. Date I II 3 8929 2014-12-18 4 80 4 8930 2014-12-28 8 60 5 8931 2014-12-31 9 12
Quote
#3
Thanks Yoriz, but the problem is format column date and transform it like this:
8929 18/12/2014 4 80
(day-month-year)
........
Quote
#4
Use dt.strftime to covert to wanted format.
>>> import pandas as pd

>>> df = pd.read_clipboard()
>>> df['Date'] = pd.to_datetime(df['Date'])
>>> df
   Conc.       Date  I  II
3   8929 2014-12-18  4  80
4   8930 2014-12-28  8  60
5   8931 2014-12-31  9  12

>>> df['Date'] = df['Date'].dt.strftime('%d/%m/%Y')
>>> df
   Conc.        Date  I  II
3   8929  18/12/2014  4  80
4   8930  28/12/2014  8  60
5   8931  31/12/2014  9  12 
Quote
#5
Thanks Snippsat. The conversion work, but applying "df.where" to filter df["Date"]>"30/11/2014"", the result does not change. Is valid the operator > or i do to use only operator ==?
Quote
#6
(May-13-2019, 04:05 PM)frame Wrote: The conversion work, but applying "df.where" to filter df["Date"]>"30/11/2014"", the result does not change. Is valid the operator > or i do to use only operator ==?
The code for @Yoriz should work,is that what you have problem with or are doing something else?
Doing myfilter = df["Date"] == "30/11/2014" it will only drop.
Output:
Conc. Date I II 2 8928.0 30/11/2014 3.0 15.0
Test of code:
>>> import pandas as pd

>>> df = pd.read_clipboard()
>>> df['Date'] = pd.to_datetime(df['Date'])
>>> df
   Conc.       Date  I  II
0   8926 2014-11-18  1  22
1   8927 2014-11-20  2  33
2   8928 2014-11-30  3  15
3   8929 2014-12-18  4  80
4   8930 2014-12-28  8  60
5   8931 2014-12-31  9  12

>>> myfilter = df["Date"] > "30/11/2014"
>>> df = df.where(myfilter ,axis=1).dropna()
>>> df['Date'] = df['Date'].dt.strftime('%d/%m/%Y')
>>> df
    Conc.        Date    I    II
3  8929.0  18/12/2014  4.0  80.0
4  8930.0  28/12/2014  8.0  60.0
5  8931.0  31/12/2014  9.0  12.0
Quote
#7
Now it works very well, Thanks for your patience Snippsat and Yoriz
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  Problem with date type (object to datetime) karlito 6 300 Oct-16-2019, 08:07 AM
Last Post: karlito
  Trying to Pass date to pandas search from input prompt curranjohn46 1 149 Oct-10-2019, 10:01 AM
Last Post: curranjohn46
  Need help passing date to pandas query curranjohn46 1 268 Oct-10-2019, 09:59 AM
Last Post: curranjohn46
  How to resolve scipy differences? Oliver 2 258 Oct-08-2019, 08:40 PM
Last Post: Oliver
  Pandas converting date to epoch randor 2 404 Jul-16-2019, 02:41 AM
Last Post: scidam
  replace nan values by mean group by date.year, date.month wissam1974 4 698 Jul-06-2019, 02:00 AM
Last Post: wissam1974
  Working with date indexes (pandas) dervast 0 396 Apr-05-2019, 01:29 PM
Last Post: dervast
  Problem while showing a Pandas DataFrame Sirduke1 0 515 Aug-29-2018, 06:14 AM
Last Post: Sirduke1
  Error: Slicing a single date from Pandas cheerful 0 654 Jul-22-2018, 08:09 PM
Last Post: cheerful
  Finding date count from a list of date range in pandas trillerducas72 0 702 May-24-2018, 02:30 AM
Last Post: trillerducas72

Forum Jump:


Users browsing this thread: 1 Guest(s)