Python Forum
Filter dataframe Not Working
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Filter dataframe Not Working
#1
The followings are my code and prints. Why the df_sample is empty? The ydate variable is '2022-12-31' and the original df As_of_date have '2022-12-31'. Thanks.

Code:
sql = """
select *
from working.Temp
where As_of_date = '2022-12-31'"""

df = pd.read_sql(sql,cnxn)

date1 = datetime.datetime.now() - timedelta(days = 5)
ydate = (date1.strftime("%Y-%m-%d"))

df_sample = df.loc[(df['As_of_date'] == ydate)]
print(ydate)
print('df_sample')
print(df_sample)
print('df')
print(df)

Output:
2022-12-31
df_sample
Empty DataFrame
Columns: [As_of_date, Term, Current_Bal, Amount1, Amount2]

Index: []
df
As_of_date Term Current_Bal Amount1 Amount2
0 2022-12-31 144.0 18665.43 186467.6457 12953808.42
1 2022-12-31 144.0 14476.56 144620.8344 11364099.60
2 2022-12-31 84.0 8473.60 59230.4640 6134886.40
3 2022-12-31 180.0 11744.23 82092.1677 7833401.41
4 2022-12-31 144.0 10481.00 157110.1900 6802169.00
Reply
#2
Use Code tags and no color in code.
You most look at what type As_of_date is first,now is a object type,and then is just text and will not work with dates.
If i read your output and give a demo.
import pandas as pd
import datetime
from datetime import timedelta

date1 = datetime.datetime.now() - timedelta(days = 6)
ydate = (date1.strftime("%Y-%m-%d"))
df = pd.read_clipboard()
>>> df
   As_of_date   Term  Current_Bal      Amount1      Amount2
0  2022-12-31  144.0     18665.43  186467.6457  12953808.42
1  2022-12-31  144.0     14476.56  144620.8344  11364099.60
2  2022-12-31   84.0      8473.60   59230.4640   6134886.40
3  2022-12-31  180.0     11744.23   82092.1677   7833401.41
4  2022-12-31  144.0     10481.00  157110.1900   6802169.00

>>> df.dtypes
As_of_date      object
Term           float64
Current_Bal    float64
Amount1        float64
Amount2        float64
dtype: object

# Convert to Pandas datetime
>>> df['As_of_date'] = pd.to_datetime(df['As_of_date'])

# Now see that is correct type
>>> df.dtypes
As_of_date     datetime64[ns]
Term                  float64
Current_Bal           float64
Amount1               float64
Amount2               float64
dtype: object

# Then it work
>>> df_sample = df.loc[(df['As_of_date'] == ydate)]
>>> df_sample
  As_of_date   Term  Current_Bal      Amount1      Amount2
0 2022-12-31  144.0     18665.43  186467.6457  12953808.42
1 2022-12-31  144.0     14476.56  144620.8344  11364099.60
2 2022-12-31   84.0      8473.60   59230.4640   6134886.40
3 2022-12-31  180.0     11744.23   82092.1677   7833401.41
4 2022-12-31  144.0     10481.00  157110.1900   6802169.00
and dee like this post
Reply
#3
Thank you very much snippsat.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Filter data into new dataframe as main dataframe is being populated cubangt 8 1,014 Oct-23-2023, 12:43 AM
Last Post: cubangt
  Filter dataframe by datetime.date column glidecode 2 5,192 Dec-05-2021, 12:51 AM
Last Post: glidecode

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020