Python Forum

Full Version: Filter dataframe Not Working
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
The followings are my code and prints. Why the df_sample is empty? The ydate variable is '2022-12-31' and the original df As_of_date have '2022-12-31'. Thanks.

Code:
sql = """
select *
from working.Temp
where As_of_date = '2022-12-31'"""

df = pd.read_sql(sql,cnxn)

date1 = datetime.datetime.now() - timedelta(days = 5)
ydate = (date1.strftime("%Y-%m-%d"))

df_sample = df.loc[(df['As_of_date'] == ydate)]
print(ydate)
print('df_sample')
print(df_sample)
print('df')
print(df)

Output:
2022-12-31
df_sample
Empty DataFrame
Columns: [As_of_date, Term, Current_Bal, Amount1, Amount2]

Index: []
df
As_of_date Term Current_Bal Amount1 Amount2
0 2022-12-31 144.0 18665.43 186467.6457 12953808.42
1 2022-12-31 144.0 14476.56 144620.8344 11364099.60
2 2022-12-31 84.0 8473.60 59230.4640 6134886.40
3 2022-12-31 180.0 11744.23 82092.1677 7833401.41
4 2022-12-31 144.0 10481.00 157110.1900 6802169.00
Use Code tags and no color in code.
You most look at what type As_of_date is first,now is a object type,and then is just text and will not work with dates.
If i read your output and give a demo.
import pandas as pd
import datetime
from datetime import timedelta

date1 = datetime.datetime.now() - timedelta(days = 6)
ydate = (date1.strftime("%Y-%m-%d"))
df = pd.read_clipboard()
>>> df
   As_of_date   Term  Current_Bal      Amount1      Amount2
0  2022-12-31  144.0     18665.43  186467.6457  12953808.42
1  2022-12-31  144.0     14476.56  144620.8344  11364099.60
2  2022-12-31   84.0      8473.60   59230.4640   6134886.40
3  2022-12-31  180.0     11744.23   82092.1677   7833401.41
4  2022-12-31  144.0     10481.00  157110.1900   6802169.00

>>> df.dtypes
As_of_date      object
Term           float64
Current_Bal    float64
Amount1        float64
Amount2        float64
dtype: object

# Convert to Pandas datetime
>>> df['As_of_date'] = pd.to_datetime(df['As_of_date'])

# Now see that is correct type
>>> df.dtypes
As_of_date     datetime64[ns]
Term                  float64
Current_Bal           float64
Amount1               float64
Amount2               float64
dtype: object

# Then it work
>>> df_sample = df.loc[(df['As_of_date'] == ydate)]
>>> df_sample
  As_of_date   Term  Current_Bal      Amount1      Amount2
0 2022-12-31  144.0     18665.43  186467.6457  12953808.42
1 2022-12-31  144.0     14476.56  144620.8344  11364099.60
2 2022-12-31   84.0      8473.60   59230.4640   6134886.40
3 2022-12-31  180.0     11744.23   82092.1677   7833401.41
4 2022-12-31  144.0     10481.00  157110.1900   6802169.00
Thank you very much snippsat.