Python Forum
Need help passing date to pandas query - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Data Science (https://python-forum.io/forum-44.html)
+--- Thread: Need help passing date to pandas query (/thread-21686.html)



Need help passing date to pandas query - curranjohn46 - Oct-09-2019

How do I pass the output of this prompt to a pandas search by date in excel?

import pandas as pd
TestedDateBegin = pd.to_datetime(input('Input date in mm-dd-yyyy format: '))
For example, if I input 2019-09-08 into above input prompt and run TestedDateBegin I get this output:

Timestamp('2019-09-08 00:00:00')

This search with the date hard coded works fine.

data = df.loc[df['emr_first_access_date'] >= '2019-09-08', ['site_name','subs_num','emr_id', ```'emr_first_access_date']]
But how do I pass the date inputted from the prompt so the user can search by any date?

This doesn't work:

data = df.loc[df['emr_first_access_date'] >= 'TestedDateBegin', ['site_name','subs_num','emr_id',
and throws a exception:


---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
pandas/_libs/tslibs/conversion.pyx in pandas._libs.tslibs.conversion.convert_str_to_tsobject()

pandas/_libs/tslibs/np_datetime.pyx in pandas._libs.tslibs.np_datetime._string_to_dts()

ValueError: Error parsing datetime string "TestedDateBegin" at position 0

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
pandas/_libs/tslibs/conversion.pyx in pandas._libs.tslibs.conversion.convert_str_to_tsobject()

pandas/_libs/tslibs/parsing.pyx in pandas._libs.tslibs.parsing.parse_datetime_string()

~\AppData\Local\Continuum\anaconda3\lib\site-packages\dateutil\parser\_parser.py in parse(timestr, parserinfo, **kwargs)
   1357     else:
-> 1358         return DEFAULTPARSER.parse(timestr, **kwargs)
   1359 

~\AppData\Local\Continuum\anaconda3\lib\site-packages\dateutil\parser\_parser.py in parse(self, timestr, default, ignoretz, tzinfos, **kwargs)
    648         if res is None:
--> 649             raise ValueError("Unknown string format:", timestr)
    650 

ValueError: ('Unknown string format:', 'TestedDateBegin')

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\arrays\datetimes.py in wrapper(self, other)
    144             try:
--> 145                 other = _to_M8(other, tz=self.tz)
    146             except ValueError:

~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\arrays\datetimes.py in _to_M8(key, tz)
     77         # this also converts strings
---> 78         key = Timestamp(key)
     79         if key.tzinfo is not None and tz is not None:

pandas/_libs/tslibs/timestamps.pyx in pandas._libs.tslibs.timestamps.Timestamp.__new__()

pandas/_libs/tslibs/conversion.pyx in pandas._libs.tslibs.conversion.convert_to_tsobject()

pandas/_libs/tslibs/conversion.pyx in pandas._libs.tslibs.conversion.convert_str_to_tsobject()

ValueError: could not convert string to Timestamp

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
<ipython-input-2-702fd23c14bb> in <module>
----> 1 data = df.loc[df['emr_first_access_date'] >= 'TestedDateBegin', ['site_name','subs_num','emr_id', 'emr_first_access_date']]

~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\ops.py in wrapper(self, other, axis)
   1714 
   1715             res_values = dispatch_to_index_op(op, self, other,
-> 1716                                               pd.DatetimeIndex)
   1717 
   1718             return self._constructor(res_values, index=self.index,

~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\ops.py in dispatch_to_index_op(op, left, right, index_class)
   1189         left_idx = left_idx._shallow_copy(freq=None)
   1190     try:
-> 1191         result = op(left_idx, right)
   1192     except NullFrequencyError:
   1193         # DatetimeIndex and TimedeltaIndex with freq == None raise ValueError

~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\indexes\datetimelike.py in wrapper(self, other)
    115                 other = other._values
    116 
--> 117             result = op(self._data, maybe_unwrap_index(other))
    118             return result
    119 

~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\arrays\datetimes.py in wrapper(self, other)
    146             except ValueError:
    147                 # string that cannot be parsed to Timestamp
--> 148                 return ops.invalid_comparison(self, other, op)
    149 
    150             result = op(self.asi8, other.view('i8'))

~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\ops.py in invalid_comparison(left, right, op)
   1056     else:
   1057         raise TypeError("Invalid comparison between dtype={dtype} and {typ}"
-> 1058                         .format(dtype=left.dtype, typ=type(right).__name__))
   1059     return res_values
   1060 

TypeError: Invalid comparison between dtype=datetime64[ns] and str



RE: Need help passing date to pandas query - curranjohn46 - Oct-10-2019

This is resolved.

I had to remove the single quotes around TestedDateBegin as python, of course, interpreted that as a string and not a variable. Duh. So simple.

data = df.loc[df[b]['emr_first_access_date'] >= TestedDateBegin,['site_name','subs_num','emr_id', 'emr_first_access_date']]