Python Forum
Select column between to dates CSV data
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Select column between to dates CSV data
#1
Hello,

My wish is to select the values in a column (for example column FG) between two dates.
======
This is a sample of the CSV data:

STN,YYYYMMDD,DDVEC,FHVEC, FG, FHX, FHXH, FHN, FHNH, FXX, FXXH, TG, TN, TNH, TX, TXH, T10N,T10NH, SQ, SP, Q, DR, RH, RHX, RHXH, PG, PX, PXH, PN, PNH, VVN, VVNH, VVX, VVXH, NG, UG, UX, UXH, UN, UNH, EV24
348,20140101, 173, 75, 77, 110, 22, 50, 4, 170, 22, 71, 45, 7, 93, 22, , , 29, 37, 260, 36, 19, 9, 24,10028,10076, 5, 9937, 24, 39, 1, 75, 13, , 87, 93, 1, 78, 14, 4
348,20140102, 191, 58, 65, 90, 1, 50, 16, 140, 11, 84, 57, 24, 101, 4, , , 18, 23, 156, 14, 7, 4, 11, 9979,10036, 21, 9925, 4, 57, 10, 70, 1, , 88, 93, 10, 82, 12, 2
348,20140103, 202, 83, 91, 130, 20, 50, 16, 230, 17, 86, 53, 17, 119, 13, , , 23, 29, 222, 40, 115, 59, 17,10003,10047, 24, 9983, 6, 26, 15, 75, 14, , 85, 95, 7, 71, 17, 3
=========

This is my program code:

import matplotlib.pyplot as plt
import pandas as pd
from pandas.plotting import register_matplotlib_converters
register_matplotlib_converters()

df = pd.read_csv('KNMI_20191231.csv', index_col='YYYYMMDD')

date = df['date'] = pd.to_datetime(df['YYYYMMDD'], format='%Y%m%d')

df.head

df_maskA = df.loc[(df['date'] >= '2014-01-8') & (df['date'] <= '2014-01-12'), 'FG']
df_maskB = (df['date'] >= '2019-01-8') & (df['date'] <= '2019-01-12')



print(df.loc[df_maskA])
print(df.loc[df_maskB])

=======================
When i run this program i get the error code:

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2897, in get_loc
return self._engine.get_loc(key)
File "pandas/_libs/index.pyx", line 107, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 131, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1607, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1614, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'YYYYMMDD'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/jdvlot/PycharmProjects/KNMI/Testknmi2.py", line 9, in <module>
date = df['date'] = pd.to_datetime(df['YYYYMMDD'], format='%Y%m%d')
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pandas/core/frame.py", line 2980, in __getitem__
indexer = self.columns.get_loc(key)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2899, in get_loc
return self._engine.get_loc(self._maybe_cast_indexer(key))
File "pandas/_libs/index.pyx", line 107, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 131, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1607, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1614, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'YYYYMMDD'

Process finished with exit code 1


whit most regards.

J.D. Vlot
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Make unique id in vectorized way based on text data column with similarity scoring ill8 0 894 Dec-12-2022, 03:22 AM
Last Post: ill8
  What if a column has about 90% of data as outliers? Asahavey17 1 1,825 Aug-23-2021, 04:55 PM
Last Post: jefsummers
  Pandas Data frame column condition check based on length of the value aditi06 1 2,700 Jul-28-2021, 11:08 AM
Last Post: jefsummers
  Adding a new column to a Panda Data Frame rsherry8 2 2,127 Jun-06-2021, 06:49 PM
Last Post: jefsummers
  Redistributing column data metro17 2 1,687 Nov-28-2020, 05:53 PM
Last Post: metro17
  How to shift data frame rows of specified column Mekala 0 1,906 Jul-21-2020, 02:42 PM
Last Post: Mekala
  Filter data based on a value from another dataframe column and create a file using lo pawanmtm 1 4,292 Jul-15-2020, 06:20 PM
Last Post: pawanmtm
  How can I convert time-series data in rows into column srvmig 0 2,074 Apr-11-2020, 05:40 AM
Last Post: srvmig
  add formatted column to pandas data frame alkaline3 0 1,679 Mar-22-2020, 06:44 PM
Last Post: alkaline3
  Select data from between two timestamps Makada 18 27,604 Feb-07-2020, 09:56 AM
Last Post: Makada

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020