Python Forum
Insert missing data in a dataframe
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Insert missing data in a dataframe
#1
Hello question about a time series again xd

I have a long time series, where I have data of rain intensity with a deltatime of 2 minutes, it does however not fill out the time where no rain was detected. I need both as I use the data to be able to calculate the amount of water stored in a bassin, and the period where the amount of rain is equal to zero is therefore as important. I am not sure how to attack this. One idea I had was to see if the datetime before was different from the next datetime+ a timestep of two minutes:
[ Time ] [intensity]
1997-07-01 12:22:00 , 1
1997-07-01 12:28:00 , 1
1997-07-01 12:30:00 , 1
as the difference between the first time and the next is larger than 2 the output should be:
[ Time ] [intensity]
1997-07-01 12:22:00 , 1
1997-07-01 12:24:00 , 0 (inserted timestep)
1997-07-01 12:26:00 , 0 (inserted timestep)
1997-07-01 12:28:00 , 1
1997-07-01 12:30:00 , 1
I cant however not figure out how to do this in practice..
Another way I thought I could do it would be to construct a time series of the same length and then insert zero intensities, and merch the two time series
The problem with that is however that the data is registered at even and uneven times, and this method would result in cases where the delta time would become 1 minute instead of two minutes:

[ Time ] [intensity]
1997-07-01 12:22:00 , 1
1997-07-01 12:24:00 , 0 (inserted timestep)
1997-07-01 12:26:00 , 0 (inserted timestep)
1997-07-01 12:28:00 , 1
1997-07-01 12:30:00 , 0 (inserted timestep)
1997-07-01 12:31:00 , 1
1997-07-01 12:32:00 , 0 (inserted timestep)
1997-07-01 12:33:00 , 1
1997-07-01 12:34:00 , 0 (inserted timestep)
1997-07-01 12:35:00 , 1

As it can be seen here, the times inserted from 12:32 should not be inserted as the timestep should be 2 minutes.
If anyone has a way to approach this, it would be greatly appreciated
Looking forward to hearing from you!

Attached Files

.csv   example.csv (Size: 22.66 KB / Downloads: 229)
Reply


Messages In This Thread
Insert missing data in a dataframe - by amdi40 - Jan-16-2022, 02:15 PM
RE: Insert missing data in a dataframe - by amdi40 - Jan-17-2022, 02:40 PM
RE: Insert missing data in a dataframe - by amdi40 - Jan-19-2022, 07:47 AM
RE: Insert missing data in a dataframe - by amdi40 - Jan-19-2022, 08:55 AM

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020