Python Forum
Creating new rows and adding them to empty data frame
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Creating new rows and adding them to empty data frame
#1
I have data frame(df1) of start and end dates look like this.
Start Date End Date
1875-01-01 1877-09-30
1881-07-01 1886-03-31
1888-01-01 1889-06-30
1890-10-01 1890-12-31
.
.
.
2016-10-01 2018-12-31




I have a different set of data frame (df2) which consists of daily time series (Date and Value). For example,

Date Value
1875-01-01 7.21
1875-01-02 7.23
1875-01-03 7.22
1875-01-04 7.12
.
.
.
2018-12-31 3.12

I set dates as an index for df2.

I am trying to make stats table based on df2 using df1. First I created an empty data frame to add the values. For example,

outputtable = pd.DataFrame(columns = ('Max','Min','Ave'))
for i in df1.index:
    try:
        df3 = df2.loc[df1['Start Date'][i]:df1['End Date'][i]]
        minimum = df3['Value'].min()
        maximum = df3['Value'].max()
        average = df3['Value'].mean()
        outputtable[-1]= [minimum, maximum, average]

    except:
        pass
I am using try because some of the dates in df1 are not in df2. In that case, I want the code to ignore and move onto the next set of dates.

I want the code to go through every row of start and end dates in df1 and do the stats (min, max and mean) and put them into the outputtable to do further calculations. So far the code above is not working. Help would be much appreciated.

Desired output

Start Date End Date Min Max Ave
1875-01-01 1877-09-30 7 8 7.2
1881-07-01 1886-03-31 1 4 2.2
1888-01-01 1889-06-30 2 6.5 3
1890-10-01 1890-12-31 3 5 4.2
.
.
.
2016-10-01 2018-12-31 1 2 1.7
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Grouping in pandas/multi-index data frame Aleqsie 3 662 Jan-06-2024, 03:55 PM
Last Post: deanhystad
  Merging rows and adding columns based on matching index pythonnewbie78 3 809 Dec-24-2023, 11:51 AM
Last Post: Pedroski55
  Filtering Data Frame, with another value NewBiee 9 1,390 Aug-21-2023, 10:53 AM
Last Post: NewBiee
  Exporting data frame to excel dyerlee91 0 1,626 Oct-05-2021, 11:34 AM
Last Post: dyerlee91
  Pandas Data frame column condition check based on length of the value aditi06 1 2,689 Jul-28-2021, 11:08 AM
Last Post: jefsummers
  Adding a new column to a Panda Data Frame rsherry8 2 2,123 Jun-06-2021, 06:49 PM
Last Post: jefsummers
  grouped data frame glitter 0 1,596 Feb-02-2021, 11:22 AM
Last Post: glitter
  how to filter data frame dynamically with the columns psahay 0 2,402 Aug-24-2020, 01:10 PM
Last Post: psahay
  Dropping Rows From A Data Frame Based On A Variable JoeDainton123 1 2,215 Aug-03-2020, 02:05 AM
Last Post: scidam
  How to shift data frame rows of specified column Mekala 0 1,896 Jul-21-2020, 02:42 PM
Last Post: Mekala

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020