Python Forum
Different Correlation Coefficents with different Time Ranges
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Different Correlation Coefficents with different Time Ranges
#1
I built a Time-Series that displays the price of the Electricty Price in South Italy and two of their most important commodities (commodities, gas) used to produce the eletrical energy. So I ordered all these data into DataFrame where there are the following data in details:

  1. First Column - Daily Price of Petroil Future during N Day;
  2. Second Column - Daily Price of Gas Future during N Day;
  3. Third Column - Daily Price of Dau-Ahead Eletricity Market in Italy;
The data are taken from 2010 to 2022 time range, so 12 years of historical time data. The DataFrame head looks like this:

PETROIL GAS ELECTRICITY
0 64.138395 2.496172 68.608696
1 65.196161 2.482612 113.739130
2 64.982403 2.505938 112.086957
3 64.272606 2.500000 110.043478
4 65.993436 2.521739 95.260870

So on this DataFrame I tried to build the Correlation Matrix throught the Pandas metod .corr() (using the Pearson method) and faced one big issue:

If I take all 12 years as data I get:

  1. almost Zero as correlation between Electricity and Petroil price;
  2. low correlation (0.12) between Electricity and Gas price;

If I try to split in three time range (2010-2014; 2014-2018; 2018-2022) I get for each interval, really high correlation for both pair (electricity-gas, electricity-petroil) in a range around 0.60 to 0.90.

So I am here asking these two questions:

  1. Why I get this so high difference when I split the time ranges?
  2. Considering I am doing this kind of analysis to use Petroil and Gas prices to predict the electricity price, which of these two analysis should I consider? The first one (with low correlation) that considers the entire time range or the second one (with higher correlation) that is split into different time ranges?
    Thank you for your answers.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Efficient way to mark entries in df with overlap in time ranges Andreas_Python 0 1,523 Mar-24-2021, 02:10 PM
Last Post: Andreas_Python
  Cross-correlation between 2 planes in a 3D array Mark3232 2 4,557 May-16-2019, 09:46 AM
Last Post: Mark3232
  do you know a code that will print all correlation values using numpty and panda? crispybluewaffle88 1 2,423 Mar-06-2019, 12:45 PM
Last Post: scidam
  Help with correlation coefficient mattjb84 7 4,859 Jun-29-2018, 09:56 PM
Last Post: Larz60+
  Pandas dataframe: sum of exponentially weighted correlation matrices per row vvvcvvcv 1 3,210 May-29-2018, 01:09 AM
Last Post: scidam
  Mistake by correlation (x, y) Jack_Sparrow 2 2,693 May-10-2018, 02:23 PM
Last Post: volcano63
  Creating a graph with ranges Chris1986 3 4,058 Apr-24-2017, 12:22 PM
Last Post: Chris1986

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020