Linear Regression on Time Series

[karlito]

Hi,

I'm trying this time to use a simple linear regression on my time series dataset to linearly predict data. But I got this error and I don't know how to handle it. Any ideas?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

# print df.head()                    eie Date_Time   2017-11-10    4470.76 2017-11-11    5465.72 2017-11-12    15465.72 2017-11-13    25465.72 2017-11-14    21480.59     y = np.array(df.values, dtype=float) x = np.array(pd.to_datetime(df['eie']).index.values, dtype=float)   slope, intercept, r_value, p_value, std_err =sp.linregress(x,y)   xf = np.linspace(min(x),max(x),100) xf1 = xf.copy() xf1 = pd.to_datetime(xf1) yf = (slope*xf)+intercept   print('r = ', r_value, '\n', 'p = ', p_value, '\n', 's = ', std_err)

# Error

Error:
ValueError                                Traceback (most recent call last)
<ipython-input-13-5d30a02ce6af> in <module>
      1 y = np.array(df.values, dtype=float)
----> 2 x = np.array(pd.to_datetime(df['eie']).index.values, dtype=float)
      3 
      4 slope, intercept, r_value, p_value, std_err =sp.linregress(x,y)
      5 

ValueError: could not convert string to float: '2017-11-10'

[buran]

The error is clear - string '2017-11-10' could not be converted to float (obviously)

[karlito]

The error is clear - string '2017-11-10' could not be converted to float (obviously)

Yes I can read :) but for regression purpose, I read that all dates should be passed through pandas 'to_datetime()' function to convert it to float numeric because corresponding dates will be saved in the 'x' variable.
nb: before setting Date_Time as index it was already converted to 'to_datetime()'. I'm kind of lost. Any ideas?

[buran]

maybe

1 2 3 4 5 6 7

import pandas as pd import numpy as np df = pd.DataFrame([['2017-11-10', 4470.76], ['2017-11-11', 5465.72], ['2017-11-12', 15465.72]], columns=['Date_Time', 'eie']) y = np.array(df['eie'], dtype=float) x = np.array(pd.to_datetime(df['Date_Time'], format='%Y-%m-%d'), dtype=float) print(y) print(x)

Output:
[ 4470.76  5465.72 15465.72]
[1.5102720e+18 1.5103584e+18 1.5104448e+18]

[karlito]

maybe

1 2 3 4 5 6 7

import pandas as pd import numpy as np df = pd.DataFrame([['2017-11-10', 4470.76], ['2017-11-11', 5465.72], ['2017-11-12', 15465.72]], columns=['Date_Time', 'eie']) y = np.array(df['eie'], dtype=float) x = np.array(pd.to_datetime(df['Date_Time'], format='%Y-%m-%d'), dtype=float) print(y) print(x)

Output:
[ 4470.76  5465.72 15465.72]
[1.5102720e+18 1.5103584e+18 1.5104448e+18]

Thanks for your effort but it doesn't really helps me, sorry. I wish something like

Output:
[ 4470.76  5465.72 15465.72]
[1 2 3]

or even better

Output:
[ 4470.76  5465.72 15465.72]
[2017-11-10 2017-11-11 2017-11-12]

[buran]

I think there is some confusion in your understanding, but anyway

1 2 3 4 5 6 7

import pandas as pd import numpy as np df = pd.DataFrame([['2017-11-10', 4470.76], ['2017-11-11', 5465.72], ['2017-11-12', 15465.72]], columns=['Date_Time', 'eie']) y = np.array(df['eie'], dtype=float) x = np.array(pd.to_datetime(df['Date_Time'], format='%Y-%m-%d'), dtype='datetime64[D]') print(y) print(x)

Output:
[ 4470.76  5465.72 15465.72]
['2017-11-10' '2017-11-11' '2017-11-12']

or

1 2 3 4 5 6 7

import pandas as pd import numpy as np df = pd.DataFrame([['2017-11-10', 4470.76], ['2017-11-11', 5465.72], ['2017-11-12', 15465.72]], columns=['Date_Time', 'eie']) y = np.array(df['eie'], dtype=float) x = np.array(pd.to_datetime(df['Date_Time'].index.values+1, format='%Y-%m-%d'), dtype=int) print(y) print(x)

Output:
[ 4470.76  5465.72 15465.72]
[1 2 3]