ARIMA.Fit memoryError - wissam1974 - Feb-20-2019
hi All python Forum experts
i am using the software pyCharm2018.1.1
i have tried to build ARIMA model in python, my model has been identified by the parameters
(p=0, d=0, q=367), here is the code:
def arima_Model_Static_PlotErrorAC_PAC(series):
train, expctd =series , series
arima_orders = (0, 0, 367)
model = ARIMA(series, order=arima_orders)
results_MA = model.fit(disp=-1, start_params=[.1 for i in range(1 + arima_orders[2])])
yhatList=results_MA.fittedvalues
residuals = [expctd[i] - yhatList[i] for i in range(len(expctd))]
mse = mean_squared_error(expctd, yhatList)
rmse = sqrt(mse)
print(results_MA.summary())
print(rmse)
this model is called as follow:
series=DataSetDiff #DataSetDiff is a series with a length of 3652 values
outputResidualError=arima_Model_Static_PlotErrorAC_PAC(series) an error is loaded with this high q order whici is:
Error: C:\109_personel\112_pyCharmArima\venv\Scripts\python.exe C:/109_personel/112_pyCharmArima/Presentation_Vers2_ModelOneFunct_3_5.py
Traceback (most recent call last):
File "C:/109_personel/112_pyCharmArima/Presentation_Vers2_ModelOneFunct_3_5.py", line 243, in arima_Model_Static_PlotErrorAC_PAC
results_MA = model.fit(disp=-1, start_params=[.1 for i in range(1 + arima_orders[2])], solver='bfgs')
File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\tsa\arima_model.py", line 959, in fit
callback=callback, **kwargs)
File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\base\model.py", line 466, in fit
full_output=full_output)
File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\base\optimizer.py", line 191, in _fit
hess=hessian)
File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\base\optimizer.py", line 327, in _fit_bfgs
disp=disp, retall=retall, callback=callback)
File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\scipy\optimize\optimize.py", line 916, in fmin_bfgs
res = _minimize_bfgs(f, x0, args, fprime, callback=callback, **opts)
File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\scipy\optimize\optimize.py", line 970, in _minimize_bfgs
gfk = myfprime(x0)
File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\scipy\optimize\optimize.py", line 300, in function_wrapper
return function(*(wrapper_args + args))
File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\base\model.py", line 451, in score
return -self.score(params, *args) / nobs
File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\tsa\arima_model.py", line 583, in score
return approx_fprime_cs(params, self.loglike, args=(False,))
File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\tools\numdiff.py", line 202, in approx_fprime_cs
for i, ih in enumerate(increments)]
File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\tools\numdiff.py", line 202, in <listcomp>
for i, ih in enumerate(increments)]
File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\tsa\arima_model.py", line 780, in loglike
return self.loglike_kalman(params, set_sigma2)
File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\tsa\arima_model.py", line 790, in loglike_kalman
return KalmanFilter.loglike(params, self, set_sigma2)
File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\tsa\kalmanf\kalmanfilter.py", line 654, in loglike
R_mat, T_mat)
File "kalman_loglike.pyx", line 359, in statsmodels.tsa.kalmanf.kalman_loglike.kalman_loglike_complex
File "kalman_loglike.pyx", line 228, in statsmodels.tsa.kalmanf.kalman_loglike.kalman_filter_complex
File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\numpy\core\numeric.py", line 2200, in identity
return eye(n, dtype=dtype)
File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\numpy\lib\twodim_base.py", line 186, in eye
m = zeros((N, M), dtype=dtype, order=order)
MemoryError
Process finished with exit code 1
my model is well running and forecasting until q order MA 150 that mean(0,0,150). the error memoryError raised on the selection of q=367 as order
is any one can help me to solve this error, i have googled this error many time and i did not found a suitable solutions
Thank you for any help
RE: ARIMA.Fit memoryError - scidam - Feb-22-2019
It seems that you operate with big arrays somewhere, or such arrays occur during computations. zeros((N, M)) in Numpy tries to allocate N*M*8 bytes (8 is default, depends on dtype) in RAM, but you don't have enough RAM.
RE: ARIMA.Fit memoryError - wissam1974 - Feb-23-2019
(Feb-22-2019, 02:25 AM)scidam Wrote: It seems that you operate with big arrays somewhere, or such arrays occur during computations. zeros((N, M)) in Numpy tries to allocate N*M*8 bytes (8 is default, depends on dtype) in RAM, but you don't have enough RAM.
hi Mr. I have extended my physical RAM to 20G as a max limit that can hold my computer, and then I re-executed my model until q order equal to 180 as an example "ARIMA (0,0,180)" because my main objective is to go up to 367 as q order. I had kept my computer running for hours (20) no error had been loaded, but the execution of my model had not finished and no output coming. some suggestions found for this kind of problems are recommended to change the program, may be this means I should make some modifications in the predefined ARIMA.fit python function to get best optimization? because it is not an easy choice? if yes, can you help me trying it?
|