Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
ARIMA.Fit memoryError
#1
hi All python Forum experts
i am using the software pyCharm2018.1.1

i have tried to build ARIMA model in python, my model has been identified by the parameters
(p=0, d=0, q=367), here is the code:

def arima_Model_Static_PlotErrorAC_PAC(series): 
    train, expctd =series , series 
    arima_orders = (0, 0, 367)
    model = ARIMA(series, order=arima_orders)
    results_MA = model.fit(disp=-1, start_params=[.1 for i in range(1 + arima_orders[2])])
    yhatList=results_MA.fittedvalues 
    residuals = [expctd[i] - yhatList[i] for i in range(len(expctd))] 
    mse = mean_squared_error(expctd, yhatList)
    rmse = sqrt(mse)
    print(results_MA.summary())
    print(rmse)

this model is called as follow:

series=DataSetDiff #DataSetDiff is a series with a length of 3652 values
outputResidualError=arima_Model_Static_PlotErrorAC_PAC(series)
an error is loaded with this high q order whici is:
Error:
C:\109_personel\112_pyCharmArima\venv\Scripts\python.exe C:/109_personel/112_pyCharmArima/Presentation_Vers2_ModelOneFunct_3_5.py Traceback (most recent call last): File "C:/109_personel/112_pyCharmArima/Presentation_Vers2_ModelOneFunct_3_5.py", line 243, in arima_Model_Static_PlotErrorAC_PAC results_MA = model.fit(disp=-1, start_params=[.1 for i in range(1 + arima_orders[2])], solver='bfgs') File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\tsa\arima_model.py", line 959, in fit callback=callback, **kwargs) File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\base\model.py", line 466, in fit full_output=full_output) File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\base\optimizer.py", line 191, in _fit hess=hessian) File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\base\optimizer.py", line 327, in _fit_bfgs disp=disp, retall=retall, callback=callback) File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\scipy\optimize\optimize.py", line 916, in fmin_bfgs res = _minimize_bfgs(f, x0, args, fprime, callback=callback, **opts) File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\scipy\optimize\optimize.py", line 970, in _minimize_bfgs gfk = myfprime(x0) File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\scipy\optimize\optimize.py", line 300, in function_wrapper return function(*(wrapper_args + args)) File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\base\model.py", line 451, in score return -self.score(params, *args) / nobs File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\tsa\arima_model.py", line 583, in score return approx_fprime_cs(params, self.loglike, args=(False,)) File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\tools\numdiff.py", line 202, in approx_fprime_cs for i, ih in enumerate(increments)] File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\tools\numdiff.py", line 202, in <listcomp> for i, ih in enumerate(increments)] File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\tsa\arima_model.py", line 780, in loglike return self.loglike_kalman(params, set_sigma2) File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\tsa\arima_model.py", line 790, in loglike_kalman return KalmanFilter.loglike(params, self, set_sigma2) File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\statsmodels\tsa\kalmanf\kalmanfilter.py", line 654, in loglike R_mat, T_mat) File "kalman_loglike.pyx", line 359, in statsmodels.tsa.kalmanf.kalman_loglike.kalman_loglike_complex File "kalman_loglike.pyx", line 228, in statsmodels.tsa.kalmanf.kalman_loglike.kalman_filter_complex File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\numpy\core\numeric.py", line 2200, in identity return eye(n, dtype=dtype) File "C:\109_personel\112_pyCharmArima\venv\lib\site-packages\numpy\lib\twodim_base.py", line 186, in eye m = zeros((N, M), dtype=dtype, order=order) MemoryError Process finished with exit code 1
my model is well running and forecasting until q order MA 150 that mean(0,0,150). the error memoryError raised on the selection of q=367 as order

is any one can help me to solve this error, i have googled this error many time and i did not found a suitable solutions
Thank you for any help
Reply
#2
It seems that you operate with big arrays somewhere, or such arrays occur during computations. zeros((N, M)) in Numpy tries to allocate N*M*8 bytes (8 is default, depends on dtype) in RAM, but you don't have enough RAM.
Reply
#3
(Feb-22-2019, 02:25 AM)scidam Wrote: It seems that you operate with big arrays somewhere, or such arrays occur during computations. zeros((N, M)) in Numpy tries to allocate N*M*8 bytes (8 is default, depends on dtype) in RAM, but you don't have enough RAM.

hi Mr. I have extended my physical RAM to 20G as a max limit that can hold my computer, and then I re-executed my model until q order equal to 180 as an example "ARIMA (0,0,180)" because my main objective is to go up to 367 as q order. I had kept my computer running for hours (20) no error had been loaded, but the execution of my model had not finished and no output coming. some suggestions found for this kind of problems are recommended to change the program, may be this means I should make some modifications in the predefined ARIMA.fit python function to get best optimization? because it is not an easy choice? if yes, can you help me trying it?
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Demand Forecasting using Arima LukasBen123 6 1,167 Jul-21-2023, 11:58 PM
Last Post: Pedroski55
  Parallelizing Run ARIMA Model wissam1974 0 2,926 Mar-02-2019, 07:20 PM
Last Post: wissam1974
  ARIMA error with hight MA order wissam1974 0 4,247 Feb-16-2019, 02:57 PM
Last Post: wissam1974
  arima model error in python wissam1974 0 4,174 Jan-23-2019, 09:37 AM
Last Post: wissam1974

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020