Python Forum
sklearn.neural_network MLPClassifier forecast variances - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Data Science (https://python-forum.io/forum-44.html)
+--- Thread: sklearn.neural_network MLPClassifier forecast variances (/thread-30639.html)



sklearn.neural_network MLPClassifier forecast variances - CK1960 - Oct-29-2020

Issue described below


RE: sklearn.neural_network MLPClassifier forecast variances - CK1960 - Oct-29-2020

Hi, So the issue is historic forecasts change when data last data input from a time series is deleted. So data is 1500 data point time series starting with 1.31238 ending with 1.32875 varying within 1.3 to 1.45. I prepare 5 lag bins which are converted to directional bins. So if the next data point is less than previous point the bin loads with a 1. If greater the bin loads a 2, if no change the bin loads a zero.

So I run in Jupiter Notebook the following script: (This follows the bin preparation scripts.)

In [108]: from sklearn.neural_network import MLPClassifier
In [109]: model = MLPClassifier(solver='lbfgs', alpha=1e-5,
hidden_layer_sizes=3 * [200],
random_state=5)

In [110]: %time model.fit(data[cols_bin], data['direction'])



The results are fine.

If I then delete the last data point and run the entire 100 lines of script, I get similar results but several of the 1499 predictions are different to the predictions from running the 1500 data points. varying from 6-8& differences or 120-140 differences.

The forecast is either 1 or -1.

It seems a bit fatal to then run the forecast every time a new data point is available if every time the historic or previously generated predictions vary to such an extent.

How do I continue calculating accurate forecasts ( generally 56-58% accuracy) without the historical predictions changing?