Python Forum
RemoteDataError (PredictingCrpytoPrices)
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
RemoteDataError (PredictingCrpytoPrices)
#1
Smile 
Good afternoon to the community Smile .
I am using Google Colaboratory.
I want to predict the price of cryptocurrencies.
But the console shows me this error:
Error:
RemoteDataError Traceback (most recent call last) <ipython-input-21-66861f1bbce0> in <module>() 14 end = dt.datetime.now() 15 ---> 16 data = web.DataReader(f'{crypto_currency}-{against_currency}', 'yahoo', start, end) 17 18 #Prepare Data 4 frames /usr/local/lib/python3.7/dist-packages/pandas/util/_decorators.py in wrapper(*args, **kwargs) 197 else: 198 kwargs[new_arg_name] = new_arg_value --> 199 return func(*args, **kwargs) 200 201 return cast(F, wrapper) /usr/local/lib/python3.7/dist-packages/pandas_datareader/data.py in DataReader(name, data_source, start, end, retry_count, pause, session, api_key) 382 retry_count=retry_count, 383 pause=pause, --> 384 session=session, 385 ).read() 386 /usr/local/lib/python3.7/dist-packages/pandas_datareader/base.py in read(self) 251 # If a single symbol, (e.g., 'GOOG') 252 if isinstance(self.symbols, (string_types, int)): --> 253 df = self._read_one_data(self.url, params=self._get_params(self.symbols)) 254 # Or multiple symbols, (e.g., ['GOOG', 'AAPL', 'MSFT']) 255 elif isinstance(self.symbols, DataFrame): /usr/local/lib/python3.7/dist-packages/pandas_datareader/yahoo/daily.py in _read_one_data(self, url, params) 151 url = url.format(symbol) 152 --> 153 resp = self._get_response(url, params=params) 154 ptrn = r"root\.App\.main = (.*?);\n}\(this\)\);" 155 try: /usr/local/lib/python3.7/dist-packages/pandas_datareader/base.py in _get_response(self, url, params, headers) 179 msg += "\nResponse Text:\n{0}".format(last_response_text) 180 --> 181 raise RemoteDataError(msg) 182 183 def _get_crumb(self, *args): RemoteDataError: Unable to read URL: https://finance.yahoo.com/quote/ETH-USD/history?period1=1434340800&period2=1632542399&interval=1d&frequency=1d&filter=history Response Text: b'<!DOCTYPE html>\n <html lang="en-us"><head>\n <meta http-equiv="content-type" content="text/html; charset=UTF-8">\n <meta charset="utf-8">\n <title>Yahoo</title>\n <meta name="viewport" content="width=device-width,initial-scale=1,minimal-ui">\n <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">\n <style>\n html {\n height: 100%;\n }\n body {\n background: #fafafc url(https://s.yimg.com/nn/img/sad-panda-201402200631.png) 50% 50%;\n background-size: cover;\n height: 100%;\n text-align: center;\n font: 300 18px "helvetica neue", helvetica, verdana, tahoma, arial, sans-serif;\n }\n table {\n height: 100%;\n width: 100%;\n table-layout: fixed;\n border-collapse: collapse;\n border-spacing: 0;\n border: none;\n }\n h1 {\n font-size: 42px;\n font-weight: 400;\n color: #400090;\n }\n p {\n color: #1A1A1A;\n }\n #message-1 {\n font-weight: bold;\n margin: 0;\n }\n #message-2 {\n display: inline-block;\n *display: inline;\n zoom: 1;\n max-width: 17em;\n _width: 17em;\n }\n </style>\n <script>\n document.write(\'<img src="//geo.yahoo.com/b?s=1197757129&t=\'+new Date().getTime()+\'&src=aws&err_url=\'+encodeURIComponent(document.URL)+\'&err=%<pssc>&test=\'+encodeURIComponent(\'%<{Bucket}cqh[:200]>\')+\'" width="0px" height="0px"/>\');var beacon = new Image();beacon.src="//bcn.fp.yahoo.com/p?s=1197757129&t="+ne...
Please find my code below:
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import pandas_datareader as web
import datetime as dt
from sklearn.preprocessing import MinMaxScaler
from tensorflow.keras.layers import Dense, Dropout, LSTM
from tensorflow.keras.models import Sequential

crypto_currency = 'ETH'
against_currency = 'USD'

start = dt.datetime (2015,6,15)
end  = dt.datetime.now()

data = web.DataReader(f'{crypto_currency}-{against_currency}', 'yahoo', start, end)

#Prepare Data
scaler = MinMaxScaler(feature_range(0,1))
scaled_data = scaler.fit_transform(data['Close'].values.reshape(-1,1))

prediction_days = 60

x_train, y_train = [], []

for x in range(prediction_days, len(scaled_data)):
  x_train.append(scaled_data[x-prediction_days:x, 0])
  y_train.append(scaled_data[x, 0])
            
x_train, y_train = np.array(x_train), np.array(y_train)
x_train = np.reshape(x_train, (x_train.shape[0], x_train.shape[1], 1))

#Create neural network

model = Sequential()

model.add(LSTM(units=50, return_sequences=True, input_shape=(x_train.shape[1], 1)))
model.add(Dropout(0.2))
model.add(LSTM(units=50, return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(units=50))
model.add(Dropout(0.2))
model.add(Dense(units=1))

model.compile(optimizer='adam', loss='mean_squared_error')
model.fit(x_train, y_train, epochs=25, batch_size=32 )


#Testing the model
test_start = dt.datetime(2020,1,1)
test_end = dt.datetime.now()

test_data = web.DataReader(f'{crypto_currency}-{against_currency}', 'yahoo', test_tart, test_end)
actual_prices = test_data['Close'].values

total_dataset = pd.concat((data['Close'], test_data['Close']), axis = 0)

model_inputs = total_dataset[len(total_dataset)-len(test_data) - prediction_days:].values
model_inputs = model_inputs.reshape(-1,1)
model_inputs = scaler.fit_transform(model_inputs)

x_test = []

for x in range(prediction_days, len(model_inputs)):
  x_test.append(model_inputs[x-prediction_days:x,0])

x_test = np.array(x_test)
x_test = np.reshape(x_test, (x_test.reshape[0], x_test.shape[1], 1))

prediction_prices = model.predict(x_test)
prediction_prices = scaler.inverse_transform(prediction_prices)

plt.plot(actual_prices, color = 'black', label='Actual Prices')
plt.plot(prediction_prices, color='green', label ='Predicted Prices')
plt.title(f'{crypto_currency}price_prediction')
plt.xlabel('Time')
plt.ylabel('Price')
plt.legend(loc='upper left')
plt.show()
Thank your for your help.
Reply
#2
the error is: RemoteDataError: Unable to read URL:
however the URL shown is valid.
Is your internet connection working?

EDIT
I did notice this:
Quote:The access_key keyword argument of DataReader has been deprecated in favor of api_key.
here: https://pandas-datareader.readthedocs.io..._data.html
Reply
#3
Ok. I will try some manipulations by looking yahoo finance api_key.
Reply
#4
I tried some manipulations by looking yahoo finance api_key. But I have still an error.
Reply
#5
There's another package called TF Quant Finance: TensorFlow based Quant Finance Library,
I've never used it , but it looks interesting.
nightly updates:
PyPi: https://pypi.org/project/tff-nightly/
GitHub: https://github.com/google/tf-quant-finance
Reply
#6
Thank you for your reply Larz60+.
I prefer to keep the database of yahoo finance. What is Quant Finance library? (it's free or no). I really don't want to use Quant Finance Library at the first stage.
Reply
#7
The problem seems to be with pandas interpreting the page. I tested the URL and got a nicely formatted table on a page. The table is quite long, takes several loads, so thought might be related to that. Tried shortening the interval by changing the start year to 2021. Still error. Changed to using pd.read_html() instead of the data reader. Pandas came back with an error 404. Weird, so read the docs (often a good idea). read_html does not work with https apparently.

In any case, tried a request using your code from Naver. That worked (failed later as different format data from what you expect). So it is explicitly yahoo finance and the connection there that is the problem.

Not an answer for you, but might give some ideas.
Reply
#8
Thank you for your explanation jefsummers. It it narrows the number of causes of error.
Reply
#9
(Sep-27-2021, 08:00 AM)lulu43366 Wrote: Thank you for your explanation jefsummers. It narrows the number of causes of error.
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020