Hello,
I am starting to create a webscraper for work, I am very inexperienced with Python and I am running into a strange issue. I am starting with very basic concepts, for example, finding and pulling the title from a webpage. When I use a HTTP site like Wikipedia.org I can pull the title no problem. However, when I use a website starting with HTTPS I get the following error:
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
Here is my code if that helps, I have replaced my proxy names with fake values. I am also using Python 3.7, any guidance is much appreciated!
So I've done some experimenting and I have found some HTTPS websites that I can work with. Any leads on why StockX won't work would still be valuable.
I am starting to create a webscraper for work, I am very inexperienced with Python and I am running into a strange issue. I am starting with very basic concepts, for example, finding and pulling the title from a webpage. When I use a HTTP site like Wikipedia.org I can pull the title no problem. However, when I use a website starting with HTTPS I get the following error:
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
Here is my code if that helps, I have replaced my proxy names with fake values. I am also using Python 3.7, any guidance is much appreciated!
import requests from bs4 import BeautifulSoup http_proxy = 'http://abcd.org:1234 https_proxy = 'http://abcd.org:1234 url = 'https://www.stockx.com' proxies = { 'http': http_proxy, 'https': https_proxy } headers = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36'} r = requests.get(url, headers = headers, proxies = proxies) soup = BeautifulSoup(r.text, 'html.parser') results = soup.find('title') products = results.text.strip() print(products)
So I've done some experimenting and I have found some HTTPS websites that I can work with. Any leads on why StockX won't work would still be valuable.