Python Forum

Full Version: [split] Web Crawler Error
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Today I am all off the sudden getting this error message after the crawler made its way through a few pages. 

Error:
    raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))
Is this possibly that the website is blocking something? It seems to randomly happen. Most of the time i am able to extract everything without problem.
A website can block an IP if there is more than some number of requests per second. Print the response code.