Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
[split] Web Crawler Error
#1
Today I am all off the sudden getting this error message after the crawler made its way through a few pages. 

Error:
    raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))
Is this possibly that the website is blocking something? It seems to randomly happen. Most of the time i am able to extract everything without problem.
Reply
#2
A website can block an IP if there is more than some number of requests per second. Print the response code.
"As they say in Mexico 'dosvidaniya'. That makes two vidaniyas."
https://freedns.afraid.org
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  [split] Name error ktck001122 3 2,712 Sep-02-2019, 01:50 AM
Last Post: ktck001122
  [split] Python beginner: Weird Syntax Error mnsaathvika 1 2,127 Jul-22-2019, 06:14 AM
Last Post: buran
  [split] No Error, and No Output vishal2894 1 1,705 Jul-02-2019, 03:20 PM
Last Post: nilamo
  [split] Name error/is not defined rajkiran 1 3,962 Dec-27-2018, 06:47 PM
Last Post: micseydel
  [split] Python error jazzy 1 2,146 Dec-20-2018, 07:35 PM
Last Post: nilamo
  Why is my crawler not recording 404s I think it may be an error in the logic RickyWilson 3 3,369 Dec-18-2017, 12:57 PM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020