Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
[split] Web Crawler Error
#1
Today I am all off the sudden getting this error message after the crawler made its way through a few pages. 

Error:
    raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))
Is this possibly that the website is blocking something? It seems to randomly happen. Most of the time i am able to extract everything without problem.
Reply
#2
A website can block an IP if there is more than some number of requests per second. Print the response code.
"As they say in Mexico 'dosvidaniya'. That makes two vidaniyas."
https://freedns.afraid.org
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  [split] Name error ktck001122 3 2,733 Sep-02-2019, 01:50 AM
Last Post: ktck001122
  [split] Python beginner: Weird Syntax Error mnsaathvika 1 2,146 Jul-22-2019, 06:14 AM
Last Post: buran
  [split] No Error, and No Output vishal2894 1 1,722 Jul-02-2019, 03:20 PM
Last Post: nilamo
  [split] Name error/is not defined rajkiran 1 3,977 Dec-27-2018, 06:47 PM
Last Post: micseydel
  [split] Python error jazzy 1 2,161 Dec-20-2018, 07:35 PM
Last Post: nilamo
  Why is my crawler not recording 404s I think it may be an error in the logic RickyWilson 3 3,400 Dec-18-2017, 12:57 PM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020