Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Urllib error
#1
Hello all,

I am using urllib to check for availablity of URLs.
I am also handling exceptions for URLError and HTTPError.

I found that my program crashes when we get the error - as ConnectionResetError: [Errno 104] Connection reset by peer
I am using Multiprocessing but the error is definitely not due to multipocessing
Is there any way to handle "ConnectionResetError: [Errno 104] Connection reset by peer" using urllib?


Below are the error details.

Error:
multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/usr/lib64/python3.4/multiprocessing/pool.py", line 119, in worker result = (True, func(*args, **kwds)) File "/usr/lib64/python3.4/multiprocessing/pool.py", line 44, in mapstar return list(map(*args)) File "FOLDERNAME/aaaaa_functions.py", line 138, in url_check rc = urllib.request.urlopen(url, timeout=10).read().decode('utf-8') File "/usr/lib64/python3.4/urllib/request.py", line 161, in urlopen return opener.open(url, data, timeout) File "/usr/lib64/python3.4/urllib/request.py", line 464, in open response = self._open(req, data) File "/usr/lib64/python3.4/urllib/request.py", line 482, in _open '_open', req) File "/usr/lib64/python3.4/urllib/request.py", line 442, in _call_chain result = func(*args) File "/usr/lib64/python3.4/urllib/request.py", line 1211, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib64/python3.4/urllib/request.py", line 1186, in do_open r = h.getresponse() File "/usr/lib64/python3.4/urllib/request.py", line 1186, in do_open r = h.getresponse() File "/usr/lib64/python3.4/http/client.py", line 1247, in getresponse response.begin() File "/usr/lib64/python3.4/http/client.py", line 396, in begin version, status, reason = self._read_status() File "/usr/lib64/python3.4/http/client.py", line 358, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/lib64/python3.4/socket.py", line 378, in readinto return self._sock.recv_into(b) ConnectionResetError: [Errno 104] Connection reset by peer """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "FOLDERNAME/all_urls_check.py", line 91, in <module> p.map(url_check,array) File "/usr/lib64/python3.4/multiprocessing/pool.py", line 260, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/usr/lib64/python3.4/multiprocessing/pool.py", line 608, in get raise self._value ConnectionResetError: [Errno 104] Connection reset by peer
Reply
#2
Sorry. This is my first post.
Reply
#3
please show your code.
Reply
#4
Hello all,

Here is the function

def url_check(array):
    url=array[0]
    table_name=array[1]
    try:
        rc = urllib.request.urlopen(url, timeout=10).read().decode('utf-8')
    except (HTTPError,URLError) as e:
        print("URL unreachable")
    except timeout:
        print("URL unreachable")
I need a way to handle the exception -> "ConnectionResetError: [Errno 104] Connection reset by peer."
The above code is not able to handle this exception "ConnectionResetError: [Errno 104] Connection reset by peer"
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Connectivity issue with urllib ronen1m 7 4,756 Nov-22-2018, 01:03 PM
Last Post: ronen1m

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020