Python Forum

Hello all,

I am using urllib to check for availablity of URLs.
I am also handling exceptions for URLError and HTTPError.

I found that my program crashes when we get the error - as ConnectionResetError: [Errno 104] Connection reset by peer
I am using Multiprocessing but the error is definitely not due to multipocessing
Is there any way to handle "ConnectionResetError: [Errno 104] Connection reset by peer" using urllib?

Below are the error details.

Error:multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib64/python3.4/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib64/python3.4/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "FOLDERNAME/aaaaa_functions.py", line 138, in url_check
    rc = urllib.request.urlopen(url, timeout=10).read().decode('utf-8')
  File "/usr/lib64/python3.4/urllib/request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib64/python3.4/urllib/request.py", line 464, in open
    response = self._open(req, data)
  File "/usr/lib64/python3.4/urllib/request.py", line 482, in _open
    '_open', req)
  File "/usr/lib64/python3.4/urllib/request.py", line 442, in _call_chain
    result = func(*args)
  File "/usr/lib64/python3.4/urllib/request.py", line 1211, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib64/python3.4/urllib/request.py", line 1186, in do_open
    r = h.getresponse()
  File "/usr/lib64/python3.4/urllib/request.py", line 1186, in do_open
    r = h.getresponse()
  File "/usr/lib64/python3.4/http/client.py", line 1247, in getresponse
    response.begin()
  File "/usr/lib64/python3.4/http/client.py", line 396, in begin
    version, status, reason = self._read_status()
  File "/usr/lib64/python3.4/http/client.py", line 358, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib64/python3.4/socket.py", line 378, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [Errno 104] Connection reset by peer
"""


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "FOLDERNAME/all_urls_check.py", line 91, in <module>
    p.map(url_check,array)
  File "/usr/lib64/python3.4/multiprocessing/pool.py", line 260, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/lib64/python3.4/multiprocessing/pool.py", line 608, in get
    raise self._value
ConnectionResetError: [Errno 104] Connection reset by peer

Sorry. This is my first post.

please show your code.

Hello all,

Here is the function

def url_check(array):
    url=array[0]
    table_name=array[1]
    try:
        rc = urllib.request.urlopen(url, timeout=10).read().decode('utf-8')
    except (HTTPError,URLError) as e:
        print("URL unreachable")
    except timeout:
        print("URL unreachable")

I need a way to handle the exception -> "ConnectionResetError: [Errno 104] Connection reset by peer."
The above code is not able to handle this exception "ConnectionResetError: [Errno 104] Connection reset by peer"

wintenrod

wintenrod

Larz60+

wintenrod