+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html)
+--- Thread: disable proxy with requests module (/thread-2718.html)
disable proxy with requests module - metulburr - Apr-05-2017
doing a simple request with the requests module causes the error below. This error is due to me using a proxy for web browsers only. However i dont want my requests module to use that proxy. Using selenium, or the standard urllib modules does not do the same.
Error:Traceback (most recent call last):
File "test2.py", line 9, in <module>
soup = bs(requests.get(url).text, 'html.parser')
File "/usr/local/lib/python2.7/dist-packages/requests-2.11.1-py2.7.egg/requests/api.py", line 70, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests-2.11.1-py2.7.egg/requests/api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests-2.11.1-py2.7.egg/requests/sessions.py", line 488, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests-2.11.1-py2.7.egg/requests/sessions.py", line 609, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests-2.11.1-py2.7.egg/requests/adapters.py", line 390, in send
conn = self.get_connection(request.url, proxies)
File "/usr/local/lib/python2.7/dist-packages/requests-2.11.1-py2.7.egg/requests/adapters.py", line 290, in get_connection
proxy_manager = self.proxy_manager_for(proxy)
File "/usr/local/lib/python2.7/dist-packages/requests-2.11.1-py2.7.egg/requests/adapters.py", line 184, in proxy_manager_for
**proxy_kwargs
File "/usr/local/lib/python2.7/dist-packages/requests-2.11.1-py2.7.egg/requests/adapters.py", line 43, in SOCKSProxyManager
raise InvalidSchema("Missing dependencies for SOCKS support.")
requests.exceptions.InvalidSchema: Missing dependencies for SOCKS support.2) Is there a better way to handle it than to do something like....
import requests requests = requests.Session() requests.trust_env = False ...
RE: disable proxy with requests module - wavic - Apr-05-2017
You can pass a proxy as a parameter to get() method. I've never used it.
See this in SO: http://stackoverflow.com/questions/30837839/how-can-i-set-a-single-proxy-for-a-requests-session-object
Or you want to bypass it?
RE: disable proxy with requests module - metulburr - Apr-05-2017
that is if you want to add a proxy to your request. Isnt it? I dont want to have a proxy on my request at all, but it is using my browser proxy anyways.