Oct-29-2020, 06:32 PM
Im struggling with pywebcopy and proxies. Its no problem to scrape a Onion site with BeautifulSoup(not the below code).
But im trying to use pywebcopy and cant figure it out, any help would be appreciated. It works fine with "ordinary sites" but not with Onion urls.
But im trying to use pywebcopy and cant figure it out, any help would be appreciated. It works fine with "ordinary sites" but not with Onion urls.
from pywebcopy import WebPage from pywebcopy import save_webpage import requests import os proxies = { 'http': 'socks5h://localhost:9150', 'https': 'socks5h://localhost:9150', } proxy = 'socks5h://localhost:9150' os.environ['http_proxy'] = proxy os.environ['HTTP_PROXY'] = proxy os.environ['https_proxy'] = proxy os.environ['HTTPS_PROXY'] = proxy save_webpage( url='http://qrmfuxwgyzk5jdjz.onion', project_folder='C:/temp/PyClone', )When trying with Onion url i get the following:
Error:Exception has occurred: AccessError
Access is not allowed by the site of url http://qrmfuxwgyzk5jdjz.onion
File "C:\temp\PyClone\testreq2.py", line 20, in <module>
project_folder='C:/temp/PyClone',