Python Forum
Clone with pywebcopy with proxy
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Clone with pywebcopy with proxy
#1
Im struggling with pywebcopy and proxies. Its no problem to scrape a Onion site with BeautifulSoup(not the below code).

But im trying to use pywebcopy and cant figure it out, any help would be appreciated. It works fine with "ordinary sites" but not with Onion urls.

from pywebcopy import WebPage 
from pywebcopy import save_webpage
import requests
import os

proxies = {
  'http': 'socks5h://localhost:9150',
  'https': 'socks5h://localhost:9150',
}

proxy = 'socks5h://localhost:9150'

os.environ['http_proxy'] = proxy 
os.environ['HTTP_PROXY'] = proxy
os.environ['https_proxy'] = proxy
os.environ['HTTPS_PROXY'] = proxy

save_webpage(
    url='http://qrmfuxwgyzk5jdjz.onion',
    project_folder='C:/temp/PyClone',
)
When trying with Onion url i get the following:

Error:
Exception has occurred: AccessError Access is not allowed by the site of url http://qrmfuxwgyzk5jdjz.onion File "C:\temp\PyClone\testreq2.py", line 20, in <module> project_folder='C:/temp/PyClone',
Reply
#2
As the error shows.
the specified URL does not exist, or is down.

also, you are not showing where you execute the code in your listing.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Clone GitHub to Local sridhar 2 1,710 Apr-21-2020, 07:57 PM
Last Post: Larz60+
  Clone a project saisankalpj 1 2,939 Jul-29-2018, 05:20 PM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020