Jul-13-2018, 05:43 PM
edited for clarity:
Python Selenium Passing Scraped Proxy .text object Variable to Firefox Profile? I can get the proxy working with a string as the proxy but not the selenium text object scraped from the proxy website. I'm Still getting my local IP when passing the Selenium .text object. I can't understand why this isn't passing the proxy that is being printed in terminal as the variable for the proxy? I've included the code below with some of the solutions I've tried commented out.
Any help is much appreciated. I've been working on this project for way to long.
Ok found the answer here from
Running Selenium Webdriver with a proxy in Python @erm3nda
solution is this. Port has to be an integer the proxy can apparently be a string.
Python Selenium Passing Scraped Proxy .text object Variable to Firefox Profile? I can get the proxy working with a string as the proxy but not the selenium text object scraped from the proxy website. I'm Still getting my local IP when passing the Selenium .text object. I can't understand why this isn't passing the proxy that is being printed in terminal as the variable for the proxy? I've included the code below with some of the solutions I've tried commented out.
Any help is much appreciated. I've been working on this project for way to long.
###############################################################################
# MIPYTHON .com Python PROXY SCRAPE and GO SCRIPT in SELENIUM
###############################################################################
from selenium import webdriver
import time
print("*" * 60)
print("MIPYTHON .com Python PROXY SCRAPE and GO SCRIPT in SELENIUM")
print("*" * 60)
browser = webdriver.Firefox()
proxy_website = "https://free-proxy-list.net/"
target_website = "https://icanhazip.com" #"https://www.whatismyip.com"
### SCRAPE PROXY
browser.get(proxy_website)
browser.find_element_by_xpath('/html/body/header/div[1]/div/div[1]/div/div/a[2]').click()
proxy = browser.find_element_by_xpath('/html/body/section[1]/div/div[2]/div/div[2]/div/table/tbody/tr[1]/td[1]').text
print(proxy)
proxy_port = browser.find_element_by_xpath('/html/body/section[1]/div/div[2]/div/div[2]/div/table/tbody/tr[1]/td[2]').text
print(proxy_port)
### USE PROXY
#proxy = "89.23.19.143"
#proxy_port = 8080
#proxy = int(proxy)
#proxy_port = int(proxy_port)
#proxy = str(proxy)
#proxy_port = str(proxy_port)
proxy_profile = webdriver.FirefoxProfile()
proxy_profile.set_preference("network.proxy.type", 1)
proxy_profile.set_preference("network.proxy.http", proxy )
proxy_profile.set_preference("network.proxy.http_port", proxy_port)
proxy_profile.set_preference("network.proxy.ssl", proxy )
proxy_profile.set_preference("network.proxy.ssl_port", proxy_port)
browser = webdriver.Firefox(firefox_profile=proxy_profile)
target_website = browser.get(target_website)edit again:Ok found the answer here from
Running Selenium Webdriver with a proxy in Python @erm3nda
solution is this. Port has to be an integer the proxy can apparently be a string.
proxy_port = int(proxy_port) proxy = str(proxy)