(Jul-02-2020, 09:52 PM)apollo Wrote: i have created a test-account for the demo-testing of this:The login work with my code,can parse with both with BS and Selenium.
login: pluginfan pass: testpasswd123
The
Session
is open as long as not close Selenium browser.close()
.from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.common.keys import Keys from bs4 import BeautifulSoup import time #--| Setup options = Options() #options.add_argument("--headless") #options.add_argument("--window-size=1980,1020") browser = webdriver.Chrome(executable_path=r'C:\cmder\bin\chromedriver.exe', options=options) #--| Parse or automation browser.get("https://login.wordpress.org/?locale=en_US") time.sleep(2) user_name = browser.find_element_by_css_selector('#user_login') user_name.send_keys("pluginfan") password = browser.find_element_by_css_selector('#user_pass') password.send_keys("testpasswd123") time.sleep(3) submit = browser.find_elements_by_css_selector('#wp-submit')[0] submit.click() time.sleep(3) # Example using selenium for parse title = browser.find_elements_by_xpath('//*[@id="home-welcome"]/header/h1') print(title[0].text) print('-' * 50) # Example using BeautifulSoup for parse soup = BeautifulSoup(browser.page_source, 'lxml') use_bs4 = soup.select_one('#home-welcome > section.showcase > p.subheading') print(use_bs4.text)
Output:Meet WordPress
--------------------------------------------------
35% of the web uses WordPress, from hobby blogs to the biggest news sites online.
apollo Wrote:answer 3No,you should not at all use urllib2
Try this instead of urllib2

snippsat Wrote:urllib,mechanize,cookiejar is older stuff that i don't use anymore.
Requests has taken over there task in better way.