Oct-12-2018, 11:09 PM
(Oct-12-2018, 09:50 PM)Truman Wrote: p.s. now running snippsat's code, it only opens yahoo page.Nope it also search(it's your code i only added agree push button),if add some more lines i can get links from search.
from selenium import webdriver from selenium.webdriver.common.keys import Keys import bs4 import time browser = webdriver.Chrome() browser.get('http://www.yahoo.com') assert 'Yahoo' in browser.title agree = browser.find_element_by_xpath('/html/body/div[1]/div[2]/div[4]/div/div[2]/form[1]/div/input') agree.click() elem = browser.find_element_by_name('p') # find the search box res = elem.send_keys('seleniumhq' + Keys.RETURN) #print(repr(res)) time.sleep(5) soup = bs4.BeautifulSoup(browser.page_source, "lxml") link_a = soup.find_all('a', class_=" ac-algo fz-l ac-21th lh-24") for link in link_a: print(link.text)
Output:Selenium - Web Browser Automation
Downloads - Selenium
Selenium (software) - Wikipedia
Selenium (@SeleniumHQ) | Twitter
Selenium WebDriver - docs.seleniumhq.org
SeleniumHQ/selenium - GitHub
Selenium Projects
Selenium Sponsors
Selenium IDE
Selenium Documentation — Selenium Documentation