Python Forum
Thread Rating:
  • 3 Vote(s) - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Web-scraping part-2
#11
(Oct-15-2018, 11:59 PM)metulburr Wrote: Does that mean its better to use add_argument('--headless') rather than set_headless()?
Yes that's the new way and set_headless() is Deprecated.

I post same example here that show it can be used --headless or just comment out for not --headless.
This load browser,and search for car and show images.
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.common.keys import Keys
import time
 
#--| Setup
options = Options()
#options.add_argument("--headless")
caps = webdriver.DesiredCapabilities().FIREFOX
caps["marionette"] = True
browser = webdriver.Firefox(options=options, capabilities=caps, executable_path=r"geckodriver.exe")
#--| Parse or automation
browser.get('https://duckduckgo.com')
input_field = browser.find_elements_by_css_selector('#search_form_input_homepage')
input_field[0].send_keys('car' + Keys.RETURN)
time.sleep(3)
images_link = browser.find_elements_by_link_text('Images')
images_link[0].click()
time.sleep(5)
browser.quit()
--headless parse a value,also this do not load Browser.
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
import time
 
#--| Setup
options = Options()
options.add_argument("--headless")
caps = webdriver.DesiredCapabilities().FIREFOX
caps["marionette"] = True
browser = webdriver.Firefox(options=options, capabilities=caps, executable_path=r"geckodriver.exe")
#--| Parse
browser.get('https://duckduckgo.com')
logo = browser.find_elements_by_css_selector('#logo_homepage_link')
print(logo[0].text)
browser.quit()
Output:
About DuckDuckGo
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Web-Scraping part-1 snippsat 2 34,605 Jun-08-2017, 10:55 PM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020