Python Forum
Problem with searching over Beautiful Soap object
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Problem with searching over Beautiful Soap object
#23
This is the old way browser.find_elements_by_xpath(Deprecated) when use Selenium 4 is like this.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
import time

#--| Setup
options = Options()
options.add_argument("--headless")
options.add_argument("--window-size=1920,1080")
options.add_experimental_option('excludeSwitches', ['enable-logging'])
ser = Service(r"C:\cmder\bin\chromedriver.exe")
browser = webdriver.Chrome(service=ser, options=options)
#--| Parse or automation
url = "https://www.amazon.com/Advanced-Artificial-Intelligence-Robo-Justice-Georgios-ebook/dp/B0B1H2MZKX/ref=sr_1_1?keywords=9783030982058&qid=1653563461&sr=8-1"
browser.get(url)
title = browser.find_element(By.CSS_SELECTOR, '#productTitle')
print(title.text)
# with CSS selector
publisher = browser.find_element(By.CSS_SELECTOR, '#detailBullets_feature_div > ul > li:nth-child(2) > span > span:nth-child(2)')
print(publisher.text)
# With XPath
publisher1 = browser.find_element(By.XPATH, '//*[@id="detailBullets_feature_div"]/ul/li[2]/span/span[2]')
print(publisher1.text) 
Output:
Advanced Artificial Intelligence and Robo-Justice Springer (May 16, 2022) Springer (May 16, 2022)
Reply


Messages In This Thread
RE: Problem with searching over Beautiful Soap object - by snippsat - May-28-2022, 02:15 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Beautifull Soap. Split page using a value and not a tag. lillo123 5 3,611 Apr-21-2021, 09:11 AM
Last Post: lillo123
  Beautiful Soap can't find a specific section on the page Pavel_47 1 2,535 Jan-18-2021, 02:18 PM
Last Post: snippsat
  Beautiful soup and tags starter_student 11 6,668 Jul-08-2019, 03:41 PM
Last Post: starter_student
  Beautiful Soup find_all() kirito85 2 3,532 Jun-14-2019, 02:17 AM
Last Post: kirito85
  form.populate_obj problem "object has no attribute translate" pascale 0 3,747 Jun-12-2019, 07:30 PM
Last Post: pascale
  Need help with Beautiful Soup - table jlkmb 9 6,226 Dec-20-2018, 01:10 AM
Last Post: jlkmb
  Type Not Found error on python soap call using suds library wellborn 1 4,738 Dec-19-2017, 07:53 PM
Last Post: micseydel
  Help with beautiful soup Larz60+ 5 4,614 Jul-18-2017, 08:19 PM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020