Python Forum

Pages: 1 2

Hi guys,

I am trying to get some text content from html using Selenium and Chrome webdriver. The snippet of html is:

<span class="explorer__titlesegment explorer__titlesegment--type">Brokerage Account</span>

I am trying to extract the text 'Brokerage Account' which changes depending on situation...

My code in my script is:

browser = webdriver.Chrome(chromedriver.exe')
        browser.get(url)
        time.sleep(5)
        text = browser.find_element_by_class_name('explorer__titlesegment explorer__titlesegment--type').text
        print(text)

Output:

Output:Error: Message: no such element: Unable to locate element: {"method":"css selector","selector":".explorer__titlesegment explorer__titlesegment--type"}
  (Session info: headless chrome=81.0.4044.122)

Any ideas what I am doing wrong please?

thanks

The shorting in class name.

text = browser.find_element_by_class_name('explorer__titlesegment').text
print(text)

I tried that and the output:

Output:Error: Message: no such element: Unable to locate element: {"method":"css selector","selector":".explorer__titlesegment"}
  (Session info: headless chrome=81.0.4044.122)

Try find tag bye CSS selector or xPath.
Shorting class name did work for my,here a demo of both.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

#--| Setup
options = Options()
options.add_argument("--headless")
options.add_argument("window-size=1024,960")
browser = webdriver.Chrome(executable_path=r'C:\cmder\bin\chromedriver.exe', options=options)
#--| Parse or automation
browser.get('file:///E:/div_code/scrape/local.html')
text = browser.find_element_by_class_name('explorer__titlesegment').text
text_selector = browser.find_elements_by_css_selector('body > span')[0].text
print(text)
print(text_selector)

Output:Brokerage Account
Brokerage Account

local.html that i test with.

<!DOCTYPE html>
<html>
  <head>
    <meta charset="UTF-8">
    <title>Title of the document</title>
  </head>
  <body>
    <span class="explorer__titlesegment explorer__titlesegment--type">Brokerage Account</span>
  </body>
</html>

Output:
Error: list index out of range

This is the output (above) if I use the same version as you for css selector.

If I leave this code out by the way the rest of it scrapes the web page perfectly so I am lost at the moment.

Then it dos not find the tag.
Hard to tell what's going on without looking at the site.

Yes I did include the index brackets [0].

This is the block of html:

<strong class="explorer__titlesegment explorer__titlesegment--title">
<span class="icon icon--explorer"></span>EUR/USD - LIVE - Manual/Mobile (200%+)
</strong>
<span class="explorer__titlesegment explorer__titlesegment--pipe">|</span>
<span class="explorer__titlesegment explorer__titlesegment--type">Brokerage Account</span>
<span class="explorer__titlesegment explorer__titlesegment--accountnumber">XXXXXXXXXXXXXXX01</span>
<span class="explorer__titlesegment explorer__titlesegment--customstart">Starting Mar 24, 2019</span>

We usually do not to help on PM as principle.
So here is the solution,without sharing the url public .

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import time

#--| Setup
options = Options()
options.add_argument("--headless")
options.add_argument("window-size=1024,960")
browser = webdriver.Chrome(executable_path=r'chromedriver.exe', options=options)
#--| Parse or automation
browser.get('url')
time.sleep(3)
text_type = browser.find_elements_by_css_selector('span.explorer__titlesegment.explorer__titlesegment--type')
print(text_type[0].text)

Thank you this works great. The format for the css selector command was not obvious (after having read numerous other forums and tutorials on it!) - how did you arrive at the working version?

(Apr-26-2020, 07:35 AM)WiPi Wrote: [ -> ]how did you arrive at the working version?

Just looking at the tag and then write CSS selector for it.
In browser(inspect dev tools) you can right click on tag the Copy --> Copy selector or Copy XPath.

Learn how this work bye testing out stuff, eg look at CSS selectors, Xpath cheatsheet.
Example

<!DOCTYPE html>
<html>
  <head>
    <meta charset="UTF-8">
    <title>Title of the document</title>
  </head>
  <body>
    <p><span class="foo">Brokerage Account</span></p>    
  </body>
</html>

So the selector and XPath for this would be.

css_selector = browser.find_elements_by_css_selector('span.foo')
xpath_selector = browser.find_elements_by_xpath('//span[@class="foo"]')
print(css_selector[0].text)
print(xpath_selector[0].text)

Output:Brokerage Account
Brokerage Account

Pages: 1 2

WiPi

snippsat

WiPi

snippsat

WiPi

snippsat

WiPi

snippsat

WiPi

snippsat