Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Getting text using Selenium
#1
Hi guys,

I am trying to get some text content from html using Selenium and Chrome webdriver. The snippet of html is:

<span class="explorer__titlesegment explorer__titlesegment--type">Brokerage Account</span>

I am trying to extract the text 'Brokerage Account' which changes depending on situation...

My code in my script is:

browser = webdriver.Chrome(chromedriver.exe')
        browser.get(url)
        time.sleep(5)
        text = browser.find_element_by_class_name('explorer__titlesegment explorer__titlesegment--type').text
        print(text)
Output:

Output:
Error: Message: no such element: Unable to locate element: {"method":"css selector","selector":".explorer__titlesegment explorer__titlesegment--type"} (Session info: headless chrome=81.0.4044.122)
Any ideas what I am doing wrong please?

thanks
Reply
#2
The shorting in class name.
text = browser.find_element_by_class_name('explorer__titlesegment').text
print(text) 
Reply
#3
I tried that and the output:

Output:
Error: Message: no such element: Unable to locate element: {"method":"css selector","selector":".explorer__titlesegment"} (Session info: headless chrome=81.0.4044.122)
Reply
#4
Try find tag bye CSS selector or xPath.
Shorting class name did work for my,here a demo of both.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options

#--| Setup
options = Options()
options.add_argument("--headless")
options.add_argument("window-size=1024,960")
browser = webdriver.Chrome(executable_path=r'C:\cmder\bin\chromedriver.exe', options=options)
#--| Parse or automation
browser.get('file:///E:/div_code/scrape/local.html')
text = browser.find_element_by_class_name('explorer__titlesegment').text
text_selector = browser.find_elements_by_css_selector('body > span')[0].text
print(text)
print(text_selector)
Output:
Brokerage Account Brokerage Account
local.html that i test with.
<!DOCTYPE html>
<html>
  <head>
    <meta charset="UTF-8">
    <title>Title of the document</title>
  </head>
  <body>
    <span class="explorer__titlesegment explorer__titlesegment--type">Brokerage Account</span>
  </body>
</html>
Reply
#5
Output:
Error: list index out of range
This is the output (above) if I use the same version as you for css selector.

If I leave this code out by the way the rest of it scrapes the web page perfectly so I am lost at the moment.
Reply
#6
Then it dos not find the tag.
Hard to tell what's going on without looking at the site.
Reply
#7
Yes I did include the index brackets [0].

This is the block of html:

<strong class="explorer__titlesegment explorer__titlesegment--title">
<span class="icon icon--explorer"></span>EUR/USD - LIVE - Manual/Mobile (200%+)
</strong>
<span class="explorer__titlesegment explorer__titlesegment--pipe">|</span>
<span class="explorer__titlesegment explorer__titlesegment--type">Brokerage Account</span>
<span class="explorer__titlesegment explorer__titlesegment--accountnumber">XXXXXXXXXXXXXXX01</span>
<span class="explorer__titlesegment explorer__titlesegment--customstart">Starting Mar 24, 2019</span>
Reply
#8
We usually do not to help on PM as principle.
So here is the solution,without sharing the url public .
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import time

#--| Setup
options = Options()
options.add_argument("--headless")
options.add_argument("window-size=1024,960")
browser = webdriver.Chrome(executable_path=r'chromedriver.exe', options=options)
#--| Parse or automation
browser.get('url')
time.sleep(3)
text_type = browser.find_elements_by_css_selector('span.explorer__titlesegment.explorer__titlesegment--type')
print(text_type[0].text)
Reply
#9
Thank you this works great. The format for the css selector command was not obvious (after having read numerous other forums and tutorials on it!) - how did you arrive at the working version?
Reply
#10
(Apr-26-2020, 07:35 AM)WiPi Wrote: how did you arrive at the working version?
Just looking at the tag and then write CSS selector for it.
In browser(inspect dev tools) you can right click on tag the Copy --> Copy selector or Copy XPath.

Learn how this work bye testing out stuff, eg look at CSS selectors, Xpath cheatsheet.
Example
<!DOCTYPE html>
<html>
  <head>
    <meta charset="UTF-8">
    <title>Title of the document</title>
  </head>
  <body>
    <p><span class="foo">Brokerage Account</span></p>    
  </body>
</html>
So the selector and XPath for this would be.
css_selector = browser.find_elements_by_css_selector('span.foo')
xpath_selector = browser.find_elements_by_xpath('//span[@class="foo"]')
print(css_selector[0].text)
print(xpath_selector[0].text)
Output:
Brokerage Account Brokerage Account
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  selenium wait for text question Larz60+ 3 2,501 Oct-25-2021, 09:50 AM
Last Post: Larz60+
  How to get specific TD text via Selenium? euras 3 8,656 May-14-2021, 05:12 PM
Last Post: snippsat
  Selenium extract id text xzozx 1 2,078 Jun-15-2020, 06:32 AM
Last Post: Larz60+
  Selenium returning web element instead of desired text newbie_programmer 1 5,144 Dec-11-2019, 06:37 AM
Last Post: Malt
  Error in Selenium: CRITICAL:root:Selenium module is not installed...Exiting program. AcszE 1 3,588 Nov-03-2017, 08:41 PM
Last Post: metulburr

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020