Python Forum

I just started web scrapping and decided to play around with Selenium. Below is the code I wrote to just login to the website and attempt to grab some data.

from selenium import webdriver

driver = webdriver.Chrome(r"C:\Users\hones\Desktop\MWO data\chromedriver.exe")

login_screen = driver.get("https://mwomercs.com/profile/leaderboards")
username = driver.find_element_by_id("email").send_keys("login_here")
password = driver.find_element_by_id("password").send_keys("some_password")
submit = driver.find_element_by_xpath("//*[@id='loginForm']/form/button").click()

stats = driver.find_elements_by_xpath("//*[@id='contentBody']/div[2]/table/tbody/tr[1]")
print(stats)

Everything works fine and no error message is presented. However, instead of returning the number I am trying to scrape, it returns something entirely different and it is displayed below.

Output:
[<selenium.webdriver.remote.webelement.WebElement (session="ebff9ddc3b927111b738557f1767aa58", element="d02991c8-c093-4d57-97e7-53b5bc8f75bd")>]

How would I go about getting past this and getting the data I am after? Thanks in advance for any feedback that is given.

Quote:[<selenium.webdriver.remote.webelement.WebElement (session="ebff9ddc3b927111b738557f1767aa58", element="d02991c8-c093-4d57-97e7-53b5bc8f75bd")>]

This is being return by your print statement,

stats = driver.find_elements_by_xpath("//*[@id='contentBody']/div[2]/table/tbody/tr[1]")
print(stats)

The above method will return all the possible elements which matches with the xpath you mentioned in list form. You can loop through that variable stats and print each element to see

Hello,

I have just found this post and run into the same issue.

The fix is pretty easy for anyone else who comes across this post with the not so helpful (in my opinion anyway!) answer previously.

If, like me you are just trying to get some information scraped from a website but it returns the session ID, all you need to do is put

.text

at the end of your scraping request.

So for example if you wanted to scrape https://example.com then you would do the following

from selenium import webdriver
from selenium.webdriver.common.by import By

# create webdriver object
driver = webdriver.Firefox()

#get website 
driver.get("https://example.com")

#get element
element = driver.find_element(By.CLASS_NAME, "p").text

print(f"the element I want is {p}")

Output:
the element I want is This domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission.

Now, clearly this is a redundant example but it works. I found this information on the selenium docs here https://www.selenium.dev/documentation/w...formation/