Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Unable to Extract a web element
#1
Photo 
Hello,
I have this Selenium web scrapping project I've been working on for while, everything have been awesome till recently the website got updated and some Xpath values changed.
I updated the script to match the New Xpath values, everything went back to working as expected apart from one section.Here is the script
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
import time
from selenium.webdriver.support.select import Select

from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as ec
from selenium.webdriver.common.action_chains import ActionChains
   
from webdriver_manager.chrome import ChromeDriverManager


driver = webdriver.Chrome(ChromeDriverManager().install())
driver.get("https://odibets.com/")
time.sleep(8)


userName = driver.find_element_by_xpath('/html/body/div[1]/div[1]/div[1]/div[1]/div[3]/div/div[3]/a/img')
driver.execute_script("arguments[0].click();", userName)
time.sleep(3)

# navigate to  League
userName = driver.find_element_by_xpath('/html/body/div[1]/div[1]/div[1]/div[2]/div[1]/div[1]/ul/li[3]/a[2]/span[1]/img')
driver.execute_script("arguments[0].click();", userName)
time.sleep(6)


#Go To Results                                              
userName = driver.find_element_by_xpath('/html/body/div[1]/div[1]/div/div[2]/div[1]/div[1]/ul[2]/li[2]/button')
driver.execute_script("arguments[0].click();", userName)
time.sleep(7)

#check results in cell 1               
resultA=driver.find_element(By.XPATH,'/html/body/div[1]/div[1]/div/div[2]/div[1]/div[2]/div[1]/div/div/div[1]/div/table/tr[2]/td[2]')
#resultA=WebDriverWait(driver, 20).until(ec.visibility_of_element_located((By.XPATH, '/html/body/div[1]/div[1]/div/div[2]/div[1]/div[2]/div[1]/div/div/div[1]/div/table/tr[2]/td[2]')))
print (resultA.text)
print ('Results')
The part
print (resultA.text)
does not print anything and doesn't throw any Error
I have checked the Xpath countless times to a certain i have the right one, I have tried the Wait method but the problem remains.

İmage

Basically I want to get only the results of the first teams i.e 2:1 etc .
Kindly Help
Reply
#2
I cant see attached image. If you show what you want get with images, I will try create script.
law likes this post
Reply
#3
Hello,

Kindly view the image on this link:-

https://drive.google.com/file/d/1TozKdDv...sp=sharing

I'm aiming to get the values (results) encircled in red. Thanks
Reply
#4
Hello,

Check if this is right for you:

import json

import requests


def odibets():
    response = requests.post(f'https://odibets.com/api/fv', json={
        "competition_id": 1,  # 1 - English League
        "period": "",
        "sub_type_id": "",
        "tab": "results"
    })

    data = json.loads(response.content)

    data = data['data']

    match = next(iter(next(iter(data['results']))["matches"]))
    score = match['result'].split(':')

    print(f'{match["home_team"]} {score[0]} : {score[1]} {match["away_team"]}')


if __name__ == "__main__":
    try:
        odibets()
    except Exception as e:
        print("Unexpected error:", e)
law likes this post
Reply
#5
Thumbs Up 
Thanks Master, this works perfectly Surfing . I have to admit I don't quite understand what that code says but it works better than expected.
I'ma go n start learning Json immediately, next time I'll be better prepared for such challenges. If u know of any learning material that can help reduces my learning curve in this, kindly let me know.
Again thanks a million, you saved me big time Thumbs Up
Reply
#6
I suggest, take look at requests library https://requests.readthedocs.io/en/lates...uickstart/ and read about GET and POST. JSON is last thing. Selenium is overhead in this particular case.
What requests send to server you can see in browser dev tools network tab.
law likes this post
Reply
#7
Thanks Man, this has been super helpful
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Unable to click element in web page Kalpana 0 1,862 Jun-25-2020, 05:20 AM
Last Post: Kalpana

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020