Python Forum
Selenium suddenly fails to find element
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Selenium suddenly fails to find element
#4
(Sep-02-2022, 05:05 PM)snippsat Wrote: Amazon has some best protection against scraper,bot...ect.
So it natural that it fails sometime,also if element is not loaded on page then scrape to early will fail.
If work most of the times schedule a new try.
Can use Waits for this or as first test i just throw in time.sleep sometime .
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import re

.....
tag_url = WebDriverWait(browser, 10).until(EC.visibility_of_element_located((By.CSS_SELECTOR, '.a-size-mini > a:nth-child(1)')))
print(tag_url.get_property('href'))

Thanks. Tried your approach. Doesn't work.
It seems that if the number of accesses (or overall access time) to Amazon exceeds a certain limit, access is blocked for a certain period of time.
Surprisingly, using VPN cannot solve the problem either.
Reply


Messages In This Thread
RE: Selenium suddenly fails to find element - by Pavel_47 - Sep-04-2022, 11:06 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  How to find element in a deeply nested html structure Orientation_group 5 1,387 Oct-09-2023, 10:13 AM
Last Post: Larz60+
  find a hyperlink in Gmail body python 3(imap and selenium) taomihiranga 1 8,223 Dec-30-2020, 05:31 PM
Last Post: Gamer1057
  Beautiful Soup (suddenly) doesn't get full webpage html j.crater 8 17,131 Jul-11-2020, 04:31 PM
Last Post: j.crater
  Find element using xpath engli 3 3,988 May-20-2020, 08:56 AM
Last Post: Larz60+
  selenium click in iframe fails 3Pinter 6 5,190 Apr-29-2020, 12:59 PM
Last Post: Larz60+
  find element...click() dont work windows11 6 3,202 Apr-23-2020, 11:13 PM
Last Post: law
  Find Dynamic Input Element Class Problem wazacko 0 1,627 Apr-03-2020, 11:46 AM
Last Post: wazacko
  bypassing find element click() windows11 1 1,917 Apr-02-2020, 11:55 AM
Last Post: Larz60+
  How find link element zinho 3 3,159 Mar-31-2020, 12:29 PM
Last Post: snippsat
  Clicking on element not triggering event in Selenium Python (Event Key is not in data dkaeloredo 2 4,309 Feb-16-2020, 05:50 AM
Last Post: dkaeloredo

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020