Sep-04-2022, 11:06 AM
(Sep-02-2022, 05:05 PM)snippsat Wrote: Amazon has some best protection against scraper,bot...ect.
So it natural that it fails sometime,also if element is not loaded on page then scrape to early will fail.
If work most of the times schedule a new try.
Can use Waits for this or as first test i just throw intime.sleep
sometime .
from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC import re ..... tag_url = WebDriverWait(browser, 10).until(EC.visibility_of_element_located((By.CSS_SELECTOR, '.a-size-mini > a:nth-child(1)'))) print(tag_url.get_property('href'))
Thanks. Tried your approach. Doesn't work.
It seems that if the number of accesses (or overall access time) to Amazon exceeds a certain limit, access is blocked for a certain period of time.
Surprisingly, using VPN cannot solve the problem either.