Python Forum
Scraping a Flexible Element - works at first, and then crashes
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Scraping a Flexible Element - works at first, and then crashes
#1
import selenium
import os
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import time

executable_path = "C:\Program Files (x86)\chromedriver.exe"
os.environ["webdriver.chrome.driver"] = executable_path

chrome_options = Options()
chrome_options.add_extension('C:\\Program Files (x86)\\adguard.crx')

driver = webdriver.Chrome(executable_path=executable_path, chrome_options=chrome_options)


driver.get("https://pinterest.com")

element = driver.find_element_by_xpath('//div[contains(text(),"Live Posts")]')
element.click()


time.sleep(6)
posts = driver.find_elements_by_class_name('_3MKTqZAzt9ZahG8uCIFoOb')

for post in posts:
		title = post.find_element_by_class_name('_1h0SwjHGr__FljITS6J2FA').text
		time_ago = post.find_element_by_class_name('_94ZmeYYiS8oR1zgGtS0xI').text	
		value= post.find_element_by_class_name('_1jlBA3TjmrUfFXXCyWbfpV').text
		print(title, value, time_ago)
This code works and scrapes the first several lines, but then it crashes with this error:

Quote:selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document

The element I am trying to scrape is flex, and gets reloaded every dew seconds (at that time when it reloads, the script crashes. the reload takes less than 1 second.
This is image of the flex element:
[Image: 1onk0sd]
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Understanding and debugging memory error crashes with python3.10.10 Arkaik 5 2,073 Apr-18-2023, 03:22 AM
Last Post: Larz60+
  Pydroid3 app crashes on xiaomi poco F3 JMD 2 1,235 Nov-27-2022, 11:56 AM
Last Post: JMD
  HTML file crashes program mikefirth 12 3,808 Dec-31-2021, 03:57 AM
Last Post: Pedroski55
  Bug that crashes the Python interpreter 3.7 on Mac OS X Stef 1 3,684 Feb-18-2019, 06:30 PM
Last Post: micseydel
  Unable to locate element no such element gahhon 6 4,466 Feb-18-2019, 02:09 PM
Last Post: gahhon
  Change single element in 2D list changes every 1D element AceScottie 9 12,066 Nov-13-2017, 07:05 PM
Last Post: Larz60+
  Threading timer crashes my App panoss 4 6,493 Jun-05-2017, 05:57 PM
Last Post: nilamo

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020