Hello Everyone,
I have a problem with the continuous down scrolling of page as it does not stop at the scrape number found. The following is what I have so far but can't seem to finish it so that it stop at the number found from class Polaris-Heading_1brcv. What do I need to modify to finish this script? Thanks
I have a problem with the continuous down scrolling of page as it does not stop at the scrape number found. The following is what I have so far but can't seem to finish it so that it stop at the number found from class Polaris-Heading_1brcv. What do I need to modify to finish this script? Thanks
def scroll_down(elem, num): for _ in range(num): time.sleep(.01) elem.send_keys(Keys.PAGE_DOWN) SCROLL_PAUSE_TIME = 10 elem = browser.find_element_by_tag_name("body") prev_height = elem.get_attribute("scrollHeight") for i in range(0, 100000000000): edtx = WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.CLASS_NAME, "Polaris-Heading_1brcv"))) et= edtx.text print('This is text from Records',et) ett= re.findall('\d+',str(etm)) ea = ett[0] browser.find_element_by_class_name("crNG1").click() scroll_down(elem,100) # Wait to allow new items to load time.sleep(SCROLL_PAUSE_TIME) #check to see if scrollable space got larger #also we're waiting until the second iteration to give time for the initial loading print('This is ea', ea) if str(ea) ==str(200): break