Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Continuous Scrolling Loop
#1
Hello Everyone,

I have a problem with the continuous down scrolling of page as it does not stop at the scrape number found. The following is what I have so far but can't seem to finish it so that it stop at the number found from class Polaris-Heading_1brcv. What do I need to modify to finish this script? Thanks


def scroll_down(elem, num):
    for _ in range(num):
        time.sleep(.01)
        elem.send_keys(Keys.PAGE_DOWN)
        
SCROLL_PAUSE_TIME = 10
elem = browser.find_element_by_tag_name("body")
prev_height = elem.get_attribute("scrollHeight")


for i in range(0, 100000000000):
    edtx =  WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.CLASS_NAME, "Polaris-Heading_1brcv")))
    et= edtx.text
    print('This is text from Records',et)
    ett= re.findall('\d+',str(etm))
    ea = ett[0]

    browser.find_element_by_class_name("crNG1").click()

    
    scroll_down(elem,100)
    # Wait to allow new items to load
    time.sleep(SCROLL_PAUSE_TIME)

    #check to see if scrollable space got larger
    #also we're waiting until the second iteration to give time for the initial loading
    print('This is ea', ea)
    if str(ea)  ==str(200):
    
        break
    
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020