Python Forum
Can't get method to scroll down page. - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html)
+--- Thread: Can't get method to scroll down page. (/thread-19160.html)



Can't get method to scroll down page. - caarsonr - Jun-15-2019

This method is supposed to find the comment box on youtube, right now, my scrolling method does not seem to be working. I can't seem to figure out why. Any ideas?

    def find_comment_box(self):
        try:
            self.browser.get(self.url)
            SCROLL_PAUSE_TIME = 0.5

            # Get scroll height
            last_height = self.browser.execute_script("return document.body.scrollHeight")

            while True:
                # Scroll down to bottom
                self.browser.execute_script("window.scrollTo(0, document.body.scrollHeight);")

                # Wait to load page
                time.sleep(SCROLL_PAUSE_TIME)

                # Calculate new scroll height and compare with last scroll height
                new_height = self.browser.execute_script("return document.body.scrollHeight")
                if new_height == last_height:
                    break
                if last_height != new_height:
                    pass

            print("hello")
            self.comment_box = self.browser.find_element_by_id("contenteditable-root")
        except Exception as internalerror:
            print(internalerror)
            print(self.error)



RE: Can't get method to scroll down page. - metulburr - Jun-16-2019

Expain in more detail? Do you get an error? Is it in a different window? Does it scroll at least once, or not at all? Try a timesleep right after scrolling before loading the element ID to attempt to see if it does anything.

The best way to get an answer is to make an example script that goes to youtube and attempts to scroll so wecan just run it


RE: Can't get method to scroll down page. - caarsonr - Jun-17-2019

(Jun-16-2019, 04:49 PM)metulburr Wrote: def find_comment_box(self):
    try:
        self.browser.get(self.url)
        SCROLL_PAUSE_TIME = 0.5
 
        # Get scroll height
        last_height = self.browser.execute_script("return document.body.scrollHeight")
 
        while True:
            # Scroll down to bottom
            self.browser.execute_script("window.scrollTo(0, document.body.scrollHeight);")
 
            # Wait to load page
            time.sleep(SCROLL_PAUSE_TIME)
 
            # Calculate new scroll height and compare with last scroll height
            new_height = self.browser.execute_script("return document.body.scrollHeight")
            if new_height == last_height:
                break
            if last_height != new_height:
                pass
 
        print("hello")
        self.comment_box = self.browser.find_element_by_id("contenteditable-root")
    except Exception as internalerror:
        print(internalerror)
        print(self.error)
going to make you guys a script that does this for debugging, thank you!


RE: Can't get method to scroll down page. - caarsonr - Jun-20-2019

(Jun-16-2019, 04:49 PM)metulburr Wrote: Expain in more detail? Do you get an error? Is it in a different window? Does it scroll at least once, or not at all? Try a timesleep right after scrolling before loading the element ID to attempt to see if it does anything. The best way to get an answer is to make an example script that goes to youtube and attempts to scroll so wecan just run it
# imports
import random
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys


class Commenter:
# declares variables for commentor script
    def __init__(self):
        self.error = "[!] There was an error!"
        self.url =  "https://www.youtube.com/watch?v=SEpmYLu-CCA"
        self.browser = webdriver.Chrome(executable_path = '/Users/carsonrhodes/Desktop/Chrome Driver/chromedriver')
        self.comment_box = " "

# finds comment box so that the script can comment.
    def find_comment_box(self):
        try:
            self.browser.get(self.url)
            SCROLL_PAUSE_TIME = 0.5

            # Get scroll height
            last_height = self.browser.execute_script("return document.body.scrollHeight")

            while True:
                # Scroll down to bottom
                self.browser.execute_script("window.scrollTo(0, document.body.scrollHeight);")

                # Wait to load page
                time.sleep(SCROLL_PAUSE_TIME)

                # Calculate new scroll height and compare with last scroll height
                new_height = self.browser.execute_script("return document.body.scrollHeight")
                if new_height == last_height:
                    break
                if last_height != new_height:
                    pass

            print("hello")
            self.comment_box = self.browser.find_element_by_id("contenteditable-root")
        except Exception as internalerror:
            print(internalerror)
            print(self.error)


Commenter = Commenter() #creates class (cannot call methods without calling class.

Commenter.find_comment_box()
this should be a working script.


RE: Can't get method to scroll down page. - metulburr - Jun-20-2019

Im not sure why this doesnt work. But page down does work on this site

class Commenter:
# declares variables for commentor script
    def __init__(self):
        self.error = "[!] There was an error!"
        self.url =  "https://www.youtube.com/watch?v=SEpmYLu-CCA"
        self.browser = webdriver.Chrome('/home/metulburr/chromedriver')
        self.comment_box = " "
        self.browser.get(self.url)
        
        bg = self.browser.find_element_by_css_selector('body')
        for _ in range(3):
            bg.send_keys(Keys.PAGE_DOWN)
            time.sleep(.5)
This gets access to the input box of commenting


RE: Can't get method to scroll down page. - caarsonr - Jun-20-2019

(Jun-20-2019, 08:20 PM)metulburr Wrote:         bg = self.browser.find_element_by_css_selector('body')
        for _ in range(3):
            bg.send_keys(Keys.PAGE_DOWN)
            time.sleep(.5)
elegent and beautiful solution thanks!