+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html)
+--- Thread: scrolldown + beautifulsoup? (/thread-21087.html)
scrolldown + beautifulsoup? - zarize - Sep-13-2019
Hi guys,
Is it possible to extract all data from the site, while site is showing more results each time when user is scrolling down?
I would love to do this by using BeautifulSoup.
I have tried Selenium, but couldn't even once get a text from any element :(
RE: scrolldown + beautifulsoup? - metulburr - Sep-13-2019
It is javascript that is showing results based on whether it is scrolled down, requiring you to use selenium to scroll and obtain the results.
What is your code and the website? Most likely you are parsing the HTML incorrectly.