Python Forum
How to Caputre Data After Selenium Scroll
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How to Caputre Data After Selenium Scroll
#1
Hello

I made a small python script that loads a website and then scrolls to the bottom of the page. I now would like to count all the instances in the page which have text ', no run,' and preferably show them in a column or data frame format. Can someone please tell me how can I do it and how can I load the fully scrolled data into a python variable?

Thanks! P.S. Below is the script that I wrote.

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time


driver = webdriver.Firefox()
driver.get('https://www.espncricinfo.com/series/11380/commentary/860281/sri-lanka-vs-pakistan-2nd-t20i-pakistan-tour-of-sri-lanka-2015')


elm = driver.find_element_by_tag_name('html')
elm.send_keys(Keys.END)
time.sleep(3)
elm.send_keys(Keys.END)
time.sleep(3)
elm.send_keys(Keys.END)
time.sleep(3)
elm.send_keys(Keys.END)
time.sleep(3)
time.sleep(3)
elm.send_keys(Keys.END)
time.sleep(3)
time.sleep(3)
elm.send_keys(Keys.END)
time.sleep(3)
time.sleep(3)
elm.send_keys(Keys.END)
time.sleep(3)
time.sleep(3)
elm.send_keys(Keys.END)
time.sleep(3)
time.sleep(3)
elm.send_keys(Keys.END)
time.sleep(3)
time.sleep(3)
elm.send_keys(Keys.END)
time.sleep(3)
time.sleep(3)
elm.send_keys(Keys.END)
time.sleep(3)
Reply


Messages In This Thread
How to Caputre Data After Selenium Scroll - by ahmedwaqas92 - Jul-30-2019, 02:32 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Python Selenium (Dropdown-) data Robin_at_Cantelli 2 6,212 Dec-29-2021, 03:16 AM
Last Post: ondreweil
  Scroll Down Post Instagram Likes rmiguelantunes74 0 3,164 Oct-05-2020, 07:38 PM
Last Post: rmiguelantunes74
  Extract data with Selenium and BeautifulSoup nestor 3 3,931 Jun-06-2020, 01:34 AM
Last Post: Larz60+
  Webelement scroll rove76 1 2,108 Mar-17-2020, 05:38 PM
Last Post: rove76
  Clicking on element not triggering event in Selenium Python (Event Key is not in data dkaeloredo 2 4,288 Feb-16-2020, 05:50 AM
Last Post: dkaeloredo
  Selenium get data from newly accessed page hoff1022 2 2,955 Oct-09-2019, 06:52 PM
Last Post: hoff1022
  Unable to access javaScript generated data with selenium and headless FireFox. pjn4 0 2,554 Aug-04-2019, 11:10 AM
Last Post: pjn4
  Can't get method to scroll down page. caarsonr 5 4,277 Jun-20-2019, 09:14 PM
Last Post: caarsonr
  Selenium Data Scrubbing - Need Some Help HalPlz 1 2,365 Feb-26-2018, 11:06 PM
Last Post: Larz60+
  Error in Selenium: CRITICAL:root:Selenium module is not installed...Exiting program. AcszE 1 3,640 Nov-03-2017, 08:41 PM
Last Post: metulburr

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020