Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How do I scrape a web page?
#3
You can use the Python module selenium to scrape this web page. Selenium will starts your browser and interact with it through Python. Selenium needs a driver, if you use Firefox you need GeckoDriver and for Chrome you need ChromeDriver.

Then you can start it like so

from selenium import webdriver
 
driver = webdriver.Firefox()
driver.get("https://dev.to")
and find the element by id or name

You can find the element id in html, the table has the id "id="grid_table_642496". Then it would be

element=browser.find_element(By.ID,"grid_table_642496")
from there grab the data you need
Reply


Messages In This Thread
How do I scrape a web page? - by oradba4u - Dec-23-2020, 04:46 AM
RE: How do I scrape a web page? - by Larz60+ - Dec-23-2020, 12:23 PM
RE: How do I scrape a web page? - by codeto - Dec-23-2020, 12:35 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  how to scrape page that works dynamicaly? samuelbachorik 0 746 Sep-23-2023, 10:38 AM
Last Post: samuelbachorik
  to scrape wiki-page: getting back the results - can i use pandas also apollo 2 2,691 Feb-09-2021, 03:57 PM
Last Post: apollo
  Beautifulsoup doesn't scrape page (python 2.7) Hikki 0 2,022 Aug-01-2020, 05:54 PM
Last Post: Hikki
  use Xpath in Python :: libxml2 for a page-to-page skip-setting apollo 2 3,670 Mar-19-2020, 06:13 PM
Last Post: apollo
  scrape data 1 go to next page scrape data 2 and so on alkaline3 6 5,280 Mar-13-2020, 07:59 PM
Last Post: alkaline3
  How do i scrape website whose page changes using javsacript _dopostback function and Prince_Bhatia 1 7,281 Aug-06-2018, 09:45 AM
Last Post: wavic
  Scrape Facebook page user posts text stockholm 6 8,477 May-08-2017, 12:24 PM
Last Post: Joseph_f2

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020