Python Forum
Webscraping news articles by using selenium
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Webscraping news articles by using selenium
#4
(Aug-24-2023, 08:53 AM)cate16 Wrote: Since I am very new to Python, I haven't understand what you mean. Do I have to put this in what part of the code?
Now you shall open cmd and type echo %PATH:;=&echo.% with <enter> then will get a list of folders.
So in one those folder you most place chromedriver.exe,these are Environment Variables Path

Here a working example.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
import time

# Setup
#https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/116.0.5795.0/win64/chromedriver-win64.zip
options = Options()
options.add_argument("--headless=new")
ser = Service(r"C:\cmder\bin\chromedriver.exe")
browser = webdriver.Chrome(service=ser, options=options)
# Parse or automation
url = 'https://www.palottery.state.pa.us/Draw-Games/Treasure-Hunt.aspx'
browser.get(url)
lotto_number = browser.find_element(By.CSS_SELECTOR, 'div.details')
print(lotto_number.text)
Output:
0106091523
So i have C:\cmder\bin in my OS Path.
It i use cmd,see that folder is there eg also folder C:\WINDOWS would work to place chromedriver.exe in.
Microsoft Windows [Version 10.0.19045.3324]
(c) Microsoft Corporation. Med enerett.

C:\>%PATH:;=&echo.%
.....
C:\cmder\bin
C:\WINDOWS\system32
C:\WINDOWS
....

C:\>
Reply


Messages In This Thread
RE: Webscraping news articles by using selenium - by snippsat - Aug-24-2023, 01:04 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Webscraping with beautifulsoup cormanstan 3 2,201 Aug-24-2023, 11:57 AM
Last Post: snippsat
  Webscraping returning empty table Buuuwq 0 1,466 Dec-09-2022, 10:41 AM
Last Post: Buuuwq
  WebScraping using Selenium library Korgik 0 1,090 Dec-09-2022, 09:51 AM
Last Post: Korgik
  How to get rid of numerical tokens in output (webscraping issue)? jps2020 0 1,996 Oct-26-2020, 05:37 PM
Last Post: jps2020
  Python Webscraping with a Login Website warriordazza 0 2,672 Jun-07-2020, 07:04 AM
Last Post: warriordazza
  Help with basic webscraping Captain_Snuggle 2 4,034 Nov-07-2019, 08:07 PM
Last Post: kozaizsvemira
  Can't Resolve Webscraping AttributeError Hass 1 2,370 Jan-15-2019, 09:36 PM
Last Post: nilamo
  How to exclude certain links while webscraping basis on keywords Prince_Bhatia 0 3,292 Oct-31-2018, 07:00 AM
Last Post: Prince_Bhatia
  Webscraping homework Ghigo1995 1 2,716 Sep-23-2018, 07:36 PM
Last Post: nilamo
  Intro to WebScraping d1rjr03 2 3,511 Aug-15-2018, 12:05 AM
Last Post: metulburr

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020