Python Forum
Web Page not opening while web scraping through python selenium
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Web Page not opening while web scraping through python selenium
#1
I am writing codes to do web crawling in windows environment, and using google chrome and mozila firefox as browser. For google chrome I downloaded latest chromedriver and the latest geckodriver for firefox and added path of these drivers in the environment variables.
A sample code is given below which I am executing for google chrome -

from selenium import webdriver

driver = webdriver.Chrome("C:\\ChromeDriver\\chromedriver.exe")
driver.get("https://www.google.com")
In this case the web browser is opening but the google page didn't opened and the whole page remains white after staying like this for certain times the below error is coming. The image of the browser is attached.
Error:
c:\ChromeDriver>python sparrow.py [4064:8324:1119/150817.243:ERROR:install_util.cc(691)] Failed to read HKLM\SOFTWARE\Policies\Google\Chrome\MachineLevelUserCloudPolicyEnrollmentToken: The system cannot find the file specified. (0x2) DevTools listening on ws://127.0.0.1:50164/devtools/browser/f68ff514-1e83-403b-b8a3-61adfc50b15d Traceback (most recent call last): File "sparrow.py", line 3, in <module> driver = webdriver.Chrome("C:\\ChromeDriver\\chromedriver.exe") File "C:\Users\CSURV5\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 81, in __init__ desired_capabilities=desired_capabilities) File "C:\Users\CSURV5\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 157, in __init__ self.start_session(capabilities, browser_profile) File "C:\Users\CSURV5\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 252, in start_session response = self.execute(Command.NEW_SESSION, parameters) File "C:\Users\CSURV5\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute self.error_handler.check_response(response) File "C:\Users\CSURV5\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.WebDriverException: Message: chrome not reachable (Driver info: chromedriver=2.43.600210 (68dcf5eebde37173d4027fa8635e332711d2874a),platform=Windows NT 10.0.17134 x86_64)
The below code is for the firefox -
from selenium import webdriver

driver = webdriver.Firefox("C:\\ChromeDriver\\")
driver.get("https://www.google.com")
I am not getting any output after executing the code, even the browser is not also opening. After some time this error message has been thrown.
Error:
Traceback (most recent call last): File "sparrow.py", line 3, in <module> driver = webdriver.Firefox("C:\\ChromeDriver\\") File "C:\Users\CSURV5\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\firefox\webdriver.py", line 164, in __init__ self.service.start() File "C:\Users\CSURV5\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\common\service.py", line 104, in start raise WebDriverException("Can not connect to the Service %s" % self.path) selenium.common.exceptions.WebDriverException: Message: Can not connect to the Service geckodriver
Please let me know the exact reason, why I am getting these error?

Attached Files

Thumbnail(s)
   
Reply
#2
see: https://python-forum.io/Thread-Web-scraping-part-2
Reply
#3
Quote:driver = webdriver.Chrome("C:\\ChromeDriver\\chromedriver.exe")
what is the permissions of C:\ChromeDirver?

Quote:
driver = webdriver.Firefox("C:\\ChromeDriver\\")
As for firefox it appears you are just passing the same directory to it. Where is geckodriver?
Recommended Tutorials:
Reply
#4
(Nov-19-2018, 12:23 PM)metulburr Wrote:
Quote:what is the permissions of C:\ChromeDirver?

The directory ChromeDriver has full permission.

Quote:As for firefox it appears you are just passing the same directory to it. Where is geckodriver?
geckodriver and chromedriver is in the same directory C:\ChromeDriver
Reply
#5
(Nov-19-2018, 10:57 AM)sumandas89 Wrote: Failed to read HKLM\SOFTWARE\Policies\Google\Chrome\MachineLevelUserCloudPolicyEnrollmentToken:
It complains that you don't have this key is register.
I check i don't have this key,but work fine without for me.
Try add key to register follow this.

Also basic stuff if you make a folder ChromeDriver with chrome and gecko driver in it,you will need to add that path(C:\ChromeDriver) to Windows Path(Restart Pc).
How to Edit/Add Your System PATH
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Need help opening pages when web scraping templeowls 1 303 Feb-29-2024, 06:45 PM
Last Post: snippsat
  Scraping div tags with selenium, need help hfakoor2 1 1,066 Mar-12-2023, 08:31 AM
Last Post: hfakoor2
  I am scraping a web page but got an Error Sarmad54 3 1,452 Mar-02-2023, 08:20 PM
Last Post: Sarmad54
  Click on a button on web page using Selenium Pavel_47 7 4,677 Jan-05-2023, 04:20 AM
Last Post: ellapurnellrt
  Scraping the page without distorting content oleglpts 5 2,486 Dec-16-2021, 05:08 PM
Last Post: oleglpts
  Scraping a page with log in data (security, proxies) iamaghost 0 2,143 Mar-27-2021, 02:56 PM
Last Post: iamaghost
  Selenium/Helium loads up a blank web page firaki12345 0 2,043 Mar-23-2021, 11:51 AM
Last Post: firaki12345
  Scraping .aspx page Larz60+ 21 51,138 Mar-18-2021, 10:16 AM
Last Post: Larz60+
  Saving html page and reloading into selenium while developing all xpaths Larz60+ 4 4,171 Feb-04-2021, 07:01 AM
Last Post: jonathanwhite1
  opening your scraping with a csv file in excel?? Nobelium 0 1,423 Jan-27-2021, 02:31 PM
Last Post: Nobelium

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020