Python Forum

Selenium is a good idea.

I try run your code but got the exception as follow:
...
WebDriverException: ChromeDriver executable needs to be available in the path. Please download from http://chromedriver.storage.googleapis.com/index.html and read up at http://code.google.com/p/selenium/wiki/ChromeDriver
...

Quote:...
If i had a working log this would probably work,because now it simulate a browser.
...

What a working log do you mean here?

(Sep-01-2017, 03:16 PM)thomas2004ch Wrote: [ -> ]I try run your code but got the exception as follow:

You need chromedriver.exe in Windows Path or in folder where you run code.
Same phantomjs.exe if want to test it.
It's in link error message you got,unzip chromedriver_win32.zip

Quote:What a working log do you mean here?

I did mean a working login credential to access site.

I try again and this time I got following exception:
...
WebDriverException: unknown error: Runtime.executionContextCreated has invalid 'context': {"auxData":{"frameId":"6304.1","isDefault":true},"id":1,"name":"","origin":"://"}
(Session info: chrome=60.0.3112.113)
(Driver info: chromedriver=2.8.241075,platform=Windows NT 6.3 x86_64)
...

I am not sure if this is the version problem of chronmedriver? The chromedriver I downloaded is 32bit. But my Windows is 64-bit.

Why you use the SubID and Subname?

(Sep-01-2017, 04:54 PM)thomas2004ch Wrote: [ -> ]The chromedriver I downloaded is 32bit. But my Windows is 64-bit.

There's only 32-bit release of Chromedriver,it work for both 32 and 64 bit versions of Windows. 
You can thy the FireFox driver.
browser = webdriver.Firefox()

Quote:Why you use the SubID and Subname?

I am using CSS selector to find input fields and button.

I changed to using the FireFox. This time I got no error. The FireFox browser is open with an empty page. What does this mean?

After a while the browser is closed itself and I got following message and exception:

Error:
runfile('D:/Temp/Selenium.py', wdir='D:/Temp')
Traceback (most recent call last):

  File "<ipython-input-9-7d93cfa31c34>", line 1, in <module>
    runfile('D:/Temp/Selenium.py', wdir='D:/Temp')

  File "C:\ProgramData\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 880, in runfile
    execfile(filename, namespace)

  File "C:\ProgramData\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "D:/Temp/Selenium.py", line 14, in <module>
    browser = webdriver.Firefox()

  File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\webdriver.py", line 59, in __init__
    self.binary, timeout),

  File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\extension_connection.py", line 47, in __init__
    self.binary.launch_browser(self.profile)

  File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\firefox_binary.py", line 64, in launch_browser
    self._wait_until_connectable()

  File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\firefox_binary.py", line 108, in _wait_until_connectable
    self.profile.path, self._get_firefox_output()))

WebDriverException: Can't load the profile. Profile Dir: C:\Users\ADMINI~1\AppData\Local\Temp\2\tmpk2k3o7ox Firefox output: None

I usually use the chrome driver.

This did work for me.
Add geckodriver.exe the 64-bit version to Windows Path or folder you run code.
In code:

caps = webdriver.DesiredCapabilities().FIREFOX
caps["marionette"] = False
browser = webdriver.Firefox(capabilities=caps)
web_url = 'http://technical.traders.com/sub/sublogin2.asp'
browser.get(web_url)

It doesn't help. The result is the same as before. Now here the whole code:

from selenium import webdriver
from bs4 import BeautifulSoup
import time
 
# Activate Phantom and deactivate Chrome to not load browser

caps = webdriver.DesiredCapabilities().FIREFOX
caps["marionette"] = False
browser = webdriver.Firefox(capabilities=caps)
web_url = 'http://technical.traders.com/sub/sublogin2.asp'
browser.get(web_url)

user_name = browser.find_element_by_css_selector('#SubID > input[type="text"]')
user_name.send_keys("Foo")
password = browser.find_element_by_css_selector('#SubName > input[type="text"]')
password.send_keys("Bar")
time.sleep(2)
submit = browser.find_element_by_css_selector('#SubButton > input[type="submit"]')
submit.click()
time.sleep(2)
 
# Give source code to BeautifulSoup
soup = BeautifulSoup(browser.page_source, 'lxml')
log_in = soup.find('h2')
print(log_in.text)

---

Here the out put after the browser closes:

runfile('D:/Temp/Selenium.py', wdir='D:/Temp')
Traceback (most recent call last):

File "<ipython-input-12-7d93cfa31c34>", line 1, in <module>
runfile('D:/Temp/Selenium.py', wdir='D:/Temp')

File "C:\ProgramData\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 880, in runfile
execfile(filename, namespace)

File "C:\ProgramData\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "D:/Temp/Selenium.py", line 20, in <module>
browser = webdriver.Firefox(capabilities=caps)

File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\webdriver.py", line 59, in __init__
self.binary, timeout),

File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\extension_connection.py", line 47, in __init__
self.binary.launch_browser(self.profile)

File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\firefox_binary.py", line 64, in launch_browser
self._wait_until_connectable()

File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\firefox_binary.py", line 108, in _wait_until_connectable
self.profile.path, self._get_firefox_output()))

WebDriverException: Can't load the profile. Profile Dir: C:\Users\ADMINI~1\AppData\Local\Temp\2\tmpzuvl2fst Firefox output: None

I not sure what your error is.
I have anaconda3,so i can test there to.

Install selenuim into Anaconda3:

G:\Anaconda3\Scripts
λ pip install -U selenium
Collecting selenium
  Using cached selenium-3.5.0-py2.py3-none-any.whl
Installing collected packages: selenium
Successfully installed selenium-3.5.0

Run code:

G:\Anaconda3\Scripts
λ cd ..

G:\Anaconda3
λ python log_test.py
Log In Error

It start FireFox and fill in fields,so work as it should.

Try running from command line as i do here,name file something else than Selenium.py.
Run cmd as administrator.
Make sure that geckodriver.exe is in same folder as you run code.

Hi,

I use also Anaconda3. I did download and install the Selenium module with pip as you did. It must be coorrect, otherwise I will get error. And I rename my code to test_log.py as yours. And I copy the gecodriver.exe (64-bit) to the dir where the test_log.py is. Nothing more.

Then I open the cmd as Admin and start the code as you described. But the result is the same as before. Here the message after the browser closes itself.

C:\ProgramData\Anaconda3>python log_test.py
Traceback (most recent call last):
File "log_test.py", line 20, in <module>
browser = webdriver.Firefox(capabilities=caps)
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\webdriver.py", line 59, in __init__
self.binary, timeout),
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\extension_connection.py", line 47, in __init__
self.binary.launch_browser(self.profile)
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\firefox_binary.py", line 64, in launch_browser
self._wait_until_connectable()
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\firefox_binary.py", line 108, in _wait_until_connectable
self.profile.path, self._get_firefox_output()))
selenium.common.exceptions.WebDriverException: Message: "Can't load the profile. Profile Dir: C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\2\\tmpd9p5x5gv Firefox output: None"

C:\ProgramData\Anaconda3>


Info: I use Windows Server 2016, same as Windows 10. But I think this should not the reason, right? Or maybe it is lack off something by me?

---

I reinstall the Selenium again as follow:

Quote:C:\ProgramData\Anaconda3\Scripts>pip install -U selenium
Collecting selenium
Downloading selenium-3.5.0-py2.py3-none-any.whl (921kB)
100% |████████████████████████████████| 921kB 936kB/s
Installing collected packages: selenium
Found existing installation: selenium 2.43.0
Uninstalling selenium-2.43.0:
Successfully uninstalled selenium-2.43.0
Successfully installed selenium-3.5.0

Then I run my code again:

Quote:C:\ProgramData\Anaconda3\Scripts>cd ..

C:\ProgramData\Anaconda3>python.exe log_test.py

The FireFox browser is open. After a while it is closed itself. I see from the CMD console following messages. This time somewhat different than before. It tells the reason could be the uncompotability of FireFox:

Quote:C:\ProgramData\Anaconda3>python.exe log_test.py
Traceback (most recent call last):
File "log_test.py", line 20, in <module>
browser = webdriver.Firefox(capabilities=caps)
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\webdriver.py", line 171, in __init__
self.binary, timeout)
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\extension_connection.py", line 52, in __init__
self.binary.launch_browser(self.profile, timeout=timeout)
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\firefox_binary.py", line 73, in launch_browser
self._wait_until_connectable(timeout=timeout)
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\firefox_binary.py", line 114, in _wait_until_connectable
% (self.profile.path))
selenium.common.exceptions.WebDriverException: Message: Can't load the profile. Possible firefox version mismatch. You must use GeckoDriver instead for Firefox 48+. Profile Dir: C:\Users\ADMINI~1\AppData\Local\Temp\tmpsquk65eh If you specified a log_file in the FirefoxBinary constructor, check it for details.

---

What a FireFox version do you have? I ahve the 55.0.3(64-bit)

(Sep-01-2017, 07:58 PM)thomas2004ch Wrote: [ -> ]What a FireFox version do you have? I ahve the 55.0.3(64-bit)

Try change to:
caps["marionette"] = True
This i have to do for newer version of FireFox.