Posts: 17
Threads: 1
Joined: Aug 2017
Selenium is a good idea.
I try run your code but got the exception as follow:
...
WebDriverException: ChromeDriver executable needs to be available in the path. Please download from http://chromedriver.storage.googleapis.com/index.html and read up at http://code.google.com/p/selenium/wiki/ChromeDriver
...
Quote:...
If i had a working log this would probably work,because now it simulate a browser.
...
What a working log do you mean here?
Posts: 7,106
Threads: 122
Joined: Sep 2016
Sep-01-2017, 04:13 PM
(This post was last modified: Sep-01-2017, 04:13 PM by snippsat.)
(Sep-01-2017, 03:16 PM)thomas2004ch Wrote: I try run your code but got the exception as follow: You need chromedriver.exe in Windows Path or in folder where you run code.
Same phantomjs.exe if want to test it.
It's in link error message you got,unzip chromedriver_win32.zip
Quote:What a working log do you mean here?
I did mean a working login credential to access site.
Posts: 17
Threads: 1
Joined: Aug 2017
Sep-01-2017, 04:54 PM
(This post was last modified: Sep-01-2017, 04:54 PM by thomas2004ch.)
I try again and this time I got following exception:
...
WebDriverException: unknown error: Runtime.executionContextCreated has invalid 'context': {"auxData":{"frameId":"6304.1","isDefault":true},"id":1,"name":"","origin":"://"}
(Session info: chrome=60.0.3112.113)
(Driver info: chromedriver=2.8.241075,platform=Windows NT 6.3 x86_64)
...
I am not sure if this is the version problem of chronmedriver? The chromedriver I downloaded is 32bit. But my Windows is 64-bit.
Why you use the SubID and Subname?
Posts: 7,106
Threads: 122
Joined: Sep 2016
(Sep-01-2017, 04:54 PM)thomas2004ch Wrote: The chromedriver I downloaded is 32bit. But my Windows is 64-bit. There's only 32-bit release of Chromedriver,it work for both 32 and 64 bit versions of Windows.
You can thy the FireFox driver.
browser = webdriver.Firefox()
Quote:Why you use the SubID and Subname?
I am using CSS selector to find input fields and button.
Posts: 17
Threads: 1
Joined: Aug 2017
Sep-01-2017, 06:02 PM
(This post was last modified: Sep-01-2017, 06:27 PM by metulburr.)
I changed to using the FireFox. This time I got no error. The FireFox browser is open with an empty page. What does this mean?
After a while the browser is closed itself and I got following message and exception:
Error: runfile('D:/Temp/Selenium.py', wdir='D:/Temp')
Traceback (most recent call last):
File "<ipython-input-9-7d93cfa31c34>", line 1, in <module>
runfile('D:/Temp/Selenium.py', wdir='D:/Temp')
File "C:\ProgramData\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 880, in runfile
execfile(filename, namespace)
File "C:\ProgramData\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "D:/Temp/Selenium.py", line 14, in <module>
browser = webdriver.Firefox()
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\webdriver.py", line 59, in __init__
self.binary, timeout),
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\extension_connection.py", line 47, in __init__
self.binary.launch_browser(self.profile)
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\firefox_binary.py", line 64, in launch_browser
self._wait_until_connectable()
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\firefox_binary.py", line 108, in _wait_until_connectable
self.profile.path, self._get_firefox_output()))
WebDriverException: Can't load the profile. Profile Dir: C:\Users\ADMINI~1\AppData\Local\Temp\2\tmpk2k3o7ox Firefox output: None
Posts: 7,106
Threads: 122
Joined: Sep 2016
I usually use the chrome driver.
This did work for me.
Add geckodriver.exe the 64-bit version to Windows Path or folder you run code.
In code:
caps = webdriver.DesiredCapabilities().FIREFOX
caps["marionette"] = False
browser = webdriver.Firefox(capabilities=caps)
web_url = 'http://technical.traders.com/sub/sublogin2.asp'
browser.get(web_url)
Posts: 17
Threads: 1
Joined: Aug 2017
Sep-01-2017, 06:34 PM
(This post was last modified: Sep-01-2017, 06:35 PM by thomas2004ch.)
It doesn't help. The result is the same as before. Now here the whole code:
from selenium import webdriver
from bs4 import BeautifulSoup
import time
# Activate Phantom and deactivate Chrome to not load browser
caps = webdriver.DesiredCapabilities().FIREFOX
caps["marionette"] = False
browser = webdriver.Firefox(capabilities=caps)
web_url = 'http://technical.traders.com/sub/sublogin2.asp'
browser.get(web_url)
user_name = browser.find_element_by_css_selector('#SubID > input[type="text"]')
user_name.send_keys("Foo")
password = browser.find_element_by_css_selector('#SubName > input[type="text"]')
password.send_keys("Bar")
time.sleep(2)
submit = browser.find_element_by_css_selector('#SubButton > input[type="submit"]')
submit.click()
time.sleep(2)
# Give source code to BeautifulSoup
soup = BeautifulSoup(browser.page_source, 'lxml')
log_in = soup.find('h2')
print(log_in.text)
Here the out put after the browser closes:
runfile('D:/Temp/Selenium.py', wdir='D:/Temp')
Traceback (most recent call last):
File "<ipython-input-12-7d93cfa31c34>", line 1, in <module>
runfile('D:/Temp/Selenium.py', wdir='D:/Temp')
File "C:\ProgramData\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 880, in runfile
execfile(filename, namespace)
File "C:\ProgramData\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "D:/Temp/Selenium.py", line 20, in <module>
browser = webdriver.Firefox(capabilities=caps)
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\webdriver.py", line 59, in __init__
self.binary, timeout),
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\extension_connection.py", line 47, in __init__
self.binary.launch_browser(self.profile)
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\firefox_binary.py", line 64, in launch_browser
self._wait_until_connectable()
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\firefox_binary.py", line 108, in _wait_until_connectable
self.profile.path, self._get_firefox_output()))
WebDriverException: Can't load the profile. Profile Dir: C:\Users\ADMINI~1\AppData\Local\Temp\2\tmpzuvl2fst Firefox output: None
Posts: 7,106
Threads: 122
Joined: Sep 2016
Sep-01-2017, 07:22 PM
(This post was last modified: Sep-01-2017, 07:22 PM by snippsat.)
I not sure what your error is.
I have anaconda3,so i can test there to.
Install selenuim into Anaconda3:
G:\Anaconda3\Scripts
λ pip install -U selenium
Collecting selenium
Using cached selenium-3.5.0-py2.py3-none-any.whl
Installing collected packages: selenium
Successfully installed selenium-3.5.0 Run code:
G:\Anaconda3\Scripts
λ cd ..
G:\Anaconda3
λ python log_test.py
Log In Error It start FireFox and fill in fields,so work as it should.
Try running from command line as i do here,name file something else than Selenium.py .
Run cmd as administrator.
Make sure that geckodriver.exe is in same folder as you run code.
Posts: 17
Threads: 1
Joined: Aug 2017
Sep-01-2017, 07:58 PM
(This post was last modified: Sep-01-2017, 08:18 PM by thomas2004ch.)
Hi,
I use also Anaconda3. I did download and install the Selenium module with pip as you did. It must be coorrect, otherwise I will get error. And I rename my code to test_log.py as yours. And I copy the gecodriver.exe (64-bit) to the dir where the test_log.py is. Nothing more.
Then I open the cmd as Admin and start the code as you described. But the result is the same as before. Here the message after the browser closes itself.
C:\ProgramData\Anaconda3>python log_test.py
Traceback (most recent call last):
File "log_test.py", line 20, in <module>
browser = webdriver.Firefox(capabilities=caps)
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\webdriver.py", line 59, in __init__
self.binary, timeout),
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\extension_connection.py", line 47, in __init__
self.binary.launch_browser(self.profile)
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\firefox_binary.py", line 64, in launch_browser
self._wait_until_connectable()
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\firefox_binary.py", line 108, in _wait_until_connectable
self.profile.path, self._get_firefox_output()))
selenium.common.exceptions.WebDriverException: Message: "Can't load the profile. Profile Dir: C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\2\\tmpd9p5x5gv Firefox output: None"
C:\ProgramData\Anaconda3>
Info: I use Windows Server 2016, same as Windows 10. But I think this should not the reason, right? Or maybe it is lack off something by me?
I reinstall the Selenium again as follow:
Quote:C:\ProgramData\Anaconda3\Scripts>pip install -U selenium
Collecting selenium
Downloading selenium-3.5.0-py2.py3-none-any.whl (921kB)
100% |████████████████████████████████| 921kB 936kB/s
Installing collected packages: selenium
Found existing installation: selenium 2.43.0
Uninstalling selenium-2.43.0:
Successfully uninstalled selenium-2.43.0
Successfully installed selenium-3.5.0
Then I run my code again:
Quote:C:\ProgramData\Anaconda3\Scripts>cd ..
C:\ProgramData\Anaconda3>python.exe log_test.py
The FireFox browser is open. After a while it is closed itself. I see from the CMD console following messages. This time somewhat different than before. It tells the reason could be the uncompotability of FireFox:
Quote:C:\ProgramData\Anaconda3>python.exe log_test.py
Traceback (most recent call last):
File "log_test.py", line 20, in <module>
browser = webdriver.Firefox(capabilities=caps)
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\webdriver.py", line 171, in __init__
self.binary, timeout)
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\extension_connection.py", line 52, in __init__
self.binary.launch_browser(self.profile, timeout=timeout)
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\firefox_binary.py", line 73, in launch_browser
self._wait_until_connectable(timeout=timeout)
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\firefox\firefox_binary.py", line 114, in _wait_until_connectable
% (self.profile.path))
selenium.common.exceptions.WebDriverException: Message: Can't load the profile. Possible firefox version mismatch. You must use GeckoDriver instead for Firefox 48+. Profile Dir: C:\Users\ADMINI~1\AppData\Local\Temp\tmpsquk65eh If you specified a log_file in the FirefoxBinary constructor, check it for details.
What a FireFox version do you have? I ahve the 55.0.3(64-bit)
Posts: 7,106
Threads: 122
Joined: Sep 2016
(Sep-01-2017, 07:58 PM)thomas2004ch Wrote: What a FireFox version do you have? I ahve the 55.0.3(64-bit) Try change to:
caps["marionette"] = True
This i have to do for newer version of FireFox.
|