Posts: 20
Threads: 3
Joined: Apr 2019
Would I get the cookies from the page before clicking the submit button (step 3) or when I launch Chrome (step 2)?
here are the steps I take
1) Selenium Launches Chrome -> 2) Navigate to Login Screen -> 3) Navigate to Page with Submit Button (html below)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
<div class="mc2" id="user_input">
<form id="request" name="request" onsubmit="return checkAndSubmit(this.tNumber.value)" action="/ebilling/request.action" method="post">
<input type="hidden" name="CSRFToken" value="8be3e045-99b4-46f6-a6d9-f34e70017b96">
<table class="modulebackgroundcolor" width="100%" cellspacing="0" cellpadding="0" border="0">
<tbody><tr>
</tr>
<tr>
<td>Please enter the TNumber (XXXXXXXXXXXXXXX or 99999999999) below to submit your request.</td>
</tr>
<tr>
<td id="errorMessage" style="color: red;">
</td>
</tr>
<tr>
<td>
<input type="text" name="tNumber" size="30" maxlength="30" value="" id="request_tNumber"></td>
</tr>
<tr>
<td>Note: XYZ.</td>
</tr>
<tr>
<td>*XXX</td>
</tr>
<tr>
<td> </td>
</tr>
</tbody></table>
<div class="mb" align="right">
<h4>
<span><input class="bSwan" type="button" value="Cancel" onclick="location.href='/ebilling/dashBoard.action?reportId=dashboardIncentiveRpt'"></span>
<span><input type="submit" value="Submit" id="request_0" class="bSwan">
</span>
</h4>
</div>
<br>
<input type="hidden" name="_csrf" value="e7634769-8942-415f-b939-39e748359b81"></form>
</div>
|
Posts: 5,151
Threads: 396
Joined: Sep 2016
I would do it right after logging in
Recommended Tutorials:
Posts: 20
Threads: 3
Joined: Apr 2019
Aug-02-2019, 01:34 PM
(This post was last modified: Aug-02-2019, 01:34 PM by NSearch.)
Hi metulburr,
I am just now getting back to this project and I attempted the code you posted, but I get the following error:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
Traceback (most recent call last):
File "C:/Users/SA/PycharmProjects/xyz.py", line 96, in <module>
c = [driver.add_cookie(c) for c in response_cookies_browser]
File "C:/Users/SA/PycharmProjects/xyz.py", line 96, in <listcomp>
c = [driver.add_cookie(c) for c in response_cookies_browser]
File "C:\Users\SA\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 894, in add_cookie
self.execute(Command.ADD_COOKIE, {'cookie': cookie_dict})
File "C:\Users\SA\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\SA\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: missing 'value'
(Session info: chrome=75.0.3770.142)
(Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729@{
|
Would you please take a look and advise what I might be doing wrong.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
driver.get(url)
driver.maximize_window()
driver.find_element_by_id("email").send_keys(cca_username)
driver.find_element_by_id("pwd").send_keys(cca_password)
driver.find_element_by_id("submitBtn").click()
request_cookies_browser = driver.get_cookies()
params = {'os_username': 'username', 'os_password': 'password'}
s = requests.Session()
c = [s.cookies.set(c['name'], c['value']) for c in request_cookies_browser]
resp = s.post(url, params)
dict_resp_cookies = resp.cookies.get_dict()
response_cookies_browser = [{'name': name, 'value': value} for name, value in dict_resp_cookies.items()]
c = [driver.add_cookie(c) for c in response_cookies_browser]
|
Thank you
Posts: 5,151
Threads: 396
Joined: Sep 2016
does this return anything?
1 |
dict_resp_cookies.items()
|
Recommended Tutorials:
Posts: 20
Threads: 3
Joined: Apr 2019
1 |
dict_items([('Secure', None)])
|
Posts: 20
Threads: 3
Joined: Apr 2019
metulburr,
Just want to check in and see if you had anymore advice.
Thanks
Posts: 5,151
Threads: 396
Joined: Sep 2016
Aug-06-2019, 10:02 PM
(This post was last modified: Aug-06-2019, 10:03 PM by metulburr.)
At this point my guess is just as good as yours. Its hard to identify a problem with selenium unless i can actually run the full code, and tinker with it on the fly. And even then i can still be just as stumped as you are. Ive never came across your issue before when using selenium. Is this a site you are able to make a dummy account, enough to mimic the problem, and pass that login info here?
Recommended Tutorials:
Posts: 7,319
Threads: 123
Joined: Sep 2016
Hi i have seen your PM,i do not do help with TeamViewer.
As mention bye @ metulburr it's hard to help when not can test this live
In theory should CSRF token be send automatically as Selenium WebDriver just automated a user using the browser.
Just to show a basic log in example with Selenium using this site.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup
import time
options = Options()
browser = webdriver.Chrome(executable_path=r'C:\cmder\bin\chromedriver.exe', options=options)
user_name = browser.find_elements_by_xpath("//input[@name='username']")[0]
user_name.send_keys("snippsat")
password = browser.find_elements_by_xpath("//input[@name='password']")[0]
password.send_keys("xxxxxxxx")
browser.implicitly_wait(2)
browser.find_elements_by_xpath('//*[@id="content"]/div/form/div/input')[0].click()
browser.implicitly_wait(2)
soup = BeautifulSoup(browser.page_source, 'lxml')
welcome = soup.find('span', class_="welcome").text
print(welcome)
|
Output: Welcome back, snippsat. You last visited: Today, 02:46 PM Log Out
Same with Request in this post.
Posts: 5,151
Threads: 396
Joined: Sep 2016
Aug-09-2019, 03:55 PM
(This post was last modified: Aug-09-2019, 03:55 PM by metulburr.)
I would like to add that this is not a login issue. This is afterwords; putting in a number to search while being logged in. Login works successfully. I can verify that manually clicking the search button works, while when doing it selenium creates an error on the website (if you go back a page after the error you can click it manually and works). The website somehow detects when selenium clicks it? The button Id is correct, it correctly clicks it, there is no iframe, etc. It does feel like there is some sort of javascript detecting that the browser is automated (not sure if that is possible or not) and blocking it because of that.
The confusion lies as to if selenium was being blocked...wouldnt they block you at login instead? Why do it there when they could of blocked you from logging in completely. However there is a method to check if that is the case.
@OP
1) Do you need to be logged in to put in that number and search?
2) Can you create a dummy account there to hand it over publicly here?
3) Teamviewer is kind of laggy. And its quite uncomfortable doing it there when if i had code i could do it leisurely on my own time, even sporadically.
Recommended Tutorials:
Posts: 20
Threads: 3
Joined: Apr 2019
Thank you snippsat for the reply! and again, metlburr, you're awesome -- thank you very much for your attempts so far.
1) Do you need to be logged in to put in that number and search?
Yes, you need to be logged in.
2) Can you create a dummy account there to hand it over publicly here?
I am going to attempt to do this. Right now I am working with a client's login information, which is why I have not shared it.
3) Teamviewer is kind of laggy. And its quite uncomfortable doing it there when if i had code i could do it leisurely on my own time, even sporadically.
I completely understand, and normally wouldn't have asked for assistance like this. It's just the client information and I didn't feel comfortable sharing it. I actually tried to create a dummy account before asking for assistance using TeamViewer, but I couldn't create one (I will try again).
|
