Python Forum

I've tried this in the past using requests and or selenium, as I recall requests was good a filling in the forms and selenium was okay and mimicking a browser but I haven't found a good tool to do both and I don't believe I can make them share a connection variable/object.

This is the rundown of what I need:

connect to URL

ask user to enter username and store in username-variable
ask user to enter password and store in password-variable

input data from username-variable into username field:

<input id="inptUserId" maxlength="8"
name="txt_inptUserId" onkeyup="autoSkipNN(event.which,this);" onfocus="saveFocusedControl(&quot;inptUserId&quot;);" style="background-color:Window;font-size:11px;height:19px;width:67px;line-height:11px;left:275px;padding-bottom:0;font-family:Arial;border-width:2px;position:absolute;border-style:inset;top:101px;color:000000;padding-top:0;" tabindex="26"
title="" type="text" value="">

input data from password-variable into password field:

<input id="password" maxlength="8"
name="txt_password" onkeyup="autoSkipNN(event.which,this);" onfocus="saveFocusedControl(&quot;password&quot;);" style="background-color:Window;font-size:11px;height:19px;width:67px;line-height:11px;left:445px;padding-bottom:0;font-family:Arial;border-width:2px;position:absolute;border-style:inset;top:101px;color:000000;padding-top:0;" tabindex="28"
title="" type="password" value="">

click button:

<input alt="Enter" id="btnOK"
name="btn_btnOK" onfocus="saveFocusedControl(&quot;btnOK&quot;);" style="font-family: Arial; font-size: 12px; font-weight: Bold; position: absolute; left: 19px; top: 292px; width: 49px; height: 23px; color: #ffffcc; background-color: transparent; border-width: 0px; text-align:left;padding-left:0px;padding-right: -5px;cursor: pointer; padding-left:0px;padding-right:0px;" tabindex="-1"
title="" type="submit" value="Enter">

Any suggestions on the best way to accomplish this? Thanks.

Quote:I've tried this in the past using requests and or selenium, as I recall requests was good a filling in the forms and selenium was okay and mimicking a browser but I haven't found a good tool to do both and I don't believe I can make them share a connection variable/object.

I use a combination all the time. Just posted a simple example that uses Selenium to expand javascript, then beautifulsoup to parse. Although I don't do it in this simple script, there's no reason that you can't use selenium to click , fill in search boxes (and/or password info), and on target page extract links and get those pages using beautifulsoup and requests: https://python-forum.io/Thread-get-hotel...7#pid78167

[quote='Larz60+' pid='78458' dateline='1556132810']

Quote:I've tried this in the past using requests and or selenium, as I recall requests was good a filling in the forms and selenium was okay and mimicking a browser but I haven't found a good tool to do both and I don't believe I can make them share a connection variable/object.

I finally got it running headless!

I'm new to Beautiful Soup, when preforming a search for buttons/text fields should I be searching via html 'id' or 'name'? Thanks.

for parser, i like to use lxml
soup = BeautifulSoup(page, 'lxml')