Python Forum
HOWTO? Login DSL Modem with Python Requests: need Click "Apply" Button
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
HOWTO? Login DSL Modem with Python Requests: need Click "Apply" Button
#1
Esteemed Forum Participants & Lurkers:
======================================
NOTE: Inexperienced with Web Development ...

I am trying to make a simple tool to track my Inet usage so that I don't hit my ISP usage cap. My goal is a web scraper to fetch the "Total Usage" value out of the Browser-based control panel for my DSL modem/router (ZyXEL PK5001Z).

I found "Python Requests" and it seemed to be a reasonable tool for my task, and I got started with it. The big problem is that the modem login can NOT be completed with a <Return> character ... there is an "Apply" button that must be "clicked" in order to complete the login. I don't know if the login page even accepts my request into the appropriate text entry boxes, or how to activate the "Apply" button function.
>>> import requests
>>> r = requests.get('<http modem URL>', auth=('<webtest>', '<modem-pswd>'))
>>> r.status_code
200
>>> r.headers['content-type']
'text/html; charset=iso-8859-1'
>>> r.encoding
'iso-8859-1'
>>> r.text
<Entire page code of>: <title>Century Link  Modem Configurator</title>
I actually have already figured out how to fetch the parameter I need ... I was playing around with the Developer pane in Firefox and found that AFTER I get logged in, I only need to access a specific page URL that returns the data that loads the DSL Status page! I can parse that easily. Here is the beginning of the modem login page AFTER I issued the above "request.get":
LOGIN PAGE UPLOADED:
 . . . . .
<title>Century Link  Modem Configurator</title>
<link href="../_css/basic.css" rel="stylesheet" type="text/css" />\r\n<script language="Javascript"
 . . . . .
? SHOULD BE WELCOME PAGE ?:
 . . . . .
<title>Century Link Modem Configurator</title>
<link href="../_css/index.css" . . . 
For the next step, the modem does NOT accept the password with a <Return> keystroke ... you MUST click the "Apply" button in order to actually log in. I think this is the relevant code on the login page:
<div class="stepWrapper">
  <p><strong> 2. Click &quot;Apply&quot; to log in.</strong></p>
  <a href="#" onclick="ButtonClick(1)" class="btn apply_btn">Apply</a>
</div>
Roger Shruber over on the #python chat tried to help and suggested that I might try Selenium, but I couldn't figure out how to get it installed into Linux Mint 17.3.

Thank you for any and all comments, suggestions, and gracious assistance.
Blessings in abundance, all the best, & ENJOY!
Art in Carlisle, PA USA
Reply


Messages In This Thread
HOWTO? Login DSL Modem with Python Requests: need Click "Apply" Button - by Webtest - Jul-29-2017, 10:29 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  cant click button by script at page michael1834 1 993 Dec-08-2023, 04:44 PM
Last Post: SpongeB0B
  Click on a button on web page using Selenium Pavel_47 7 4,561 Jan-05-2023, 04:20 AM
Last Post: ellapurnellrt
  POST requests - different requests return the same response Default_001 3 1,899 Mar-10-2022, 11:26 PM
Last Post: Default_001
  Show HTML in Python application and handle click SamHobbs 2 2,660 Sep-28-2021, 06:27 PM
Last Post: SamHobbs
  Login and download an exported csv file within a ribbon/button in a website Alekhya 0 2,616 Feb-26-2021, 04:15 PM
Last Post: Alekhya
  button click error rafarangel 2 3,081 Feb-11-2021, 08:19 PM
Last Post: buran
  Problem with logging in on website - python w/ requests GoldeNx 6 5,208 Sep-25-2020, 10:52 AM
Last Post: snippsat
  Log In Button Won't Click - Python Selenium Webdriver samlee916 2 3,769 Jun-07-2020, 04:42 PM
Last Post: samlee916
  Python Webscraping with a Login Website warriordazza 0 2,571 Jun-07-2020, 07:04 AM
Last Post: warriordazza
  How to click facebook message button JanelleGuthrie 2 2,368 May-14-2020, 06:02 PM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020