working with lxml and requests

[gentoobob]

So after looking over your links here is what I have now...

import requests
from bs4 import BeautifulSoup

response = requests.get('https://10.10.10.1/vmrest/users?rowsPerPage=2000\&pageNumber=1', verify=False, auth=('user', 'pass'))

soup = BeautifulSoup(response.content, 'html.parser')
print(soup.find('title').text)

and I get the following output...

Error:
Warning (from warnings module):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 845
    InsecureRequestWarning)
InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
            Cisco System - Error report   

It does support GET. There is an old perl script that uses GET to do the same thing, however, the old script doesnt work on a new Debian server and none of us know Perl, just basic Python for scripting access to routers and switches. The perl script uses the following modules...
LWP::UserAgent
XML::Simple

The code in perl looks like this...

use LWP::Simple;
use XML::Simple;

my $xml = new XML::Simple;
my @userdata;

$page = 1;

while(1)
{ 
  my $url = "https://USER:pass\@10.10.10.1/vmrest/users?rowsPerPage=2000\&pageNumber=$page";
  my $content = get($url);
  die "error getting $url" unless defined $content;

  my $data = $xml->XMLin($content);

# if we dont get at least one user end loop

  if(@{$data->{User}} < 1)
{
  last;
}
# build the userdata array, each entry contains "username,extension"

$start =(($page-1) * 2000);
for ($i=$start;$i<=$start + @{$data->{User}} - 1;$i++)
{
  push(@userdata,"$data->{User}->[$i-$start]->{Alias},$data->{User}->[$i-$start]->{DtmfAccessId}");
 }
$page++

# Dump the results to a file

open(UNITY,"/usr/scripts/unityLDAP/$timestamp-unity.csv");
for(@userdata)
 {
 print UNITY "$_\n";
}
close(UNITY);