Python Forum
how to add a login to a bs4 parser-script
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
how to add a login to a bs4 parser-script
#11
(Jul-02-2020, 09:52 PM)apollo Wrote: i have created a test-account for the demo-testing of this:
login: pluginfan
pass: testpasswd123
The login work with my code,can parse with both with BS and Selenium.
The Session is open as long as not close Selenium browser.close().
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup
import time

#--| Setup
options = Options()
#options.add_argument("--headless")
#options.add_argument("--window-size=1980,1020")
browser = webdriver.Chrome(executable_path=r'C:\cmder\bin\chromedriver.exe', options=options)
#--| Parse or automation
browser.get("https://login.wordpress.org/?locale=en_US")
time.sleep(2)
user_name = browser.find_element_by_css_selector('#user_login')
user_name.send_keys("pluginfan")
password = browser.find_element_by_css_selector('#user_pass')
password.send_keys("testpasswd123")
time.sleep(3)
submit = browser.find_elements_by_css_selector('#wp-submit')[0]
submit.click()

time.sleep(3)
# Example using selenium for parse
title = browser.find_elements_by_xpath('//*[@id="home-welcome"]/header/h1')
print(title[0].text)

print('-' * 50)
# Example using BeautifulSoup for parse
soup = BeautifulSoup(browser.page_source, 'lxml')
use_bs4 = soup.select_one('#home-welcome > section.showcase > p.subheading')
print(use_bs4.text)
Output:
Meet WordPress -------------------------------------------------- 35% of the web uses WordPress, from hobby blogs to the biggest news sites online.
apollo Wrote:answer 3
Try this instead of urllib2
No,you should not at all use urllib2 Hand
snippsat Wrote:urllib,mechanize,cookiejar is older stuff that i don't use anymore.
Requests has taken over there task in better way.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  little parser-script crashes after doing good work for some time apollo 0 1,627 Feb-03-2021, 10:48 AM
Last Post: apollo
  Python-selenium script for automated web-login does not work hectorKJ 2 3,982 Sep-10-2019, 01:29 PM
Last Post: buran
  html parser tjnichols 9 30,849 Mar-17-2018, 11:00 PM
Last Post: tjnichols

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020