Python Forum
how to add a login to a bs4 parser-script
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
how to add a login to a bs4 parser-script
#9
You can not create a new session with Requests,most work with the session that Selenium create.
Delete all code from line 35 to end,do simpler test first to make sure login work.
Find content that only visible when logged in,and try to parse it.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup
import time

#--| Setup
options = Options()
#options.add_argument("--headless")
#options.add_argument("--window-size=1980,1020")
browser = webdriver.Chrome(executable_path=r'C:\cmder\bin\chromedriver.exe', options=options)
#--| Parse or automation
browser.get("https://login.wordpress.org/?locale=en_US")
time.sleep(2)
user_name = browser.find_element_by_css_selector('#user_login')
user_name.send_keys("Test_user")
password = browser.find_element_by_css_selector('#user_pass')
password.send_keys("123456")
time.sleep(3)
submit = browser.find_elements_by_css_selector('#wp-submit')[0]
submit.click()

# Example send page source to BeautifulSoup or selenium for parse
soup = BeautifulSoup(browser.page_source, 'lxml')
use_bs4 = soup.find('title')
print(use_bs4.text)

# Example if i use this forum as example
post_thread = browser.find_elements_by_xpath('//*[@id="panel"]/div[2]/div/ul[1]/li[1]/a')
print(post_thread[0].text)
Output:
Welcome back, snippsat. You last visited: Today, 07:50 PM Log Out User CP
The output show here could i only get if login works.
You most find own content as shown with eg Xpath,CSS selector,find...ect
Can not use what i have last in code as a example,
and i can of course not look at contend after logged in as i am not member at Wordpress forum.
Reply


Messages In This Thread
RE: how to add a login to a bs4 parser-script - by snippsat - Jun-24-2020, 07:38 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  little parser-script crashes after doing good work for some time apollo 0 1,660 Feb-03-2021, 10:48 AM
Last Post: apollo
  Python-selenium script for automated web-login does not work hectorKJ 2 4,094 Sep-10-2019, 01:29 PM
Last Post: buran

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020