(Nov-14-2016, 11:06 AM)buran Wrote: Please, note that there are 2 more hidden fields that you must supply as parameters to the POST request - form_id and form_build_id
Also username field is actually 'email', not 'username'
import requests from bs4 import BeautifulSoup USERNAME = '[email protected]' PASSWORD = 'mypassword' FORM_BUILD_ID='form-c4aeea083a82fdae7d43562ee8cafeb7' FORM_ID = 'packt_user_login_form' BASE_URL = 'https://www.packtpub.com' PROMO_URL = 'https://www.packtpub.com/packt/offers/free-learning' session = requests.session() headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2810.1 Safari/537.36'} session.post(BASE_URL, {"email": USERNAME, "password": PASSWORD, 'form_build_id':FORM_BUILD_ID, 'form_id':FORM_ID}, headers=headers) #' response = session.get(PROMO_URL, headers=headers) soup = BeautifulSoup(response.text, 'html.parser') current_offer_href = BASE_URL + soup.find("div", {"class": "free-ebook"}).a['href'] print(current_offer_href) print(session.get(current_offer_href, headers=headers)) my_account_url = BASE_URL+ '/account' #https://www.packtpub.com/account response = session.get(my_account_url, headers=headers) soup = BeautifulSoup(response.text, 'html.parser') print soup.find('div', class_='menu-account').find('h1').textI'm not sure if form_build_id change over time
Output:https://www.packtpub.com/freelearning-claim/8294/21478 <Response [200]> Your Name
Thank you. I really didn't spot the form_build_id and form_id fields. It's now fully working. Just like you, I also don't know if those fields change over time, but it appears they do, because mine is different than yours. I'm getting them on each call, so it doesn't really matter. My code:
#!/usr/bin/env python # -*- coding: utf-8 -*- import requests from bs4 import BeautifulSoup USERNAME = '[email protected]' PASSWORD = 'secret' BASE_URL = 'https://www.packtpub.com' PROMO_URL = 'https://www.packtpub.com/packt/offers/free-learning' headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2810.1 Safari/537.36'} session = requests.session() root_page = session.get(BASE_URL, headers=headers) soup = BeautifulSoup(root_page.text, 'html.parser') FORM_BUILD_ID = soup.find("input", {"name": "form_build_id"})['value'] FORM_ID = soup.find("input", {"id": "edit-packt-user-login-form"})['value'] session.post(BASE_URL, {"email": USERNAME, "password": PASSWORD, 'form_build_id': FORM_BUILD_ID, 'form_id': FORM_ID}, headers=headers) promo_page = session.get(PROMO_URL, headers=headers) soup = BeautifulSoup(promo_page.text, 'html.parser') current_offer_href = BASE_URL + soup.find("div", {"class": "free-ebook"}).a['href'] print(session.get(current_offer_href, headers=headers))Output:
<Response [200]> Process finished with exit code 0