Help with python3 (BeautifulSoup) - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html) +--- Thread: Help with python3 (BeautifulSoup) (/thread-12826.html) |
Help with python3 (BeautifulSoup) - freaknez - Sep-14-2018 import requests from termcolor import * import colorama from bs4 import BeautifulSoup colorama.init() headers = { "Cookie": "PHPSESSID=Testing", "User-Agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.81 Safari/537.36" } url = "http://naruto.lt/kovos.php?kur=mob&kas=kzm_Futa%20Kagetsu" r = requests.get(url, headers=headers) soup = BeautifulSoup(r.content, 'html.parser')r.content shows link (http://naruto.lt/kovos.php?kur=begin&ref=[GENERATING RANDOM NUMBERS]&kas=kzm_Futa%20Kagetsu) how i can get $_GET['ref'] ? i tried to do in many ways but nothing worked sorry for bad english RE: Help with python3 (BeautifulSoup) - Larz60+ - Sep-14-2018 1st you should modify: r = requests.get(url, headers=headers) soup = BeautifulSoup(r.content, 'html.parser')to response = requests.get(url, headers=headers) if response.status_code == 200: soup = BeautifulSoup(response.content, 'html.parser') links = soup.find_all('a') for link in links: print(link.get('href')) print(link.text) else: print('Unable to open URL: {}'.format(url)) |