Scrape multiple urls LXML - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html) +--- Thread: Scrape multiple urls LXML (/thread-22042.html) |
Scrape multiple urls LXML - santdoyle - Oct-26-2019 Hi i want scrape multiple urls using LXML. The code im using allow me scrape one url a a time, but i want to do it in bulk. I tried to doit with a for loop, but i couldnt. The single url code is: from lxml import html import requests page = requests.get('https://miwebsite.com') tree = html.fromstring(page.content) Titulo = tree.xpath('//div[@class="tituladorVIVO"]/text()') Fuentes = tree.xpath('//audio/@src') print ('Fuentes: ', Fuentes) print ('Titulo', Titulo)How can i do it with a list of urls. Regards RE: Scrape multiple urls LXML - snippsat - Oct-26-2019 Can show a example of one way to do it. from lxml import html import requests url_lst = ['https://www.cnn.com', 'https://python-forum.io'] for page in url_lst: page = requests.get(page) tree = html.fromstring(page.content) title = tree.xpath('//title') print('-' * 20) print(title[0].text)
|