Python Forum

Full Version: Scrape multiple urls LXML
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hi i want scrape multiple urls using LXML. The code im using allow me scrape one url a a time, but i want to do it in bulk. I tried to doit with a for loop, but i couldnt. The single url code is:

from lxml import html
import requests
page = requests.get('https://miwebsite.com')
tree = html.fromstring(page.content)
Titulo = tree.xpath('//div[@class="tituladorVIVO"]/text()')
Fuentes = tree.xpath('//audio/@src')
print ('Fuentes: ', Fuentes)
print ('Titulo', Titulo)
How can i do it with a list of urls. Regards
Can show a example of one way to do it.
from lxml import html
import requests

url_lst = ['https://www.cnn.com', 'https://python-forum.io']
for page in url_lst:
    page = requests.get(page)
    tree = html.fromstring(page.content)
    title = tree.xpath('//title')
    print('-' * 20)
    print(title[0].text)
Output:
-------------------- CNN International - Breaking News, US News, World News and Video -------------------- Python Forum