Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Parse a URL list stored in a CSV
#1
"""
hello any help greatly appreciated
urls stored in the CSV Coloum A lines 1-100, i would like to parse the links in each url to output to another CSV, so i can parse these links for certain data shown.

#working code below for one url
link_set = set()
for link in soup.find_all('a',{'class' : 'horse'}):
web_links = link.get("href")
print(web_links)
link_set.add(web_links)
"""

[python]
import requests
from bs4 import BeautifulSoup
import csv
import requests
import time

#the csv file lines example
#https://gg.co.uk/racing/07-feb-2020/dundalk-1700
#https://gg.co.uk/racing/07-feb-2020/chelmsford-city-1745
#ect, ect , ect

#prints screen urls in csv file
with open("course_links.csv") as infile:
reader = csv.DictReader(infile)
for link in reader:
res = requests.get(link['course_links'])
#time.sleep(5)
print(res.url)

#prints screen urls in csv file
with open("course_links.csv", 'r') as file:
csv_file = csv.DictReader(file)
for row in csv_file:
print(dict(row))
[\python]
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Python/BeautiifulSoup. list of urls ->parse->extract data to csv. getting ERROR IanTheLMT 2 2,219 Jul-04-2019, 02:31 AM
Last Post: IanTheLMT

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020