Mar-22-2020, 06:10 AM
Hello Python Web Scrapers,
This is what I am currently up against and was hoping somebody could point me in the right direction.
Python3 + BeautifulSoup4 + lxml (HTML -> CSV):
How to loop to the next HTML URL and save as new CSV Row in the existing .csv that the current code scrapes from.
For instance: How would I make this URL do the above
Next HTML URL: https://law.justia.com/cases/federal/app...66/308423/
Python3 Code:
Best Regards,
Brandon Kastning
P.S. - Everyone be safe!
This is what I am currently up against and was hoping somebody could point me in the right direction.
Python3 + BeautifulSoup4 + lxml (HTML -> CSV):
How to loop to the next HTML URL and save as new CSV Row in the existing .csv that the current code scrapes from.
For instance: How would I make this URL do the above
Next HTML URL: https://law.justia.com/cases/federal/app...66/308423/
Python3 Code:
from urllib.request import urlopen from bs4 import BeautifulSoup html = urlopen("http://law.justia.com/cases/federal/appellate-courts/F2/999/663/308588/") bsObj = BeautifulSoup(html.read()) allOpinion = bsObj.findAll(id="opinion") import requests from bs4 import BeautifulSoup url = "http://law.justia.com/cases/federal/appellate-courts/F2/999/663/308588/" allTitle = bsObj.findAll({"title"}) allURL = url print(allOpinion) print(allTitle) print(allURL) import csv csvRow = [allOpinion,allTitle,allURL] csvfile = "current_F2_opinion_with_tags_current.csv" with open(csvfile, "a") as fp: wr = csv.writer(fp, dialect='excel') wr.writerow(csvRow) print(allOpinion[0].get_text(),url) import csv csvRow = [allOpinion[0].get_text(),allTitle[0].get_text(),allURL] csvfile = "current_F2_opinion_without_tags_current.csv" with open(csvfile, "a") as fp: wr = csv.writer(fp, dialect='excel') wr.writerow(csvRow)Thank you!
Best Regards,
Brandon Kastning
P.S. - Everyone be safe!
“And one of the elders saith unto me, Weep not: behold, the Lion of the tribe of Juda, the Root of David, hath prevailed to open the book,...” - Revelation 5:5 (KJV)
“And oppress not the widow, nor the fatherless, the stranger, nor the poor; and ...” - Zechariah 7:10 (KJV)
#LetHISPeopleGo
“And oppress not the widow, nor the fatherless, the stranger, nor the poor; and ...” - Zechariah 7:10 (KJV)
#LetHISPeopleGo