Apr-01-2023, 07:08 PM
Hey everyone,
I was wondering if I could get you guys to help me out a little. I'm attempting to make a web scraper to scrape a site for some strings of numbers. I'm first trying to scrape a list of links and then join those URLs and scrape the strings of numbers that I'm looking for. At the end of that I'm just trying to save the scraped strings of numbers to a .txt file for later use.
I'm surprisingly not getting any errors, but my .txt file is not being populated with the scraped data. I'm thinking that maybe I'm not directing it correctly through the html via html tags/classes?
here is my code:
I was wondering if I could get you guys to help me out a little. I'm attempting to make a web scraper to scrape a site for some strings of numbers. I'm first trying to scrape a list of links and then join those URLs and scrape the strings of numbers that I'm looking for. At the end of that I'm just trying to save the scraped strings of numbers to a .txt file for later use.
I'm surprisingly not getting any errors, but my .txt file is not being populated with the scraped data. I'm thinking that maybe I'm not directing it correctly through the html via html tags/classes?
here is my code:
import requests from bs4 import BeautifulSoup from urllib.parse import urljoin # Scrape the links to each monthly results page url = "http://www.calotteryx.com/Fantasy-5/drawing-results-calendar.htm" response = requests.get(url) content = response.content soup = BeautifulSoup(content, 'html.parser') links = [] for link in soup.find_all('a', class_='noline'): anchor = link.find('a') if anchor and (href := anchor.get('href')): links.append(urljoin(url, href)) # Scrape the winning numbers for each monthly results page winning_numbers = [] for link in links: response = requests.get(link) content = response.content soup = BeautifulSoup(content, 'html.parser') for tag in soup.find_all('div', class_='ball blue5 fcblack1'): numbers = tag.text.strip().split() winning_numbers.append(numbers) # Write the winning numbers to a file with open('winning_numbers.txt', 'w') as f: for numbers in winning_numbers: f.write(''.join(numbers) + '\n')I joined a while ago, but I'm still not good at coding lol, so please forgive my ignorance, or my bad looking code. Any help in pointing out my mistakes and how to fix them would be greatly appreciated please.