+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html)
+--- Thread: Scraping aspx (/thread-7586.html)
Scraping aspx - asnyder - Jan-16-2018
Hello,
I'm trying to scrap an aspx page and not sure how to get started with it. Specifically this site: http://www.quikstatsiowa.com/Default.aspx
It looks like the site is also using jquery
I'm trying to grab data from each of the sports for a group of schools to be able to easily compare data in our conference and then post it to a word press page. I've tried a WordPress scraping plugin but it doesn't pull the data very nicely so my next move is python.
This is where I've started but might be going in the wrong direction.
from bs4 import BeautifulSoup
import requests
search = input("Enter search term: ")
params = {"q": search}
r = requests.get("hhttp://www.quikstatsiowa.com/Default.aspx", params=params)
soup = BeautifulSoup(r.text, "html.parser")
print(soup.prettify())
results = soup.find("ol", {"id": "b_results"})
links = results.findall("li", {"class": "b_algo"})
for item in links:
item_text = item.find("a").text
item_href = item.find("a").attrs("href")
print(item_text)
print(item_href)
RE: Scraping aspx - wavic - Jan-16-2018
You may try Selenium. Requests module can't handle JavaScript.
Here you can see how it works: https://python-forum.io/Thread-Web-scraping-part-2