Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Scraping aspx
#1
Hello,

I'm trying to scrap an aspx page and not sure how to get started with it. Specifically this site: http://www.quikstatsiowa.com/Default.aspx
It looks like the site is also using jquery

I'm trying to grab data from each of the sports for a group of schools to be able to easily compare data in our conference and then post it to a word press page. I've tried a WordPress scraping plugin but it doesn't pull the data very nicely so my next move is python.

This is where I've started but might be going in the wrong direction.

from bs4 import BeautifulSoup

import requests

search = input("Enter search term: ")
params = {"q": search}

r = requests.get("hhttp://www.quikstatsiowa.com/Default.aspx", params=params)

soup = BeautifulSoup(r.text, "html.parser")
print(soup.prettify())
results = soup.find("ol", {"id": "b_results"})
links = results.findall("li", {"class": "b_algo"})

for item in links:
    item_text = item.find("a").text
    item_href = item.find("a").attrs("href")
    print(item_text)
    print(item_href)
Reply
#2
You may try Selenium. Requests module can't handle JavaScript.
Here you can see how it works: https://python-forum.io/Thread-Web-scraping-part-2
"As they say in Mexico 'dosvidaniya'. That makes two vidaniyas."
https://freedns.afraid.org
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Scraping .aspx page Larz60+ 21 50,850 Mar-18-2021, 10:16 AM
Last Post: Larz60+
  How to get registeration data from a website that uses .aspx? Help me brothers. humble_coder 1 2,424 Feb-18-2021, 06:03 PM
Last Post: Larz60+
  Scrape ASPX data with python... hoff1022 0 4,500 Feb-26-2019, 06:16 PM
Last Post: hoff1022
  Scrapping .aspx websites boxingowl88 3 8,142 Oct-10-2018, 05:35 PM
Last Post: stranac
  How do i loop through list of data from CSV file and post requests in aspx dynamics w Prince_Bhatia 1 6,041 Nov-09-2017, 02:53 PM
Last Post: heiner55

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020