Python Forum
Extract data from sports betting sites
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Extract data from sports betting sites
#1
Hi, as the title suggests, I would like to extract data from sports betting sites, with this code I download the html of the site
from bs4 import BeautifulSoup
import urllib.request

url = "https://sports.bwin.com/"
try:
    page = urllib.request.urlopen(url)
except:
    print("An error occured.")

soup = BeautifulSoup(page, 'html.parser')
print(soup)
then I can't go on, if I want to extract the participating teams I tried with the re module and this code but it doesn't work
from bs4 import BeautifulSoup
import urllib.request
import re
url = "https://sports.bwin.com/"
try:
    page = urllib.request.urlopen(url)
except:
    print("An error occured.")

soup = BeautifulSoup(page, 'html.parser')
#print(soup)

regex = re.compile("participant")
content_lis = soup.find_all('div', attrs={'class': regex})
print(content_lis)
thank you who will help me
Reply
#2
That site uses a lot of javascript.
You will need to use selenium to expose the javascript, after you do this, you can finish with Beautiful Soup
Or do it all in selenium
there are good selenium tutorial within the web scraping tutorial on this forum
see:
web scraping part 1
web scraping part 2
Reply
#3
I'm working on a similar Project, Selenium will easily do that work for you especially if you encounter a site using Ajax
Reply
#4
The tutorial links in post #2 are still quick and valid.
They apply to all types of sites, including sports sites.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Extract data from a table Bob_M 3 1,121 Aug-14-2020, 03:36 PM
Last Post: Bob_M
  Extract data with Selenium and BeautifulSoup nestor 3 1,736 Jun-06-2020, 01:34 AM
Last Post: Larz60+
  Extract json-ld schema markup data and store in MongoDB Nuwan16 0 1,293 Apr-05-2020, 04:06 PM
Last Post: Nuwan16
  Extract data from a webpage cycloneseb 5 1,459 Apr-04-2020, 10:17 AM
Last Post: alekson
  Cannot Extract data through charts online AgileAVS 0 840 Feb-01-2020, 01:47 PM
Last Post: AgileAVS
  Cannot extract data from the next pages nazmulfinance 4 1,330 Nov-11-2019, 08:15 PM
Last Post: nazmulfinance
  How to use Python to extract data from Zoho Creator software on the web dan7055 2 2,346 Jul-05-2019, 05:11 PM
Last Post: DeaD_EyE
  Python/BeautiifulSoup. list of urls ->parse->extract data to csv. getting ERROR IanTheLMT 2 2,241 Jul-04-2019, 02:31 AM
Last Post: IanTheLMT
  Help to extract data from web prasadmathe 4 1,718 May-20-2019, 10:59 PM
Last Post: michalmonday
  [Python 3] - Extract specific data from a web page using lxml module Takeshio 9 4,508 Aug-25-2018, 08:46 AM
Last Post: leotrubach

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020