+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html)
+--- Thread: Extract data from sports betting sites (/thread-25949.html)
Extract data from sports betting sites - nestor - Apr-16-2020
Hi, as the title suggests, I would like to extract data from sports betting sites, with this code I download the html of the site
from bs4 import BeautifulSoup
import urllib.request
url = "https://sports.bwin.com/"
try:
page = urllib.request.urlopen(url)
except:
print("An error occured.")
soup = BeautifulSoup(page, 'html.parser')
print(soup)then I can't go on, if I want to extract the participating teams I tried with the re module and this code but it doesn't workfrom bs4 import BeautifulSoup
import urllib.request
import re
url = "https://sports.bwin.com/"
try:
page = urllib.request.urlopen(url)
except:
print("An error occured.")
soup = BeautifulSoup(page, 'html.parser')
#print(soup)
regex = re.compile("participant")
content_lis = soup.find_all('div', attrs={'class': regex})
print(content_lis)thank you who will help me
RE: Extract data from sports betting sites - Larz60+ - Apr-16-2020
That site uses a lot of javascript.
You will need to use selenium to expose the javascript, after you do this, you can finish with Beautiful Soup
Or do it all in selenium
there are good selenium tutorial within the web scraping tutorial on this forum
see:
web scraping part 1
web scraping part 2
RE: Extract data from sports betting sites - law - Apr-18-2020
I'm working on a similar Project, Selenium will easily do that work for you especially if you encounter a site using Ajax
RE: Extract data from sports betting sites - Larz60+ - Mar-30-2021
The tutorial links in post #2 are still quick and valid.
They apply to all types of sites, including sports sites.