Extract data from sports betting sites

[nestor]

Hi, as the title suggests, I would like to extract data from sports betting sites, with this code I download the html of the site

1 2 3 4 5 6 7 8 9 10 11

from bs4 import BeautifulSoup import urllib.request   url = "https://sports.bwin.com/" try:     page = urllib.request.urlopen(url) except:     print("An error occured.")   soup = BeautifulSoup(page, 'html.parser') print(soup)

then I can't go on, if I want to extract the participating teams I tried with the re module and this code but it doesn't work

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

from bs4 import BeautifulSoup import urllib.request import re url = "https://sports.bwin.com/" try:     page = urllib.request.urlopen(url) except:     print("An error occured.")   soup = BeautifulSoup(page, 'html.parser') #print(soup)   regex = re.compile("participant") content_lis = soup.find_all('div', attrs={'class': regex}) print(content_lis)

thank you who will help me

[Larz60+]

That site uses a lot of javascript.
You will need to use selenium to expose the javascript, after you do this, you can finish with Beautiful Soup
Or do it all in selenium
there are good selenium tutorial within the web scraping tutorial on this forum
see:
web scraping part 1
web scraping part 2

[law]

I'm working on a similar Project, Selenium will easily do that work for you especially if you encounter a site using Ajax

[Larz60+]

The tutorial links in post #2 are still quick and valid.
They apply to all types of sites, including sports sites.