Python Forum

Full Version: Python Web Scraping can not getting all HTML content
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
I am very new to web scraping in Python, and I am using BeautifulSoup for parsing. After obtaining the HTML data, I will try to access some content under "< div id="root">.</div>", but I don’t have all the HTML sites that will be displayed when I actually click "Check". How can I access, or is the way the website prevents me from accessing the information on the page?

If it doesn’t make sense, I’m going to say "." in that div instead of more subcategories for me to view (I will see when I click "check" on the web page).

This is my beautiful soup code... Undecided Undecided

from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup

myurl = 'https://www.coolbet.com/en/sports/incoming-bets'

#open connecting and grab content
uClient = uReq(myurl)
page_html = uClient.read()
uClient.close()

#html parsing
page_soup = soup(page_html, "html.parser")

#grabs each product
containers = page_soup.div.findAll("div", {"class":"sc-iuJeZd iJcGXh"})

print(containers)