Jan-28-2021, 03:16 AM
Hello,
I have this problem trying to scrape a website using Beautifulsoup.
I'm trying to find a "span" in multiple "div", but I can't find anything deeper than the very first div
Here's my code
if I call page_soup.body i get this result:
I have this problem trying to scrape a website using Beautifulsoup.
I'm trying to find a "span" in multiple "div", but I can't find anything deeper than the very first div
Here's my code
from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup
my_url = 'https://www.newegg.com/p/pl?d=graphicscard'
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()
page_soup = soup(page_html, "html.parser")
containers = page_soup.findAll("span", {"class":"fs-13"})and here's the result I have in my console>>> containers = page_soup.findAll("div",{"id":"app"})
>>> len(containers)
1
>>> containers
[<div id="app"></div>]
>> containers = page_soup.findAll("span",{"class":"fs-11"})
>>> len(containers)
0see the <div id="app"> is the very first div, but there's a whole bunch of stuff in this div. I can see it when I inspect the webpage, but if I try to find the <span class="fs-11"> using the function findAll i get nothingif I call page_soup.body i get this result:
>>> page_soup.body
<body>
<div id="app"></div>
<div id="modal"></div>
<script>
if (window.location.port !== '80') window.__env__ = 'dev';
</script>
<script>
window.appHash = 'b0b815fdc589074946ba';
</script>
<script src="https://polyfill.io/v3/polyfill.min.js"></script>
<script src="https://cdn.polyfill.io/v......(cut for the sake of brievety)So my question is: How do I scrape a <span> in a website which is embedded in multiple <div>?