Python Forum
trying to scrape a span inside a div using beautifulsoup
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
trying to scrape a span inside a div using beautifulsoup
#3
If i look can not find a class=fs-11 on that site.
Can look at this test,see that i use Requests and not urllib.
So this will take out product info and price for the first graphics card.
import requests
from bs4 import BeautifulSoup

url = 'https://www.newegg.com/p/pl?d=graphicscard'
headers = {"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.75 Safari/537.36"}
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.content, 'lxml')
prod_info = soup.select_one('div.item-info > a')
print(prod_info.text)
print('-' * 25)
price = soup.select_one('ul > li.price-current')
print(price.text)
Output:
XFX Radeon RX 580 DirectX 12 RX-580P8DFD6 XXX Edition 8GB 256-Bit GDDR5 PCI Express 3.0 CrossFireX Support Video Card ------------------------- $696.98 (4 Offers)–
Reply


Messages In This Thread
RE: trying to scrape a span inside a div using beautifulsoup - by snippsat - Jan-28-2021, 01:20 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Beautifulsoup doesn't scrape page (python 2.7) Hikki 0 2,006 Aug-01-2020, 05:54 PM
Last Post: Hikki
  select all the span text with same attribute JennyYang 2 2,154 Jul-28-2020, 02:56 PM
Last Post: snippsat
  scrape data 1 go to next page scrape data 2 and so on alkaline3 6 5,216 Mar-13-2020, 07:59 PM
Last Post: alkaline3
  Scrap a dynamic span hefaz 0 2,709 Mar-07-2020, 02:56 PM
Last Post: hefaz
  Cannot get contents from ul.li.span.string LLLLLL 8 4,024 Nov-29-2019, 10:30 AM
Last Post: LLLLLL
  selenium click a span tag metulburr 1 21,971 Nov-30-2016, 05:47 PM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020