Aug-18-2020, 10:05 AM
As mention you most check better that name of class is correct,it is only
Also not all product has a rating so need a fix for this.
Most specify which parser that BS shall use or get a Warning message,preferably use
As you have loop move append lines into the loop,or only get first product in the lists.
hGSR34
and not hGSR34 _2beYZw
.Also not all product has a rating so need a fix for this.
Most specify which parser that BS shall use or get a Warning message,preferably use
lxml
.As you have loop move append lines into the loop,or only get first product in the lists.
soup = BeautifulSoup(browser.page_source, 'lxml') products = [] #List to store name of the product prices = [] #List to store price of the product ratings = [] #List to store rating of the product browser.get("https://www.flipkart.com/search?q=laptop&otracker=search&otracker1=search&marketplace=FLIPKART&as-show=off&as=off") content = browser.page_source soup = BeautifulSoup(content, 'lxml') for a in soup.findAll('a', href=True, class_="_31qSD5"): name = a.find('div', class_="_3wU53n") price = a.find('div', class_="_1vC4OE _2rQ-NK") rating = a.find('div',class_="hGSR34") products.append(name.text) prices.append(price.text) try: ratings.append(rating.text) except AttributeError: pass
[>>> ratings ['4.2', '4.4', '4.5', '4.3', '4.4', '4', '4.2', '3.7', '4.6', '4.4', '4.5', '4', '4.5', '4.4', '4.7', '4', '4.5', '4.6', '4.2', '4.6', '4.3', '4.3', '5'] >>> prices ['₹39,990', '₹1,20,990', '₹35,990', '₹56,990', '₹52,990', '₹59,990', '₹32,990', '₹39,990', '₹43,990', '₹52,990', '₹59,990', '₹60,990', '₹35,990', '₹39,990', '₹38,990', '₹73,990', '₹24,990', '₹55,990', '₹69,990', '₹1,01,990', '₹59,990', '₹54,990', '₹61,990', '₹75,990']