Python Forum
Getting 'list index out of range' while fetching product details using BeautifulSoup?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Getting 'list index out of range' while fetching product details using BeautifulSoup?
#6
First Issue-
check_merchant_product_id = soup.find('div', attrs={'class': 'description'}).findAll('span')[3]
                if check_merchant_product_id is not None:
                    merchant_product_id = check_merchant_product_id.text
                if merchant_product_id is not None:
                    prod_details['merchant_product_id'] = merchant_product_id
Error:
getting 'Product Code:' instead of value
Second Issue-
prod_details['merchant_image_urls'] = ",".join(list(filter(None, map(lambda x: x['href'].replace(",", "%2C"),
                                                                                         soup.find('div', attrs={
                                                                                             'class': 'left'}).findAll(
                                                                                             'a')))))
Error:
getting broken image urls
Third Issue-
check_price = soup.find('span', attrs={"class": "price-old"})
                    if check_price is not None:
                        prod_details['price'] = check_price.text.replace("SGD $", "")
                    check_sale_price = soup.find('span', attrs={"class": "price-new"})
                    if check_sale_price is not None:
                        prod_details['sale_price'] = check_sale_price.text.replace("SGD $", "")
Error:
getting NaN for some products
Reply


Messages In This Thread
RE: Getting 'list index out of range' while fetching product details using BeautifulSoup? - by PrateekG - Jun-06-2018, 10:25 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Fetching Images from DB in Django Dexty 2 1,754 Mar-15-2024, 08:43 AM
Last Post: firn100
  All product links to products on a website MarionStorm 0 1,106 Jun-02-2022, 11:17 PM
Last Post: MarionStorm
  IndexError: list index out of range" & "TypeError: The view function f: Flask Web App joelbeater992 5 3,570 Aug-31-2021, 08:08 PM
Last Post: joelbeater992
  Python BeautifulSoup IndexError: list index out of range rhat398 1 6,276 May-28-2021, 09:09 PM
Last Post: Daring_T
  fetching, parsing data from Wikipedia apollo 2 3,571 May-06-2021, 08:08 PM
Last Post: snippsat
  How to make data coming from a database clickable giving more details newbie1 8 3,785 May-29-2020, 11:19 PM
Last Post: newbie1
  IndexError: tuple index out of range ? JohnnyCoffee 4 3,426 Jan-22-2020, 06:54 AM
Last Post: JohnnyCoffee
  Fetching and Parsing XML Data FalseFact 3 3,294 Apr-01-2019, 10:21 AM
Last Post: Larz60+
  My Django 2.0.6 logging is not working while product merging PrateekG 0 2,187 Jul-26-2018, 02:24 PM
Last Post: PrateekG
  from List to BeautifulSoup , Homework RPC 6 7,088 Jul-03-2018, 12:17 AM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020