Jun-28-2018, 09:05 AM
Hi All,
I am fetching product description(with html tags) from a site using BeautifulSoup+Python3.6.
My code is as below-
Like in above code product2 description: is blank.Sample output-
I am fetching product description(with html tags) from a site using BeautifulSoup+Python3.6.
My code is as below-
def get_soup(url): try: response = requests.get(url) if response.status_code == 200: html = response.content return BeautifulSoup(html, "html.parser") except Exception as ex: print("error from " + url + ": " + str(ex)) def get_product_details(url): try: desc_list = soup.select('p ~ ul') prod_details['description'] = ''.join([str(i) for i in desc_list]) return prod_details except Exception as ex: logger.warning('%s - %s', ex, url) if __name__ == '__main__': print("product1 description:") get_product_details("http://www.aprisin.com.sg/p-748-littletikespoptunesguitar.html") print("product2 description:") get_product_details("http://www.aprisin.com.sg/p-1052-172083littletikesclassiccastle.html")The problem with my above code is that it is not able to fetch description for some product urls.
Like in above code product2 description: is blank.Sample output-
product1 description: <ul> <li>Freestyle</li> <li>Play along with 5 pre-set tunes: </li> </ul><ul> <li>Each string will play a note</li> <li>Guitar has a whammy bar</li> <li>2-in-1 volume control and power button </li> <li>Simple and easy to use </li> <li>Helps develop music appreciation </li> <li>Requires 3 "AA" alkaline batteries (included)</li> </ul> product2 description:So what changes I need to make here so that it may work for all types of product?