How to use BeautifulSoup4 with pandas series type of html data?

PrateekG · (This post was last modified: Apr-24-2018, 11:03 AM by PrateekG.)

Hi All,

I have some html data in the form of pandas Series.
For example I am storing this data in a variable-html_series

Now when I try to apply BeautifulSoup here as -

soup = BeautifulSoup(html_series, "html.parser")
print(soup.prettify())

I am getting below error-
ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

Can you please tell me what I am missing here?

Thanks!

**buran** · Apr-25-2018, 06:54 AM

what exactly is html data in the form of pandas Series? This sounds non-sense to me

PrateekG · (This post was last modified: Apr-25-2018, 09:34 AM by PrateekG.)

Let me explain-
1. Using shoppify api I fetched the json from an ecommerce site.
2. I normalized this json data into dataframe by-

df = json_normalize(result)

3. From this dataframe I take out the html content by-

html_data = df['body_html']

4. Now when I use below code I got the error-

soup = BeautifulSoup(html_data, "html.parser")
print(soup.prettify())

Hope I mentioned everything here.

***snippsat*** · Apr-25-2018, 09:51 AM

It look like you try to put json data into BeautifulSoup.
What is the contented html_data?
For it to work it's has to be html.

from bs4 import BeautifulSoup

html_data = '''\
<!DOCTYPE html>
<html>
  <head>
    <title>Title of document</title>
  </head>
  <body>
    <p>Content of the document</p>
  </body>
</html'''

soup = BeautifulSoup(html_data, 'lxml')
print(soup.select('head > title')[0].text)

Output:
Title of document

PrateekG · Apr-26-2018, 07:33 AM

I have resolved the above error it was due to dataframe normalization.
Now I have raise another ticket with below url-
https://python-forum.io/Thread-How-to-cl...Python-3-6

Please see once and let me know if you can help.
Thanks!

Possibly Related Threads…
Thread		Author	Replies	Views	Last Post
	Trying to scrape data from HTML with no identifiers	pythonpaul32	2	844	Dec-02-2023, 03:42 AM Last Post: pythonpaul32
	Beautifulsoup4 help	samuelbachorik	1	1,351	Feb-05-2022, 10:44 PM Last Post: snippsat
	Post HTML Form Data to API Endpoints	Dexty	0	1,400	Nov-11-2021, 10:51 PM Last Post: Dexty
	HTML multi select HTML listbox with Flask/Python	rfeyer	0	4,623	Mar-14-2021, 12:23 PM Last Post: rfeyer
	Cleaning HTML data using Jupyter Notebook	jacob1986	7	4,131	Mar-05-2021, 10:44 PM Last Post: snippsat
	Any way to remove HTML tags from scraped data? (I want text only)	SeBz2020uk	1	3,460	Nov-02-2020, 08:12 PM Last Post: Larz60+
	html data cell attribute issue	delahug	5	3,145	May-31-2020, 09:18 AM Last Post: delahug
	Extracting html data using attributes	WiPi	14	5,472	May-04-2020, 02:04 PM Last Post: snippsat
	How to crawl schema markup data type using scrapy?	Nuwan16	1	3,078	Mar-31-2020, 03:42 PM Last Post: stranac
	extrat data from a button html	windows11	1	1,976	Mar-24-2020, 03:39 PM Last Post: Larz60+

How to use BeautifulSoup4 with pandas series type of html data?

User Panel Messages

Announcements