Python Forum
'soup.findAll()' help - Want to retrieve multiple attribute values
Thread Rating:
  • 3 Vote(s) - 2 Average
  • 1
  • 2
  • 3
  • 4
  • 5
'soup.findAll()' help - Want to retrieve multiple attribute values
#1
Hello:

I am extremely new to python and am working on developing basic web crawlers. One data point I am interested in retrieving from a particular website is 'engagement'; however, when I inspect the webpage element, the HTML reveals that the engagement number has multiple attribute values depending upon how much it is being engaged with. For example, the element is 'span' and the attribute is 'class' but there are multiple attribute values associated with it ('hot', 'warm').

Here is my (partial) code:

for link in soup.findAll('span', {'class': 'warm' 'hot'}):
            views = link.string
            print(views)
If i choose just one, i do get some print out. However, I want to get the engagement data from every article I crawl.

Therefore, my question is this: How do I incorporate multiple attribute values into a single soup.findAll so that I don't clutter up my print out?

Thanks for the help!

I have also tried the following:

for link in soup.findAll('span', {'class': ['warm','hot']}):
            views = link.string
            print(views)
This doesn't produce any errors, but it doesn't print anything out
Reply
#2
you second code is the one that should work.
Quote:soup.findAll('span', {'class': ['warm','hot']}):
Work your way backwards. Instead of printing out link.string, see what link actually contains. If link is empty then work you way backwards even more.

However is there an outer HTML tag that this is nested in, to grab each span element?
Recommended Tutorials:
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  __init__() got multiple values for argument 'schema' dawid294 4 2,246 Jan-03-2024, 09:42 AM
Last Post: buran
  Can I retrieve 2 values in a for ? Kakalok 3 520 Nov-06-2023, 10:35 PM
Last Post: menator01
  regex findall() returning weird result Radical 1 636 Oct-15-2023, 08:47 PM
Last Post: snippsat
  Python: re.findall to find multiple instances don't work but search worked Secret 1 1,203 Aug-30-2022, 08:40 PM
Last Post: deanhystad
  How to combine multiple column values into 1? cubangt 15 2,801 Aug-11-2022, 08:25 PM
Last Post: cubangt
  Trouble selecting attribute with beautiful soup bananatoast 3 1,614 Jan-30-2022, 10:01 AM
Last Post: bananatoast
  AttributeError: 'list' object has no attribute 'values' ilknurg 4 14,936 Jan-19-2022, 08:33 AM
Last Post: menator01
  Function - Return multiple values tester_V 10 4,435 Jun-02-2021, 05:34 AM
Last Post: tester_V
  Xlsxwriter: Create Multiple Sheets Based on Dataframe's Sorted Values KMV 2 3,481 Mar-09-2021, 12:24 PM
Last Post: KMV
  regex.findall that won't match anything xiaobai97 1 2,008 Sep-24-2020, 02:02 PM
Last Post: DeaD_EyE

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020