Python Forum
Web Crawler: How to find all divs that starts with...
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Web Crawler: How to find all divs that starts with...
#1
Hello!

I'm using BeautifulSoup to make a web crawler and i would like to know how can i get the list of all divs that starts with a certain name.

Here what i tried:
divs = soup.findAll('div', {'class':'postContainer*'})
Reply
#2
(Sep-30-2016, 11:25 PM)amandacstr Wrote: Here what i tried:
divs = soup.findAll('div', {'class':'postContainer*'})
No you can not do it like this.

You can use CSS selector.
soup.select('div[class^="foo"]')
So this will match all div and css class name that start with foo.
<div class="foo_something">
<div class="foo123">
I have a tutorial on this site,
where i give a demo of using CSS selector "Web-Scraping part-1".
Reply
#3
you can always use your lambda approach  :P 

divs = soup.findAll('div', {'class':lambda x: x and x.startswith('postContainer')})
or re

divs = soup.findAll('div', {'class':re.compile('postContainer.*')})
Recommended Tutorials:
Reply
#4
Thanks for helping me guys! Problem fixed.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  python selenium - get info from nested divs damian0612 3 13,106 Feb-26-2021, 08:58 AM
Last Post: law
  Web Crawler help Mr_Mafia 2 1,847 Apr-04-2020, 07:20 PM
Last Post: Mr_Mafia
  how to iterate through a divs tag yokaso 3 3,086 Nov-01-2019, 07:48 AM
Last Post: buran
  Web Crawler help takaa 39 26,862 Apr-26-2019, 12:14 PM
Last Post: stateitreal

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020