Python Forum

Full Version: Web Crawler: How to find all divs that starts with...
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hello!

I'm using BeautifulSoup to make a web crawler and i would like to know how can i get the list of all divs that starts with a certain name.

Here what i tried:
divs = soup.findAll('div', {'class':'postContainer*'})
(Sep-30-2016, 11:25 PM)amandacstr Wrote: [ -> ]Here what i tried:
divs = soup.findAll('div', {'class':'postContainer*'})
No you can not do it like this.

You can use CSS selector.
soup.select('div[class^="foo"]')
So this will match all div and css class name that start with foo.
<div class="foo_something">
<div class="foo123">
I have a tutorial on this site,
where i give a demo of using CSS selector "Web-Scraping part-1".
you can always use your lambda approach  :P 

divs = soup.findAll('div', {'class':lambda x: x and x.startswith('postContainer')})
or re

divs = soup.findAll('div', {'class':re.compile('postContainer.*')})
Thanks for helping me guys! Problem fixed.