Mar-09-2021, 08:26 AM
Hi Everyone,
I have been trying to scrap this website and I run into some NoneType errors that I'm having a really hard time figuring out the root cause of the error.
Here is my code:
Unfortunately, every time I try to pull that class into a variable so that I can iterate over the listings. It either spits out an error for NoneType, or I get an empty list [].
To take a step back...the core of what I'm trying to do is pull the title of a position, the location of that position, and the type of position (full-time, etc). From this website: First Mode Careers
Looping through the listings until I have captured all the individual job positions.
If people have an idea of how to do that with bs4 that is completely different from what I have currently, I am all ears, I'm just trying to learn as much as possible.
Thanks for any help!
CK
I have been trying to scrap this website and I run into some NoneType errors that I'm having a really hard time figuring out the root cause of the error.
Here is my code:
import requests from bs4 import BeautifulSoup url = 'https://firstmode.bamboohr.com/jobs/' page = requests.get(url) print(page.status_code) soup = BeautifulSoup(page.content, 'html.parser') results = soup.find_all(id='resultDiv') listing = soup.find('div', class_='ResAts__card-content ResAts__listing')I am able to run everything up to line 9 in my debugger and see that the variable "results" is populated with information from the page. My goal is to pull individual listings from this webpage and after inspecting the page I think the class that I have on line 10 is the one I'm after.
Unfortunately, every time I try to pull that class into a variable so that I can iterate over the listings. It either spits out an error for NoneType, or I get an empty list [].
To take a step back...the core of what I'm trying to do is pull the title of a position, the location of that position, and the type of position (full-time, etc). From this website: First Mode Careers
Looping through the listings until I have captured all the individual job positions.
If people have an idea of how to do that with bs4 that is completely different from what I have currently, I am all ears, I'm just trying to learn as much as possible.
Thanks for any help!
CK