Python Forum
My first Web Scraping project
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
My first Web Scraping project
#1
So, recently, I have been learning web scraping with tutorials and videos and other things like that. Here's my first go at a simple project that helps in displaying software developer jobs available in New York from the Monster.com site
import requests
from bs4 import BeautifulSoup

URL = "https://www.monster.com/jobs/search/?q=Software-Developer&where=NewYork"
page = requests.get(URL)

soup = BeautifulSoup(page.content, 'html.parser')

results = soup.find(id='ResultsContainer')


job_elements = results.find_all('section', class_='card-content')

for job_element in job_elements:
    title_element = job_element.find('h2', class_='title')
    company_element = job_element.find('div', class_='company')
    location_element = job_element.find('div', class_='location')
    if None in (title_element, company_element, location_element):
        continue
    print(title_element.text.strip())
    print(company_element.text.strip())
    print(location_element.text.strip())
    print()
Any feedback would be appreciated!
(And yes, I had to take some help for this but please - I'm a beginner in this topic)
pyzyx3qwerty
"The greatest glory in living lies not in never falling, but in rising every time we fall." - Nelson Mandela
Need help on the forum? Visit help @ python forum
For learning more and more about python, visit Python docs
Reply


Messages In This Thread
My first Web Scraping project - by pyzyx3qwerty - Jun-08-2020, 10:29 AM
RE: My first Web Scraping project - by metulburr - Jun-08-2020, 10:42 AM
RE: My first Web Scraping project - by pyzyx3qwerty - Jun-08-2020, 10:50 AM
RE: My first Web Scraping project - by buran - Jun-08-2020, 01:16 PM
RE: My first Web Scraping project - by pyzyx3qwerty - Jun-08-2020, 02:56 PM

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020