My first Web Scraping project - Printable Version

My first Web Scraping project - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: General (https://python-forum.io/forum-1.html)
+--- Forum: Code Review (https://python-forum.io/forum-46.html)
+--- Thread: My first Web Scraping project (/thread-27479.html)

My first Web Scraping project - pyzyx3qwerty - Jun-08-2020

So, recently, I have been learning web scraping with tutorials and videos and other things like that. Here's my first go at a simple project that helps in displaying software developer jobs available in New York from the Monster.com site

import requests
from bs4 import BeautifulSoup

URL = "https://www.monster.com/jobs/search/?q=Software-Developer&where=NewYork"
page = requests.get(URL)

soup = BeautifulSoup(page.content, 'html.parser')

results = soup.find(id='ResultsContainer')


job_elements = results.find_all('section', class_='card-content')

for job_element in job_elements:
    title_element = job_element.find('h2', class_='title')
    company_element = job_element.find('div', class_='company')
    location_element = job_element.find('div', class_='location')
    if None in (title_element, company_element, location_element):
        continue
    print(title_element.text.strip())
    print(company_element.text.strip())
    print(location_element.text.strip())
    print()

Any feedback would be appreciated!
(And yes, I had to take some help for this but please - I'm a beginner in this topic)

RE: My first Web Scraping project - metulburr - Jun-08-2020

Pretty clean compared to others i have seen. Web scraping can get messy quickly. So it is important to keep it clean and organized.

RE: My first Web Scraping project - pyzyx3qwerty - Jun-08-2020

Well, I had learnt that not keeping it messy was one of the most important steps in web scrapping, so I did the best I could

RE: My first Web Scraping project - buran - Jun-08-2020

elements = (title_element, company_element, location_element)
if all(elements):
    for element in elements:
        print(element.text.strip())
    print()

the part where you search for each element can also be shortened with a loop if you have a container type with what you search for

RE: My first Web Scraping project - pyzyx3qwerty - Jun-08-2020

Thanks for your suggestion @buran!