Python Forum
Python + request from specific website - please help
Thread Rating:
  • 1 Vote(s) - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Python + request from specific website - please help
#7
Another "lazier" approach is to use Selenium.
This deal with JavaScript,so we get whole page source back and not a Waring that JavaScripts most be turned on.
Example:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.keys import Keys
import time
from bs4 import BeautifulSoup

#--| Setup
options = Options()
options.add_argument("--headless")
browser = webdriver.Chrome(executable_path=r'C:\cmder\bin\chromedriver.exe', options=options)
#--| Parse or automation
browser.get('https://ucr.gov/enforcement/1712583')
soup = BeautifulSoup(browser.page_source, 'lxml')
status = soup.find('div', class_="sc-epnACN fEroud")
print(status.text)
Output:
UNREGISTERED
For more about this look at Web-scraping part-2.
Reply


Messages In This Thread
RE: Python + request from specific website - please help - by snippsat - Feb-06-2019, 07:59 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Extracting content from a website using Python? SandraYokum 2 501 May-27-2024, 03:30 AM
Last Post: Davidleo
Question Python request (post/get) Drunknmonkie 1 2,833 Jan-19-2023, 02:02 PM
Last Post: prvncpa
  Retrieve website content using Python? Vadanane 1 1,432 Jan-16-2023, 09:55 AM
Last Post: Axel_Erfurt
  I want to create an automated website in python mkdhrub1 2 2,622 Dec-27-2021, 11:27 PM
Last Post: Larz60+
  Python to build website Methew324 1 2,334 Dec-15-2020, 05:57 AM
Last Post: buran
  Scraping all website text using Python MKMKMKMK 1 2,176 Nov-26-2020, 10:35 PM
Last Post: Larz60+
  Python Webscraping with a Login Website warriordazza 0 2,727 Jun-07-2020, 07:04 AM
Last Post: warriordazza
  Python tool based on website? zarize 2 2,565 Mar-21-2020, 02:25 PM
Last Post: zarize
  hi new at python , trying to get urls from website dviry 6 4,956 Feb-24-2018, 07:34 PM
Last Post: metulburr

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020