Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
an issue with bs4 scraping
#4
The site is not best,there are lot better site with free API to get weather info in eg JSON.
The site has language detection.
Just to get temp as a demo,
i use PhatomJS(to no load a browser) as a driver for selenium.
This is the simple way to get away not looking at JavaScript source for reverse engineer as mention bye @stranac.
from selenium import webdriver
from bs4 import BeautifulSoup
import time

browser = webdriver.PhantomJS()
url = 'http://freemeteo.no/vaer/oslo/daglig-vaermelding/idag/?gid=3143244&language=norwegian&country=norway'
browser.get(url)
time.sleep(3)

# Give source code to BeautifulSoup
soup = BeautifulSoup(browser.page_source, 'lxml')
temp = soup.select('#content > div.right-col > div.weather-now > div.today.clearfix > a.section.first > span.temp > strong')
print(temp)
print(temp[0].text)
Output:
[<strong>2<em>°C</em></strong>] 2°C
Reply


Messages In This Thread
an issue with bs4 scraping - by komarek - Oct-10-2017, 12:53 PM
RE: an issue with bs4 scraping - by stranac - Oct-10-2017, 01:06 PM
RE: an issue with bs4 scraping - by metulburr - Oct-10-2017, 01:10 PM
RE: an issue with bs4 scraping - by snippsat - Oct-10-2017, 02:08 PM
RE: an issue with bs4 scraping - by komarek - Oct-11-2017, 03:42 PM
RE: an issue with bs4 scraping - by buran - Oct-11-2017, 06:31 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Size scraping issue scrapemasta 0 528 Feb-09-2024, 10:26 AM
Last Post: scrapemasta
Thumbs Up Issue facing while scraping the data from different websites in single script. Balamani 1 2,228 Oct-20-2020, 09:56 AM
Last Post: Larz60+
  POST request with form data issue web scraping hoff1022 1 2,778 Aug-14-2020, 10:25 AM
Last Post: kashcode
  Strange BS4 Scraping Issue digitalmatic7 1 2,471 Jan-14-2018, 04:34 PM
Last Post: wavic

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020