Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
an issue with bs4 scraping
#1
i have this script running on my raspberry that is scraping weather report from an url:
http://it.freemeteo.com/tempo/
this is the snippet of the weather prevision html:

<div class="pred"><script type="text/javascript">
          document.write(Icons.GetDescription(1,'CurrentWeather'));
        </script>SUNNY DAY</div>

i'm using bs4
 i do something like this: 

r = requests.get(urlWeather)
soup = BeautifulSoup(r.content, 'html.parser')
soup.body.find('div', class_="pred")
but the result is
<div class="pred"><script type="text/javascript">
          document.write(Icons.GetDescription(1,'CurrentWeather'));
        </script></div>

the Weather part vanishes
it is probably a problem of the javascript
anyone knows the solution?
cheers
Reply


Messages In This Thread
an issue with bs4 scraping - by komarek - Oct-10-2017, 12:53 PM
RE: an issue with bs4 scraping - by stranac - Oct-10-2017, 01:06 PM
RE: an issue with bs4 scraping - by metulburr - Oct-10-2017, 01:10 PM
RE: an issue with bs4 scraping - by snippsat - Oct-10-2017, 02:08 PM
RE: an issue with bs4 scraping - by komarek - Oct-11-2017, 03:42 PM
RE: an issue with bs4 scraping - by buran - Oct-11-2017, 06:31 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Size scraping issue scrapemasta 0 383 Feb-09-2024, 10:26 AM
Last Post: scrapemasta
Thumbs Up Issue facing while scraping the data from different websites in single script. Balamani 1 2,139 Oct-20-2020, 09:56 AM
Last Post: Larz60+
  POST request with form data issue web scraping hoff1022 1 2,706 Aug-14-2020, 10:25 AM
Last Post: kashcode
  Strange BS4 Scraping Issue digitalmatic7 1 2,414 Jan-14-2018, 04:34 PM
Last Post: wavic

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020