Python Forum
Web scraping Possible JavaScript issue
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Web scraping Possible JavaScript issue
#1
Hi,

I'm trying to scrape the values from the following url 'https://sports.williamhill.com/betting/en-gb/in-play/football' and using requests.get to call it.

When I inspect the element and search for some text next to the value in the response, it has what looks like to be a java script function.

In the attached, I'm trying to extract the score of the match that's 1-4. As per the right hand side, I'm searching the response for the text '__livescore-item team-a">'

Browser:
https://pasteboard.co/JWIGpGp.png

Response:
https://pasteboard.co/JWII0JL.png

I've also tried render from HTMLSession but that didn't seem to work either :(

Cheers,
J
Reply
#2
Don't they have an API you can use instead of scraping the page?
Reply
#3
(Apr-11-2021, 04:30 AM)ndc85430 Wrote: Don't they have an API you can use instead of scraping the page?

They do but you need to be part of some developer network. I applied for that but was rejected :( That was some time ago but looking as of late, they've stopped issuing api keys :(
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Scraping Issue with BS muzikman 11 3,016 Dec-10-2021, 08:49 AM
Last Post: muzikman
Brick Javascript based web page scraping amjadraza26 1 1,426 Oct-21-2021, 09:36 AM
Last Post: Larz60+
  Web scraping: webbrowser issue Truman 10 6,956 Jul-11-2018, 11:57 PM
Last Post: snippsat
  Scraping a javascript RSSI indicator jon333 8 5,838 Aug-10-2017, 07:11 PM
Last Post: jon333

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020