Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
scraping in a text/javascript
#2
I dont think BS4 has a way to parse the top level element like that? It's not meant to do that. There is nothing stopping you from grabbing the html source and regex finding what you need out of it though, except if the script element itself is modified by javascript, then you must use selenium to even obtain the proper element.

If there is no way with BS4 that i am unaware of, i would probably parse out spConfig into proper JSON and then load it like a dictionary. As i think if you omit var spConfig = new Product.Config() it leaves what appears to be a dictionary/JSON. Then it would leave for less room for error, or if they append to that it would not break your script.
Recommended Tutorials:
Reply


Messages In This Thread
scraping in a text/javascript - by saasyp - Aug-30-2019, 01:52 PM
RE: scraping in a text/javascript - by metulburr - Aug-31-2019, 11:39 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Scraping all website text using Python MKMKMKMK 1 2,053 Nov-26-2020, 10:35 PM
Last Post: Larz60+
  Scraping text from application? kamix 1 1,549 Sep-25-2020, 10:53 PM
Last Post: Larz60+
  Web Scraping on href text Superzaffo 11 7,242 Nov-16-2019, 10:52 AM
Last Post: Superzaffo
  scraping javascript websites with selenium DoctorEvil 1 3,315 Jun-08-2018, 06:40 PM
Last Post: DoctorEvil

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020