Python Forum
Webscraping a site that uses javascript?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Webscraping a site that uses javascript?
#3
Thanks. I will look into Selenium now and report back here when I get stuck or accomplish my task.

Actually, I noticed something which is confusing me.  When I use the code soup.findall('p')[-2], the list of tickers seem to be embedded in the returned output. 

 
 

<p><script src="https://d33t3vvu2t2yu5.cloudfront.net/tv.js" type="text/javascript"></script><br/>
<script type="text/javascript">
new TradingView.MiniWidget({
  "container_id": "tv-miniwidget-c316c",
  "tabs": [
    "Micro Cap Biotech"
  ],
  "symbols": {
    "Micro Cap Biotech": [
      [
        "Abeona Thera",
        "NASDAQ:ABEO|3m"
      ],
      [
        "Actinium Pharma",
        "AMEX:ATNM|3m"
      ],
      [
        "ADMA Biologics",
        "NASDAQ:ADMA|3m"
      ],
      [
        "Adverum Biotech",
        "NASDAQ:ADVM|3m"
      ],
      [
        "Aeglea",
        "NASDAQ:AGLE|3m"
      ],
      [
        "Affimed",
        "NASDAQ:AFMD|3m"
      ],
      [
        "Akari Therapeutics",
        "NASDAQ:AKTX|3m"
      ],
      [
        "Alcobra",
        "NASDAQ:ADHD|3m"
      ],
      [
        "Actinium Pharma",
        "AMEX:ATNM|3m"
      ],
      [
        "ADMA Biologics",
        "NASDAQ:ADMA|3m"
      ],
      [
        "Adverum Biotech",
        "NASDAQ:ADVM|3m"
      ],
      [
        "Aeglea",
        "NASDAQ:AGLE|3m"
      ],
      [
        "Affimed",
        "NASDAQ:AFMD|3m"
      ],
      [
        "Akari Therapeutics",
        "NASDAQ:AKTX|3m"
      ],
      [
        "Alcobra",
        "NASDAQ:ADHD|3m"
Does this mean, it's still scrapable with Beautiful Soup?

Thank you.
Reply


Messages In This Thread
RE: Webscraping a site that uses javascript? - by bigmit37 - Apr-29-2017, 08:35 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Webscraping news articles by using selenium cate16 7 3,271 Aug-28-2023, 09:58 AM
Last Post: snippsat
  Webscraping with beautifulsoup cormanstan 3 2,097 Aug-24-2023, 11:57 AM
Last Post: snippsat
  Webscraping returning empty table Buuuwq 0 1,442 Dec-09-2022, 10:41 AM
Last Post: Buuuwq
  WebScraping using Selenium library Korgik 0 1,074 Dec-09-2022, 09:51 AM
Last Post: Korgik
  How to get rid of numerical tokens in output (webscraping issue)? jps2020 0 1,979 Oct-26-2020, 05:37 PM
Last Post: jps2020
  Python Webscraping with a Login Website warriordazza 0 2,635 Jun-07-2020, 07:04 AM
Last Post: warriordazza
  Help with basic webscraping Captain_Snuggle 2 3,993 Nov-07-2019, 08:07 PM
Last Post: kozaizsvemira
  Can't Resolve Webscraping AttributeError Hass 1 2,345 Jan-15-2019, 09:36 PM
Last Post: nilamo
  How to exclude certain links while webscraping basis on keywords Prince_Bhatia 0 3,272 Oct-31-2018, 07:00 AM
Last Post: Prince_Bhatia
  Webscraping homework Ghigo1995 1 2,682 Sep-23-2018, 07:36 PM
Last Post: nilamo

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020