Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Web scrapping - Stopped working
#1
Hi, I wrote a small script as below

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from bs4 import BeautifulSoup

options = Options()
browser = webdriver.Chrome(executable_path=r"C:\Users\Admin\Downloads\chromedriver_win32\chromedriver.exe",options=options)
       
url = "https://www.nseindia.com/api/option-chain-equities?symbol=ACC"
    
browser.get(url)
soup = BeautifulSoup(browser.page_source,'lxml')
    
print(soup.prettify())
I got the url with query string (https://www.nseindia.com/api/option-chai...symbol=ACC) after doing Inspect and looking up the Network tab in developer tools.
The url that we are supposed to use via a browser is https://www.nseindia.com/option-chain. What I am trying to do is to read the json file with the values that the table in this page gets populated with.

This used to work and I was able to get the json file. But it seems to have stopped working and instead of an html page with the json, I am getting a message "Resource Not found"
If I copy the url from the Network tab via Inspect into a browser window, it displays the jason content. But if I put this url into my scipt, it gives the "Resource Not found" message.

Can you please help? Thank you
Reply


Messages In This Thread
Web scrapping - Stopped working - by peterjv26 - Sep-22-2020, 02:09 PM
RE: Web scrapping - Stopped working - by mlieqo - Sep-22-2020, 07:51 PM
RE: Web scrapping - Stopped working - by peterjv26 - Sep-23-2020, 08:30 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Problem with scrapping Website giddyhead 1 2,456 Mar-08-2024, 08:20 AM
Last Post: AhanaSharma
  python web scrapping mg24 1 1,184 Mar-01-2024, 09:48 PM
Last Post: snippsat
  How can I ignore empty fields when scrapping never5000 0 2,014 Feb-11-2022, 09:19 AM
Last Post: never5000
  web scrapping through Python Naheed 2 3,333 May-17-2021, 12:02 PM
Last Post: Naheed
  Website scrapping and download santoshrane 3 5,515 Apr-14-2021, 07:22 AM
Last Post: kashcode
  Image Scraper (beautifulsoup), stopped working, need to help see why woodmister 9 5,552 Jan-12-2021, 04:10 PM
Last Post: woodmister
  Newbie help with lxml scrapping chelsealoa 1 2,396 Jan-08-2021, 09:14 AM
Last Post: Larz60+
  Scrapping Sport score laplacea 1 3,047 Dec-13-2020, 04:09 PM
Last Post: Larz60+
  Web scrapping login facebook credentials kosmas9 0 2,501 Aug-17-2020, 01:33 PM
Last Post: kosmas9
  Web Scrapping Through API krishgokul29 3 3,067 Aug-11-2020, 05:56 PM
Last Post: buran

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020