Web scrapping - Stopped working

[peterjv26]

Thanks mlieqo for your help
I tried your suggestion and it worked sometimes but not consistent.

I tried to do this in a loop with multiple stock symbols as below. Now I get a different error that "You are not authorized to access...". More over it does the loop once and for the second stock in the list, the target server is refusing connection even to https://www.nseindia.com/option-chain

I guess they are trying to prevent people from scrapping this site? Is my understanding correct, or am I doing something wrong?

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from bs4 import BeautifulSoup
import time

options = Options()
browser = webdriver.Chrome(executable_path=r"C:\Users\Admin\Downloads\chromedriver_win32\chromedriver.exe",options=options)
stocklist = ['ACC','HDFC','HCLTECH','ICICIBANK','RELIANCE','SBIN',] 

for symbl in stocklist:    
    url = "https://www.nseindia.com/api/option-chain-equities?symbol={stck}".format(stck=symbl)
    
    print('Trying nse option chain')
    browser.get('https://www.nseindia.com/option-chain')
    time.sleep(3)
    print('Symbol - {stck}'.format(stck=symbl))
    print('Trying api url')
    browser.get(url)   
    time.sleep(3)
    #soup = BeautifulSoup(browser.page_source,'lxml')
    browser.quit()