Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Scrape java script web site
#1
Hi,
I just want scrap java script web site.
url : https://www.mcxindia.com/market-data/bhavcopy
Can anyone help me on this.

Thanks in advance
Reply
#2
what have you tried so far?
please show code
Reply
#3
Hello PythonHunger,
Can you provide the code so that we can get a clarity and maybe a chance to help you?
Reply
#4
Hi,
Thanks for the reply.
I just want to download csv/excel file from url https://www.mcxindia.com/market-data/bhavcopy .
[Image: excel.png]
I have write a code but got error and browser not getting close.

from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
path_to_chromedriver = 'D:\ChromeDriver\chromedriver.exe'
browser = webdriver.Chrome(executable_path=path_to_chromedriver, options=chrome_options)
url = 'https://www.mcxindia.com/market-data/bhavcopy'
browser.get(url)
f = open(browser.find_element_by_id('cph_InnerContainerRight_C001_lnkExpToCSV').click(), 'wb')
browser.quit()
getting error :
Error:
TypeError: expected str, bytes or os.PathLike object, not NoneType
Reply
#5
The error you show doesn't look like the complete error traceback. Please post verbatim as it contains valuable information.
Quote:I just want to download csv/excel file from url url:https://www.mcxindia.com/market-data/bhavcopy .
you don't need selenium for this, you can get it with a simple requests statement:
import requests

excelfile = resquests.get(url)
Also, I tried to connect to that server and timed out.
Reply
#6
First of all find_element_by_id returns None. And you cannot set the filename to None. Second of all...

you dont have to use open() at all. You only have to click the button and it should download. However you might want to wait a few seconds after the click to allow it to download.

        browser.find_element_by_id('cph_InnerContainerRight_C001_lnkExpToCSV').click()
        time.sleep(10)
Just tested with this ant it works fine.
Recommended Tutorials:
Reply
#7
Hi metulburr.
Its working fine now.
Thank you very much for the reply Smile .
Thanks you sir Smile
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Can we automate Java based Webservices PythonBeginner_2020 5 2,388 Mar-25-2020, 12:09 PM
Last Post: PythonBeginner_2020
  scrape data 1 go to next page scrape data 2 and so on alkaline3 6 5,087 Mar-13-2020, 07:59 PM
Last Post: alkaline3
  Scrape script when deployed on heroku not giving expected outcomes alok001 1 3,225 Oct-19-2019, 12:39 PM
Last Post: alok001
  Web scraping and java script yoz69100 2 1,836 Oct-14-2019, 07:41 PM
Last Post: yoz69100

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020