Scrape option chains data from finance.yahoo.com

[buran]

This was on the old forum in the Completed Scripts section and today I decided to add it also here.

http://github.com/boyank/yoc

yoc (yahoo option chains)

That's version 2 and it works with yahoo.com website design since July 2016.It can be used as a command line tool for scraping financial options prices and related data from finance.yahoo.com

In ver.2 it requests and gets response in json format. There are a lot more data available. One can easily change the script to parse the json and extract any of these data.
Check the repo for more details.

[micseydel]

I understand that they don't have an API so you're forced to scrape, but I'm curious what your strategy is in terms of breakages.

[snippsat]

I understand that they don't have an API so you're forced to scrape

Not quite,they have one of the best API out there.
Even have own language Yahoo Query Language (YQL).
Own online test of commands.
Have used before with Requests.

I don't know if buran have looked at this,
it happens that not use API and do it self can work equally okay.

[buran]

To be honest, I don't use the tool at all. A colleague of mine needed to collect data for a paper he was working on and he was trying to write some tool in matlab to scrape data daily, because at that time YQL was not working for option chains (and still not working). So he asked me how to parse the html to extaract the data he needed using pure string operations and instead I decided to write the entire tool for him. That was version 1. It was as he wanted it and based on it he wrote a tool in matlab and he got the data he needed. After that I was not following what's going on. One day there was an issue from some user that the tools is not working any more and asking if I can check. It turned out that the problem was because yahoo changed the design of their website. After couple of weeks I found time to change the code and this now is version 2.
So more or less every one is free to use the tool, but it's up to him to secure that there are no problems (e.g. problems with connection, etc.). My colleague din'dt update his tool at all. Anyway I have some regular amount of daily traffic from matlab central and SO and people who clone the repo.


http://www.mathworks.com/matlabcentral/f...ptionchain
http://stackoverflow.com/questions/12773...-using-yql

Long story short - I don't have strategy as I don't use it. But I'll be glad to provide some support if someone has specific request or needs some sort of assistance.

[micseydel]

That's actually cool, if people are cloning it then that's lots of people who can either make a pull request or fork you if it dies. I guess another way to phrase my question is: for someone seeing this post years from now, when the current code is broken, what are they to do? And you've provided a satisfactory answer :)

[Larz60+]

Just a quick note, earlier we had a thread under snippets started by nilamo. There's a lot of information there, including using
the Google real time quote engine (which is still available, although not officially). I think I may have posted a complete set of
Yahoo commands as well, and if not, it will be done if requested.