Python Forum

Hi guys-New Python user here. I enjoy working with basketball statistics and have realized I need to figure out a more efficient way to scrape data from NBA.com. I found this script on the internet and have managed to (I think) get pretty close it working, but when I open Powershell and type "python NBA2.py" (file name) I get no response. Powershell stays open but doesn't give any feedback or let me type anymore. FYI I'm using Windows 10 and Python 3.6

import requests
import csv

url = "http://stats.nba.com/stats/leaguedashplayerstats?DateFrom=&DateTo=&GameScope=&GameSegment=&LastNGames=15&LeagueID=00&Location=&MeasureType=Advanced&Month=0&OpponentTeamID=0&Outcome=&PaceAdjust=N&PerMode=Totals&Period=0&PlayerExperience=&PlayerPosition=&PlusMinus=N&Rank=N&Season=2015-16&SeasonSegment=&SeasonType=Regular+Season&StarterBench=&VsConference=&VsDivision="

data = requests.get(url)
entries = data.json()

with open('output.csv', 'wb') as f_output:
    csv_output = csv.writer(f_output)
    csv_output.writerow(entries['resultSets'][0]['headers'])
    csv_output.writerows(entries['resultSets'][0]['rowSet'])

If anyone could provide some info on why this isn't working or point me toward a website that can help it would be greatly appreciated. I've spent hours troubleshooting and can't seem to pinpoint where the problem is.

take a look at:
https://python-forum.io/Thread-Web-Scraping-part-1
and
https://python-forum.io/Thread-Web-scraping-part-2

(Oct-29-2017, 12:57 AM)Larz60+ Wrote: [ -> ]take a look at:
https://python-forum.io/Thread-Web-Scraping-part-1
and
https://python-forum.io/Thread-Web-scraping-part-2

Thanks. Those links were helpful but don't really solve my issue. I guess "scrape" isn't really the proper terminology for what I need to do (like I said, new at this ). The url in my script takes me directly to a web page with the data separated by commas. From there I just need get it into a format that I can analyze within Excel. I feel like the script I posted should do that, but I must be missing something.

If you click on the url that you have, you get the following response:

Quote:The field SeasonType must match the regular expression '^(Regular Season)|(Pre Season)|(Playoffs)|(All Star)$'.; The StarterBench property is required.; The VsConference property is required.; The VsDivision property is required.

(Oct-29-2017, 03:12 AM)Larz60+ Wrote: [ -> ]If you click on the url that you have, you get the following response:

Quote:The field SeasonType must match the regular expression '^(Regular Season)|(Pre Season)|(Playoffs)|(All Star)$'.; The StarterBench property is required.; The VsConference property is required.; The VsDivision property is required.

It works when you copy/paste. For some reason the hyperlink stops before the end of the URL.

Even so, requests for whatever reason freezes up and doesn't even timeout
I believe that the syntax that you have (for the json load) is correct, but I would still look at the errors that
are in the message above.

There is a different way to pass params, see: https://stackoverflow.com/questions/6386...-in-python

and another here: https://stackoverflow.com/questions/1751...ter-python

(Oct-29-2017, 05:08 AM)Larz60+ Wrote: [ -> ]Even so, requests for whatever reason freezes up and doesn't even timeout
I believe that the syntax that you have (for the json load) is correct, but I would still look at the errors that
are in the message above.

There is a different way to pass params, see: https://stackoverflow.com/questions/6386...-in-python

and another here: https://stackoverflow.com/questions/1751...ter-python

Shot in the dark here but could there be a problem with python reading the plus sign (+) within the link?

After altering the link I managed to get this response when running in PowerShell. Trying to figure out on my own how to fix this but haven't come up with anything.

PS C:\Users\Nick> python NBASample.py
Traceback (most recent call last):
File "NBASample.py", line 11, in <module>
csv_output.writerow(entries['resultSet'][0]['headers'])
KeyError: 0

import requests
import csv

url = "http://stats.nba.com/stats/leagueLeaders?LeagueID=00&PerMode=PerGame&Scope=S&Season=2017-18&SeasonType=Regular+Season&StatCategory=PTS"

data = requests.get(url)
entries = data.json()

with open('NBAFile.csv', 'wb') as f_output:
    csv_output = csv.writer(f_output)
    csv_output.writerow(entries['resultSet'][0]['headers'])
    csv_output.writerows(entries['resultSet'][0]['rowSet'])