+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: CSV output problem (/thread-5921.html)
CSV output problem - chisox721 - Oct-28-2017
Hi guys-New Python user here. I enjoy working with basketball statistics and have realized I need to figure out a more efficient way to scrape data from NBA.com. I found this script on the internet and have managed to (I think) get pretty close it working, but when I open Powershell and type "python NBA2.py" (file name) I get no response. Powershell stays open but doesn't give any feedback or let me type anymore. FYI I'm using Windows 10 and Python 3.6
import requests
import csv
url = "http://stats.nba.com/stats/leaguedashplayerstats?DateFrom=&DateTo=&GameScope=&GameSegment=&LastNGames=15&LeagueID=00&Location=&MeasureType=Advanced&Month=0&OpponentTeamID=0&Outcome=&PaceAdjust=N&PerMode=Totals&Period=0&PlayerExperience=&PlayerPosition=&PlusMinus=N&Rank=N&Season=2015-16&SeasonSegment=&SeasonType=Regular+Season&StarterBench=&VsConference=&VsDivision="
data = requests.get(url)
entries = data.json()
with open('output.csv', 'wb') as f_output:
csv_output = csv.writer(f_output)
csv_output.writerow(entries['resultSets'][0]['headers'])
csv_output.writerows(entries['resultSets'][0]['rowSet'])If anyone could provide some info on why this isn't working or point me toward a website that can help it would be greatly appreciated. I've spent hours troubleshooting and can't seem to pinpoint where the problem is.
RE: CSV output problem - Larz60+ - Oct-29-2017
take a look at:
https://python-forum.io/Thread-Web-Scraping-part-1
and
https://python-forum.io/Thread-Web-scraping-part-2
RE: CSV output problem - chisox721 - Oct-29-2017
(Oct-29-2017, 12:57 AM)Larz60+ Wrote: take a look at:
https://python-forum.io/Thread-Web-Scraping-part-1
and
https://python-forum.io/Thread-Web-scraping-part-2
Thanks. Those links were helpful but don't really solve my issue. I guess "scrape" isn't really the proper terminology for what I need to do (like I said, new at this
). The url in my script takes me directly to a web page with the data separated by commas. From there I just need get it into a format that I can analyze within Excel. I feel like the script I posted should do that, but I must be missing something.
RE: CSV output problem - Larz60+ - Oct-29-2017
If you click on the url that you have, you get the following response:
Quote:The field SeasonType must match the regular expression '^(Regular Season)|(Pre Season)|(Playoffs)|(All Star)$'.; The StarterBench property is required.; The VsConference property is required.; The VsDivision property is required.
RE: CSV output problem - chisox721 - Oct-29-2017
(Oct-29-2017, 03:12 AM)Larz60+ Wrote: If you click on the url that you have, you get the following response:
Quote:The field SeasonType must match the regular expression '^(Regular Season)|(Pre Season)|(Playoffs)|(All Star)$'.; The StarterBench property is required.; The VsConference property is required.; The VsDivision property is required.
It works when you copy/paste. For some reason the hyperlink stops before the end of the URL.
RE: CSV output problem - Larz60+ - Oct-29-2017
Even so, requests for whatever reason freezes up and doesn't even timeout
I believe that the syntax that you have (for the json load) is correct, but I would still look at the errors that
are in the message above.
There is a different way to pass params, see: https://stackoverflow.com/questions/6386308/http-requests-and-json-parsing-in-python
and another here: https://stackoverflow.com/questions/17518937/saving-a-json-file-to-computer-python
RE: CSV output problem - chisox721 - Oct-29-2017
(Oct-29-2017, 05:08 AM)Larz60+ Wrote: Even so, requests for whatever reason freezes up and doesn't even timeout
I believe that the syntax that you have (for the json load) is correct, but I would still look at the errors that
are in the message above.
There is a different way to pass params, see: https://stackoverflow.com/questions/6386308/http-requests-and-json-parsing-in-python
and another here: https://stackoverflow.com/questions/17518937/saving-a-json-file-to-computer-python
Shot in the dark here but could there be a problem with python reading the plus sign (+) within the link?
RE: CSV output problem - chisox721 - Oct-30-2017
After altering the link I managed to get this response when running in PowerShell. Trying to figure out on my own how to fix this but haven't come up with anything.
PS C:\Users\Nick> python NBASample.py
Traceback (most recent call last):
File "NBASample.py", line 11, in <module>
csv_output.writerow(entries['resultSet'][0]['headers'])
KeyError: 0
import requests
import csv
url = "http://stats.nba.com/stats/leagueLeaders?LeagueID=00&PerMode=PerGame&Scope=S&Season=2017-18&SeasonType=Regular+Season&StatCategory=PTS"
data = requests.get(url)
entries = data.json()
with open('NBAFile.csv', 'wb') as f_output:
csv_output = csv.writer(f_output)
csv_output.writerow(entries['resultSet'][0]['headers'])
csv_output.writerows(entries['resultSet'][0]['rowSet'])