python code for scraping - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: General Coding Help (https://python-forum.io/forum-8.html) +--- Thread: python code for scraping (/thread-2609.html) Pages:
1
2
|
python code for scraping - sirgeo - Mar-28-2017 Would like to learn more about how to scrape data using openchargemap.org/site/develop/api for the tinkering around. Specifically, extract all the point data (with attributes) from their main database. Have read through their docs / dev and don't see a clear cut way to access this data. Was given the following code on a forum to do this here, **note** i had to remove https:// from resp = requests.get('______ api.openchargemap.io/v2/poi/') since clickable links are not allowed yet. import requests resp = requests.get('api.openchargemap.io/v2/poi/') data = resp.json() for d in data: id = d['ID'] operator_id = d['OperatorID'] lat = d['AddressInfo']['Latitude'] lon = d['AddressInfo']['Longitude'] print('{0} | {1} | {2} | {3}'.format(id, operator_id, lat, lon))which produces: SyntaxError: multiple statements found while compiling a single statementand when running just import requestsget the following: Traceback (most recent call last): File "<pyshell#0>", line 1, in <module> import requests ModuleNotFoundError: No module named 'requests'Downloaded and installed Python 3.6.1 (i was on 2.7 which i did not uninstall) using Windows 7 with Python 3.6.1 Shell Any help would be appreciated. RE: python code for scraping - wavic - Mar-28-2017 Welcome! The requests module is not a module from the standard Python. You have to install it: pip install requestsI don't see while statement in the snippets you are showing us. The error has nothing with your code. RE: python code for scraping - snippsat - Mar-28-2017 Quote:ModuleNotFoundError: No module named 'requests' You most install Requests, if you have mark on Add Python36 to Path and pip under installation of 36.It work like this from cmd: Microsoft Windows [Version 10.0.14393] (c) 2016 Microsoft Corporation. Med enerett. C:\Windows\System32>cd\ C:\>pip install requests Collecting requests Using cached requests-2.13.0-py2.py3-none-any.whl Installing collected packages: requests Successfully installed requests-2.13.0 C:\>python Python 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 07:18:10) [MSC v.1900 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import requests >>> requests.__version__ '2.13.0' >>> exit() C:\> RE: python code for scraping - sirgeo - Mar-28-2017 (Mar-28-2017, 04:02 PM)wavic Wrote: Welcome!Thanks for the reply! pasted pip install requestsin Python 3.6.1 Shell gives: SyntaxError: invalid syntaxin cmd.exe Requirement already satisfied <use --upgrade to upgrade>: requets in c:\users\super\appdata\local\enthought\canopy\user\lib\site-packages Cleaning up...I recognize this enthought/canopy location from when i did a python meetup a few years ago. Should i uninstall enthought? RE: python code for scraping - snippsat - Mar-28-2017 Quote:Should i uninstall enthought?You should remove it from Environment Variables(Path) Make sure that ;C:\python36\;C:\python36\scripts is in Path,or the path you have installed Python 36 to. Restart. The the default path that 36 use is stupid c:\users\<pc name>\appdata\local\programs\python\python36\ Just uninstall/reinstall if this is the path,and choose a better Path eg C:\python36 .
RE: python code for scraping - sirgeo - Mar-28-2017 (Mar-28-2017, 04:42 PM)snippsat Wrote: Python 36Environmental Variables path update was easy enough. After downloading the Gzipped source tarball from https://www.python.org/downloads/release/python-360/ and unzipping several times and creating C:\Python36 and pasting everything in there. I double click the setup file (99kb) and Canopy opens and doesn't appear to be doing anything after "kernel died" and "restarting" What am i missing? Python 3.6.1 Shell opens fine but getting the same errors still RE: python code for scraping - snippsat - Mar-28-2017 Quote:After downloading the Gzipped source tarball from https://www.python.org/downloads/release/python-360/ You shall not use that file. You choose executable installer and under install you choose Customize Installation(here you choose path C:\python36)Look at this. RE: python code for scraping - sirgeo - Mar-28-2017 (Mar-28-2017, 06:17 PM)snippsat Wrote:Quote:After downloading the Gzipped source tarball from https://www.python.org/downloads/release/python-360/ Okay, downloaded the executable installer , did not see a way to change the default "stupid" path , just let it install there... now have a Python 3.6.0 Shell giving the same errorSyntaxError: invalid syntaxafter inputing pip install requests RE: python code for scraping - metulburr - Mar-28-2017 (Mar-28-2017, 06:32 PM)sirgeo Wrote:Python 3.6.0 Shell giving the same error pip install requests gets executed on your system command line, not python interpreterQuote:I recognize this enthought/canopy location from when i did a python meetup a few years ago.I am not sure what enthought is, but this is the correct procedure. This "enthought python" looks to already have requests installed. If its been a few years ago and your not using it i would remove it. RE: python code for scraping - sirgeo - Mar-28-2017 (Mar-28-2017, 06:37 PM)metulburr Wrote: pip install requests gets executed on your system command line, not python interpretersame error with system command line: Requirement already satisfied <use --upgrade to upgrade>: requets in c:\users\pcname\appdata\local\enthought\canopy\user\lib\site-packages Cleaning up... |