Python Forum

Full Version: python code for scraping
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2
Would like to learn more about how to scrape data using openchargemap.org/site/develop/api
for the tinkering around.
Specifically, extract all the point data (with attributes) from their main database.
Have read through their docs / dev  and don't see a clear cut way to access this data.

Was given the following code on a forum to do this here, **note** i had to remove
https://  
from resp = requests.get('______ api.openchargemap.io/v2/poi/')
since clickable links are not allowed yet.

import requests

resp = requests.get('api.openchargemap.io/v2/poi/')
data = resp.json()

for d in data:
    id = d['ID']
    operator_id = d['OperatorID']
    lat = d['AddressInfo']['Latitude']
    lon = d['AddressInfo']['Longitude']
        print('{0} | {1} | {2} | {3}'.format(id, operator_id, lat, lon))  
which produces:
SyntaxError: multiple statements found while compiling a single statement
and when running just
import requests
get the following:
Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'
Downloaded and installed Python 3.6.1 (i was on 2.7 which i did not uninstall) using Windows 7 with Python 3.6.1 Shell
Any help would be appreciated.
Welcome!
The requests module is not a module from the standard Python. You have to install it:
pip install requests
I don't see while statement in the snippets you are showing us. The error has nothing with your code.
Quote:ModuleNotFoundError: No module named 'requests'

You most install Requests,
if you have mark on Add Python36 to Path and pip under installation of 36.
It work like this from cmd:
Microsoft Windows [Version 10.0.14393]
(c) 2016 Microsoft Corporation. Med enerett.

C:\Windows\System32>cd\
C:\>pip install requests
Collecting requests
  Using cached requests-2.13.0-py2.py3-none-any.whl
Installing collected packages: requests
Successfully installed requests-2.13.0

C:\>python
Python 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 07:18:10) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.__version__
'2.13.0'
>>> exit()
C:\> 
(Mar-28-2017, 04:02 PM)wavic Wrote: [ -> ]Welcome!
The requests module is not a module from the standard Python. You have to install it:
pip install requests
I don't see while statement in the snippets you are showing us. The error has nothing with your code.
 Thanks for the reply!
pasted
pip install requests
in Python 3.6.1 Shell gives:
SyntaxError: invalid syntax
in cmd.exe
Requirement already satisfied <use --upgrade to upgrade>: requets in c:\users\super\appdata\local\enthought\canopy\user\lib\site-packages
Cleaning up...
I recognize this enthought/canopy location from when i did a python meetup a few years ago.
Should i uninstall enthought?
Quote:Should i uninstall enthought?
You should remove it from Environment Variables(Path)
Make sure that ;C:\python36\;C:\python36\scripts is in Path,
or the path you have installed Python 36 to.
Restart.

The the default path that 36 use is stupid c:\users\<pc name>\appdata\local\programs\python\python36\ 
Just uninstall/reinstall if this is the path,and choose a better Path eg C:\python36.
(Mar-28-2017, 04:42 PM)snippsat Wrote: [ -> ]Python 36
Environmental Variables path update was easy enough.

After downloading the Gzipped source tarball from https://www.python.org/downloads/release/python-360/
and unzipping several times
and creating C:\Python36
and pasting everything in there.
I double click the setup file (99kb) and Canopy opens and doesn't appear to be doing anything after "kernel died" and "restarting"
What am i missing?

Python 3.6.1 Shell opens fine but getting the same errors still
Quote:After downloading the Gzipped source tarball from https://www.python.org/downloads/release/python-360/

You shall not use that file.
You choose executable installer and under install you choose Customize Installation(here you choose path C:\python36)
Look at this.
(Mar-28-2017, 06:17 PM)snippsat Wrote: [ -> ]
Quote:After downloading the Gzipped source tarball from https://www.python.org/downloads/release/python-360/

You shall not use that file.
You choose executable installer and under install you choose Customize Installation(here you choose path C:\python36)
Look at this.

Okay, downloaded the executable installer  , did not see a way to change the default "stupid" path , just let it install there... now have a Python 3.6.0 Shell giving the same error
SyntaxError: invalid syntax
after inputing pip install requests
(Mar-28-2017, 06:32 PM)sirgeo Wrote: [ -> ]
Error:
SyntaxError: invalid syntax
Python 3.6.0 Shell giving the same error
pip install requests gets executed on your system command line, not python interpreter

Quote:I recognize this enthought/canopy location from when i did a python meetup a few years ago.

Should i uninstall enthought?
I am not sure what enthought is, but this is the correct procedure. This "enthought python" looks to already have requests installed. If its been a few years ago and your not using it i would remove it.
(Mar-28-2017, 06:37 PM)metulburr Wrote: [ -> ]pip install requests gets executed on your system command line, not python interpreter
same error with system command line:
Requirement already satisfied <use --upgrade to upgrade>: requets in c:\users\pcname\appdata\local\enthought\canopy\user\lib\site-packages
Cleaning up...
Pages: 1 2