Python Forum
Thread Rating:
  • 1 Vote(s) - 3 Average
  • 1
  • 2
  • 3
  • 4
  • 5
python code for scraping
#1
Would like to learn more about how to scrape data using openchargemap.org/site/develop/api
for the tinkering around.
Specifically, extract all the point data (with attributes) from their main database.
Have read through their docs / dev  and don't see a clear cut way to access this data.

Was given the following code on a forum to do this here, **note** i had to remove
https://  
from resp = requests.get('______ api.openchargemap.io/v2/poi/')
since clickable links are not allowed yet.

import requests

resp = requests.get('api.openchargemap.io/v2/poi/')
data = resp.json()

for d in data:
    id = d['ID']
    operator_id = d['OperatorID']
    lat = d['AddressInfo']['Latitude']
    lon = d['AddressInfo']['Longitude']
        print('{0} | {1} | {2} | {3}'.format(id, operator_id, lat, lon))  
which produces:
SyntaxError: multiple statements found while compiling a single statement
and when running just
import requests
get the following:
Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'
Downloaded and installed Python 3.6.1 (i was on 2.7 which i did not uninstall) using Windows 7 with Python 3.6.1 Shell
Any help would be appreciated.
Reply
#2
Welcome!
The requests module is not a module from the standard Python. You have to install it:
pip install requests
I don't see while statement in the snippets you are showing us. The error has nothing with your code.
"As they say in Mexico 'dosvidaniya'. That makes two vidaniyas."
https://freedns.afraid.org
Reply
#3
Quote:ModuleNotFoundError: No module named 'requests'

You most install Requests,
if you have mark on Add Python36 to Path and pip under installation of 36.
It work like this from cmd:
Microsoft Windows [Version 10.0.14393]
(c) 2016 Microsoft Corporation. Med enerett.

C:\Windows\System32>cd\
C:\>pip install requests
Collecting requests
  Using cached requests-2.13.0-py2.py3-none-any.whl
Installing collected packages: requests
Successfully installed requests-2.13.0

C:\>python
Python 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 07:18:10) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.__version__
'2.13.0'
>>> exit()
C:\> 
Reply
#4
(Mar-28-2017, 04:02 PM)wavic Wrote: Welcome!
The requests module is not a module from the standard Python. You have to install it:
pip install requests
I don't see while statement in the snippets you are showing us. The error has nothing with your code.
 Thanks for the reply!
pasted
pip install requests
in Python 3.6.1 Shell gives:
SyntaxError: invalid syntax
in cmd.exe
Requirement already satisfied <use --upgrade to upgrade>: requets in c:\users\super\appdata\local\enthought\canopy\user\lib\site-packages
Cleaning up...
I recognize this enthought/canopy location from when i did a python meetup a few years ago.
Should i uninstall enthought?
Reply
#5
Quote:Should i uninstall enthought?
You should remove it from Environment Variables(Path)
Make sure that ;C:\python36\;C:\python36\scripts is in Path,
or the path you have installed Python 36 to.
Restart.

The the default path that 36 use is stupid c:\users\<pc name>\appdata\local\programs\python\python36\ 
Just uninstall/reinstall if this is the path,and choose a better Path eg C:\python36.
Reply
#6
(Mar-28-2017, 04:42 PM)snippsat Wrote: Python 36
Environmental Variables path update was easy enough.

After downloading the Gzipped source tarball from https://www.python.org/downloads/release/python-360/
and unzipping several times
and creating C:\Python36
and pasting everything in there.
I double click the setup file (99kb) and Canopy opens and doesn't appear to be doing anything after "kernel died" and "restarting"
What am i missing?

Python 3.6.1 Shell opens fine but getting the same errors still
Reply
#7
Quote:After downloading the Gzipped source tarball from https://www.python.org/downloads/release/python-360/

You shall not use that file.
You choose executable installer and under install you choose Customize Installation(here you choose path C:\python36)
Look at this.
Reply
#8
(Mar-28-2017, 06:17 PM)snippsat Wrote:
Quote:After downloading the Gzipped source tarball from https://www.python.org/downloads/release/python-360/

You shall not use that file.
You choose executable installer and under install you choose Customize Installation(here you choose path C:\python36)
Look at this.

Okay, downloaded the executable installer  , did not see a way to change the default "stupid" path , just let it install there... now have a Python 3.6.0 Shell giving the same error
SyntaxError: invalid syntax
after inputing pip install requests
Reply
#9
(Mar-28-2017, 06:32 PM)sirgeo Wrote:
Error:
SyntaxError: invalid syntax
Python 3.6.0 Shell giving the same error
pip install requests gets executed on your system command line, not python interpreter

Quote:I recognize this enthought/canopy location from when i did a python meetup a few years ago.

Should i uninstall enthought?
I am not sure what enthought is, but this is the correct procedure. This "enthought python" looks to already have requests installed. If its been a few years ago and your not using it i would remove it.
Recommended Tutorials:
Reply
#10
(Mar-28-2017, 06:37 PM)metulburr Wrote: pip install requests gets executed on your system command line, not python interpreter
same error with system command line:
Requirement already satisfied <use --upgrade to upgrade>: requets in c:\users\pcname\appdata\local\enthought\canopy\user\lib\site-packages
Cleaning up...
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020