Python Forum

Full Version: error zomato scraping data
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
hey, i just learned about zomato data scraping. then I practice using one of the codes on github.

import requests
import csv

with open("output_restaurant.csv", "a", newline='') as fp:
    wr = csv.writer(fp, dialect='excel')
    url = "https://developers.zomato.com/api/v2.1/search?entity_id=4&entity_type=city&q=Bangalore&start=80"
    header = {"Accept": "application/json", "user-key": "API_KEY", "User-agent": "curl/7.43.0"}
    resp = requests.get(url, headers=header).json()
    for i in range(0, 20):
        rest = resp['restaurants'][i]
        res_id = rest['restaurant']['id']
        name = rest['restaurant']['name']
        locality = rest['restaurant']['location']['locality']
        cuisines = rest['restaurant']['cuisines']
        average_cost_for_two = rest['restaurant']['average_cost_for_two']
        rating = rest['restaurant']['user_rating']['aggregate_rating']
        votes = rest['restaurant']['user_rating']['votes']
        list_ = [res_id, name, locality, cuisines, average_cost_for_two, rating, votes]
        wr.writerow(list_)
when run the first time an error like this appears
Output:
Traceback (most recent call last): File "Zomato_API.py", line 10, in <module> rest = resp['restaurants'][i] KeyError: 'restaurants'
then some time the error changes to
Output:
Traceback (most recent call last): File "Zomato_API.py", line 10, in <module> rest = resp['restaurants'][i] IndexError: list index out of range
from the error, what is the way to fix it?
You could replace the for i in range... loop by
if 'restaurants' in resp:
    for i, res in enumerate(resp['restaurants']):
        ...
(Jun-22-2020, 08:09 AM)Gribouillis Wrote: [ -> ]You could replace the for i in range... loop by
if 'restaurants' in resp:
    for i, res in enumerate(resp['restaurants']):
        ...

i try this and problem solved, thanks bro.
can you explain why 'for i in range' error in this code ?
Because the range uses all the integers 0, 1, ..., 19. If the list rest['restaurants'] has less than 20 elements, the lookup will fail.