Python Forum
Getting Index Error - list index out of range
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Getting Index Error - list index out of range
#1
Hi,
I am new to Python and doing a web scraping project and trying to web scrape the data from restaurant aggregator site (https://www.zomato.com/mumbai) using Jupyter notebook and want to scrape different types of restaurants from each neighborhood in the city but I am getting IndexError: list index out of range -->buttons[0].click().

Below is my code.

import pandas as pd
!pip install -U selenium
import requests
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.keys import Keys
import time
chrome_path = r'C:\Users\Klsingh\Desktop\chromedriver.exe'

# Url for each neighborhood
url = 'https://www.zomato.com/mumbai/lower-parel-restaurants' 
# neighborhood
city = "lower_parel_"
wd = webdriver.Chrome(executable_path=r'C:\Users\Klsingh\Desktop\chromedriver.exe')
wd.get(url)
rest_types = wd.find_element_by_xpath('//div[@class="search-filter-label pb5"]')
rest_types = rest_types.text.split("\n")
print(rest_types)
wd.quit()
# This function returns all the buttons for navigating each category
def rest_types_buttons():
    buttons = wd.find_elements_by_xpath('//span[@class="zred"]')
    return buttons
# This function returns the link, name and address of restaurant for each page
def name_link_add():
    rest_link = []
    rest_name = []
    restaurants = wd.find_elements_by_xpath('//a[@class="result-title hover_feedback zred bold ln24   fontsize0 "]')
    for name in restaurants:
        rest_link.append(name.get_attribute('href'))
        rest_name.append(name.text)
    restaurants_address =  wd.find_elements_by_xpath('//div[@class="col-m-16 search-result-address grey-text nowrap ln22"]')
    rest_address = []
    for rest_add in restaurants_address:
        rest_address.append(rest_add.text)
    return rest_link, rest_name, rest_address
# This function returns the all the data from an individual category (all pages combined)
def get_data_rest_type(rest_type):
    try:
        prev_link,prev_name,prev_add = None, None, None
        rest_link, rest_name, rest_address = name_link_add()
        link = []
        name = []
        address = []
        while(prev_link != rest_link):
            prev_link,prev_name,prev_add = rest_link, rest_name, rest_address
            link = link + prev_link
            name = name + prev_name
            address = address + prev_add
            next_page_button = wd.find_element_by_xpath('//i[@class="right angle icon"]')
            next_page_button.click()
            wd.switch_to_window(wd.window_handles[0])
            rest_link, rest_name, rest_address = name_link_add()
            # Below two if conditions are for debugging
            if((len(rest_address) == len(rest_link) == len(rest_name)) == False):
                print("need to see, name link address mismatch")
                break
            if(len(rest_name) == 0):
                print("Empty found")
    except:
        print("unknown error")
    return link, name, address
# For each of the category
type_ = ("_".join(rest_types[0].lower().split(' ')))
print(type_)
# Go inside the category by clicking on the button on the main page
wd = webdriver.Chrome(chrome_path)
wd.get(url)
wd.switch_to_window(wd.window_handles[0])
buttons = rest_types_buttons()
buttons[0].click()
wd.switch_to_window(wd.window_handles[0])
This portion of the code
Quote:# Go inside the category by clicking on the button on the main page
is giving me Index error as mentioned below.

Error:
C:\Users\Klsingh\Anaconda3\lib\site-packages\ipykernel_launcher.py:4: DeprecationWarning: use driver.switch_to.window instead after removing the cwd from sys.path. --------------------------------------------------------------------------- IndexError Traceback (most recent call last) <ipython-input-12-422a6bba8955> in <module> 4 wd.switch_to_window(wd.window_handles[0]) 5 button = rest_types_button() ----> 6 button[0].click() 7 driver.switch_to_window(wd.window_handles[0]) IndexError: list index out of range
Please help.
Thank you all in advance.
Reply


Messages In This Thread
Getting Index Error - list index out of range - by RahulSingh - Feb-03-2020, 06:27 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Grouping in pandas/multi-index data frame Aleqsie 3 607 Jan-06-2024, 03:55 PM
Last Post: deanhystad
  Merging rows and adding columns based on matching index pythonnewbie78 3 750 Dec-24-2023, 11:51 AM
Last Post: Pedroski55
  Index out of range error standenman 0 1,040 May-22-2023, 10:35 PM
Last Post: standenman
  multi index issue of one hot encoder preprocessing aupres 0 1,057 Jun-10-2022, 11:23 AM
Last Post: aupres
  IndexError: invalid index to scalar variable. jyotib2610 3 2,994 Mar-10-2022, 09:55 AM
Last Post: jyotib2610
  Inverted Index thewal 0 1,282 Feb-10-2022, 03:54 PM
Last Post: thewal
  IndexError: index 0 is out of bounds for axis 0 with size 0 atomxkai 2 5,381 Mar-03-2021, 08:26 AM
Last Post: atomxkai
  [split] Getting Index Error - list index out of range krishna 2 2,567 Jan-09-2021, 08:29 AM
Last Post: buran
  Interpolating DataFrame method=‘index’ help tlewick1 1 1,822 Oct-22-2020, 12:48 AM
Last Post: scidam
  Cycle through Numpy range within another range(?) Zero01 0 1,994 Jul-31-2020, 02:37 PM
Last Post: Zero01

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020