Can test attribute of button,in BS can do this with
Example i would use
.attrs
Example i would use
.get()
as .attrs
return a dictionary,then can add as default value eg more pages.import requests from bs4 import BeautifulSoup import time # First url = "https://www.homeadvisor.com/c.Garage-Garage-Doors.Atlanta.GA.-12036.html" # Last #url = "https://www.homeadvisor.com/c.Garage-Garage-Doors.Atlanta.GA.-12036.html?startingIndex=50" headers = { "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)\ AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.75 Safari/537.36" } response = requests.get(url, headers=headers) soup = BeautifulSoup(response.content, "lxml") button = soup.select('button.page-next.\@px-1.\@ml-1')Test
# First page,or all pages before last page will return this >>> button[0].attrs.get('disabled', 'More pages') 'More pages' >>> # Last page >>> button[0].attrs.get('disabled', 'More pages') 'disabled'