Jun-30-2020, 08:14 PM
First you need to find out how many pages there is and then you can jump through individual pages using the
_p
url query parameter number_of_pages = 200 # just and example number url = "https://abom.learningbuilder.com/public/membersearch?model.FirstName=&model.LastName=&model.UniqueId=&model.City=&model.State=&performSearch=true&_p={page_number}&_s=20&_o=LastName&_d=" for page_number in range(number_of_pages): soup = make_soup(url.format(page_number=page_number)) for record in soup.findAll('tr'): mydata = "" for data in soup.findAll('td'): mydata = mydata + "," + data.text mydata_saved = mydata_saved + "\n" + mydata[1:] print(mydata_saved)