Mar-05-2020, 06:09 PM
Hi everyone, I'm fairly new to Python and brand new to web scraping. any help from the forum is greatly appreciated
I have some code here that gets me what I want from page 1 of my website.
Then i would like to summarize all of the outputs together.
How can I do so? Thank you very much

I have some code here that gets me what I want from page 1 of my website.
response = requests.get(url) response print(response.text) soup = BeautifulSoup(r.text, 'html.parser') results = soup.find(id='preblockBody') results print(results.prettify()) job_elems = results.find_all('table', class_='pbListingTable') for job_elem in job_elems: title_elem2 = job_elem.find_all('tr', class_='pbListingTable1') for pbListingTable1 in job_elem.find_all('tr', {'class':'pbListingTable1'}): print(pbListingTable1.text) title_elem = job_elem.find_all('tr', class_='pbListingTable0') for pbListingTable0 in job_elem.find_all('tr', {'class':'pbListingTable0'}): print(pbListingTable0.text)Next, I would like to go to page 2 and do the same thing then page 3 and so on. I'm unsure how to do so because the inspection indicates
Quote:<a href="javascript:gotoNextPage(2)"> 2 </a>
Then i would like to summarize all of the outputs together.
How can I do so? Thank you very much