Python Forum

Full Version: Python beautifulsoup pagination error
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
I want to make a pagination but page is not changed
import requests 
from bs4 import BeautifulSoup as bs
from colorama import init 

init(autoreset=True) 

#Header parametreleri
header_param = {"User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.162 Safari/537.36"}

#Site Urlsi
url = "https://eksisozluk.com/"

#Aranacak entry 
word = input("Aranacak kelime:")


r = requests.get(url+word,headers=header_param)

soup = bs(r.content,"lxml")

#Aranan konuya ait sayfa sayisini almak 
pages = soup.find("div",attrs={"class":"pager"}).get("data-pagecount")

#Sayfa sayisi 
num  = 1
sayfa = "?p="

for num in range(1,int(pages)+1):
	#Her sayfa icin ayri bir istek gonderiyoruz
	pageRequest = requests.get("https://eksisozluk.com/"+word+"?p="+str(num),headers=header_param)
	
	pageSource = bs(pageRequest.content,"lxml")
	
	print(pageRequest.url)
	#Entrylerin bulundugu ul tagindaki tum li elementlerini aliyoruz
	
	entryler = pageSource.find(id="entry-item-list").find_all("li")
	
	for entry in entryler:
		#Entry paylasan kisi
		name = entry.find(class_='entry-author').get_text(strip=True)
		#Entry icerigi
		content = entry.find(class_='content').get_text(strip=True)
		#Entry Paylasilma Zamani
		publish = entry.find(class_='entry-date').get_text(strip=True)
		
		#Bilgileri Ekrana Yaziyoruz
		print(name,content,publish,sep="\n")
		print("\n")
		
		
		
what would your search word be?
comment out line 30, then
after line 29, add:
    newurl = f"https://eksisozluk.com/{word}?p={str(num)}"
    print(newurl)
    pageRequest = requests.get(newurl, headers=header_param)
what do you get?
does the newurl look proper?
(Apr-08-2020, 03:06 AM)Larz60+ Wrote: [ -> ]what would your search word be?
comment out line 30, then
after line 29, add:
    newurl = f"https://eksisozluk.com/{word}?p={str(num)}"
    print(newurl)
    pageRequest = requests.get(newurl, headers=header_param)
what do you get?
does the newurl look proper?

Page number is changed but page content is not changed
what would your search word be?
(Apr-08-2020, 09:29 AM)Larz60+ Wrote: [ -> ]what would your search word be?
Thank you i solve the problem
Please share how solved with the forum, so others may benefit.