Python Forum
Thread Rating:
  • 2 Vote(s) - 3 Average
  • 1
  • 2
  • 3
  • 4
  • 5
BeautifulSoup and pagination.
#2
Can do it in a loop with string formatting.
import time

for page in range(1,5):
    time.sleep(2)
    url = 'https://muzofond.org/search/e%20mantra/{}'.format(page)
    print(url)
Output:
https://muzofond.org/search/e%20mantra/1 https://muzofond.org/search/e%20mantra/2 https://muzofond.org/search/e%20mantra/3 https://muzofond.org/search/e%20mantra/4
If something block when downloading more than one page,
then it's fine task to train on launching parallel tasks.
Reply


Messages In This Thread
BeautifulSoup and pagination. - by Mike Ru - Sep-22-2017, 08:43 AM
RE: BeautifulSoup and pagination. - by snippsat - Sep-22-2017, 10:15 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  BeautifulSoup pagination using href rhat398 1 2,438 Jun-30-2021, 10:55 AM
Last Post: snippsat
  Python beautifulsoup pagination error The61 5 3,515 Apr-09-2020, 09:17 PM
Last Post: Larz60+
  Pagination prejni 2 2,423 Nov-18-2019, 10:45 AM
Last Post: alekson
  Scrapy Javascript Pagination (next_page) nazmulfinance 2 3,063 Nov-18-2019, 01:01 AM
Last Post: nazmulfinance
  pagination for non standarded pages zarize 12 6,073 Sep-02-2019, 12:35 PM
Last Post: zarize
  Python - Scrapy Javascript Pagination (next_page) Baggelhsk95 3 10,045 Oct-08-2018, 01:20 PM
Last Post: stranac
  Filtering and pagination garynobles 0 38,339 Jun-14-2018, 08:11 PM
Last Post: garynobles

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020