Python Forum
Scrapy Javascript Pagination (next_page)
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Scrapy Javascript Pagination (next_page)
#1
Dear Members,

I am trying to scrap the first table in all pages from the following website: https://www.secform4.com/insider-trading/320193.htm. The website uses Javascript to do the pagination. I know I need to use Scrapy Splash for the pagination but do not know how to code it. With my codes, I get the data from the first page only. It means my next_page command does not work. Could you please help me to do the pagination using Scrapy Splash?
Reply
#2
what have you tried so far?
Please show code
Reply
#3
Dear Larz60+

I find a solution on Youtube. Thank you for your help.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  BeautifulSoup pagination using href rhat398 1 2,354 Jun-30-2021, 10:55 AM
Last Post: snippsat
  Python beautifulsoup pagination error The61 5 3,401 Apr-09-2020, 09:17 PM
Last Post: Larz60+
  Pagination prejni 2 2,361 Nov-18-2019, 10:45 AM
Last Post: alekson
  the next_page command using Scrapy Splash is not working nazmulfinance 0 2,043 Nov-16-2019, 03:47 PM
Last Post: nazmulfinance
  pagination for non standarded pages zarize 12 5,898 Sep-02-2019, 12:35 PM
Last Post: zarize
  Python - Scrapy Javascript Pagination (next_page) Baggelhsk95 3 9,919 Oct-08-2018, 01:20 PM
Last Post: stranac
  Filtering and pagination garynobles 0 34,311 Jun-14-2018, 08:11 PM
Last Post: garynobles
  BeautifulSoup and pagination. Mike Ru 1 7,849 Sep-22-2017, 10:15 AM
Last Post: snippsat
  Scrapy-cut: Advanced Cookiecutter Scrapy Templating scriptso 2 4,612 Feb-02-2017, 07:57 PM
Last Post: scriptso

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020