Python Forum
How to handle tables splitted across multiple web pages
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How to handle tables splitted across multiple web pages
#1
Hi
I am trying to print value from a table using the following code

row = browser.find_element_by_xpath("//table[@cols='8']/tbody/tr[last()]/td[2]/div").text
print(row)

But the table is splitted across multiple web pages and pagination is being used to access various parts of table. So the above code is returning the text in last row of table on first web page, not the actual last row of table.. How to access actual last row of table
Reply
#2
Xpath is different for every part of every page, so it varies completely from page to page.
You need specific Xpath for page in question.
Please let us know URL, and what part of page you are targeting.

Also, learn how to use element inspector which is available on Firefox and chrome.
basically hover over area of page that you want Xpath for, right click, select inspect element,
move cursor until exact target is highlighted, Then right click on highlighted html code in inspector, select copy ==> Xpath
Reply
#3
Thanks for your reply. It is working now after I used sleep() function.. The structure of table is same on all pages. Except that it needs to wait for a few seconds before the last page is loaded.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Scrape table from multiple pages Nhattanktnn 1 860 Jun-07-2023, 09:35 AM
Last Post: Larz60+
  Web scrap multiple pages anilacem_302 3 3,828 Jul-01-2020, 07:50 PM
Last Post: mlieqo
  scraping multiple pages from table bandar 1 2,687 Jun-27-2020, 10:43 PM
Last Post: Larz60+
  Beginner help - Leap Year Issue Feb 29 and multiple pages warriordazza 3 2,701 May-10-2020, 01:14 AM
Last Post: warriordazza
  Scraping Multiple Pages mbadatanut 1 4,220 May-08-2020, 02:30 AM
Last Post: Larz60+
  Looping through multiple pages with changing url Qaruri 2 2,576 Jan-17-2020, 01:55 PM
Last Post: Qaruri
  scraping multiple pages of a website. Blue Dog 14 22,402 Jun-21-2018, 09:03 PM
Last Post: Blue Dog

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020