Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
selenium wait for text question
#1
I am trying to wait for certain text to show up in the web page after I click on the 'Next' button.
if pageno == 1, the code works as expected.
when pageno != 1, I don't know how to wait for the <li tag's value to be equal to 'expected_text' which assures that
the page I am loading is fully loaded before continuing.
The page does indeed actually get loaded, but since I am only waiting for the ".pageinfo" css selector, which is already there from the previous page,
code continues too soon. How do I get it to wait for the text of <li tag to be equal to 'expected_text'?

code:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
from pathlib import Path
import os
import sys


class ByCSS_SELECTOR:
    def __init__(self) -> None:
        os.chdir(os.path.abspath(os.path.dirname(__file__)))
        self.HomePath = Path(".")

        self.NewHampshireBusinessListing = 'https://quickstart.sos.nh.gov/online/BusinessInquire/LandingPageBusinessSearch'
        self.browser = None
        self.browser_running = False
        self.page = None
    
    def find_page(self, pageno):
        if not self.browser_running:
            self.start_browser()

        if pageno == 1:
            self.browser.get(self.NewHampshireBusinessListing)
            try:
                elements = [my_elem.get_attribute("innerHTML") for my_elem in WebDriverWait(self.browser, 5). \
                    until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, ".pageinfo")))]
                print(f"found: {elements}")
                self.page = self.browser.page_source
            except TimeoutException:
                print("Query timed out")
        else:
            self.browser.find_element(By.CSS_SELECTOR, "li.next:nth-child(9) > a:nth-child(1)").click()
            try:
                expected_text = "Page {pageno} of"
                elements = [my_elem.get_attribute("innerHTML") for my_elem in WebDriverWait(self.browser, 5). \
                    until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, ".pageinfo")))]
                print(f"Length elements: {len(elements)}")
                print(f"elements: {elements}")
                self.page = self.browser.page_source
            except TimeoutException:
                print("Query timed out")

    def start_browser(self):
        caps = webdriver.DesiredCapabilities().FIREFOX
        caps["marionette"] = True
        self.browser = webdriver.Firefox(capabilities=caps)
        self.browser_running = True

    def stop_browser(self):
        self.browser.close()
        self.browser_running = False


def main():
    bcs = ByCSS_SELECTOR()
    bcs.start_browser()
    bcs.find_page(1)
    bcs.find_page(2)
    if bcs.browser_running:
        bcs.stop_browser()


if __name__ == '__main__':
    main()
Reply
#2
I am a beginner to Python and Selenium but if JavaScript is updating the page then you probably need to wait for a DocumentComplete event. I am not sure (I forget) what event fires at the beginning of the download that results in a DocumentComplete event but perhaps that will help you find what you need.
Reply
#3
The problem is that Selenium can't catch redirects via running script or AJAX, it doesn't wait for them to finish.
In addition, you can't catch them via readyState, it waits for a while, but will signal complete long before AJAX content is downloaded.
The most reliable way to know, is to check for the line where Next button is located.
It changes for each loaded page, which I know in advance, thus the expected_text = "Page {pageno} of" statement. (line 37)
This text will be inserted into the HTML after the page has been fetched, contained in the
<li class="pageinfo">
    Page 1 of 28581, records 1 to 25 of 714504
</li>
and therefore is what I want to wait for.
All i need to identify, is the 'Page n' part.
My question was how to wait specifically for that text value in the <li tag.
Reply
#4
BUMP
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Need Help with Selenium Explicit Wait gw1500se 6 2,897 Nov-28-2021, 09:44 PM
Last Post: Larz60+
  How to get specific TD text via Selenium? euras 3 8,775 May-14-2021, 05:12 PM
Last Post: snippsat
  Question about Webscrabbing with Selenium DasD 1 1,784 Jun-29-2020, 05:36 PM
Last Post: HarleyQuin
  Selenium extract id text xzozx 1 2,114 Jun-15-2020, 06:32 AM
Last Post: Larz60+
  Getting text using Selenium WiPi 10 4,815 Apr-27-2020, 08:58 AM
Last Post: WiPi
  question about using javascript on python selenium Kai 1 1,884 Apr-12-2020, 04:28 AM
Last Post: Larz60+
  drive wait in try windows11 0 1,821 Apr-07-2020, 01:50 PM
Last Post: windows11
  wait for element and click windows11 2 2,652 Mar-21-2020, 09:23 PM
Last Post: windows11
  Selenium returning web element instead of desired text newbie_programmer 1 5,186 Dec-11-2019, 06:37 AM
Last Post: Malt
  wait.until(EC.element_to_be_clickable) failed to click gahhon 4 7,986 Feb-23-2019, 04:58 AM
Last Post: gahhon

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020