Python Forum

Full Version: Suggestion request for scrapping html table
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hi,

I am struggling to scrap a html table.
When the date is selected if there is data, the table will be displayed, i am scrapping it using the xpath
table I'd and no issues with this.

But if the selected date has no data, no table will be displayed. Here my code stops and waiting for the table to appear.

How to specify timeout and continue without exceptions.
Please show latest code (working or not).
----snip-----
driver.get("https://website/list.php");

num_rows = 0
num_cols = 0
num_rows = len(driver.find_elements(By.XPATH,'/html/body/form/div[4]/div/table/tbody/tr'))
num_cols = len(driver.find_elements(By.XPATH, '//*[@id="gv_output"]/tbody/tr[2]/td'))
print("Num rows is {0} , num_col is {1}" .format(num_rows, num_cols))
table_output_reading = []
if num_rows > 1:
    for i in range (2,num_rows):
        table_zone = driver.find_element(By.XPATH, "/html/body/form/div[4]/div/table/tbody/tr["+str(i)+"]/td[3]").text
        table_product = driver.find_element(By.XPATH, "/html/body/form/div[4]/div/table/tbody/tr["+str(i)+"]/td[4]").text
        table_price = driver.find_element(By.XPATH, "/html/body/form/div[4]/div/table/tbody/tr["+str(i)+"]/td[5]").text
        table_output_reading.append(str(table_zone)+str(table_product)+str(table_price))
        print("table lists is", table_output_reading)

--snip----
This works fine if there is a table listed.

Suppose when i select another date, and on that date if there is no data no html table is displayed.
and this script here waits endlessly for the table
i tried giving num_rows >1 not working
The table id is gv_output

I want this script to run only if table is present

thanks in advance
What do the error traceback look like?