May-14-2022, 06:32 AM
Hi , I was trying to extract search results but has the error "AttributeError: 'list' object has no attribute 'text'[/output]"
Please suggest.
Error
Please suggest.
# import libraries import urllib.request from bs4 import BeautifulSoup from selenium import webdriver import time import pandas as pd # specify the url urlpage = 'https://RP&recordedDateRange=18000101%2C20220506&searchOcrText=false&searchType=quickSearch' print(urlpage) # run chrome webdriver from executable path of your choice driver=webdriver.Chrome(r'C:\Users\Desktop\webdrivers\chromedriver_win32\chromedriver.exe') # get web page driver.get(urlpage) # execute script to scroll down the page driver.execute_script("window.scrollTo(0, document.body.scrollHeight);var lenOfPage=document.body.scrollHeight;return lenOfPage;") # sleep for 30s time.sleep(30) # driver.quit() # find elements by xpath results = driver.find_elements_by_xpath("/html/body/div[2]/main/div[2]/div/div/article/div[3]/div/div[2]/div[1]/table/tbody/tr[1]/td[3]").text print('Number of results', len(results)) # create empty array to store data data = [] # loop over results for result in results: product_list = result.text # close driver driver.quit() # save to pandas dataframe df = pd.DataFrame(results) print(df) # write to csv df.to_csv(r'C:\Users\\Desktop\list.csv')
Error
Output:runfile('C:/Users/.spyder-py3/untitled1.py', wdir='C:/Users/.spyder-py3')
https:/results?department=RP&recordedDateRange=18000101%2C20220506&searchOcrText=false&searchType=quickSearch
C:\Users\.spyder-py3\untitled1.py:11: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
driver=webdriver.Chrome(r'C:\Users\Desktop\webdrivers\chromedriver_win32\chromedriver.exe')
C:\Users\.spyder-py3\untitled1.py:23: DeprecationWarning: find_elements_by_xpath is deprecated. Please use find_elements(by=By.XPATH, value=xpath) instead
results = driver.find_elements_by_xpath("/html/body/div[2]/main/div[2]/div/div/article/div[3]/div/div[2]/div[1]/table/tbody/tr[1]/td[3]").text
Traceback (most recent call last):
File "C:\Users\.spyder-py3\untitled1.py", line 23, in <module>
results = driver.find_elements_by_xpath("/html/body/div[2]/main/div[2]/div/div/article/div[3]/div/div[2]/div[1]/table/tbody/tr[1]/td[3]").text
AttributeError: 'list' object has no attribute 'text'