Python Forum

Full Version: Loop through tags inside tags in Selenium/Python
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
I am trying to use selenium to loop through a list of properties on a web page and return the property address and auction time. I have the following python code so far and html for the web page below.

I'm able to return the links to every property in the list, but can't seen to return the values I need from the "H4" tags. I think I'm doing something wrong with getting the elements by Xpath but I can't seem to figure it out.

Any help would be greatly appreciated!

HTML:
<div data-elem-id="asset_list_content">
	<a href="/details/123-memory-lane">
		<div data-elm-id="asset_2352111_address" class="styles__address-container--2l39p styles__u-mr-1--3qZyj">
			<h4 data-elm-id="asset_2352111_address_content_1" class="styles__asset-font-big--vQU7K">123 memory-lane</h4>
			<label data-elm-id="asset_2352111_address_content_2" class="styles__asset-font-small--2JgrX">POWDER SPRINGS, GA 30127, Cobb County</label>
		</div>
		<div class="styles__auction-container--45DZU styles__u-ml-1--34mF_">
			<h4 data-elm-id="asset_2352111_auction_date" class="styles__asset-font-big--vQU7K">Apr 04, 10:00am</h4>
		</div>
	</a>
	<a href="/details/456-memory-lane">
		<div data-elm-id="asset_8463157_address" class="styles__address-container--2l39p styles__u-mr-1--3qZyj">
			<h4 data-elm-id="asset_8463157_address_content_1" class="styles__asset-font-big--vQU7K">456 memory-lane</h4>
			<label data-elm-id="asset_8463157_address_content_2" class="styles__asset-font-small--2JgrX">POWDER SPRINGS, GA 30127, Cobb County</label>
		</div>
		<div class="styles__auction-container--45DZU styles__u-ml-1--34mF_">
			<h4 data-elm-id="asset_8463157_auction_date" class="styles__asset-font-big--vQU7K">March 10, 10:00am</h4>
		</div>
	</a>
</div>
Python (Selenium):
propertyList = browser.find_elements_by_xpath('//div[@data-elm-id="asset_list_content"]')

for element in propertyList:
    propertyLinks = element.find_elements_by_tag_name('a')
    for propertyLink in propertyLinks:
        propertyAddress = propertyLink.get_element_by_xpath('//h4[1]')
        propertyAuctionTime = propertyLink.get_element_by_xpath('//h4[2]')
        print(propertyAddress).text
        print(propertyAuctionTime).text

Output:
Output:
propertyAddress = propertyLink.get_element_by_xpath('//h4[1]') AttributeError: 'WebElement' object has no attribute 'get_element_by_xpath'
What is the website URL?
It doesn't look like Selenium is needed here, and BeautifulSoup might be better.
If you post the URL, I'll take a look at it.