Python Forum
Loop through tags inside tags in Selenium/Python
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Loop through tags inside tags in Selenium/Python
#1
I am trying to use selenium to loop through a list of properties on a web page and return the property address and auction time. I have the following python code so far and html for the web page below.

I'm able to return the links to every property in the list, but can't seen to return the values I need from the "H4" tags. I think I'm doing something wrong with getting the elements by Xpath but I can't seem to figure it out.

Any help would be greatly appreciated!

HTML:
<div data-elem-id="asset_list_content">
	<a href="/details/123-memory-lane">
		<div data-elm-id="asset_2352111_address" class="styles__address-container--2l39p styles__u-mr-1--3qZyj">
			<h4 data-elm-id="asset_2352111_address_content_1" class="styles__asset-font-big--vQU7K">123 memory-lane</h4>
			<label data-elm-id="asset_2352111_address_content_2" class="styles__asset-font-small--2JgrX">POWDER SPRINGS, GA 30127, Cobb County</label>
		</div>
		<div class="styles__auction-container--45DZU styles__u-ml-1--34mF_">
			<h4 data-elm-id="asset_2352111_auction_date" class="styles__asset-font-big--vQU7K">Apr 04, 10:00am</h4>
		</div>
	</a>
	<a href="/details/456-memory-lane">
		<div data-elm-id="asset_8463157_address" class="styles__address-container--2l39p styles__u-mr-1--3qZyj">
			<h4 data-elm-id="asset_8463157_address_content_1" class="styles__asset-font-big--vQU7K">456 memory-lane</h4>
			<label data-elm-id="asset_8463157_address_content_2" class="styles__asset-font-small--2JgrX">POWDER SPRINGS, GA 30127, Cobb County</label>
		</div>
		<div class="styles__auction-container--45DZU styles__u-ml-1--34mF_">
			<h4 data-elm-id="asset_8463157_auction_date" class="styles__asset-font-big--vQU7K">March 10, 10:00am</h4>
		</div>
	</a>
</div>
Python (Selenium):
propertyList = browser.find_elements_by_xpath('//div[@data-elm-id="asset_list_content"]')

for element in propertyList:
    propertyLinks = element.find_elements_by_tag_name('a')
    for propertyLink in propertyLinks:
        propertyAddress = propertyLink.get_element_by_xpath('//h4[1]')
        propertyAuctionTime = propertyLink.get_element_by_xpath('//h4[2]')
        print(propertyAddress).text
        print(propertyAuctionTime).text

Output:
Output:
propertyAddress = propertyLink.get_element_by_xpath('//h4[1]') AttributeError: 'WebElement' object has no attribute 'get_element_by_xpath'
Reply
#2
What is the website URL?
It doesn't look like Selenium is needed here, and BeautifulSoup might be better.
If you post the URL, I'll take a look at it.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Variable definitions inside loop / could be better? gugarciap 2 363 Jan-09-2024, 11:11 PM
Last Post: deanhystad
  How to create a variable only for use inside the scope of a while loop? Radical 10 1,473 Nov-07-2023, 09:49 AM
Last Post: buran
  why doesn't it replace all html tags? Melcu54 3 678 Jul-05-2023, 04:47 AM
Last Post: Melcu54
  BeautifulSoup - I can't translate html tags that contain <a href=..</a> OR <em></em> Melcu54 10 1,547 Oct-27-2022, 08:58 AM
Last Post: wavic
  Help adding a loop inside a loop Extra 31 4,253 Oct-23-2022, 12:16 AM
Last Post: Extra
  How to find tags using specific text (timestamps) in a url? q988988 1 1,317 Mar-08-2022, 08:09 AM
Last Post: buran
  Get text from within h3 html tags Pedroski55 8 4,114 Jan-05-2022, 06:50 AM
Last Post: Larz60+
  Too many if statements working as "tags" zeek 3 1,913 Sep-20-2021, 05:22 PM
Last Post: deanhystad
  Parsing link from html tags with Python Melcu54 0 1,567 Jun-14-2021, 09:25 AM
Last Post: Melcu54
  Delimiters - How to skip some html tags from being translate Melcu54 0 1,610 May-26-2021, 06:21 AM
Last Post: Melcu54

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020