Python Forum

Full Version: cant loop through scraped site
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
hi there, I have the following scraping code which gets the information I want for the first game but when I try to loop through the page, to get the other game information on the same page, the script fails with ...

    
for data in datas:
TypeError: 'Element' object is not iterable
Any help appreciated,

TIA





from requests_html import HTML, HTMLSession

session = HTMLSession()
 
r = session.get("https://www.afl.com.au/fixture")
r.html.render()


data = r.html.find('section')[11]
#print(data.text)

#for data in datas:
date = data.find('.match-list__group-date', first=True).text
print(date)
lteam = data.find('.match-team__name')[0].text
ltscore = data.find('.match-scoreboard__score-total', first=True).text
rteam = data.find('.match-team__name')[1].text
rtscore = data.find('.match-scoreboard__score-total')[1].text
mtime = data.find('.match-scoreboard__status-label', first=True).text

print(lteam + '[' + ltscore + ']' + ' ' + 'VS' + ' ' + rteam + '[' + rtscore + ']' + ' ' + '(' + mtime + ')')
print()
Nowhere in your code is datas defined. Please show the code you actually have with the loop. Otherwise, we have to guess and that doesn't help anyone.
ok fair call, here is the code with the error.

from requests_html import HTML, HTMLSession

session = HTMLSession()
 
r = session.get("https://www.afl.com.au/fixture")
r.html.render()


datas = r.html.find('section')[11]
#print(data.text)

for data in datas:
    date = data.find('.match-list__group-date', first=True).text
    print(date)
    lteam = data.find('.match-team__name')[0].text
    ltscore = data.find('.match-scoreboard__score-total', first=True).text
    rteam = data.find('.match-team__name')[1].text
    rtscore = data.find('.match-scoreboard__score-total')[1].text
    mtime = data.find('.match-scoreboard__status-label', first=True).text
    
    print(lteam + '[' + ltscore + ']' + ' ' + 'VS' + ' ' + rteam + '[' + rtscore + ']' + ' ' + '(' + mtime + ')')
    print()
Why would you think datas should be iterable? From line 9 it's one item in a sequence and is likely to be an object representing whatever HTML element that is. You probably need to do a find on it to get the children, but check the API docs to confirm that.