cant loop through scraped site

cant loop through scraped site - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html)
+--- Thread: cant loop through scraped site (/thread-28967.html)

cant loop through scraped site - matt42 - Aug-12-2020

hi there, I have the following scraping code which gets the information I want for the first game but when I try to loop through the page, to get the other game information on the same page, the script fails with ...

    
for data in datas:
TypeError: 'Element' object is not iterable

Any help appreciated,

TIA

from requests_html import HTML, HTMLSession

session = HTMLSession()
 
r = session.get("https://www.afl.com.au/fixture")
r.html.render()


data = r.html.find('section')[11]
#print(data.text)

#for data in datas:
date = data.find('.match-list__group-date', first=True).text
print(date)
lteam = data.find('.match-team__name')[0].text
ltscore = data.find('.match-scoreboard__score-total', first=True).text
rteam = data.find('.match-team__name')[1].text
rtscore = data.find('.match-scoreboard__score-total')[1].text
mtime = data.find('.match-scoreboard__status-label', first=True).text

print(lteam + '[' + ltscore + ']' + ' ' + 'VS' + ' ' + rteam + '[' + rtscore + ']' + ' ' + '(' + mtime + ')')
print()

RE: cant loop through scraped site - ndc85430 - Aug-12-2020

Nowhere in your code is datas defined. Please show the code you actually have with the loop. Otherwise, we have to guess and that doesn't help anyone.

RE: cant loop through scraped site - matt42 - Aug-12-2020

ok fair call, here is the code with the error.

from requests_html import HTML, HTMLSession

session = HTMLSession()
 
r = session.get("https://www.afl.com.au/fixture")
r.html.render()


datas = r.html.find('section')[11]
#print(data.text)

for data in datas:
    date = data.find('.match-list__group-date', first=True).text
    print(date)
    lteam = data.find('.match-team__name')[0].text
    ltscore = data.find('.match-scoreboard__score-total', first=True).text
    rteam = data.find('.match-team__name')[1].text
    rtscore = data.find('.match-scoreboard__score-total')[1].text
    mtime = data.find('.match-scoreboard__status-label', first=True).text
    
    print(lteam + '[' + ltscore + ']' + ' ' + 'VS' + ' ' + rteam + '[' + rtscore + ']' + ' ' + '(' + mtime + ')')
    print()

RE: cant loop through scraped site - ndc85430 - Aug-12-2020

Why would you think datas should be iterable? From line 9 it's one item in a sequence and is likely to be an object representing whatever HTML element that is. You probably need to do a find on it to get the children, but check the API docs to confirm that.