Code stops after 20min+ with no output

Code stops after 20min+ with no output - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html)
+--- Thread: Code stops after 20min+ with no output (/thread-25570.html)

Code stops after 20min+ with no output - JacobK - Apr-03-2020

Hi,

I'm trying to get the 'src' from 500 profile pictures on Transfermarkt. I've managed to get the URL:s for every player in a list, but when I'm iterating through them to get the 'src' for their profile picture, the code just goes on and on, for 20min+ and then stops, without any output.

List of players

Please help.

from bs4 import BeautifulSoup
import requests
import pandas as pd


playerID = []
playerImgSrc = []


result = []

for page in range(1, 21):

    r = requests.get("https://www.transfermarkt.com/spieler-statistik/wertvollstespieler/marktwertetop?land_id=0&ausrichtung=alle&spielerposition_id=alle&altersklasse=alle&jahrgang=0&kontinent_id=0&plus=1",
        params= {"page": page},
        headers= {"User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:74.0) Gecko/20100101 Firefox/74.0"}
    )
    soup = BeautifulSoup(r.content, "html.parser")

    links = soup.select('a.spielprofil_tooltip')
    
    for i in range(len(links)):
        playerID.append(links[i].get('id'))

    playerProfile = ["https://www.transfermarkt.com/josh-maja/profil/spieler/" + x for x in playerID]

    for p in playerProfile:
        html = requests.get(p).text
        soup = BeautifulSoup(html, "html.parser")
        
        link = soup.select('div.dataBild')

    for i in range(len(link)):
        playerImgSrc.append(link[i].get('src'))
print(playerImgSrc)

RE: Code stops after 20min+ with no output - Larz60+ - Apr-03-2020

line 33:
for i in range(len(link)):
shouldn't this be:
for i in range(len(links)):

or if not, lines 33-34 need to be indented