Apr-03-2020, 03:14 PM
Hi,
I'm trying to get the 'src' from 500 profile pictures on Transfermarkt. I've managed to get the URL:s for every player in a list, but when I'm iterating through them to get the 'src' for their profile picture, the code just goes on and on, for 20min+ and then stops, without any output.
List of players
Please help.
I'm trying to get the 'src' from 500 profile pictures on Transfermarkt. I've managed to get the URL:s for every player in a list, but when I'm iterating through them to get the 'src' for their profile picture, the code just goes on and on, for 20min+ and then stops, without any output.
List of players
Please help.
from bs4 import BeautifulSoup import requests import pandas as pd playerID = [] playerImgSrc = [] result = [] for page in range(1, 21): r = requests.get("https://www.transfermarkt.com/spieler-statistik/wertvollstespieler/marktwertetop?land_id=0&ausrichtung=alle&spielerposition_id=alle&altersklasse=alle&jahrgang=0&kontinent_id=0&plus=1", params= {"page": page}, headers= {"User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:74.0) Gecko/20100101 Firefox/74.0"} ) soup = BeautifulSoup(r.content, "html.parser") links = soup.select('a.spielprofil_tooltip') for i in range(len(links)): playerID.append(links[i].get('id')) playerProfile = ["https://www.transfermarkt.com/josh-maja/profil/spieler/" + x for x in playerID] for p in playerProfile: html = requests.get(p).text soup = BeautifulSoup(html, "html.parser") link = soup.select('div.dataBild') for i in range(len(link)): playerImgSrc.append(link[i].get('src')) print(playerImgSrc)