Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Unexpected output
#1
Hi!

I'm having problems with my code. I've managed to scrape a player ID for 500 football players, which I then want to add to a URL and save in a list. It should look something like this: https://www.transfermarkt.com/chaher-idj...ler/342228

The problem is that my loop adds the same player ID, 500 times, and creating 500 identical URL:s. What am I doing wrong?

from bs4 import BeautifulSoup
import requests
import pandas as pd

playerID = []
playerImage = []
playerProfile = []
result = []

for page in range(1, 21):

    r = requests.get("https://www.transfermarkt.com/spieler-statistik/wertvollstespieler/marktwertetop? land_id=0&ausrichtung=alle&spielerposition_id=alle&altersklasse=alle&jahrgang=0&kontinent_id=0&plus=1",
        params= {"page": page},
        headers= {"User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:74.0) Gecko/20100101 Firefox/74.0"}
    )
    soup = BeautifulSoup(r.content, "html.parser")

    links = soup.select('a.spielprofil_tooltip')
    
    for i in range(len(links)):
        playerID.append(links[i].get('id'))
    
    for j in playerID[i]:
        playerProfile.append("https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/" + playerID[i])
        print(playerProfile)
Output:
'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789', 'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/131789']
Reply
#2
Lines 23-24: are you using the right loop variable?
Reply
#3
(Apr-03-2020, 07:36 AM)ndc85430 Wrote: Lines 23-24: are you using the right loop variable?

Should it be i again? Sorry, I'm bad at this, could you help me out?
Reply
#4
Try this.
from bs4 import BeautifulSoup
import requests
import pandas as pd

playerID = []
playerImage = []
playerProfile = []
result = []
for page in range(1, 3):
    r = requests.get(
        "https://www.transfermarkt.com/spieler-statistik/wertvollstespieler/marktwertetop? land_id=0&ausrichtung=alle&spielerposition_id=alle&altersklasse=alle&jahrgang=0&kontinent_id=0&plus=1",
        params={"page": page},
        headers={"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:74.0) Gecko/20100101 Firefox/74.0"},
    )
    soup = BeautifulSoup(r.content, "html.parser")
    links = soup.select("a.spielprofil_tooltip")
    for tag in links:
        tag_id = tag.get("id")
        playerID.append(tag_id)
        playerProfile.append(f'https://www.transfermarkt.com/chaher-idjihadi/profil/spieler/{tag_id}')
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020