Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Help copying data from a website
#1
Hi, I am trying to get the data from this website ino an excel (Actually libre office) file. https://sc2pulse.nephest.com/sc2/?season...ladder-top

This website shows the top players, and their mmr (match making rank). That's all I Want, a list of their names + their mmr number. I'm new to coding, I just have this code into a notepad file, converted it into a python file (.py) and run it from cmd. I have python installed. It does make the file, but the file has zero data on it. What do I need to change? Thank you very much.

import os
import requests
from bs4 import BeautifulSoup
import pandas as pd

# Define the URL of the website
url = "https://sc2pulse.nephest.com/sc2/?season=56&queue=LOTV_1V1&team-type=ARRANGED&us=true&eu=true&kr=true&cn=true&bro=true&sil=true&gol=true&pla=true&dia=true&mas=true&gra=true&page=0&type=ladder&ratingAnchor=99999&idAnchor=0&count=1#ladder-top"

try:
    # Send an HTTP request and get the webpage content
    response = requests.get(url)

    if response.status_code == 200:
        # Parse the HTML content of the page
        soup = BeautifulSoup(response.text, "html.parser")

        # Find the data you want by inspecting the webpage's HTML structure
        # Extract and store the data in a list or a pandas DataFrame

        # Example: Extracting player names and MMR
        players = []
        mmr = []

        for player_row in soup.select(".ladder-player-row"):
            player_name = player_row.find("span", class_="player-name").text
            player_mmr = player_row.find("span", class_="player-mmr").text

            players.append(player_name)
            mmr.append(player_mmr)

        # Create a pandas DataFrame
        df = pd.DataFrame({"Player Name": players, "MMR": mmr})

        # Define the path to save the Excel file in "My Documents"
        my_documents_path = os.path.expanduser("~\Documents")
        excel_file_path = os.path.join(my_documents_path, "top_100_players_mmr.xlsx")

        # Save the DataFrame to an Excel file in "My Documents"
        df.to_excel(excel_file_path, index=False)
        print(f"Data has been successfully saved to {excel_file_path}")
    else:
        print("Failed to fetch the webpage. Status code:", response.status_code)
except Exception as e:
    print("An error occurred:", str(e))
EDIT: Sorry it made two threads for some reason
Reply


Messages In This Thread
Help copying data from a website - by awere34a4e3 - Nov-07-2023, 02:59 AM
RE: Help copying data from a website - by snippsat - Nov-07-2023, 02:35 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Scraping lender data from Ren Ren Dai website using Python. I will pay for that 200$ Hafedh_2021 1 2,788 May-18-2021, 08:41 PM
Last Post: snippsat
  How to get registeration data from a website that uses .aspx? Help me brothers. humble_coder 1 2,493 Feb-18-2021, 06:03 PM
Last Post: Larz60+
  Exporting excel data to website bdarragh00 1 2,647 Jun-15-2018, 08:57 PM
Last Post: micseydel
  Selenium to pick data from csv and enter into website Prince_Bhatia 1 6,395 Sep-08-2017, 10:58 AM
Last Post: hbknjr

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020