Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Scraping Data from Website
#1
Hi, I am trying to learn Scraping Data from Website with python and i tried extract that list ( List of largest companies by revenue - Wikipedia ) but it shows 60 columns instead of 8. I added the picture where i confused. ( ‘USD millions’ should be the last column but it continues like 1, 2, 3…). and i added the code. How should i fix it?
That's the code:
from bs4 import BeautifulSoup
import requests

url = 'https://en.wikipedia.org/wiki/List_of_largest_companies_by_revenue'
page = requests.get(url)


soup = BeautifulSoup(page.text, 'html')


print(soup)

soup.find_all('table')




soup.find('table', class_ = 'wikitable sortable ')




table = soup.find_all('table')[1]

print(table)

world_titles = table.find_all('th')

world_titles

world_table_titles = [title.text.strip() for title in world_titles]
print ( world_table_titles)

import pandas as pd

df = pd.DataFrame(columns = world_table_titles)
df

column_data = table.find_all('tr')

for row in column_data[2:]: #baştaki boşluk gitti
    row_data = row.find_all('td')
    individual_row_Data = [data.text.strip() for data in row_data]
    lenght = len(df)
    df.loc[lenght] == individual_row_Data

df
[Image: Whats-App-Image-2023-09-21-at-16-57-48.jpg]
snippsat write Sep-21-2023, 02:58 PM:
Added code tag in your post,look at BBCode on how to use.
Reply


Messages In This Thread
Scraping Data from Website - by melkaray - Sep-21-2023, 02:00 PM
RE: Scraping Data from Website - by snippsat - Sep-21-2023, 04:16 PM
RE: Scraping Data from Website - by Larz60+ - Sep-21-2023, 10:38 PM
RE: Scraping Data from Website - by melkaray - Sep-22-2023, 12:41 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Error 403 Scraping website cartonics 17 4,840 Oct-25-2023, 02:54 PM
Last Post: snippsat
  Code to retrieve data from a website charlie13255 0 1,663 Jul-07-2022, 07:53 PM
Last Post: charlie13255
  Is this possible to write a script for checking data from website? WanW 2 1,784 Jun-02-2022, 02:31 AM
Last Post: Larz60+
  Extracting data from a website tgottsc1 2 2,948 Jan-09-2021, 08:14 PM
Last Post: tgottsc1
  extracting data from json on website larry2311 2 6,348 Feb-09-2018, 01:27 AM
Last Post: larry2311
  header of website but no data Brian1210 2 4,704 Oct-07-2017, 11:59 PM
Last Post: Brian1210
  sending data from my raspberry pi to my website mohitsangavikar 2 18,974 Sep-05-2017, 06:55 PM
Last Post: wrybread
  Twitter scraping exclude some data Robbert 6 6,431 Sep-02-2017, 09:44 PM
Last Post: nilamo

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020