Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
table from wikipedia
#2
You are a little on wrong track when you start to manually parse the table.
Change your first code to this.
import pandas as pd
import requests
from bs4 import BeautifulSoup

url_cntr = 'https://en.wikipedia.org/wiki/List_of_countries_and_dependencies_by_population'
t = requests.get(url_cntr)
html_content = t.content
html_soup = BeautifulSoup(html_content, 'html.parser')
sov_tables = html_soup.find('table', class_="wikitable sortable")
At this point you have the table needed in sov_tables.
Now want to use Pandas with pd.read_html(),in this case you could also only used this method to get table and drop all over.

To bring sov_tables into pandas,it need to be file object or string.
So can use str()
df = pd.read_html(str(sov_tables))
df = df[0]
df
It also get easier if you use Jupyter Notebook,then the table also look nice.
[Image: VM444n.jpg]
At this point you can continue with task,need convert to columns to right format int,datetime..ect
Reply


Messages In This Thread
table from wikipedia - by flow50 - Jun-27-2019, 10:31 AM
RE: table from wikipedia - by snippsat - Jun-27-2019, 12:47 PM
RE: table from wikipedia - by flow50 - Jun-28-2019, 03:02 PM
RE: table from wikipedia - by snippsat - Jun-28-2019, 05:22 PM
RE: table from wikipedia - by flow50 - Jul-01-2019, 12:16 PM
RE: table from wikipedia - by snippsat - Jul-01-2019, 07:12 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
Question Scraping Wikipedia Article (Name in 1 column & URL in 2nd column) ->CSV! Anyone? BrandonKastning 4 2,056 Jan-27-2022, 04:36 AM
Last Post: Larz60+
  fetching, parsing data from Wikipedia apollo 2 3,572 May-06-2021, 08:08 PM
Last Post: snippsat
  Need help scraping wikipedia table bborusz2 6 3,282 Dec-01-2020, 11:31 PM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020