You are a little on wrong track when you start to manually parse the table.
Change your first code to this.
Now want to use Pandas with pd.read_html(),in this case you could also only used this method to get table and drop all over.
To bring
So can use
At this point you can continue with task,need convert to columns to right format int,datetime..ect
Change your first code to this.
import pandas as pd import requests from bs4 import BeautifulSoup url_cntr = 'https://en.wikipedia.org/wiki/List_of_countries_and_dependencies_by_population' t = requests.get(url_cntr) html_content = t.content html_soup = BeautifulSoup(html_content, 'html.parser') sov_tables = html_soup.find('table', class_="wikitable sortable")At this point you have the table needed in
sov_tables
.Now want to use Pandas with pd.read_html(),in this case you could also only used this method to get table and drop all over.
To bring
sov_tables
into pandas,it need to be file object or string.So can use
str()
df = pd.read_html(str(sov_tables)) df = df[0] dfIt also get easier if you use Jupyter Notebook,then the table also look nice.
At this point you can continue with task,need convert to columns to right format int,datetime..ect