Python Forum
Want to extract 4 tables from webpage - Nubee Stuck :(
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Want to extract 4 tables from webpage - Nubee Stuck :(
#1
Hi, I am trying to extract the 4 tables of cases and deaths (new+total) from this page:

https://coronavirus.data.gov.uk/#local-authorities

I have read all numerous websites about how BS works but seem to be failing at each stage. Very frustrating but I'm sure down to my own inabilities!

So, am hoping I can get some help here?

The following seem to be the code that works least badly..

import requests
from bs4 import BeautifulSoup

import pandas as pd 
import numpy as np 

url = "https://coronavirus.data.gov.uk/#local-authorities"
r = requests.get(url)


soup = BeautifulSoup(r.content, 'html.parser')

table = soup.find_all('table')
rows = table.find('cell')
row_list = list()

for tr in rows:
    td = tr.find_all('td')
    row = [i.text for i in td]
    row_list.append(row)
    

df_bs = pd.DataFrame(row_list,columns=['City','Country','Notes'])
df_bs.set_index('Country',inplace=True)
df_bs.to_csv('beautifulsoup.csv')
It gives me the following error on the line "rows = table.find('cell')"
"ResultSet object has no attribute '%s'. You're probably treating a list of items like a single item. Did you call find_all() when you meant to call find()?" % key

AttributeError: ResultSet object has no attribute 'find'. You're probably treating a list of items like a single item. Did you call find_all() when you meant to call find()?

Am tearing my hair out!

Thanks Andrew
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  How to extract links from grid located on webpage Pavel_47 5 1,373 Aug-04-2023, 12:43 PM
Last Post: Gaurav_Kumar
  Extract data from a webpage cycloneseb 5 2,820 Apr-04-2020, 10:17 AM
Last Post: alekson

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020