Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Table data with BeatifulSoup
#8
Hi All,

just wanted to let you know that I was able to write the code :)
Thanks for all your help!!

import requests
from bs4 import BeautifulSoup

d=dict()

def parsepage(page):
    soup = BeautifulSoup(page, 'lxml')
    table = soup.find_all('table')[7]
    if table is not None:
        trs = table.find_all('tr')[8:15]
        for n, tr in enumerate(trs):
            tds = tr.find_all('td')[:2]
            for n1, td in enumerate(tds):
    #            print(f"\n------------------------------ tr_{n}, td_{n1} ------------------------------")
    #            print(f"{td.prettify}")
#                print(td.contents)
                td = str(td)
                if td.find('LIBOR')>0:
                    spos = td.find('">')
                    epos = td.find('</a>')
                    title = td[spos+2:epos]
                    d[title]=d.get(title,0)

                if td.find('%')>0:
                    spos = td.find('>')
                    epos = td.find('%')
                    rate = td[spos+1:epos-1]
                    d[title] = rate



    else:
        print(f"Cound not find table")




def scrape_url(url):
    response = requests.get(url)
    if response.status_code == 200:
        page = response.content
        parsepage(page)
    else:
        print(f"unable to retreive {url}")

if __name__ == '__main__':
    url = 'https://www.global-rates.com/interest-rates/libor/libor.aspx'
    scrape_url(url)

print(d)
Output:
{'Euro LIBOR - overnight': '-0.56971', 'Euro LIBOR - 1 week': '-0.54743', 'Euro LIBOR - 2 weeks': 0, 'Euro LIBOR - 1 month': '-0.50200', 'Euro LIBOR - 2 months': '-0.44600', 'Euro LIBOR - 3 months': '-0.42529'}
Reply


Messages In This Thread
Table data with BeatifulSoup - by gerry84 - Sep-24-2019, 04:17 PM
RE: Table data with BeatifulSoup - by buran - Sep-24-2019, 04:34 PM
RE: Table data with BeatifulSoup - by gerry84 - Sep-24-2019, 04:38 PM
RE: Table data with BeatifulSoup - by Larz60+ - Sep-24-2019, 04:57 PM
RE: Table data with BeatifulSoup - by gerry84 - Sep-24-2019, 05:30 PM
RE: Table data with BeatifulSoup - by metulburr - Sep-24-2019, 07:19 PM
RE: Table data with BeatifulSoup - by gerry84 - Sep-24-2019, 08:07 PM
RE: Table data with BeatifulSoup - by gerry84 - Sep-24-2019, 10:48 PM
RE: Table data with BeatifulSoup - by gerry84 - Oct-22-2019, 06:00 PM
RE: Table data with BeatifulSoup - by Larz60+ - Oct-22-2019, 06:09 PM
RE: Table data with BeatifulSoup - by gerry84 - Oct-22-2019, 09:03 PM
RE: Table data with BeatifulSoup - by Larz60+ - Oct-23-2019, 10:09 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Scraping data from table into existing dataframe vincer58 1 2,045 Jan-09-2022, 05:15 PM
Last Post: vincer58
  Inserting data from a table to another (in same db) firebird 5 2,517 Oct-05-2020, 06:04 AM
Last Post: buran
  Extract data from a table Bob_M 3 2,717 Aug-14-2020, 03:36 PM
Last Post: Bob_M
  Scraping a dynamic data-table in python through AJAX request filozofo 1 3,912 Aug-14-2020, 10:13 AM
Last Post: kashcode
  Want to scrape a table data and export it into CSV format tahir1990 9 5,324 Oct-22-2019, 08:03 AM
Last Post: buran
  Using flask to add data to sqlite3 table with PRIMARY KEY catafest 1 3,776 Sep-09-2019, 07:00 AM
Last Post: buran
  sqlalchemy DataTables::"No data available in table" when using self-joined table Asma 0 2,606 Nov-22-2018, 02:46 PM
Last Post: Asma
  beatifulsoup scrap td tag. piuk3man 1 3,904 Jun-11-2018, 06:16 AM
Last Post: buran
  Insert data in a table after a user is created from djando admin prithvi 0 3,569 Aug-11-2017, 06:25 PM
Last Post: prithvi
  Installation of bs4 and BeatifulSoup landlord1984 7 8,788 Jan-09-2017, 07:41 AM
Last Post: landlord1984

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020