Python Forum
Help Scraping links and table from link
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Help Scraping links and table from link
#1
from bs4 import BeautifulSoup
from bs4.dammit import EncodingDetector
import requests

parser = 'html.parser'  # or 'lxml' (preferred) or 'html5lib', if installed
resp = requests.get("https://www.sbostats.com/soccer/league/italy/serie-a")
http_encoding = resp.encoding if 'charset' in resp.headers.get('content-type', '').lower() else None
html_encoding = EncodingDetector.find_declared_encoding(resp.content, is_html=True)
encoding = html_encoding or http_encoding
soup = BeautifulSoup(resp.content, parser, from_encoding=encoding)

#print (soup)

##for link in soup.select('a[href^="/soccer/stats?"]'):
##    #print ('https://www.sbostats.com/soccer/stats?country=italy&league=serie-a&quote=1.44&direction=away&id=Mzk5OTk5MQ==')
##    href1 = ['href']
##    # a"e
##    c = ('https://www.sbostats.com'+link['href'])
##    x = c.replace('"e', "&quote")
##    print (x)


data = []
table = soup.find_all('table',attrs={'class':'updated_next_results_table'}) #, 

print (table)


rows = soup.find_all('tr')
for row in rows:
    cols = row.find_all('td') #, attrs={'class':'widget-results__team-name match-name'}
    cols = [ele.text.strip() for ele in cols]
    data.append([ele for ele in cols if ele]) # Get rid of empty values

print (data)

i am able to take links and datas but my expected result is
from this link https://www.sbostats.com/soccer/league/italy/serie-a

for each match have values of names of teams from the table and the relative link.
Reply


Messages In This Thread
Help Scraping links and table from link - by cartonics - Oct-06-2023, 08:32 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Scraping data from table into existing dataframe vincer58 1 2,046 Jan-09-2022, 05:15 PM
Last Post: vincer58
  Need help scraping wikipedia table bborusz2 6 3,293 Dec-01-2020, 11:31 PM
Last Post: snippsat
  Web Scraping Inquiry (Extracting content from a table in asubdomain) DustinKlent 3 3,771 Aug-17-2020, 10:10 AM
Last Post: snippsat
  Scraping a dynamic data-table in python through AJAX request filozofo 1 3,917 Aug-14-2020, 10:13 AM
Last Post: kashcode
  scraping multiple pages from table bandar 1 2,736 Jun-27-2020, 10:43 PM
Last Post: Larz60+
  get link and link text from table metulburr 5 6,356 Jun-13-2019, 07:50 PM
Last Post: snippsat
  webscrapping links and then enter those links to scrape data kirito85 2 3,246 Jun-13-2019, 02:23 AM
Last Post: kirito85
  Error while scraping links with beautiful soup mgtheboss 4 8,441 Dec-22-2017, 12:41 PM
Last Post: mgtheboss
  Web scraping "fancy" table acehole60 2 4,945 Dec-16-2016, 09:17 AM
Last Post: acehole60

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020