Python Forum
Help Scraping links and table from link
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Help Scraping links and table from link
#6
from bs4 import BeautifulSoup
from bs4.dammit import EncodingDetector
import requests
 
parser = 'html.parser'  # or 'lxml' (preferred) or 'html5lib', if installed
resp = requests.get("https://www.sbostats.com/soccer/league/italy/serie-a")
http_encoding = resp.encoding if 'charset' in resp.headers.get('content-type', '').lower() else None
html_encoding = EncodingDetector.find_declared_encoding(resp.content, is_html=True)
encoding = html_encoding or http_encoding
soup = BeautifulSoup(resp.content, parser, from_encoding=encoding)
table = soup.find_all('table',attrs={'class':'updated_next_results_table'})
 
table = table[0]
tr = table.find_all('tr')
for row in tr:
    if row.text == None:
        pass
    if row.find('a') == None:
        pass
    else:
        #print(row.text)
        #print(' '.join(row.text.replace('STATS', '-').split()))
        #print(f"{row.find('a')['href']}\n")
        y= f"{row.find('a')['href']}\n"
        x= ' '.join(row.text.replace('STATS', '-').split())
        q= ''.join([i for i in x if not i.isdigit()])
        c = ('*https://www.sbostats.com' + y)
        z = c.replace('"e', "&quote")
        #print(x + z)
        f = open("matches.txt", "a")
    #f.write([x] +[y])
        f.write(str(q) + ' ' + str(z))
        f.close()
i edited the link for my needs i have only to understand how to remove all numbers of odds

tryed this q= ''.join([i for i in x if not i.isdigit()]) but in output i find . points that remains from decimals
so i added q1= ' '.join(q.replace('.', '').split())
does the work but i think is a very dirty solution.. i think that is the worst solution :)
Reply


Messages In This Thread
RE: Help Scraping links and table from link - by cartonics - Oct-10-2023, 08:38 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Scraping data from table into existing dataframe vincer58 1 2,103 Jan-09-2022, 05:15 PM
Last Post: vincer58
  Need help scraping wikipedia table bborusz2 6 3,406 Dec-01-2020, 11:31 PM
Last Post: snippsat
  Web Scraping Inquiry (Extracting content from a table in asubdomain) DustinKlent 3 3,830 Aug-17-2020, 10:10 AM
Last Post: snippsat
  Scraping a dynamic data-table in python through AJAX request filozofo 1 3,975 Aug-14-2020, 10:13 AM
Last Post: kashcode
  scraping multiple pages from table bandar 1 2,782 Jun-27-2020, 10:43 PM
Last Post: Larz60+
  get link and link text from table metulburr 5 6,448 Jun-13-2019, 07:50 PM
Last Post: snippsat
  webscrapping links and then enter those links to scrape data kirito85 2 3,327 Jun-13-2019, 02:23 AM
Last Post: kirito85
  Error while scraping links with beautiful soup mgtheboss 4 8,547 Dec-22-2017, 12:41 PM
Last Post: mgtheboss
  Web scraping "fancy" table acehole60 2 4,987 Dec-16-2016, 09:17 AM
Last Post: acehole60

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020