Python Forum
sports Stats > table output loop problems
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
sports Stats > table output loop problems
#1
would like the code to loop over all profile links and show the table text but will only parse one page and not loop over all profiles, please see below code, please can you advise

import requests
from bs4 import BeautifulSoup
import csv
 
page = 'https://gg.co.uk/tips/today'
tree = requests.get(page, headers = headers)
soup = BeautifulSoup(tree.content, 'html.parser')
 
courseLinks = []
links = soup.select("a.winning-post")
 
for i in range(0,1):   
    courseLinks.append(links[i].get("href"))
    
    #For each location that we have taken, add the website before it - this allows us to call it later
for i in range(len(courseLinks)):
    courseLinks[i] = "https://gg.co.uk"+courseLinks[i]  
#output
['https://gg.co.uk/racing/15-jul-2020/great-yarmouth-1200']
['https://gg.co.uk/racing/15-jul-2020/great-yarmouth-1240']

profileLinks = []
 
#Run the scraper through each of our links
for i in range(len(courseLinks)):
 
    page = courseLinks[i]
    tree = requests.get(page, headers = headers)
    soup = BeautifulSoup(tree.content, 'html.parser')
 
    #Extract all links
    links = soup.select("a.horse")
     
    #For each link, extract the location that it is pointing to
    for j in range(len(links)):
        profileLinks.append("https://gg.co.uk" + links[j].get("href"))
           
 
    #The page list the profile 
    profileLinks = list(set(profileLinks))
    
#output
['https://gg.co.uk/racing/form-profile-2723245',
'https://gg.co.uk/racing/form-profile-2713135',
'https://gg.co.uk/racing/form-profile-2672365',
'https://gg.co.uk/racing/form-profile-2652145',
'https://gg.co.uk/racing/form-profile-2723235']

the code works up until here - it will not process only one of the below links and output

for i in range(len(profileLinks)):
 
    page = profileLinks[i]
    tree = requests.get(page, headers = headers)
    soup = BeautifulSoup(tree.content, 'html.parser') 
 
#find tables data
tableData = soup.find_all('table', id='results-profile' )
last_links = soup.find(class_='border-bottom alt')
last_links.decompose()
 
for tables in tableData:
    for cell in tables.find_all('td'):
        print (cell.text)
#code output
10th3 10
15th Jul 2020 Good to Firm 1m 2f 23y Class 5
12:00 Great Yarmouth Canberra 94 Andrea Atzeni P W Chapple-Hyam
25/1101

#the above code will only parse one of profilelinks https://gg.co.uk/racing/form-profile-2723235, i would like it to loop over all links and output like #code output, then put into a csv file column by column
Reply
#2
In the last block of code you should indent lines 7 through 14 so they become part of the for loop.
Reply
#3
Hi paulfearn100,
Did it help you? Is it solved? Then please mark the thread as "Solved".
Reply
#4
paulfearn100,

RE: My post on programming IDEs for Python. What is the one you are using you just posted snippets from?
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Python script to summarize excel tables, then output a composite table? i'm a total n surfer349 1 215 Feb-05-2021, 04:37 PM
Last Post: nilamo
  using 'while loop' output going into infinite loop... amitkb 2 520 Oct-05-2020, 09:18 PM
Last Post: micseydel
  Fetch Oracle DB rows & print it in HTML file with table's col headers in table format tssr_2001 1 671 Sep-04-2020, 01:39 PM
Last Post: ibreeden
  Print output in single file using pramika loop deepakkhw 1 527 Jul-11-2020, 11:57 AM
Last Post: j.crater
  Loop problems Kristenl2784 0 307 Jul-09-2020, 03:57 PM
Last Post: Kristenl2784
  “While” Loop Problems Ryan_teresi 3 846 Jun-27-2020, 06:43 PM
Last Post: tanv321
  Mysql CREATE TABLE IF NOT EXISTS dynamic table name nisusavi 0 516 Apr-29-2020, 06:45 PM
Last Post: nisusavi
  Save output into a Excel Sheet with Format Table skaailet 1 617 Apr-17-2020, 11:56 PM
Last Post: thirteendec
  Need help with For Loop logic for reading docx Table column data vintysaw 2 1,154 Jan-10-2020, 06:36 AM
Last Post: vintysaw
  MYSQL how to assign a table for each user in my username table database? YoshikageKira 1 1,006 Dec-26-2019, 05:57 AM
Last Post: buran

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020