Python Forum
sports Stats > table output loop problems
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
sports Stats > table output loop problems
#1
would like the code to loop over all profile links and show the table text but will only parse one page and not loop over all profiles, please see below code, please can you advise

import requests
from bs4 import BeautifulSoup
import csv
 
page = 'https://gg.co.uk/tips/today'
tree = requests.get(page, headers = headers)
soup = BeautifulSoup(tree.content, 'html.parser')
 
courseLinks = []
links = soup.select("a.winning-post")
 
for i in range(0,1):   
    courseLinks.append(links[i].get("href"))
    
    #For each location that we have taken, add the website before it - this allows us to call it later
for i in range(len(courseLinks)):
    courseLinks[i] = "https://gg.co.uk"+courseLinks[i]  
#output
['https://gg.co.uk/racing/15-jul-2020/great-yarmouth-1200']
['https://gg.co.uk/racing/15-jul-2020/great-yarmouth-1240']

profileLinks = []
 
#Run the scraper through each of our links
for i in range(len(courseLinks)):
 
    page = courseLinks[i]
    tree = requests.get(page, headers = headers)
    soup = BeautifulSoup(tree.content, 'html.parser')
 
    #Extract all links
    links = soup.select("a.horse")
     
    #For each link, extract the location that it is pointing to
    for j in range(len(links)):
        profileLinks.append("https://gg.co.uk" + links[j].get("href"))
           
 
    #The page list the profile 
    profileLinks = list(set(profileLinks))
    
#output
['https://gg.co.uk/racing/form-profile-2723245',
'https://gg.co.uk/racing/form-profile-2713135',
'https://gg.co.uk/racing/form-profile-2672365',
'https://gg.co.uk/racing/form-profile-2652145',
'https://gg.co.uk/racing/form-profile-2723235']

the code works up until here - it will not process only one of the below links and output

for i in range(len(profileLinks)):
 
    page = profileLinks[i]
    tree = requests.get(page, headers = headers)
    soup = BeautifulSoup(tree.content, 'html.parser') 
 
#find tables data
tableData = soup.find_all('table', id='results-profile' )
last_links = soup.find(class_='border-bottom alt')
last_links.decompose()
 
for tables in tableData:
    for cell in tables.find_all('td'):
        print (cell.text)
#code output
10th3 10
15th Jul 2020 Good to Firm 1m 2f 23y Class 5
12:00 Great Yarmouth Canberra 94 Andrea Atzeni P W Chapple-Hyam
25/1101

#the above code will only parse one of profilelinks https://gg.co.uk/racing/form-profile-2723235, i would like it to loop over all links and output like #code output, then put into a csv file column by column
Reply
#2
In the last block of code you should indent lines 7 through 14 so they become part of the for loop.
Reply
#3
Hi paulfearn100,
Did it help you? Is it solved? Then please mark the thread as "Solved".
Reply
#4
paulfearn100,

RE: My post on programming IDEs for Python. What is the one you are using you just posted snippets from?
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Problems with my loop python_student 3 1,506 Feb-17-2022, 09:49 PM
Last Post: deanhystad
  Python script to summarize excel tables, then output a composite table? i'm a total n surfer349 1 2,290 Feb-05-2021, 04:37 PM
Last Post: nilamo
  using 'while loop' output going into infinite loop... amitkb 2 1,910 Oct-05-2020, 09:18 PM
Last Post: micseydel
  Print output in single file using pramika loop deepakkhw 1 2,036 Jul-11-2020, 11:57 AM
Last Post: j.crater
  Loop problems Kristenl2784 0 1,317 Jul-09-2020, 03:57 PM
Last Post: Kristenl2784
  “While” Loop Problems Ryan_teresi 3 2,822 Jun-27-2020, 06:43 PM
Last Post: tanv321
  Save output into a Excel Sheet with Format Table skaailet 1 2,455 Apr-17-2020, 11:56 PM
Last Post: thirteendec
  Need help with For Loop logic for reading docx Table column data vintysaw 2 3,834 Jan-10-2020, 06:36 AM
Last Post: vintysaw
  Output in for LOOP iterated Renym 1 1,480 Nov-17-2019, 09:15 PM
Last Post: perfringo
  Problem Table Output Phil 4 2,573 May-19-2019, 12:17 PM
Last Post: Phil

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020