Python Forum

Full Version: While loop skips multiple dropdown menu options and then works as intended
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Below is the code I am writing to try and scrape information from a website. There are multiple options to choose from on a drop down menu and each value represents the results from a different season. I want to scrape the data from every season starting with season 10 (earliest available). When the code is run, it navigates to the site, successfully logins, goes to season 10 and begins going through the data. After going through season 10 it jumps to 20, skipping 11 through 19. Once it gets done with 20, it goes through every season and stops at 41, as I intended. Regardless of the starting season I select, it will complete the same action (skip several season then proceed as intended). Any help or advice on how I could fix this loop would be much appreciated.

from selenium import webdriver
from selenium.webdriver.support.ui import Select
import time


login_screen = driver.get("https://mwomercs.com/profile/leaderboards")
username = driver.find_element_by_id("email").send_keys("login_here")
password = driver.find_element_by_id("password").send_keys("some_password")
submit = driver.find_element_by_xpath("//*[@id='loginForm']/form/button").click()

# set up variable for season dropdown selection
dropdown = driver.find_element_by_xpath("//*[@id='season']")
season = Select(dropdown)

# select the season to start on
i = 10
season.select_by_value(str(i))

# loop to change seasons at the end of just scraped season
while True:
    source = driver.page_source
    time.sleep(2)
    try:
        driver.find_element_by_partial_link_text("Next").click()
    except:
        i += 1
        if i <= 41:
            season = driver.find_element_by_xpath("//*[@id='season']/option[" + str(i) + "]").click()
        else:
            break
hello,

havent tested your code
but you are using
while true
isnt it in your case better to convert you code so you can use
something like

i=10
while i<=41:
    #rethink the code you will be writing here
    i+1