Python Forum
While loop skips multiple dropdown menu options and then works as intended
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
While loop skips multiple dropdown menu options and then works as intended
#1
Below is the code I am writing to try and scrape information from a website. There are multiple options to choose from on a drop down menu and each value represents the results from a different season. I want to scrape the data from every season starting with season 10 (earliest available). When the code is run, it navigates to the site, successfully logins, goes to season 10 and begins going through the data. After going through season 10 it jumps to 20, skipping 11 through 19. Once it gets done with 20, it goes through every season and stops at 41, as I intended. Regardless of the starting season I select, it will complete the same action (skip several season then proceed as intended). Any help or advice on how I could fix this loop would be much appreciated.

from selenium import webdriver
from selenium.webdriver.support.ui import Select
import time


login_screen = driver.get("https://mwomercs.com/profile/leaderboards")
username = driver.find_element_by_id("email").send_keys("login_here")
password = driver.find_element_by_id("password").send_keys("some_password")
submit = driver.find_element_by_xpath("//*[@id='loginForm']/form/button").click()

# set up variable for season dropdown selection
dropdown = driver.find_element_by_xpath("//*[@id='season']")
season = Select(dropdown)

# select the season to start on
i = 10
season.select_by_value(str(i))

# loop to change seasons at the end of just scraped season
while True:
    source = driver.page_source
    time.sleep(2)
    try:
        driver.find_element_by_partial_link_text("Next").click()
    except:
        i += 1
        if i <= 41:
            season = driver.find_element_by_xpath("//*[@id='season']/option[" + str(i) + "]").click()
        else:
            break
Reply
#2
hello,

havent tested your code
but you are using
while true
isnt it in your case better to convert you code so you can use
something like

i=10
while i<=41:
    #rethink the code you will be writing here
    i+1
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Generating dynamic dropdown list test 1 1,428 Aug-30-2023, 08:00 AM
Last Post: blessinguvula
  Using yt-dlp options in Python Pavel_47 8 14,207 Jun-29-2022, 12:52 PM
Last Post: Pavel_47
  Python Selenium (Dropdown-) data Robin_at_Cantelli 2 6,157 Dec-29-2021, 03:16 AM
Last Post: ondreweil
  Selenium Python for Dropdown not working gj31980 1 2,612 Oct-27-2020, 02:02 AM
Last Post: gj31980
  Dropdown interact moisesfelipee 0 1,656 May-04-2020, 01:11 AM
Last Post: moisesfelipee
  How can i scrape dropdown value ? caca 0 2,939 Nov-03-2019, 11:24 PM
Last Post: caca
  getting options from a html form pgoosen 5 3,193 Jul-03-2019, 06:07 PM
Last Post: nilamo
  Python Flask Dependent Dropdown Anfaa 3 18,366 Oct-24-2018, 09:35 PM
Last Post: nilamo
  Click dropdown menu option with Selenium PyChrome AcszE 3 5,855 Oct-26-2017, 10:07 PM
Last Post: metulburr
  Create Dictionary List (From a webpage dropdown) for Comparison to a CSV File Guttmann 5 5,851 Mar-31-2017, 01:29 AM
Last Post: Guttmann

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020