Best way to have most of my code outside of a giant for loop

[danw32]

Hi there, I'm trying to refactor my code as at the moment it is far too clunky for what I am trying to achieve. I read in a line at a time of an excel spreadsheet and use the data from it to enter into Web Pages with Selenium Webdriver. The problem I'm having is literally all of my functional code (selenium code etc.) has to be INSIDE the 'for' loop (which loops through the spreadsheet). This as you can see is a really bad way to get what I need it to do, but I literally can't think of another way to read in a line, do some selenium stuff, then read in the next row. I also have an if statement in it as a "run row" for yes or no in the first excel cell.

Is there a way I can have a very small 'for' loop where I basically run a test i.e. call another module for instance with the selenium code in it? Or failing that, staying in the same module but running the selenium code outside of the 'for' loop?

BTW in the code below "data" is the list of dictionaries of the excel sheet. Basically a list of each row of the spreadsheet. (Also there is an obvious indent issue on a long line of selenium code but I couldn't figure it out on this forum.)

I hope it makes sense! Many thanks. :)

"""Module where I read the excel sheet and put in 'data'"""
from read_sheet import *
"""this is my selenium webdriver module"""
import driver_global

class runUnitTest_class(driver_global.SeleniumTest):

    """loop through the list"""
    for i in range(len(data)):
        if data[i]['Run'] == 'Y' or data[i]['Run'] == 'y':
            print "run the below..."
            url_value = data[i]['URL']
            password_value = data[i]['password']
            username_value = data[i]['username']
            print url_value
            print password_value
            print username_value

            """Selenium coding below. Need to refactor"""

            driver_global.SeleniumTest.setUpClass()
            driver_global.SeleniumTest.driver.get(url_value)

            driver_global.SeleniumTest.driver.find_element_by_id("username").clear()
            driver_global.SeleniumTest.driver.find_element_by_id("username").send_keys(username_value)
            driver_global.SeleniumTest.driver.find_element_by_id("password").clear()
            driver_global.SeleniumTest.driver.find_element_by_id("password").send_keys(password_value)
            driver_global.SeleniumTest.driver.find_element_by_css_selector("button.button").click()
driver_global.SeleniumTest.driver.find_element_by_xpath("//header[@id='header']/div/nav/div/div/a[8]/div").click()
            driver_global.SeleniumTest.take_screenshot_now("from_runUnitTest")
            driver_global.SeleniumTest.tearDownClass()

        elif data[i]['Run'] == 'N' or data[i]['Run'] == 'n':
            print "Don't run row " + str(i + 1)
        elif data[i]['Run'] != 'N' or data[i]['Run'] != 'n' or data[i]['Run'] != 'Y' or data[i]['Run'] != 'y':
            print "enter something useful: Y or N"
        else:
            pass

[ka06059]

what if u put the selenium coding into separate, single function? just call this function below print username_value. u can always run the selenium code outside for loop by calling the function too

other suggestion is that the if statements such as data[i]['Run'] == 'Y' or data[i]['Run'] == 'y' can be shorten into if data[i]['Run'].lower() == 'y'

[danw32]

Wow, thanks man. I literally can't believe I didn't think of that! If I keep all the values like url_value etc. in the loop and then just call my selenium from a function out of the loop, then hopefully the variables (again, like url_value) will update on each pass. I tried putting these outside the loop but it obviously only ran the first row! And thanks a lot for the other suggestion, that's a fab idea. I have done that before in sql to save me hassle, so that's great.

Thanks a lot mate and I'll let you know if it works!

[danw32]

It hasn't worked :( I remember now, I did try it before but if you put the selenium code outside the 'for' loop, the variables i.e. url_value are not resolved in the selenium code. Which makes sense as they are only set in the 'for' loop. If I just wanted to do the first row of the excel sheet it would be easy, I would just set all the variables at the top of the module and hard code in row number 1 in 'data' like this:

url_value = data[1]['URL']
password_value = data[1]['password']
username_value = data[1]['username']

(using a '1' for row 1 instead of an 'i' as in the loop). But if I run the for loop first and then run the selenium when the for loop has completed, it would loop through all the rows and always set the variables to the last row in the sheet. Then the selenium code would use this successfully, but I could never run any other row other than the last one. If I put more rows in the excel sheet, I obviously don't want to hard code in each variable for each row, as that's just daft.

Is there a way to set my variables to what's in row 1, run the selenium code, then overwrite the variables to row two and run the same selenium code? This was my initial problem and it's driving me mad! I can't put the selenium code into a function and call it inside the for loop, or indeed call it from another module as the variables will not be resolved as they are not set to anything until the code in the for loop runs.

Please help! If this is still not clear please let me know and I'll try and explain again.

Many thanks - I'll be SUPER grateful if anyone can think of a solution. :)

[ka06059]

just to remind i've yet to use/import selenium related modules, but after reading the code, i can only tell the codes are just trying to set/obtain values from data.. so its pretty straight forward. is this how your selenium function looks like?...

def seleniumfunction(url_value,password_value,username_value):
    driver_global.SeleniumTest.setUpClass()
    driver_global.SeleniumTest.driver.get(url_value)
    driver_global.SeleniumTest.driver.find_element_by_id("username").clear()
    driver_global.SeleniumTest.driver.find_element_by_id("username").send_keys(username_value)
    driver_global.SeleniumTest.driver.find_element_by_id("password").clear()
    driver_global.SeleniumTest.driver.find_element_by_id("password").send_keys(password_value)
    driver_global.SeleniumTest.driver.find_element_by_css_selector("button.button").click()
    driver_global.SeleniumTest.driver.find_element_by_xpath("//header[@id='header']/div/nav/div/div/a[8]/div").click()
    driver_global.SeleniumTest.take_screenshot_now("from_runUnitTest")
    driver_global.SeleniumTest.tearDownClass()

just call the function below line 17, same indent level

print url_value
print password_value
print username_value   #line 17
seleniumfunction(url_value,password_value,username_value)

if u wish to run the whole things outside of the 'for loop' my suggestion is to create lists of list to store informations on each data in that 'for loop'. then create new looping codes outside to run selenium function on each element on that list. still, u can barely escape from iteration.

[digitalmatic7]

Disclaimer: I'm a beginner, so this advice may or may not work for your situation.

Here's how I handle lots of URLs in a CSV and navigate through them in selenium:

• Use pandas to load the CSV into Python (will be saved as an internal list - dataframe)
  
• Get the list total (based on the column that has the URLs)
  
• You can use 'pop' to take 1 URL off the list at a time in the loop
  

For the loop.. setup a counter, and run it against the total list items. Increment the counter at the end of the loop..

So example:

counter = 0
my_list = len(pandas_dataframe)      # get total list items

while counter < my_list:            # loop until counter matches total amount of urls in list

    driver.get() # pop list item off here to get URL and remove it from list at the same time
    
    if something:
        break # this should exit out of the loop I believe, not 100% sure though

    counter = counter + 1
    

[danw32]

Hey guys, after a couple of ponderous nights thinking I figured it out! Only took me a year on and off haha.

What I did was create a function with my selenium code in it, like you said. But how to resolve all the data variables? I solved it by passing them ALL in as parameters. See below:

from read_sheet import *
import driver_global


def selenium_code_lottery(url_value, username_value, password_value):
    driver_global.SeleniumTest.setUpClass()
    driver_global.SeleniumTest.driver.get(url_value)
    driver_global.SeleniumTest.driver.find_element_by_id("username").clear()
    driver_global.SeleniumTest.driver.find_element_by_id("username").send_keys(username_value)
    driver_global.SeleniumTest.driver.find_element_by_id("password").clear()
    driver_global.SeleniumTest.driver.find_element_by_id("password").send_keys(password_value)
    driver_global.SeleniumTest.take_screenshot_now("sign_in_page")
    driver_global.SeleniumTest.driver.find_element_by_css_selector("button.button").click()
    # need assert before screenshot, and the implicit wait.
    driver_global.SeleniumTest.take_screenshot_now("signed_in")
    driver_global.SeleniumTest.driver.find_element_by_xpath(
        "//header[@id='header']/div/nav/div/div/a[8]/div").click()
    driver_global.SeleniumTest.take_screenshot_now("lottery")
    driver_global.SeleniumTest.tearDownClass()

class runUnitTest_class(driver_global.SeleniumTest):

    """loop through the list"""
    for i in range(len(data)):
        # if data[i]['Run'] == 'Y' or data[i]['Run'] == 'y':
        if data[i]['Run'].lower() == 'y':
            print "run the below..."
            url_value = data[i]['URL']
            password_value = data[i]['password']
            username_value = data[i]['username']
            print url_value
            print password_value
            print username_value
	    
	    """Function as top of module but all data variables are passed in as parameters. Works like a treat!"""
            selenium_code_lottery(url_value, username_value, password_value)

        elif data[i]['Run'] == 'N' or data[i]['Run'] == 'n':
            print "Don't run row " + str(i + 1)
        elif data[i]['Run'] != 'N' or data[i]['Run'] != 'n' or data[i]['Run'] != 'Y' or data[i]['Run'] != 'y':
            print "enter something useful: Y or N"
        else:
            pass

Thanks for your help guys. It helped jig my brain. It was just a scoping problem and how to get round it.

Cheers!