While loop and read_lines()

[pooria]

Hi,

I am working on a python project with selenium for webscraping.

If anyone's worked with selenium before, you know there is a send_keys() function that whatever variable/argument is passed in the (), it will be automatically typed into a form or a textarea.

There will be a .txt file in the folder of the project. Let's say text.txt. In this text file, there are like 10, 20, or even 100 of lines. Like this for example:

Line 1
Line 2
Line 3
...

I've tried different things but I just couldn't come up with a solution.

I want a loop (for or while) that allows the script, on each loop, to type in each line in the form.

I'm guessing we should say something like:

text=[]

to declare a new array. And then open and read each line of the file. Append it to this array and then loop through it.

Another problem I faced was that sometimes the loop would just use the first line and in the next loop it would use the same line again. Another problem was it just used all the lines in one loop. I think after each loop the value of let's say a variable "text" that has our current index of that array should be deleted and the next index (actually it's value) should be used.

And since in that function I told you send_keys(), we can use only one thing. Like a variable.

Please let me know if you have any solutions

[mlieqo]

I'll be happy to help if you share some of the code you've tried

[buran]

Check our tutorial on working with files https://python-forum.io/Thread-Files

Basically you would do:

with open('some_file.txt') as f:
    for line in f:
        # do something with that line

Note that line will have new-line char at the end and you may need to strip() it

[pooria]

Check our tutorial on working with files https://python-forum.io/Thread-Files

Basically you would do:

with open('some_file.txt') as f:
    for line in f:
        # do something with that line

Note that line will have new-line char at the end and you may need to strip() it

This code doesn't do much.

It should read each line, either store all of it an array, break it up, and use each value, in each loop, as a variable for my function OR store each line in a variable and during each loop use that variable in my function.

[buran]

It should read each line, either store all of it an array, break it up, and use each value, in each loop, as a variable for my function OR store each line in a variable and during each loop use that variable in my function.

it store each line in variable line. It's up to you to use it in your function (one line at a time) as you see fit.

[pooria]

It should read each line, either store all of it an array, break it up, and use each value, in each loop, as a variable for my function OR store each line in a variable and during each loop use that variable in my function.

it store each line in variable line. It's up to you to use it in your function (one line at a time) as you see fit.

By 'each' line in variable line, do you mean for example:

A) Line 1, Line 2, Line 3...will ALL be stored in variable line
B) Line 1, Line 2, Line 3... will in each loop be stored as variable line. So, in the first loop, line = line 1, the next loop, line = line 2....and so on?

And if I pass the variable line in my function send_keys(line), it should work?

[buran]

I mean B.
Yes, it will work if you use element.send_keys(line) or element.send_keys(line.strip()). In the second one it will remove the new line char at the end of line

[pooria]

I mean B.
Yes, it will work if you use element.send_keys(line) or element.send_keys(line.strip()). In the second one it will remove the new line char at the end of line

Hey,

Thanks for the great reply.

Is there anyway I could put up my script so others could comment on it? Or could I sent it to you to just check it pretty quick?

---

And one more thing.

During each loop, after it reads each line, the function does something, I want it to delete the previous line it had read.

So, if we have line 1, line 2, line 3...when it reads line 1, the function does something, I want it to delete line 1 and then move on to line 2

[buran]

You can post your script here using python tags and ask if you have particular questions. If you consider it complete and just ask for review/advise for improvements - post it in the complete scripts section.

[pooria]

*Install Python
*Install Selenium web driver
*Install Geocke driver for firefox
*You need a quora account. An email and password to access it

If you are accessing it from some English-speaking country then comment out the function "Back_to_English()"

**I've noticed after almost 100 reruns that at some parts, there is a timeout or cannot find element. Bear in mind that I have correctly identified each element. I think the problem is with either the time.sleep or the ec. conditions that I've set. I'd appreciate if you could check it. Thank you

**This is my first script. I want to try out a personal project.

from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as ec
import time


def Login_Quora():

    Email_id = input("Enter your email:")
    Password_id = input("Enter your password:")
    #Login quora

    Email = driver.find_element_by_xpath('//div[@class="form_column"]/input[1]')
    Email.send_keys(Email_id)

    Password = driver.find_element_by_xpath('//div[@class="form_inputs"]/div[2]/input[1]')
    Password.send_keys(Password_id)

    #Login = driver.find_element_by_xpath('//div[@class="form_inputs"]/div[3]/input[1]')
    wait1 = WebDriverWait(driver, 10)
    Login = wait1.until(ec.presence_of_element_located((By.XPATH, "//div[@class='form_inputs']/div[3]/input[1]")))
    ActionChains(driver).move_to_element(Login).perform()
    time.sleep(3)
    Login.click()
    time.sleep(5)

def Back_to_English():
    #Back to english quora
    #English = driver.find_element_by_xpath('//div[@class="u-margin-top--md"]/a')
    wait2 = WebDriverWait(driver, 10)
    English = wait2.until(ec.element_to_be_clickable((By.XPATH, "//div[@class='u-margin-top--md']/a")))
    English.click()
    time.sleep(3)

def Add_question():
    #input for question
    #questions=input("Enter your question:")
    #click addquestion button

    #AddQuestion = driver.find_element_by_xpath('//div[@class="right_contents"]/div[3]/a')
    wait6 = WebDriverWait(driver, 10)
    AddQuestion = wait6.until(ec.element_to_be_clickable((By.XPATH, "//div[@class='right_contents']/div[3]/a")))
    AddQuestion.click()
    time.sleep(3)
    #question = input("Enter your question:")
    #Type in the question
    #TypeQuestion = driver.find_element_by_xpath('//div[@class="modal_wrapper normal"]/div[1]/div[@class="modal_content modal_content_tabs"]/div[1]/div[3]/div[1]/textarea[@class="selector_input text"]')
    wait3 = WebDriverWait(driver, 10)
    TypeQuestion = wait3.until(ec.visibility_of_element_located((By.XPATH, '//div[@class="modal_wrapper normal"]/div[1]/div[@class="modal_content modal_content_tabs"]/div[1]/div[3]/div[1]/textarea[@class="selector_input text"]')))
    TypeQuestion.send_keys(line.strip())
    time.sleep(3)
    #Click add question
    #AddQuestion = driver.find_element_by_xpath('//div[@class="modal_wrapper normal"]/div[1]/div[@class="modal_footer"]/div[@class="modal_actions"]/span[2]/a')
    wait4 = WebDriverWait(driver, 10)
    AddQuestion = wait4.until(ec.visibility_of_element_located((By.XPATH, "//div[@class='modal_wrapper normal']/div[1]/div[@class='modal_footer']/div[@class='modal_actions']/span[2]/a")))
    AddQuestion.click()
    time.sleep(6)
    #Click on confirm
    #Confirm = driver.find_element_by_xpath('//div[@class="modal_overlay"]/div[2]/div[1]/div[@class="modal_footer"]/div[@class="modal_actions"]/a')
    #wait = WebDriverWait(driver, 10)
    #Confirm = wait.until(ec.visibility_of_element_located((By.XPATH, "//div[@class='modal_overlay']/div[2]/div[1]/div[@class='modal_footer']/div[@class='modal_actions']/a")))
    #Confirm.click()
    #time.sleep(2)
    wait5 = WebDriverWait(driver, 10)
    Cancel = wait5.until(ec.element_to_be_clickable((By.XPATH, "//div[@class='modal_overlay']/div[2]/div[1]/div[@class='modal_footer']/div[@class='modal_actions']/span[1]/a")))
    #Cancel = driver.find_element_by_xpath("//div[@class='modal_overlay']/div[2]/div[1]/div[@class='modal_footer']/div[@class='modal_actions']/span[1]/a")
    Cancel.click()

    #Alert Box
    obj = driver.switch_to.alert
    obj.accept()
    time.sleep(2)
    question= None
    #Cross
    #Cross = driver.find_element_by_xpath('//div[@class="modal_overlay"]/div[3]/div[1]/div[1]/div[1]/a')
    #Cross.click()

driver = webdriver.Firefox()




#open quora
driver.get("http://www.quora.com")

Login_Quora()

Back_to_English()

with open('questions.txt') as f:
    for line in f:
        Add_question()

#question = []
#with open("questions.txt") as f:
    #for line in f:
        #question.append(line)
        #Add_question()