Python Forum

Full Version: How to export to csv the output of every iteration when scrapping with a loop
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hi,
I am very new to Python. I am trying to scrape a website and I have create a small code for this:

select = Select(character.find_element_by_id('character-template-choice'))
options = select.options
for index in range(0, len(options) - 0):
    select.select_by_index(index)
    option1 = character.find_elements_by_class_name('pc-stat-value')
    power = []
    for c in option1:
        power.append("Power:" + c.text)
    option2 = character.find_elements_by_class_name(
        'unit-stat-group-stat-label')
    skills = []
    for a in option2:
        skills.append(a.text)
    option3 = character.find_elements_by_class_name(
        'unit-stat-group-stat-value')
    values = []
    for b in option3:
        values.append(b.text)
    test = [power, skills, values]
The issue I have is when I try to export "test" list of lists to csv I only get the last iteration. I want to get the data for every iteration but I don't know how to do so. Can someone help me?

Thank you
Please, when posting code, provide enough to be able to run.

if you haven't already done so, import csv at top of script

# untested code, you may have to correct any errors.
select = Select(character.find_element_by_id('character-template-choice'))
options = select.options
savefile = 'savefile.txt' # Change to your desired file name
with open(savefile, 'w') as fp":
    csvwriter = csv.writer(fp)
    for index in range(0, len(options) - 0):
    select.select_by_index(index)
    option1 = character.find_elements_by_class_name('pc-stat-value')
    power = []
    for c in option1:
        power.append("Power:" + c.text)
    option2 = character.find_elements_by_class_name(
        'unit-stat-group-stat-label')
    skills = []
    for a in option2:
        skills.append(a.text)
    option3 = character.find_elements_by_class_name(
        'unit-stat-group-stat-value')
    values = []
    for b in option3:
        values.append(b.text)
    test = [power, skills, values]
        csvwriter.writerow(test)
Thank you for the help with the code. I will use the editor for future posts. I haven't posted before so I didn't know how I should do it properly.