Python Forum

Full Version: not printing soup data to csv
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hi, All really struggling with this code below, its a datepicked in Selnium/Soup. The code cycles through months selecting a from & to date, then submits to see a table of booking data. Which is then exported to CSV. The last bit of code is just to move the month forwards at the end of the month. & 'd' to just do this for 9 months.

All the code independently works fine. Except when I pasted the last step the bit to get the table data & export to CSV doesn't print to CSV. I can see the table entries from print(row) in Pycharm all comma deliminated. & when the table/CSV code is run on its own outside the main datepicker loop it prints to CSV.

Code works & cycles through dates - just doesn't print data to CSV - really struggling as this is my first bit of code to write.



for d in range(9):
    # Made a list of all elements that have "ui-state-default" on the id.
    list_days = driver.find_elements_by_class_name("ui-state-default")
    # Now I get the len of that list
    count = len(list_days) - 4
    # Loop over each element of the list count
    for i in range(count):
        # And I get the link to the element
        driver.find_element_by_class_name("ui-datepicker-trigger").click()
        python_botton = driver.find_elements_by_class_name("ui-state-default")[i]
        ActionChains(driver).click(python_botton).perform()
        time.sleep(3)
        driver.find_element_by_xpath('//*[@id="spanDateTo"]/img').click()
        python_tobutton = driver.find_elements_by_class_name("ui-state-default")[i]
        ActionChains(driver).click(python_tobutton).perform()
        submit_button = driver.find_element_by_class_name('button_blue')
        submit_button.click()
        time.sleep(3)
        soup = BeautifulSoup(driver.page_source, 'lxml')
        table = soup.find('tbody', id="transactionsBody")
        tr = table.findAll(['tr'])
        csvFile = open("booking_data.csv", 'wt', newline='', encoding='utf-8')
        writer = csv.writer(csvFile)
        for cell in tr:
            tr = table.findAll(['tr'])
            td = cell.find_all('td')
            row = [i.text.replace('\n', '') for i in td]
            writer.writerow(row)
            print(row)
        csvFile.close()
    #this section will now click the next month on both calendars
    driver.find_element_by_class_name("ui-datepicker-trigger").click()
    date_next = driver.find_element_by_class_name("ui-datepicker-next")
    ActionChains(driver).click(date_next).perform()
    driver.find_element_by_link_text("2").click()
    driver.find_element_by_xpath('//*[@id="spanDateTo"]/img').click()
    date_next_to = driver.find_element_by_class_name("ui-datepicker-next")
    ActionChains(driver).click(date_next_to).perform()
    driver.find_element_by_link_text("2").click()
Hey dude.

So using the same settings in regards to .csv i created something simple to check that there wasn't an issue there which there of course should not have been..

import csv

tr = ["434", "3434", "23", "4523423", "423", "654", "234", "34", "54", "65", "234", "654", "5647", "345", "234"]

csvFile = open("booking_data.csv", 'wt', newline='', encoding='utf-8')
writer = csv.writer(csvFile)
for cell in tr:
    writer.writerow(cell)
    print(cell)
csvFile.close()
Everything worked fine.

Are you getting the table data printing to console?
Hi apologies, just saw this reply - it prints table data to console & have been copying and pasting from there since. Ideally the CSV bit would work


(Sep-17-2020, 12:07 PM)Aspire2Inspire Wrote: [ -> ]Hey dude.

So using the same settings in regards to .csv i created something simple to check that there wasn't an issue there which there of course should not have been..

import csv

tr = ["434", "3434", "23", "4523423", "423", "654", "234", "34", "54", "65", "234", "654", "5647", "345", "234"]

csvFile = open("booking_data.csv", 'wt', newline='', encoding='utf-8')
writer = csv.writer(csvFile)
for cell in tr:
    writer.writerow(cell)
    print(cell)
csvFile.close()
Everything worked fine.

Are you getting the table data printing to console?