Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
not printing soup data to csv
#1
Hi, All really struggling with this code below, its a datepicked in Selnium/Soup. The code cycles through months selecting a from & to date, then submits to see a table of booking data. Which is then exported to CSV. The last bit of code is just to move the month forwards at the end of the month. & 'd' to just do this for 9 months.

All the code independently works fine. Except when I pasted the last step the bit to get the table data & export to CSV doesn't print to CSV. I can see the table entries from print(row) in Pycharm all comma deliminated. & when the table/CSV code is run on its own outside the main datepicker loop it prints to CSV.

Code works & cycles through dates - just doesn't print data to CSV - really struggling as this is my first bit of code to write.



for d in range(9):
    # Made a list of all elements that have "ui-state-default" on the id.
    list_days = driver.find_elements_by_class_name("ui-state-default")
    # Now I get the len of that list
    count = len(list_days) - 4
    # Loop over each element of the list count
    for i in range(count):
        # And I get the link to the element
        driver.find_element_by_class_name("ui-datepicker-trigger").click()
        python_botton = driver.find_elements_by_class_name("ui-state-default")[i]
        ActionChains(driver).click(python_botton).perform()
        time.sleep(3)
        driver.find_element_by_xpath('//*[@id="spanDateTo"]/img').click()
        python_tobutton = driver.find_elements_by_class_name("ui-state-default")[i]
        ActionChains(driver).click(python_tobutton).perform()
        submit_button = driver.find_element_by_class_name('button_blue')
        submit_button.click()
        time.sleep(3)
        soup = BeautifulSoup(driver.page_source, 'lxml')
        table = soup.find('tbody', id="transactionsBody")
        tr = table.findAll(['tr'])
        csvFile = open("booking_data.csv", 'wt', newline='', encoding='utf-8')
        writer = csv.writer(csvFile)
        for cell in tr:
            tr = table.findAll(['tr'])
            td = cell.find_all('td')
            row = [i.text.replace('\n', '') for i in td]
            writer.writerow(row)
            print(row)
        csvFile.close()
    #this section will now click the next month on both calendars
    driver.find_element_by_class_name("ui-datepicker-trigger").click()
    date_next = driver.find_element_by_class_name("ui-datepicker-next")
    ActionChains(driver).click(date_next).perform()
    driver.find_element_by_link_text("2").click()
    driver.find_element_by_xpath('//*[@id="spanDateTo"]/img').click()
    date_next_to = driver.find_element_by_class_name("ui-datepicker-next")
    ActionChains(driver).click(date_next_to).perform()
    driver.find_element_by_link_text("2").click()
Reply
#2
Hey dude.

So using the same settings in regards to .csv i created something simple to check that there wasn't an issue there which there of course should not have been..

import csv

tr = ["434", "3434", "23", "4523423", "423", "654", "234", "34", "54", "65", "234", "654", "5647", "345", "234"]

csvFile = open("booking_data.csv", 'wt', newline='', encoding='utf-8')
writer = csv.writer(csvFile)
for cell in tr:
    writer.writerow(cell)
    print(cell)
csvFile.close()
Everything worked fine.

Are you getting the table data printing to console?
Reply
#3
Hi apologies, just saw this reply - it prints table data to console & have been copying and pasting from there since. Ideally the CSV bit would work


(Sep-17-2020, 12:07 PM)Aspire2Inspire Wrote: Hey dude.

So using the same settings in regards to .csv i created something simple to check that there wasn't an issue there which there of course should not have been..

import csv

tr = ["434", "3434", "23", "4523423", "423", "654", "234", "34", "54", "65", "234", "654", "5647", "345", "234"]

csvFile = open("booking_data.csv", 'wt', newline='', encoding='utf-8')
writer = csv.writer(csvFile)
for cell in tr:
    writer.writerow(cell)
    print(cell)
csvFile.close()
Everything worked fine.

Are you getting the table data printing to console?
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  how to store data from the soup in the loop? zarize 1 1,676 Mar-02-2020, 04:17 PM
Last Post: Larz60+
  strange characters after printing soup zarize 4 2,444 Jan-28-2020, 08:57 AM
Last Post: zarize

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020