write to csv file problem

[jacksfrustration]

i have the following function:

my_list=save_ent()
    new_list=[]

    for ent in my_list:
        new_list.append(output_found_flights(ent))
    try:
        data_file=pd.read_csv("flight_data.csv")
        data_dict=data_file.to_dict(orient="records")
    except pandas.errors.EmptyDataError:
        messagebox.showerror(title="Oooops",message="There are no saved information")
        data_dict=[]
    finally:
        for ent in new_list:
            data_dict.append(ent)
        try:
            for i,ent in enumerate(data_dict):
                res= not all (ent.values())
                if str(res):
                    data_dict.pop(i)
        except AttributeError:
            data_dict.pop(i)

        finally:

            new_data_file=pd.DataFrame(data_dict)
            new_data_file.to_csv("flight_data.csv",index=False)

the problem is the first time i run the code it executes correctly. When i try to add more search results it overwrites the csv file as expected but it gets rid of the previous search result and replaces it with the following

Output:
0
""

[noisefloor]

You need to set the mode for writing the file. The default is mode='w' which truncates the file prior to writing. You need to set mode='a' for append mode:

...
new_data_file.to_csv("flight_data.csv",index=False, mode='a')

See pandas documentation on to_csv for more information.

Regards, noisefloor

[deanhystad]

Why are you reading in the file instead of appending to the end?

But assuming there is a reason for loading in the file, I have some comments on the posted code.

This deletes all the items from data_dict. Well, it would, but you get an IndexError when the value of i is greater or equal to the number items in data_dict.

            for i,ent in enumerate(data_dict):
                res= not all (ent.values())
                if str(res):
                    data_dict.pop(i)

all(ent.values()) will be True or False. str(res) will be either the str "True" or the str "False". bool(str) for any non-empty str is True, so this code tries to pop all entries in data_dict.

But that isn't the only odd thing about the loop. Let's correct the above mistake and examine further.

        try:
            for i,ent in enumerate(data_dict):
                if not all (ent.values()):
                    data_dict.pop(i)
        except AttributeError:
            data_dict.pop(i)

What is the purpose of the try/except? Are you concerned that data_dict might contain things other than dictionaries? If so, the try is in the wrong place. The code should be this:

       for i,ent in enumerate(data_dict):
            try:
                if not all (ent.values()):
                    data_dict.pop(i)
            except AttributeError:
                data_dict.pop(i)

In your code, an AttributeError kicked you out of the for loop, leaving the remainder of the data_dict unprocessed. In the modified code above, the AttributeError is inside the for loop, so the loop continues to process the data_dict.

But you still have the problem of the index error. That's probably why you have all those "finally" statements. Deleting values from something you are indexing is almost always bad. I would write the code the other way around. Instead of adding items to data_dict then removing the bad items, I would not add bad items to data_dict in the first place.

import pandas as pd

data_file = "flight_data.csv"
modified = False
data = []
try:
    # Load existing data from file
    try:
        data = pd.read_csv(data_file).to_dict(orient="records")
    except (FileNotFoundError, pd.errors.EmptyDataError):
        messagebox.showerror(title="Oooops", message="There are no saved information")


    # Add new data
    for ent in save_ent():
        try:
            ent = output_found_flights(ent)
            if all(ent.values()):
                data.append(ent)
                modified = True
        except AttributeError:
            pass

finally:  # Will catch any exceptions we might have missed.
    # Save modified data to file.
    if data and modified:
        pd.DataFrame(data).to_csv(data_file, index=False)

[deanhystad]

If you go the append route, you probably only want to write the header when the file is created.

import pandas as pd
from pathlib import Path
 

data = []
try:
    # Filter out bad entries
    for ent in save_ent():
        try:
            ent = output_found_flights(ent)
            if all(ent.values()):
                data.append(ent)
        except AttributeError:
            pass
 
finally:  # Will "catch" other types of errors.
    # Save data if we have some.  Only write the header when the file is created.
    if data:
        file = Path("flight_data.csv")
        pd.DataFrame(data).to_csv(file, index=False, mode="a", header=not file.exists())

[jacksfrustration]

i have the following function:

my_list=save_ent()
    new_list=[]

    for ent in my_list:
        new_list.append(output_found_flights(ent))
    try:
        data_file=pd.read_csv("flight_data.csv")
        data_dict=data_file.to_dict(orient="records")
    except pandas.errors.EmptyDataError:
        messagebox.showerror(title="Oooops",message="There are no saved information")
        data_dict=[]
    finally:
        for ent in new_list:
            data_dict.append(ent)
        try:
            for i,ent in enumerate(data_dict):
                res= not all (ent.values())
                if str(res):
                    data_dict.pop(i)
        except AttributeError:
            data_dict.pop(i)

        finally:

            new_data_file=pd.DataFrame(data_dict)
            new_data_file.to_csv("flight_data.csv",index=False)

the problem is the first time i run the code it executes correctly. When i try to add more search results it overwrites the csv file as expected but it gets rid of the previous search result and replaces it with the following

Output:
0
""

ok i tried changing the mode as you suggested. i have the following code now

    my_list=save_ent()
    new_list=[]

    for ent in my_list:
        
        new_list.append(output_found_flights(ent))
    new_data_file=pd.DataFrame(new_list)
    new_data_file.to_csv("flight_data.csv",index=False,mode="a")

but i still get the following addition on my csv file

Output:
0
""

in my code i have the output_found_flights function being called. the code for that follows

def output_found_flights(ent):
    '''connects to the api in order to find flights. information from entries are received
    generates values in the current entry that is supplied when the function is called
    and returns the updated dictionary file'''
    query = {
        "fly_from": ent['from_iata_code'],
        "fly_to": ent['to_iata_code'],
        "date_from": ent['from_time'],
        "date_to": ent["to_time"],
        "nights_in_dst_from": 7,
        "nights_in_dst_to": 28,
        "flight_type": "round",
        "one_for_city": 1,
        "max_stopovers": 0,
        "curr": "GBP"
    }
    headers = {
        "apikey": API_KEY
    }
    endpoint = "https://api.tequila.kiwi.com/v2/search"
    response = requests.get(url=endpoint, headers=headers, params=query)
    flights = response.json()
    try:
        lowest_price = round(float(ent["lowest_price"]), 2)
    except ValueError:
        messagebox.showerror(title="Ooops",
                             message="You have not entered appropriate input for the lowest price variable")

    for i in range(0, len(flights['data'])):
        flight_price = round(float(flights['data'][0]['price']), 2)
        if flight_price <= lowest_price:
            adult_price=flights['data'][i]['fare']['adults']
            children_price=flights['data'][i]['fare']['children']
            infants_price=flights['data'][i]['fare']['infants']
            total_price=float((adult_price * ent['adults'] ) + (children_price * ent['children']) + (infants_price * ent['infants']))
            url = flights['data'][i]['deep_link']
            date = flights['data'][i]['local_departure']
            date_separator = date.find("T")
            departure_date = date[:date_separator]
            date_list = departure_date.split("-")
            date_formatted = f"{date_list[2]}/{date_list[1]}/{date_list[0]}"
            ent['trip_date'] = date_formatted
            ent["ticket_price"] = flight_price
            ent['total_price']=total_price
            ent['url'] = url
            try:
                if messagebox.askokcancel(title=f"{ent['trip_date']} from {ent['from_city_name']} to {ent['to_city_name']}",message=f"Ticket price is £{ent['total_price']}0\n"
                                                                                                                                       f"{int(ent['adults'])} adults are travelling with {int(ent['children'])} children and {int(ent['infants'])} infants.\nPress ok to open the URL in order to buy the tickets"):
                    webbrowser.open(ent['url'])
            except ValueError:
                messagebox.showerror(title="Ooops",message="No flights were found")


            return ent

ent from the output_found_flights gets returned as empty the way i understand it. that is why i included the function code so you can help me figure out how to beat this

[deanhystad]

ent from the output_found_flights gets returned as empty

Really? output_from_flights either returns a dictionary or it returns None. None is not the same as empty. {} is empty.

When you run your code, do you see the message box asking if it is ok to open the web browser? If you don't see that, do you see this error messge?

messagebox.showerror(title="Ooops",message="No flights were found")

I think it likely the query returns nothing and your code raises a key error when it tries to use flights['data']. Even if the request was successful I think there are several errors in the function that will raise exceptions. Verify that the request is made at all.

This makes no sense:

    try:
        lowest_price = round(float(ent["lowest_price"]), 2)
    except ValueError:
        messagebox.showerror(title="Ooops",
                             message="You have not entered appropriate input for the lowest price variable")

Popping up a message window is not handling the error. Immediately after this code you send your request and process the response even though you haven't assigned a value to lowest_price. This will result in the code raising a NameError when you try to use lowest_price in the function.

Python is not C, use iterators instead of indexing. Instead of this:

    for i in range(0, len(flights['data'])):
        flight_price = round(float(flights['data'][0]['price']), 2)  # <- Notice the index error.  Should be ['data'][i]
        if flight_price <= lowest_price:

Do this:

    for flight in flights["data"]:
        if flight["price"] <= lowest_price:

Looking at your code I think you are trying to do too much at once. Break up the program into smaller parts an solve them individually. For example, write a program that only does the request part. What happens if you make an invalid request? How do can you identify and correct the problem. How do you filter the responses? Currently you look for a price lower than your entered price. This might ignore lower prices. Once you are happy with how the request and filtering work you can fold it back into your program.

[jacksfrustration]

ok so i changed the flights to print the output
the output of found flights function returns None with no pop up messages
the code for found flights follows

def output_found_flights(ent):
    '''connects to the api in order to find flights. information from entries are received
    generates values in the current entry that is supplied when the function is called
    and returns the updated dictionary file'''
    query = {
        "fly_from": ent['from_iata_code'],
        "fly_to": ent['to_iata_code'],
        "date_from": ent['from_time'],
        "date_to": ent["to_time"],
        "nights_in_dst_from": 7,
        "nights_in_dst_to": 28,
        "flight_type": "round",
        "one_for_city": 1,
        "max_stopovers": 0,
        "curr": "GBP"
    }
    headers = {
        "apikey": API_KEY
    }
    endpoint = "https://api.tequila.kiwi.com/v2/search"
    response = requests.get(url=endpoint, headers=headers, params=query)
    flights = response.json()
    try:
        lowest_price = round(float(ent["lowest_price"]), 2)
    except ValueError:
        messagebox.showerror(title="Ooops",
                             message="You have not entered appropriate input for the lowest price variable")

    for flight in flights['data']:
        flight_price = round(float(flights['data'][0]['price']), 2)
        if flight['price'] <= lowest_price:
            adult_price=flights['data'][0]['fare']['adults']
            children_price=flights['data'][0]['fare']['children']
            infants_price=flights['data'][0]['fare']['infants']
            total_price=float((adult_price * ent['adults'] ) + (children_price * ent['children']) + (infants_price * ent['infants']))
            url = flights['data'][0]['deep_link']
            date = flights['data'][0]['local_departure']
            date_separator = date.find("T")
            departure_date = date[:date_separator]
            date_list = departure_date.split("-")
            date_formatted = f"{date_list[2]}/{date_list[1]}/{date_list[0]}"
            ent['trip_date'] = date_formatted
            ent["ticket_price"] = flight_price
            ent['total_price']=total_price
            ent['url'] = url
            try:
                if messagebox.askokcancel(title=f"{ent['trip_date']} from {ent['from_city_name']} to {ent['to_city_name']}",message=f"Ticket price is £{ent['total_price']}0\n"
                                                                                                                                       f"{int(ent['adults'])} adults are travelling with {int(ent['children'])} children and {int(ent['infants'])} infants.\nPress ok to open the URL in order to buy the tickets"):
                    webbrowser.open(ent['url'])
            except ValueError:
                messagebox.showerror(title="Ooops",message="No flights were found")
        else:
            messagebox.showerror(title="No flight found",message="No flights found that fit your criteria")

i cant understand where it is going wrong. if there are no flights then according to my try statements and if statements i should get a pop up saying no flights were found. But instead all i get is return None from the above function

What am i doing wrong here?

QUick edit: after trying once more. i have found a flight using different cities but it did not save the flight in the csv file which according to the code below should have been appended to the file

my_list=save_ent()
    new_list=[]

    for ent in my_list:

        new_list.append(output_found_flights(ent))
    new_data_file=pd.DataFrame(new_list)
    new_data_file.to_csv("flight_data.csv",index=False,mode="a")

[deanhystad]

What is "flights" when there are no matching flights? When you know that you should know why your code does not identify that case.

[jacksfrustration]

What is "flights" when there are no matching flights? When you know that you should know why your code does not identify that case.

sometimes flights doesnt even get executed because the return of the save_ent function is of type None. i have tried checking the variable that i write to the csv file and sometimes it is None. I have tried writing it using pandas Dataframe as below

d_f=pd.DataFrame(new_list)
d_f.to_csv("flight_data.csv",index=False,mode="a")

i have also tried using the csv framework like

wth open("flight_data.csv","a")as file:
    writer=csv.writer(file)
    writer.writerrows(new_list)

in either occassion does it write or print out a message. In most situations i had a

Output:
0
""

being appended to my csv file or no ouput whatsoever. I have also tried both writer.writerows or writer.writerow to no avail. the weird thing is that the names of my rows are always there its the values that never get written

[deanhystad]

If there is nothing to write it doesn't matter how you try to write it. Fix the problem of there being nothing to write. Start by fixing saveEnt(). Should it ever return None?