using mutable in function defintion as optional paramter

[akbarza]

hi
I read some about using mutable(list or dict) in function definition as an optional parameter as this was discussed in the site realpython(its address is in the below code). also, I talked about this subject in this forum before.

in the below code:

#mutable_in_func_def.py
'''
problems that arise when using mutables(such as lists or dictionaries) in functions
definition as default values.
ref:https://realpython.com/python-optional-arguments/#using-python-optional-arguments-with-default-values
'''
# optional_params.py

hardware_store_list = {}
supermarket_list = {}
chimical_store_list={}

def show_list(shopping_list, include_quantities=True):
    print()
    for item_name, quantity in shopping_list.items():
        if include_quantities:
            print(f"{quantity}x {item_name}")
        else:
            print(item_name)

def add_item(item_name, quantity, shopping_list={}):
    if item_name in shopping_list.keys():
        shopping_list[item_name] += quantity
    else:
        shopping_list[item_name] = quantity

    return shopping_list   # This line technically is not needed. because:
                           # shopping_list is a dictionary so mutable, so change in it
                           # causes a change in shoping_list in the whole of the program.


supermarket_list = add_item("Bread", 1, supermarket_list)
supermarket_list = add_item("Milk", 2, supermarket_list)

show_list(supermarket_list)


print('-'*50)    #-------------------------------------------------------------
clothes_shop_list = add_item("Shirt", 3)  # with no argument corresponding to shopping_list        line39
electronics_store_list = add_item("USB cable", 1)               #line40
chimical_store_list=add_item('baloon',2)

show_list(clothes_shop_list)        #line43
show_list(electronics_store_list)
show_list(chimical_store_list)
     #oh, outputs of 3 above lines are the same!?

def fixed_add_item(item_name, quantity, shopping_list=None):
    if shopping_list is None:
        shopping_list = {}
    if item_name in shopping_list.keys():
        shopping_list[item_name] += quantity
    else:
        shopping_list[item_name] = quantity

    return shopping_list


print('*'*25,' with fixed_add_item function: ','*'*25)    #****************************
clothes_shop_list = fixed_add_item("Shirt", 3)
electronics_store_list = fixed_add_item("USB cable", 1)
show_list(clothes_shop_list)
show_list(electronics_store_list)

I debugged this code. when debugging, in line 39, clothes_shop_list was {"shirt":3}. after running and debugging line40, electronics_store_list was {"shirt":3,"USB cable":1}, but now clothes_shop_list was changed to {"shirt":3,"USB cable":1}
this also occurred in line 41 that after running this line all three chimical_store_list, electronics_store_list, and clothes_shop_list were changed to {"shirt":3,"USB cable":1,"baloon":2}.
in the above, I do not understand why by the creation of a new list, the before lists were changed.
can explain it?
thanks.

[snippsat]

You mix in example of bad version and the correct one,so it get confusing when look your code for this.
The blog post show many different version of not correct ways,and explain why.

To sum it up and show what you should use in case,can also mention that eg a class or @dataclass could maybe fit better for this.

def add_item(item_name, quantity, shopping_list=None):
    '''This effectively maintains separation between different dict,
     and avoids any issues with mutable default arguments in the function definition.
     '''
    if shopping_list is None:
        shopping_list = {}
    if item_name in shopping_list:
        shopping_list[item_name] += quantity
    else:
        shopping_list[item_name] = quantity
    return shopping_list

if __name__ == '__main__':
    supermarket_list = add_item("Bread", 2)
    supermarket_list = add_item("Milk", 3, supermarket_list)
    # Creating a new hardware store list
    hardware_store_list = add_item("Nails", 100)
    hardware_store_list = add_item("Hammer", 1, hardware_store_list)
    hardware_store_list = add_item("Saw", 2, hardware_store_list)
    print(supermarket_list)
    print(hardware_store_list)

{'Bread': 2, 'Milk': 3}
{'Nails': 100, 'Hammer': 1, 'Saw': 2}

Now that this is correct and avoid any side effects due to mutable defaults.
Can add eg a show_result function.

def show_result(**shopping_lists):
    for list_name, items in shopping_lists.items():
        print(f"\n{list_name.replace('_', ' ').title()} List:")
        for item_name, quantity in items.items():
            print(f"{quantity}x {item_name}")

def add_item(item_name, quantity, shopping_list=None):
    if shopping_list is None:
        shopping_list = {}
    if item_name in shopping_list:
        shopping_list[item_name] += quantity
    else:
        shopping_list[item_name] = quantity
    return shopping_list

if __name__ == '__main__':
    supermarket_list = add_item("Bread", 2)
    supermarket_list = add_item("Milk", 3, supermarket_list)
    # Creating a new hardware store list
    hardware_store_list = add_item("Nails", 100)
    hardware_store_list = add_item("Hammer", 1, hardware_store_list)
    hardware_store_list = add_item("Saw", 2, hardware_store_list)
    #print(supermarket_list)
    #print(hardware_store_list)
    show_result(supermarket=supermarket_list, hardware_store=hardware_store_list)

Output:
Supermarket List:
2x Bread
3x Milk

Hardware Store List:
100x Nails
1x Hammer
2x Saw

[akbarza]

hi snppsat
thanks for reply
i read the mentioned page all, but i asked why after production(or creation) the clothes_shop_list dictionary in line 39, the dictionary changes when line 40 is runed and the dictionary electronics_store_list is created.

[snippsat]

but i asked why after production(or creation) the clothes_shop_list dictionary in line 39, the dictionary changes when line 40 is runed and the dictionary electronics_store_list is created.

That's the problem link try to explain,problem arises because the way default arguments work in Python function definitions,
particularly with mutable eg objects like dictionaries,list ect.
To break it down.
When you call add_item("Shirt", 3) without a specific shopping_list,
the default shopping_list={} is used.
This default is not a new empty dictionary on each call,it's the same dictionary that was created when the function was defined.
Because of this,add_item("Shirt", 3), add_item("USB cable", 1), and add_item('baloon', 2) all modify the same dictionary.
Thus, each call does not start with an empty dictionary but continues adding to the single dictionary initialized when the function was first defined.

The solution to avoid this is as posted.

def add_item(item_name, quantity, shopping_list=None):
    if shopping_list is None:
        shopping_list = {}
    if item_name in shopping_list:
        shopping_list[item_name] += quantity
    else:
        shopping_list[item_name] = quantity
    return shopping_list

[deanhystad]

Maybe this makes it more clear.

def incorrect(item, quantity, shopping_list={}):
    shopping_list[item] = shopping_list.get(item, 0) + quantity
    return shopping_list  # Is needed else there is no way to get the "default" shopping list


default_dictionary = {}
def whats_happening(item, quantity, shopping_list=default_dictionary):
    shopping_list[item] = shopping_list.get(item, 0) + quantity
    return shopping_list  # Is needed else there is no way to get the "default" shopping list


def correct(item, quantity, shopping_list=None):
    if shopping_list is None:
        shopping_list = {}
    shopping_list[item] = shopping_list.get(item, 0) + quantity
    return shopping_list  # Is needed else there is no way to get the "default" shopping list


def test(func):
    a = func("Shirt", 3)
    b = func("USB cable", 1)
    print(func.__name__, a, b, "a is b" if a is b else "a is not b")


test(incorrect)
test(whats_happening)
test(correct)

Output:
incorrect {'Shirt': 3, 'USB cable': 1} {'Shirt': 3, 'USB cable': 1} a is b
whats_happening {'Shirt': 3, 'USB cable': 1} {'Shirt': 3, 'USB cable': 1} a is b
correct {'Shirt': 3} {'USB cable': 1} a is not b

Notice that incorrect() and whats_happening() have the same issue. The default dictionary gets reused over and over instead of creating a new dictionary each time. Dictionary a and dictionary b are the same object. Why this happens is easier to see when the dictionary is created explicitly, but explicit or implicit the result is the same. One dictionary is created as the default, and that same dictionary is used every time it is needed. In your example, every dictionary created by calling add_item() without a shopping list returns the same dictionary object.

[snippsat]

As i mention this task can be better to solve with with eg @dataclass
By using default_factory=dict,a new dictionary is created each time an instance of the dataclass is created,
ensuring that each instance has its own independent dictionary.

from dataclasses import dataclass, field

@dataclass
class ShoppingList:
    items: dict = field(default_factory=dict)

    def add_item(self, item_name: str, quantity: int):       
        if item_name in self.items:
            self.items[item_name] += quantity
        else:
            self.items[item_name] = quantity

    def show_one_list(self):    
        for item_name, quantity in self.items.items():
            print(f"{quantity}x {item_name}")
    
    @staticmethod
    def show_result(**shopping_lists):
        """Print all items in this shopping lists with quantities"""
        for list_name, shopping_list in shopping_lists.items():
            print(f"\n{list_name.replace('_', ' ').title()} List:")
            for item_name, quantity in shopping_list.items.items():
                print(f"{quantity}x {item_name}")

if __name__ == '__main__':
    supermarket = ShoppingList()
    hardware_store = ShoppingList()    
    # Add items to the supermarket list
    supermarket.add_item("Bread", 2)
    supermarket.add_item("Milk", 3)
    supermarket.add_item("Eggs", 12)
    # Add items to the hardware store list
    hardware_store.add_item("Nails", 100)
    hardware_store.add_item("Hammer", 1)
    # Display one shopping lists
    hardware_store.show_one_list()
    # Display all shopping lists   
    ShoppingList.show_result(supermarket=supermarket, hardware_store=hardware_store)

Output:
100x Nails
1x Hammer

Supermarket List:
2x Bread
3x Milk
12x Eggs

Hardware Store List:
100x Nails
1x Hammer

[Pedroski55]

For some reason the shopping list part reminds me of a joke:

"""
Wife to computer geek husband: "Buy a loaf of bread, and if they have eggs, buy 6"

Husband thinks:

def buy(something, eggs=None):
    if eggs:
        return 6 * something
    return 1 * something

Husband in shop: "Do you have eggs?"
Shop assistant: "Yes."
Husband: "6 loaves of bread please"
"""

Is there a good reason why we should not pass the dictionary we want modified by name?

def add_item(item_name, quantity, adict):
    if adict.get(item_name) == None:
        adict[item_name] = quantity
    else:
        adict[item_name] += quantity

add_item('T-shirt', 5, clothes_shop_list)

[DeaD_EyE]

Is there a good reason why we should not pass the dictionary we want modified by name?

It's not expected that a function mutates objects from the arguments. But sporadically, it's even done in the standard library. One example is heapq.heapify, which mutates the list.

The example with the dataclass is better because it has isolation. The item dict is attached as an attribute to the dataclass-instance.

[snippsat]

Is there a good reason why we should not pass the dictionary we want modified by name?

Can do something like this,but think how this solution would look if adding more shopping list.
Then have to make dictionary's global eg clothes_shop_list = {}, hardware_store = {} and add to them.
Also a ordinary class make more sense for this,if not familiar with @dataclass.

class ShoppingList:
    def __init__(self, name):
        self.name = name
        self.items = {}

    def add_item(self, item_name, quantity):
        if item_name in self.items:
            self.items[item_name] += quantity
        else:
            self.items[item_name] = quantity

    def show_list(self):
        print(f"{self.name} Shopping List:")
        for item, quantity in self.items.items():
            print(f"{quantity} x {item}")
        print("-" * 25)

# Create separate shopping list objects with names
clothes_list = ShoppingList("Clothes")
electronics_list = ShoppingList("Electronics")
# Add items to each shopping list
clothes_list.add_item('T-shirt', 5)
clothes_list.add_item('Jeans', 2)
electronics_list.add_item('USB cable', 3)
electronics_list.add_item('Headphones', 1)
# Display each shopping list
clothes_list.show_list()
electronics_list.show_list()

Output:
Clothes Shopping List:
5 x T-shirt
2 x Jeans
-------------------------
Electronics Shopping List:
3 x USB cable
1 x Headphones
-------------------------