Python Forum

Full Version: deleting select items from a list
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2
i want to delete select items from a list, such as those which have a specific value in a dictionary. this often gets pretty ugly. maybe i want to remove any string that is all digits. the problem i typically run into is when i need to delete items from an existing list instead of just rebuilding a new one. i need to use indexes and when an item is deleted the items with higher indexes change to one lower. then i need another loop to recheck the new item at the same index. it makes for some ugly code.

different use cases also need different tests to decide if the item needs to be removed, making it tough to make a common function. does anyone know of something in Python that can make this easier or more elegant?

i don't want to show code because any example would need to have specific parts, like the different cases for which items to delete, and answers may focus on that.
Process the list in reverse so the removals don't change the position of the unvisited elements.
https://python-forum.io/thread-670.html Wrote:Modifying a list (or other container) while iterating over it
for laser in lasers:
    ...
    if rm:
        lasers.remove(laser)
This will cause an IndexError when you iterate over the loop as you have just removed an index from the list that you are looping over. A catch 22. You need to loop the list in order to remove, but you cannot remove from the list as you loop. This can be easily fixed by looping a copy of the list and removing from the actual list. All you have to do to loop a copy is add [:].
for laser in lasers[:]:
    ...
    if rm:
        lasers.remove(laser)
The [:] is a shallow copy. The is identical to using the copy module for copy.copy(). If you have nested structures you may want a deep copy. This you are going to need to use copy.deepcopy() See more info here
Why do you want to mutate the list, anyway?
(Oct-08-2021, 01:53 AM)Skaperen Wrote: [ -> ]different use cases also need different tests to decide if the item needs to be removed, making it tough to make a common function. does anyone know of something in Python that can make this easier or more elegant?

Yes, this is a common thing to do and hence an abstraction already exists: filter. As well as your sequence, it takes a predicate function that should return True if an item is to be kept.

>>> def is_even(x):
...     return x % 2 == 0
... 
>>> values = [1, 2, 3, 4, 5, 6]
>>> filter(is_even, values)
<filter object at 0x7f4d539e9f98>
>>> list(filter(is_even, values))
[2, 4, 6]
>>> names = ["Alice", "Bob", "Andrew", "Charlie"]
>>> list(filter(lambda name: name.startswith("A"), names))
['Alice', 'Andrew']
>>> 
If instead, you want to keep items for which a predicate is False, filterfalse in the itertools module does that.

It sounds like you'd really benefit from learning about writing programs more declaratively - Kevlin Henney has a great talk titled "Declarative thinking, declarative practice" on this kind of stuff.
Can just use plain list comprehension for this clean and easy to read.
>>> names = ["Alice", "Bob", "Andrew", "Charlie"]
>>> [name for name in names if name.startswith('A')]
['Alice', 'Andrew']
Or the same with without.
names = ["Alice", "Bob", "Andrew", "Charlie"]
new_list = []
for name in names:
    if name.startswith('A'):
        new_list.append(name)

print(new_list)
Output:
['Alice', 'Andrew']
For me is making in new list like over,is the preferred way/solution over copy [:]: and reversed().
names = ["Alice", "Bob", "Andrew", "Charlie"]
for name in names[:]:
    if not name.startswith('A'):
        names.remove(name)

print(names)
Output:
['Alice', 'Andrew']
So it work fine,but remove() has to go over the whole list for every iteration
Big-O doesn't matter here when dealing with a lists with few items,but if a make a bigger list will see a difference.
import timeit

# Modify original list
remove = '''\
names = ["Alice", "Bob", "Andrew", "Charlie"] * 100
for name in names[:]:
    if not name.startswith('A'):
        names.remove(name)'''

# Make new list
list_comp = '''\
names = ["Alice", "Bob", "Andrew", "Charlie"] * 100
[name for name in names if name.startswith('A')]'''

print(timeit.Timer(stmt=remove).timeit(number=1000000))
So here use list comprehension 55-sec and remove use 6-minute.
In my opinion, the list-comprehension is the cleanest solution.
If you have many conditions to fulfill, you can create a function which does the job for you.

def even(user):
    """
    True if uid is even

    This function could check more.
    Just a silly example
    """
    return user["uid"] % 2 == 0


# already existing data
data = [{"uid": num, "name": f"user_{num}"} for num in range(1, 101)]

# later the new list is assigned to the name data
data = [user for user in data if even(user)]
If you work inside a function and want to assign the new list, this will need the use of global, if the list is defined outside the function. This is not very nice. A class can solve this problem of reassigning without the use of global.

class Something:
    def __init__(self):
        self.database = [{"uid": num, "name": f"user_{num}"} for num in range(1, 101)]

    def add_user(self, uid, name):
        self.database.append({"uid": uid, "name": name})

    def delete_odd_users(self):
        self.database = [user for user in self.database if self.is_even(user)]

    def is_even(self, user):
        return user["uid"] % 2 == 0
If you're using threads, you should better use locking to prevent race conditions.
(Oct-08-2021, 11:56 AM)ndc85430 Wrote: [ -> ]Why do you want to mutate the list, anyway?
to have a list of select items

and i need to do this efficiently when it is deep in nested loops in many cases i have encountered. i have found that simple actions on lists in-place tend be faster.
(Oct-08-2021, 03:10 AM)bowlofred Wrote: [ -> ]Process the list in reverse so the removals don't change the position of the unvisited elements.
i've been thinking about that also to keep the actions simpler and faster. another thing i have bee trying is to replace de-selected items with None where i can change using the list to just ignore those.
(Oct-08-2021, 05:32 AM)Yoriz Wrote: [ -> ]
https://python-forum.io/thread-670.html Wrote:Modifying a list (or other container) while iterating over it
for laser in lasers:
    ...
    if rm:
        lasers.remove(laser)
This will cause an IndexError when you iterate over the loop as you have just removed an index from the list that you are looping over. A catch 22. You need to loop the list in order to remove, but you cannot remove from the list as you loop. This can be easily fixed by looping a copy of the list and removing from the actual list. All you have to do to loop a copy is add [:].
for laser in lasers[:]:
    ...
    if rm:
        lasers.remove(laser)
The [:] is a shallow copy. The is identical to using the copy module for copy.copy(). If you have nested structures you may want a deep copy. This you are going to need to use copy.deepcopy() See more info here
what if i loop the list in reverse as suggested above? will i run into an index error while mutating the list that way? speaking of indexes, i didn't know i could get indexes that way (line 1 in your first box of code). i had been using len() and range().
Pages: 1 2