Python Forum
deleting select items from a list
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
deleting select items from a list
#1
i want to delete select items from a list, such as those which have a specific value in a dictionary. this often gets pretty ugly. maybe i want to remove any string that is all digits. the problem i typically run into is when i need to delete items from an existing list instead of just rebuilding a new one. i need to use indexes and when an item is deleted the items with higher indexes change to one lower. then i need another loop to recheck the new item at the same index. it makes for some ugly code.

different use cases also need different tests to decide if the item needs to be removed, making it tough to make a common function. does anyone know of something in Python that can make this easier or more elegant?

i don't want to show code because any example would need to have specific parts, like the different cases for which items to delete, and answers may focus on that.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#2
Process the list in reverse so the removals don't change the position of the unvisited elements.
Reply
#3
https://python-forum.io/thread-670.html Wrote:Modifying a list (or other container) while iterating over it
for laser in lasers:
    ...
    if rm:
        lasers.remove(laser)
This will cause an IndexError when you iterate over the loop as you have just removed an index from the list that you are looping over. A catch 22. You need to loop the list in order to remove, but you cannot remove from the list as you loop. This can be easily fixed by looping a copy of the list and removing from the actual list. All you have to do to loop a copy is add [:].
for laser in lasers[:]:
    ...
    if rm:
        lasers.remove(laser)
The [:] is a shallow copy. The is identical to using the copy module for copy.copy(). If you have nested structures you may want a deep copy. This you are going to need to use copy.deepcopy() See more info here
Reply
#4
Why do you want to mutate the list, anyway?
Reply
#5
(Oct-08-2021, 01:53 AM)Skaperen Wrote: different use cases also need different tests to decide if the item needs to be removed, making it tough to make a common function. does anyone know of something in Python that can make this easier or more elegant?

Yes, this is a common thing to do and hence an abstraction already exists: filter. As well as your sequence, it takes a predicate function that should return True if an item is to be kept.

>>> def is_even(x):
...     return x % 2 == 0
... 
>>> values = [1, 2, 3, 4, 5, 6]
>>> filter(is_even, values)
<filter object at 0x7f4d539e9f98>
>>> list(filter(is_even, values))
[2, 4, 6]
>>> names = ["Alice", "Bob", "Andrew", "Charlie"]
>>> list(filter(lambda name: name.startswith("A"), names))
['Alice', 'Andrew']
>>> 
If instead, you want to keep items for which a predicate is False, filterfalse in the itertools module does that.

It sounds like you'd really benefit from learning about writing programs more declaratively - Kevlin Henney has a great talk titled "Declarative thinking, declarative practice" on this kind of stuff.
Reply
#6
Can just use plain list comprehension for this clean and easy to read.
>>> names = ["Alice", "Bob", "Andrew", "Charlie"]
>>> [name for name in names if name.startswith('A')]
['Alice', 'Andrew']
Or the same with without.
names = ["Alice", "Bob", "Andrew", "Charlie"]
new_list = []
for name in names:
    if name.startswith('A'):
        new_list.append(name)

print(new_list)
Output:
['Alice', 'Andrew']
For me is making in new list like over,is the preferred way/solution over copy [:]: and reversed().
names = ["Alice", "Bob", "Andrew", "Charlie"]
for name in names[:]:
    if not name.startswith('A'):
        names.remove(name)

print(names)
Output:
['Alice', 'Andrew']
So it work fine,but remove() has to go over the whole list for every iteration
Big-O doesn't matter here when dealing with a lists with few items,but if a make a bigger list will see a difference.
import timeit

# Modify original list
remove = '''\
names = ["Alice", "Bob", "Andrew", "Charlie"] * 100
for name in names[:]:
    if not name.startswith('A'):
        names.remove(name)'''

# Make new list
list_comp = '''\
names = ["Alice", "Bob", "Andrew", "Charlie"] * 100
[name for name in names if name.startswith('A')]'''

print(timeit.Timer(stmt=remove).timeit(number=1000000))
So here use list comprehension 55-sec and remove use 6-minute.
Reply
#7
In my opinion, the list-comprehension is the cleanest solution.
If you have many conditions to fulfill, you can create a function which does the job for you.

def even(user):
    """
    True if uid is even

    This function could check more.
    Just a silly example
    """
    return user["uid"] % 2 == 0


# already existing data
data = [{"uid": num, "name": f"user_{num}"} for num in range(1, 101)]

# later the new list is assigned to the name data
data = [user for user in data if even(user)]
If you work inside a function and want to assign the new list, this will need the use of global, if the list is defined outside the function. This is not very nice. A class can solve this problem of reassigning without the use of global.

class Something:
    def __init__(self):
        self.database = [{"uid": num, "name": f"user_{num}"} for num in range(1, 101)]

    def add_user(self, uid, name):
        self.database.append({"uid": uid, "name": name})

    def delete_odd_users(self):
        self.database = [user for user in self.database if self.is_even(user)]

    def is_even(self, user):
        return user["uid"] % 2 == 0
If you're using threads, you should better use locking to prevent race conditions.
My code examples are always for Python >=3.6.0
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Reply
#8
(Oct-08-2021, 11:56 AM)ndc85430 Wrote: Why do you want to mutate the list, anyway?
to have a list of select items

and i need to do this efficiently when it is deep in nested loops in many cases i have encountered. i have found that simple actions on lists in-place tend be faster.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#9
(Oct-08-2021, 03:10 AM)bowlofred Wrote: Process the list in reverse so the removals don't change the position of the unvisited elements.
i've been thinking about that also to keep the actions simpler and faster. another thing i have bee trying is to replace de-selected items with None where i can change using the list to just ignore those.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#10
(Oct-08-2021, 05:32 AM)Yoriz Wrote:
https://python-forum.io/thread-670.html Wrote:Modifying a list (or other container) while iterating over it
for laser in lasers:
    ...
    if rm:
        lasers.remove(laser)
This will cause an IndexError when you iterate over the loop as you have just removed an index from the list that you are looping over. A catch 22. You need to loop the list in order to remove, but you cannot remove from the list as you loop. This can be easily fixed by looping a copy of the list and removing from the actual list. All you have to do to loop a copy is add [:].
for laser in lasers[:]:
    ...
    if rm:
        lasers.remove(laser)
The [:] is a shallow copy. The is identical to using the copy module for copy.copy(). If you have nested structures you may want a deep copy. This you are going to need to use copy.deepcopy() See more info here
what if i loop the list in reverse as suggested above? will i run into an index error while mutating the list that way? speaking of indexes, i didn't know i could get indexes that way (line 1 in your first box of code). i had been using len() and range().
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Getting All Items From A List knight2000 4 448 Sep-25-2021, 12:56 AM
Last Post: knight2000
  Feed List items with Integer euras 9 1,290 May-19-2021, 07:45 PM
Last Post: snippsat
  Parse String between 2 Delimiters and add as single list items lastyle 5 1,091 Apr-11-2021, 11:03 PM
Last Post: lastyle
  Sum of list items tristanfermat 2 820 Feb-06-2021, 12:04 PM
Last Post: tristanfermat
  Deleting employee from list SephMon 3 921 Jan-05-2021, 04:15 AM
Last Post: deanhystad
  Count number of occurrences of list items in list of tuples t4keheart 1 931 Nov-03-2020, 05:37 AM
Last Post: deanhystad
  concatenating 2 items at a time in a python list K11 3 916 Oct-21-2020, 09:34 AM
Last Post: buran
  Select the other of 2 items in a list Clunk_Head 7 1,437 Sep-01-2020, 05:27 PM
Last Post: Clunk_Head
  Removing items from list if containing a substring pythonnewbie138 2 931 Aug-27-2020, 10:20 PM
Last Post: pythonnewbie138
  Select correct item from list for subprocess command pythonnewbie138 6 1,306 Jul-24-2020, 09:09 PM
Last Post: pythonnewbie138

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020