Python Forum

Full Version: How to remove multiples in a list
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
How can remove something in a list if it appears more than once?
examples:

list1 = [1, 2, 3, 4, 1]
>>>>>
list1 = [1, 2, 3, 4]


or


list2 = ['a', 'b', 'a', 'a']
>>>>>
list2 = ['a', 'b']
What have you tried? Is order important or not?
I have tried the following:
old_list = ['a', 'b', 'a', 'a', 'b']
new_list = []
for i in range(len(old_list)):
    if old_list[i] not in new_list:   
        new_list.append(old_list[i])
print(new_list)
>>>>>
#This prints: ['a', 'b']
This works ofcourse, but i was wondering if there was an easier way of doing this.
Or atleast something less than 5 lines of code in order to make my code look a little bit cleaner.
I was thinking that maybe there was a command for it like

list1 = ['a', 'b', 'a', 'a', 'b']
remove_multiples(list1)
print(list1)
>>>>>
#So it will print:['a', 'b']
Order is not important.
If order is not important, you can use a set. A set can only have one of each value, so it can very efficiently strip out duplicates:

list1 = ['a', 'b', 'a', 'a', 'b']
uniques = set(list1)               # set(['a', 'b'])
uniques2 = list(set(list1))        # ['a', 'b']
Order is generally lost when you convert to a set, however.
Thanks!
jasper100125 Wrote:This works ofcourse, but i was wondering if there was an easier way of doing this.
If you don't mind using a third party library, there is
from more_itertools import unique_everseen
new_list = list(unique_everseen(old_list))
if order is important, in 3.7+ you can use order preserving feature of dicts
>>> spam = ['a', 'b', 'a', 'c', 'b', 'a']
>>> eggs = list(dict(zip(spam, spam)).keys())
>>> eggs
['a', 'b', 'c']
>>>
before 3.7 you need to use collections.OrderedDict instead.

also it's worth mentioning that this solution as well using set() will work only if list elements are hashable.