Python Forum

hi, there! i am new to python and i was trying to remove duplicate numbers in a list using for loop. but the code doesn't quite work

numbers = [3, 3, 20, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
for x in numbers:
    if numbers.count(x) > 1:
        numbers.remove(x)
print(numbers)

it doesn't remove all the duplicates.

this is what i get when i run it.

Output:
[3, 6, 6, 5, 10, 7, 19, 20]

>>> numbers = [3, 3, 20, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
>>> list(set(numbers))
[3, 5, 6, 7, 10, 19, 20]

or

>>> from collections import OrderedDict
>>> numbers = [3, 3, 20, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
>>> list(OrderedDict.fromkeys(numbers))
[3, 20, 6, 10, 5, 7, 19]

@calonia Modifying a list whilst iterating over it is a very bad idea.
The 2nd example from @metulburr is perfect as it keeps the order of the items in the original list.
If that doesn´t matter his 1st example is faster.
In addition your runtime is O(len(numbers)*len(numbers)) which is bad.
numbers.count(x) runs over the full list and counts each number.
Try to do this in O(len(numbers) which is possible. (not using above examples) :-)

Also from 3.6 and guaranteed in 3.7 so are dictionaries ordered.
This mean that can drop import of OrderedDict,and just use dict().

>>> numbers = [3, 3, 20, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
>>> list(dict.fromkeys(numbers))
[3, 20, 6, 10, 5, 7, 19]

(Jun-14-2019, 04:10 PM)ThomasL Wrote: [ -> ]@calonia Modifying a list whilst iterating over it is a very bad idea.
The 2nd example from @metulburr is perfect as it keeps the order of the items in the original list.
If that doesn´t matter his 1st example is faster.
In addition your runtime is O(len(numbers)*len(numbers)) which is bad.
numbers.count(x) runs over the full list and counts each number.
Try to do this in O(len(numbers) which is possible. (not using above examples) :-)

thanks for the reply. i really appreciate your help. but i want to know what is wrong with this code. it doesn't seem to work with some of the numbers!

Have a read here

(Jun-14-2019, 05:13 PM)ThomasL Wrote: [ -> ]Have a read here

thank you very much. that was so helpful. i now get it. modifying a list while iterating through it(adding or removing items) leads to skipping indexes.

Printing is 'cheapest' debugging tool. If one adds two print statements everything is quite self-explanatory:

>>> numbers = [3, 3, 20, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
>>> for x in numbers:
...     print(f'x is {x}')
...     if numbers.count(x) > 1:
...         numbers.remove(x)
...         print(numbers)
... 
x is 3                                         # first element in numbers
[3, 20, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
x is 20                                        # second element in numbers
[3, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
x is 10                                        # third element in numbers
[3, 6, 6, 5, 5, 7, 10, 7, 19, 19, 20]
x is 5                                         # fourth element in numbers
[3, 6, 6, 5, 7, 10, 7, 19, 19, 20]
x is 7                                         # fifth element in numbers
[3, 6, 6, 5, 10, 7, 19, 19, 20]
x is 7                                         # sixth element in numbers
x is 19                                        # seventh element in numbers
[3, 6, 6, 5, 10, 7, 19, 20]
x is 20                                        # eight and last element in numbers

(Jun-14-2019, 08:41 PM)perfringo Wrote: [ -> ]Printing is 'cheapest' debugging tool. If one adds two print statements everything is quite self-explanatory:

>>> numbers = [3, 3, 20, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
>>> for x in numbers:
...     print(f'x is {x}')
...     if numbers.count(x) > 1:
...         numbers.remove(x)
...         print(numbers)
... 
x is 3                                         # first element in numbers
[3, 20, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
x is 20                                        # second element in numbers
[3, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
x is 10                                        # third element in numbers
[3, 6, 6, 5, 5, 7, 10, 7, 19, 19, 20]
x is 5                                         # fourth element in numbers
[3, 6, 6, 5, 7, 10, 7, 19, 19, 20]
x is 7                                         # fifth element in numbers
[3, 6, 6, 5, 10, 7, 19, 19, 20]
x is 7                                         # sixth element in numbers
x is 19                                        # seventh element in numbers
[3, 6, 6, 5, 10, 7, 19, 20]
x is 20                                        # eight and last element in numbers

thanks, that further illustrates my problem.

(Jun-14-2019, 06:50 PM)calonia Wrote: [ -> ]modifying a list while iterating through it(adding or removing items) leads to skipping indexes.

You should know that you can loop over a copy of a list however
The [:] returns a shallow copy of the list, in which this case doesnt matter because all of the elements are ints

numbers = [3, 3, 20, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
for x in numbers[:]:
    if numbers.count(x) > 1:
        numbers.remove(x)
print(numbers)

Output:
[3, 6, 5, 10, 7, 19, 20]