Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 removing duplicate numbers from a list
#1
hi, there! i am new to python and i was trying to remove duplicate numbers in a list using for loop. but the code doesn't quite work

numbers = [3, 3, 20, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
for x in numbers:
    if numbers.count(x) > 1:
        numbers.remove(x)
print(numbers)
it doesn't remove all the duplicates.

this is what i get when i run it.

Output:
[3, 6, 6, 5, 10, 7, 19, 20]
Yoriz wrote Jun-14-2019, 03:28 PM:
Please post all code, output and errors (in it's entirety) between their respective tags. I did it for you this time, Here are instructions on how to do it yourself next time.
Quote
#2
>>> numbers = [3, 3, 20, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
>>> list(set(numbers))
[3, 5, 6, 7, 10, 19, 20]
or

>>> from collections import OrderedDict
>>> numbers = [3, 3, 20, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
>>> list(OrderedDict.fromkeys(numbers))
[3, 20, 6, 10, 5, 7, 19]
snippsat likes this post
Quote
#3
@calonia Modifying a list whilst iterating over it is a very bad idea.
The 2nd example from @metulburr is perfect as it keeps the order of the items in the original list.
If that doesn´t matter his 1st example is faster.
In addition your runtime is O(len(numbers)*len(numbers)) which is bad.
numbers.count(x) runs over the full list and counts each number.
Try to do this in O(len(numbers) which is possible. (not using above examples) :-)
snippsat and metulburr like this post
Quote
#4
Also from 3.6 and guaranteed in 3.7 so are dictionaries ordered.
This mean that can drop import of OrderedDict,and just use dict().
>>> numbers = [3, 3, 20, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
>>> list(dict.fromkeys(numbers))
[3, 20, 6, 10, 5, 7, 19]
metulburr likes this post
Quote
#5
(Jun-14-2019, 04:10 PM)ThomasL Wrote: @calonia Modifying a list whilst iterating over it is a very bad idea.
The 2nd example from @metulburr is perfect as it keeps the order of the items in the original list.
If that doesn´t matter his 1st example is faster.
In addition your runtime is O(len(numbers)*len(numbers)) which is bad.
numbers.count(x) runs over the full list and counts each number.
Try to do this in O(len(numbers) which is possible. (not using above examples) :-)

thanks for the reply. i really appreciate your help. but i want to know what is wrong with this code. it doesn't seem to work with some of the numbers!
Quote
#6
Have a read here
Quote
#7
(Jun-14-2019, 05:13 PM)ThomasL Wrote: Have a read here

thank you very much. that was so helpful. i now get it. modifying a list while iterating through it(adding or removing items) leads to skipping indexes.
Quote
#8
Printing is 'cheapest' debugging tool. If one adds two print statements everything is quite self-explanatory:

>>> numbers = [3, 3, 20, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
>>> for x in numbers:
...     print(f'x is {x}')
...     if numbers.count(x) > 1:
...         numbers.remove(x)
...         print(numbers)
... 
x is 3                                         # first element in numbers
[3, 20, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
x is 20                                        # second element in numbers
[3, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
x is 10                                        # third element in numbers
[3, 6, 6, 5, 5, 7, 10, 7, 19, 19, 20]
x is 5                                         # fourth element in numbers
[3, 6, 6, 5, 7, 10, 7, 19, 19, 20]
x is 7                                         # fifth element in numbers
[3, 6, 6, 5, 10, 7, 19, 19, 20]
x is 7                                         # sixth element in numbers
x is 19                                        # seventh element in numbers
[3, 6, 6, 5, 10, 7, 19, 20]
x is 20                                        # eight and last element in numbers
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy
Quote
#9
(Jun-14-2019, 08:41 PM)perfringo Wrote: Printing is 'cheapest' debugging tool. If one adds two print statements everything is quite self-explanatory:

>>> numbers = [3, 3, 20, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
>>> for x in numbers:
...     print(f'x is {x}')
...     if numbers.count(x) > 1:
...         numbers.remove(x)
...         print(numbers)
... 
x is 3                                         # first element in numbers
[3, 20, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
x is 20                                        # second element in numbers
[3, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
x is 10                                        # third element in numbers
[3, 6, 6, 5, 5, 7, 10, 7, 19, 19, 20]
x is 5                                         # fourth element in numbers
[3, 6, 6, 5, 7, 10, 7, 19, 19, 20]
x is 7                                         # fifth element in numbers
[3, 6, 6, 5, 10, 7, 19, 19, 20]
x is 7                                         # sixth element in numbers
x is 19                                        # seventh element in numbers
[3, 6, 6, 5, 10, 7, 19, 20]
x is 20                                        # eight and last element in numbers

thanks, that further illustrates my problem.
Quote
#10
(Jun-14-2019, 06:50 PM)calonia Wrote: modifying a list while iterating through it(adding or removing items) leads to skipping indexes.
You should know that you can loop over a copy of a list however
The [:] returns a shallow copy of the list, in which this case doesnt matter because all of the elements are ints
numbers = [3, 3, 20, 6, 10, 6, 5, 5, 7, 10, 7, 19, 19, 20]
for x in numbers[:]:
    if numbers.count(x) > 1:
        numbers.remove(x)
print(numbers)
Output:
[3, 6, 5, 10, 7, 19, 20]
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  huge list of whole numbers Skaperen 3 228 Jun-02-2019, 10:11 PM
Last Post: Skaperen
  Duplicate output when calling a custom function from the same file? road2knowledge 2 136 May-10-2019, 07:58 AM
Last Post: road2knowledge
  Print Numbers starting at 1 vertically with separator for output numbers Pleiades 3 127 May-09-2019, 12:19 PM
Last Post: Pleiades
  how to add the numbers of a list Pedroski55 2 162 Apr-07-2019, 03:11 PM
Last Post: perfringo
  removing tabs from input text jenya56 3 160 Mar-28-2019, 03:10 PM
Last Post: DeaD_EyE
  Sum of Numbers in a List MrGoat 8 367 Jan-27-2019, 06:03 PM
Last Post: aakashjha001
  Removing timestamps from transcriptions jehoshua 5 300 Dec-09-2018, 09:27 AM
Last Post: jehoshua
  How to add numbers in a list DragonG 3 395 Nov-17-2018, 10:27 PM
Last Post: Larz60+
  Generate a list of numbers within a list of arbitrary numbers Takeshio 4 393 Nov-08-2018, 12:29 AM
Last Post: Takeshio
  Replacing NaN with list of numbers gregory67 5 515 Oct-27-2018, 02:00 AM
Last Post: ichabod801

Forum Jump:


Users browsing this thread: 1 Guest(s)