Python Forum

i'm looking for a function to take a sequence that might have duplicates and return one of the same type with no more than one of each item. i only need it for lists for now.

set(iterable) is first that comes to mind, but as always - the devil is in the details:
- what type of elements are there (are they hashable), any nested structures and how to deal with such if present, do you keep the order, etc.

(May-10-2019, 06:56 AM)Skaperen Wrote: [ -> ]return one of the same type

Also on second read - do you mean dedup values or dedup types(i.e. only one str, int, etc...)?

i do want to keep the original order, but i need to check if i really need that, otherwise set(interable) would do the job. it is values. in my current case there are about 100 strings, mostly different.

Assuming that objective is to dedupe hashable objects in iterable while maintaining order:

def dedupe(iterable):
    seen = set()
    for item in iterable:
        if item not in seen:
            yield item
            seen.add(item)

(May-11-2019, 05:43 AM)perfringo Wrote: [ -> ]Assuming that objective is to dedupe hashable objects in iterable while maintaining order:

def dedupe(iterable):
    seen = set()
    for item in iterable:
        if item not in seen:
            yield item
            seen.add(item)

either hashable or references to the same object. so a set() won't do; it will need to be a list, at least for the non-hashables. deep leaf comparison is not needed.

so, it doesn't pre-exist.