How to Remove Duplicate Items From a List in Python

In this article, we show how to remove duplicate items from a list in Python.

So let's imagine that you have a list in Python and there are duplicate elements, maybe put in by mistake. We can create a custom function that removes these duplicate elements, so that each element in the list is unique.

So this is done below in code.

>>> def removeduplicates(items): emptyset= set() for item in items: if item not in emptyset: yield item emptyset.add(item) >>> list1= [1,2,2,3,4,4,5,6] >>> duplicatesremoved= removeduplicates(list1) >>> list(duplicatesremoved) [1, 2, 3, 4, 5, 6] >>> list2= ['Harry', 'Harry', 'Steve', 'Robert', 'John', 'John'] >>> duplicatesremoved2= removeduplicates(list2) >>> list(duplicatesremoved2) ['Harry', 'Steve', 'Robert', 'John']

So let's now go through our code.

So, first, we define a function, which we call, removeduplicates(). This takes in the parameter, items. It takes in a parameter, because we will be feeding this function a list.

Within the function, we create a variable, emptyset, which is set to set(). This creates an empty set in Python.

We then use a for loop to loop through each of the items of the list.

If the item is not in the variable, emptyset, we add it to the set.

We then go through a few examples.

We have a list, list1, which contains a list of numbers. A few of them are duplicates, the numbers 2 and 4.

We then create a variable, duplicatesremoved, which calls the removeduplicates() function. We pass into this function the list, list1.

We then use the list() function to convert it into a list, as the output of the function is not a list.

You then see that the duplicates that were present before are no longer present.

We then have a second list, list2, this time that is composed of strings rather than numbers.

There are duplicates of 'Harry' and 'John'.

After passing this list into the removeduplicates() function and making this output a list after it is run, you can seee that all the duplicates have been removed.

So this is how we can remove duplicate items from a list in Python.

Related Resources

How to Show All Tables of a MySQL Database in Python

How to Count the Number of Rows in a MySQL Table in Python