Multiple ways to remove duplicates from a list order without Python with examples

This tutorial explains about removing duplicates from the list and with or without order guaranteed

How to remove duplicates from the list

Set is a data structure used to remove duplicate elements and order is guaranteed.

  • Use Set constructor

Step 1: Convert List to Set using the constructor. Step 2: Sets remove duplicates and return unordered items.

result = list(set(numbers))

How to remove duplicate list by preserving order

Multiple ways to return unique elements from a list by preserving order

  • Python Dictionary in 3.7 and below versions

Dictionary(dict) elements are keys ordered in insertion order from Python 3.7 version,

Convert the list into a dictionary using dict.fromKeys(list) function, It removes duplicates and again converts them into a list using the list() constructor.

print(list(dict.fromkeys(numbers))); # [1,3,5,7,9]

Python 3.6 version before, you can use OrderDict class from Collections package

from collections import OrderedDict

print(list(OrderedDict.fromkeys(numbers))) # [1,3,5,7,9]
  • use Pandas package

    • The panda’s package provides Series function that accepts lists,
    • drop_duplicates() removes the duplicate elements
    • Convert to list using toList() function
import pandas as pd
  • use for loop and if conditional statements

  • Iterate a list using for loop

  • check an element using uniqueList, this list was created empty initially

  • if elements exist, append them to a unique list

uniqueList = []
for name in numbers:
    if name not in uniqueList: