How to Use LINQ to Find Duplicates in a List in C#

Finding duplicates in a list is a common task when working with collections in C#. LINQ (Language Integrated Query) provides powerful methods to identify duplicate elements in a list efficiently. In this article, we’ll explore different LINQ techniques to detect duplicates in a list based on various conditions.

---

Why Check for Duplicates?

• Data Integrity: Ensure unique records in datasets.
• Error Prevention: Prevent duplicate entries in lists or databases.
• Data Cleaning: Clean up messy data sources.

---

Example Data Set

Consider the following list of products:

List<string> products = new List<string>
{
    "Laptop", "Phone", "Tablet", "Laptop", "Monitor", "Phone"
};

---

1. Finding Duplicates Using GroupBy

The simplest way to find duplicates in a list is by using the GroupBy method in LINQ. You can group elements by their values and check where the count of elements is greater than 1.

Example Code

using System;
using System.Collections.Generic;
using System.Linq;

class Program
{
    static void Main()
    {
        List<string> products = new List<string>
        {
            "Laptop", "Phone", "Tablet", "Laptop", "Monitor", "Phone"
        };

        var duplicates = products
            .GroupBy(p => p)
            .Where(g => g.Count() > 1)
            .Select(g => new { Product = g.Key, Count = g.Count() })
            .ToList();

        Console.WriteLine("Duplicate products:");
        foreach (var item in duplicates)
        {
            Console.WriteLine($"Product: {item.Product}, Count: {item.Count}");
        }
    }
}

Output

Duplicate products:
Product: Laptop, Count: 2
Product: Phone, Count: 2

---

2. Finding Duplicate Objects by Property

If you have a list of complex objects, use GroupBy on a specific property.

Example Data Set

public class Employee
{
    public int Id { get; set; }
    public string Name { get; set; }
}

List<Employee> employees = new List<Employee>
{
    new Employee { Id = 1, Name = "Alice" },
    new Employee { Id = 2, Name = "Bob" },
    new Employee { Id = 3, Name = "Alice" },
    new Employee { Id = 4, Name = "Charlie" }
};

Example Code

var duplicateEmployees = employees
    .GroupBy(e => e.Name)
    .Where(g => g.Count() > 1)
    .Select(g => new { Name = g.Key, Count = g.Count() })
    .ToList();

Console.WriteLine("Duplicate employees:");
foreach (var employee in duplicateEmployees)
{
    Console.WriteLine($"Name: {employee.Name}, Count: {employee.Count}");
}

Output

Duplicate employees:
Name: Alice, Count: 2

---

3. Finding the First Duplicate Element

To find only the first duplicate occurrence in a list:

var firstDuplicate = products
    .GroupBy(p => p)
    .FirstOrDefault(g => g.Count() > 1)?.Key;

Console.WriteLine($"First duplicate: {firstDuplicate}");

Output

First duplicate: Laptop

---

4. Finding Unique Elements (Non-Duplicates)

If you want to find only the unique elements (those that appear once):

var uniqueProducts = products
    .GroupBy(p => p)
    .Where(g => g.Count() == 1)
    .Select(g => g.Key)
    .ToList();

Console.WriteLine("Unique products:");
foreach (var product in uniqueProducts)
{
    Console.WriteLine(product);
}

Output

Unique products:
Tablet
Monitor

---

5. Checking for Any Duplicates (Boolean Check)

If you only need to know whether duplicates exist:

bool hasDuplicates = products
    .GroupBy(p => p)
    .Any(g => g.Count() > 1);

Console.WriteLine($"Contains duplicates: {hasDuplicates}");

Output

Contains duplicates: True

---

6. Removing Duplicates from the List

To create a list with duplicates removed, use Distinct:

var distinctProducts = products.Distinct().ToList();

Console.WriteLine("Distinct products:");
foreach (var product in distinctProducts)
{
    Console.WriteLine(product);
}

Output

Distinct products:
Laptop
Phone
Tablet
Monitor

---

Summary of LINQ Techniques for Finding Duplicates

Scenario	LINQ Query
Find duplicates by value	GroupBy(p => p).Where(g => g.Count() > 1)
Find duplicates by property	GroupBy(e => e.Name).Where(g => g.Count() > 1)
Find first duplicate element	GroupBy(p => p).FirstOrDefault(g => g.Count() > 1)?.Key
Find unique elements (non-duplicates)	GroupBy(p => p).Where(g => g.Count() == 1).Select(g => g.Key)
Check if any duplicates exist	GroupBy(p => p).Any(g => g.Count() > 1)
Remove duplicates	products.Distinct()

---

Conclusion

Finding duplicates in a list using LINQ is straightforward and powerful. With methods like GroupBy, Where, and Select, you can detect duplicates, filter unique elements, and clean up your data efficiently. These LINQ techniques are essential for ensuring data quality and integrity in C# applications.