distinct

TL;DR

• distinct() is a stateful intermediate operation.
• It removes duplicate elements from the stream.
• It relies on the equals() and hashCode() methods of the objects in the stream to determine equality.

Concept

When you have a stream containing duplicate values and you only want to process unique elements, you use distinct().

Because it is a stateful operation, distinct() must maintain internal state (a set of previously seen elements) while the pipeline processes. As elements flow through, it checks its internal set. If the element is new, it passes it down the pipeline. If the element is a duplicate, it drops it.

Examples

import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;

class User {
    int id;
    String name;
    
    public User(int id, String name) { this.id = id; this.name = name; }
    
    // distinct() relies heavily on these methods!
    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;
        User user = (User) o;
        return id == user.id;
    }
    
    @Override
    public int hashCode() {
        return Integer.hashCode(id);
    }
}

public class DistinctExample {
    public static void main(String[] args) {
        
        // 1. Primitive/String distinct
        List<Integer> numbers = Arrays.asList(1, 2, 2, 3, 3, 3, 4);
        List<Integer> uniqueNums = numbers.stream()
            .distinct()
            .collect(Collectors.toList());
        System.out.println(uniqueNums); // [1, 2, 3, 4]
        
        // 2. Custom Object distinct
        List<User> users = Arrays.asList(
            new User(1, "Alice"),
            new User(1, "Alice Clone"), // Same ID, equals() returns true
            new User(2, "Bob")
        );
        
        long count = users.stream().distinct().count();
        System.out.println("Unique users: " + count); // 2
    }
}

Interview Questions

Q: How does distinct() perform in a Parallel Stream?
A: distinct() can be a major performance bottleneck in parallel streams. To guarantee it removes duplicates across multiple threads, the JVM must coordinate state between the threads, which introduces heavy synchronization overhead. If the order of the stream doesn’t matter, you can improve performance by using .unordered().distinct() on a parallel stream.

Q: What happens if you use distinct() on custom objects but forget to override equals() and hashCode()?
A: If you don’t override them, distinct() will fall back to the default Object implementation, which only checks for reference equality (memory addresses). This means two different objects with the exact same data will be treated as unique, and distinct() will fail to remove the logical duplicate.