Intermediate Operations | Engineering Interview Kit

TL;DR

Intermediate Operations are operations in a stream pipeline that return a new Stream.
They are Lazy: they do not execute until a terminal operation is called.
Examples include filter(), map(), sorted(), distinct(), limit().

Concept

Intermediate operations define what needs to be done to the data, but they don’t actually do it. They merely build up the execution blueprint.

Intermediate operations are divided into two categories:

Stateless Operations: Operations that don’t need to know about any other element in the stream to process the current element. (e.g., filter, map). These are highly efficient and easy to parallelize.
Stateful Operations: Operations that must see all elements before they can proceed. (e.g., sorted(), distinct()). A stream cannot finish sorting until it has seen every single element in the source.

Examples

import java.util.Arrays;
import java.util.List;
import java.util.stream.Stream;

public class IntermediateOps {
    public static void main(String[] args) {
        
        List<Integer> numbers = Arrays.asList(5, 1, 2, 5, 3, 4, 1);
        
        // Building the blueprint using intermediate operations.
        // Nothing is actually executed here!
        Stream<Integer> blueprint = numbers.stream()
            .distinct()               // Stateful: removes duplicates [5,1,2,3,4]
            .filter(n -> n % 2 != 0)  // Stateless: keeps odd [5,1,3]
            .sorted()                 // Stateful: sorts [1,3,5]
            .map(n -> n * 10);        // Stateless: multiplies [10,30,50]
            
        // The data is finally processed here
        blueprint.forEach(System.out::println); 
    }
}

Interview Questions

Q: Why is the order of intermediate operations important?
A: The order dictates performance. If you have a list of 10,000 items, and you want to filter out 9,000 of them and then sort the remaining 1,000, you should put filter() before sorted(). If you put sorted() first, Java will waste CPU cycles sorting all 10,000 items, only to throw away 9,000 of them in the next step.

Q: What is the difference between stateless and stateful intermediate operations?
A: Stateless operations (like map) process one element at a time, completely independently of the others. Stateful operations (like sorted or distinct) require the pipeline to pause and gather information about all previous elements before it can emit any elements to the next step. Stateful operations are bottlenecks in parallel streams.