Transform Streams | Engineering Interview Kit

TL;DR

A Transform stream is a type of Duplex stream (it is both Readable and Writable) that modifies or transforms the data as it passes through it. Examples include zipping/unzipping files or encrypting data.

Mental Model

How It Works

When implementing a Transform stream, you write to it, the stream applies some processing logic to the chunk, and then you read the modified chunk from it. It’s essentially a middleware for your data pipeline.

Common built-in Transform streams are found in the zlib (compression) and crypto (encryption) modules.

Example

const fs = require('fs');
const zlib = require('zlib');

// Compressing a file using a Transform stream
const readStream = fs.createReadStream('file.txt');
const gzipStream = zlib.createGzip(); // This is a Transform stream!
const writeStream = fs.createWriteStream('file.txt.gz');

readStream
  .pipe(gzipStream) // Data goes in raw, comes out compressed
  .pipe(writeStream)
  .on('finish', () => console.log('Compression complete!'));

Common Interview Questions

How does a Transform stream differ from a Duplex stream?

A Duplex stream is simply a stream that implements both Readable and Writable interfaces (like a TCP socket). The read and write sides don’t necessarily have to be related. In a Transform stream, the output (Readable side) is directly computed from the input (Writable side).

How do you create a custom Transform stream?

Extend the Transform class from node:stream and implement the _transform(chunk, encoding, callback) method. Inside _transform, you modify the chunk, push the result using this.push(modifiedChunk), and then call callback().