Transform & Duplex

Transform streams sit in the middle of pipelines to parse, compress, or filter - Duplex streams model two-way channels like sockets with independent read and write sides.

Recipe

import { Transform, PassThrough } from 'node:stream';
import { pipeline } from 'node:stream/promises';
 
const tap = new PassThrough();
tap.on('data', (chunk) => metrics.bytes(chunk.length));
 
await pipeline(source, tap, destination);

When to reach for this:

• NDJSON parse → object mode pipeline
• gzip compress on upload path
• Teeing one download to client and virus scanner
• Line delimiter splitting on TCP streams

Working Example

import { Transform, PassThrough, Duplex } from 'node:stream';
import { pipeline } from 'node:stream/promises';
import { Readable } from 'node:stream';
 
const splitLines = new Transform({
  transform(chunk, _enc, cb) {
    const parts = String(chunk).split('\n');
    const tail = parts.pop() ?? '';
    for (const line of parts) if (line) this.push(line);
    cb(null, tail); // buffer incomplete line in internal state via passing as chunk to next
  },
  flush(cb) {
    cb();
  },
});
 
function tee<T extends NodeJS.ReadableStream>(source: T): [T, PassThrough] {
  const branch = new PassThrough();
  source.pipe(branch);
  return [source, branch];
}
 
// Duplex conceptual - net.Socket extends Duplex
const socketLike = new Duplex({
  read() { /* pull from underlying resource */ },
  write(chunk, _enc, cb) { cb(); /* send to peer */ },
});

import { createGzip } from 'node:zlib';
 
await pipeline(
  Readable.from(['{"a":1}\n', '{"b":2}\n']),
  splitLines,
  createGzip(),
  process.stdout,
);

What this demonstrates:

• Transform may buffer partial tokens across chunks (line splitting)
• PassThrough is identity Transform - ideal for tap/tee side channels
• zlib.createGzip() is a Transform under the hood
• Duplex read/write are independent - unlike Transform's linked flow

Deep Dive

How It Works

• Transform implements _transform per chunk and optional _flush at end.
• Duplex implements _read and _write separately - TCP, TLS sockets.
• PassThrough - zero-copy-ish relay with same backpressure rules.
• Tee - multiple consumers on one source require careful backpressure (pause all if one slow).

Stream Type Pick

TypeScript Notes

import { Transform, type TransformCallback } from 'node:stream';
 
function createJsonParser(): Transform {
  let buffer = '';
  return new Transform({
    objectMode: true,
    transform(chunk: Buffer, _enc: BufferEncoding, cb: TransformCallback) {
      buffer += chunk.toString('utf8');
      const lines = buffer.split('\n');
      buffer = lines.pop() ?? '';
      try {
        for (const line of lines) if (line) this.push(JSON.parse(line));
        cb();
      } catch (err) {
        cb(err as Error);
      }
    },
  });
}

Gotchas

• Tee without handling slow branch backpressure - one slow consumer stalls all. Fix: use bounded queues or separate ingestion topics.
• Transform not calling cb on error paths - hung pipeline. Fix: always cb(err) once.
• Mixing flush state incorrectly in line parser - lost trailing line. Fix: _flush pushes remainder.
• Assuming Duplex read/write share buffer - they do not unless you wire them.
• Object mode Transform after byte gzip - order matters - gzip expects bytes.

Alternatives

FAQs

Related

• stream/promises.pipeline - wire transforms safely
• Readable & Writable Patterns - modes
• Binary Protocols - framing parsers
• Backpressure - tee slowdowns

External Related Links

• Node.js API Documentation - Complete reference for core modules and APIs.
• Node.js Best Practices (GitHub) - Curated production best practices.
• Node.js GitHub Repository - Source code, issues, contribution guides.
• Node.js Design Patterns, 4th Edition - Production-grade application patterns.
• Node.js Design Patterns, 2nd Edition - Earlier edition still useful.

Stack versions: This page was written for Node.js 24.18.0 (Active LTS), npm 10+, TypeScript 5.6+, Express 5, Fastify 5, and NestJS 11.