Debugging Best Practices

Reproduce with production-like data volume locally before tuning prod. These practices shorten Node.js incident mean time to resolution.

How to Use This List

On-call runbook links to relevant scenario pages
Staging mirrors prod Node version (24.18.0) and body size limits
Every postmortem adds one automated guard (test, alert, or lint)
Pair with On-Call Runbook Starters

A - Reproduction

Match Node major.minor to prod in staging (24.18.0 LTS).
Replay traffic with similar payload sizes - 3MB webhooks, not 30-byte fixtures.
Use autocannon or k6 to stress health endpoints while testing heavy routes.
Capture minimal repro in integration test before fixing production-only flags.
Document repro steps in ticket - another engineer can verify without Slack context.

B - Instrumentation

Structured JSON logs with requestId, traceId, route, duration.
monitorEventLoopDelay on readiness or metrics exporter.
Export pg pool total, idle, waiting on /health/db.
OpenTelemetry traces on outbound HTTP and DB spans.
Avoid console.log huge objects - stalls event loop and leaks PII.

C - Local Debugging Tools

node --inspect + Chrome DevTools for breakpoints and heap snapshots.
clinic doctor / clinic heapprofiler before blaming hardware.
Source maps enabled for TypeScript line numbers in stacks.
Remove debugger statements before merge - ESLint enforced.
NODE_OPTIONS='--unhandled-rejections=strict' in CI tests.

D - Production Safety

Do not leave --inspect=0.0.0.0 in prod without auth tunnel.
SIGUSR1 attach only via approved runbook in staging replicas.
Crash on unhandled rejection after log + drain for stateful APIs.
Heap snapshots to S3 via scheduled job, not manual prod attach, when needed.
Redact secrets in error logs - no DATABASE_URL in stack traces sent to clients.

E - Common Incident Classes

Pool exhaustion: verify release() in finally; size pools per pod math.
Event-loop stall: move sync JSON/crypto to workers or queues.
Memory leak: compare heap snapshots after load, audit global caches.
Hung requests: timeout all external I/O; log pool waiting count.
Webhook duplicates: await processing; idempotency keys on side effects.

F - Post-Incident

Postmortem blames systems, not individuals - add guard listed above.
Alert on symptom (waiting pool, event-loop p99) not only on crash.
Load test new deploy config (replica count × pool max) before peak.
Update scenario docs when a new failure mode appears twice.
Share trace + log query in runbook for this failure signature.

FAQs

printf debugging ever OK?

Local dev yes. Production use structured logs at debug level behind sampling.

How long to spend reproducing?

Timebox 30 minutes then pair. If unreproducible, add logging and wait for recurrence with better telemetry.

Debug prod directly?

Prefer staging replay. Prod: metrics, traces, controlled heap snapshot runbook only.

VS Code vs Chrome?

Either works with attach config. Standardize in team launch.json.

Flaky heisenbugs?

Enable trace sampling; use AsyncLocalStorage to tie logs to request across async hops.

When to escalate to vendor?

RDS, Stripe, or ALB metrics show external fault - still verify client timeouts and retry storms.

Junior on-call first steps?

Check /health, pool metrics, recent deploy, error rate dashboard - then open linked scenario page.

Docker local repro?

docker compose with prod-like limits on memory and pool size surfaces OOM and pool issues.

Bun/Deno debugging?

This site targets Node 24 LTS - reproduce in Node even if prod experiments with other runtimes.

Best single improvement?

--unhandled-rejections=strict in CI plus pool waiting metric alert prevents two common outage classes.

Debugging Basics - inspect and debugger
Memory Leak Hunt - heap workflow
Connection Pool Exhaustion - pool war story
Unhandled Rejection Production Outage - crash policy
Logging Best Practices - structured logs

Stack versions: This page was written for Node.js 24.18.0 (Active LTS), npm 10+, TypeScript 5.6+, Express 5, Fastify 5, and NestJS 11.

Debugging Best Practices

Reproduce with production-like data volume locally before tuning prod. These practices shorten Node.js incident mean time to resolution.

How to Use This List

On-call runbook links to relevant scenario pages
Staging mirrors prod Node version (24.18.0) and body size limits
Every postmortem adds one automated guard (test, alert, or lint)
Pair with On-Call Runbook Starters

A - Reproduction

Match Node major.minor to prod in staging (24.18.0 LTS).
Replay traffic with similar payload sizes - 3MB webhooks, not 30-byte fixtures.
Use autocannon or k6 to stress health endpoints while testing heavy routes.
Capture minimal repro in integration test before fixing production-only flags.
Document repro steps in ticket - another engineer can verify without Slack context.

B - Instrumentation

Structured JSON logs with requestId, traceId, route, duration.
monitorEventLoopDelay on readiness or metrics exporter.
Export pg pool total, idle, waiting on /health/db.
OpenTelemetry traces on outbound HTTP and DB spans.
Avoid console.log huge objects - stalls event loop and leaks PII.

C - Local Debugging Tools

node --inspect + Chrome DevTools for breakpoints and heap snapshots.
clinic doctor / clinic heapprofiler before blaming hardware.
Source maps enabled for TypeScript line numbers in stacks.
Remove debugger statements before merge - ESLint enforced.
NODE_OPTIONS='--unhandled-rejections=strict' in CI tests.

D - Production Safety

Do not leave --inspect=0.0.0.0 in prod without auth tunnel.
SIGUSR1 attach only via approved runbook in staging replicas.
Crash on unhandled rejection after log + drain for stateful APIs.
Heap snapshots to S3 via scheduled job, not manual prod attach, when needed.
Redact secrets in error logs - no DATABASE_URL in stack traces sent to clients.

E - Common Incident Classes

Pool exhaustion: verify release() in finally; size pools per pod math.
Event-loop stall: move sync JSON/crypto to workers or queues.
Memory leak: compare heap snapshots after load, audit global caches.
Hung requests: timeout all external I/O; log pool waiting count.
Webhook duplicates: await processing; idempotency keys on side effects.

F - Post-Incident

Postmortem blames systems, not individuals - add guard listed above.
Alert on symptom (waiting pool, event-loop p99) not only on crash.
Load test new deploy config (replica count × pool max) before peak.
Update scenario docs when a new failure mode appears twice.
Share trace + log query in runbook for this failure signature.

FAQs

printf debugging ever OK?

Local dev yes. Production use structured logs at debug level behind sampling.

How long to spend reproducing?

Timebox 30 minutes then pair. If unreproducible, add logging and wait for recurrence with better telemetry.

Debug prod directly?

Prefer staging replay. Prod: metrics, traces, controlled heap snapshot runbook only.

VS Code vs Chrome?

Either works with attach config. Standardize in team launch.json.

Flaky heisenbugs?

Enable trace sampling; use AsyncLocalStorage to tie logs to request across async hops.

When to escalate to vendor?

RDS, Stripe, or ALB metrics show external fault - still verify client timeouts and retry storms.

Junior on-call first steps?

Check /health, pool metrics, recent deploy, error rate dashboard - then open linked scenario page.

Docker local repro?

docker compose with prod-like limits on memory and pool size surfaces OOM and pool issues.

Bun/Deno debugging?

This site targets Node 24 LTS - reproduce in Node even if prod experiments with other runtimes.

Best single improvement?

--unhandled-rejections=strict in CI plus pool waiting metric alert prevents two common outage classes.

Debugging Basics - inspect and debugger
Memory Leak Hunt - heap workflow
Connection Pool Exhaustion - pool war story
Unhandled Rejection Production Outage - crash policy
Logging Best Practices - structured logs

Stack versions: This page was written for Node.js 24.18.0 (Active LTS), npm 10+, TypeScript 5.6+, Express 5, Fastify 5, and NestJS 11.

Debugging Best Practices

Search Documentation

Debugging Best Practices