Queues Best Practices

Q: How deep is too deep?

Sustained growth over 15 minutes is incident. Steady state near zero with spikes OK.

Q: One worker process or many?

Many pods with moderate concurrency beats one monster pod for fault tolerance.

Q: Sync fallback if queue down?

Rare. Return 503 and alert. Do not block HTTP 30s on Redis outage.

Q: BullMQ vs SQS?

BullMQ if Redis exists. SQS for AWS-managed with less Redis ops.

Q: Priority queues?

Use separate queues for premium tier instead of starving bulk work.

Q: Job ordering required?

SQS FIFO or single concurrency per MessageGroupId. BullMQ single consumer queue.

Q: Testing retries?

Mock vendor 503; assert attempts count and final DLQ placement.

Q: NestJS same rules?

Yes. Processors must be idempotent; Bull Board still recommended.

Q: Biggest outage pattern?

Queue depth silent growth until Redis OOM or SLA miss. Alert early.

Q: Job versioning?

Include schemaVersion in payload; workers handle N and N-1 during deploy.

Monitor queue depth; alert before consumers drown. Queues decouple HTTP from slow work only when ops discipline matches.

How to Use This List

Dashboard waiting/active/failed counts per queue before launch
Load test producer burst with fixed worker count
Pair with Idempotency Keys for payment paths
Run quarterly DLQ redrive drill

A - Producer (API) Rules

HTTP returns 202 with job id for work >500ms expected duration.
Enqueue is fast - no heavy serialization or DB work in enqueue path.
Stable jobId / idempotency key on client-retried mutations.
Correlation id in job payload for tracing across API and worker logs.
Separate queues by domain and blast radius (email, billing, media).

B - Worker Rules

Workers idempotent - safe if job runs twice (stalled SQS/BullMQ delivery).
Concurrency tuned per job type - I/O vs CPU bound profiles differ.
Throw on transient errors to trigger retry; catch permanent errors to fail fast.
Side effects after local idempotency check and DB commit.
Progress updates for jobs >30s user-visible wait.

C - Retries and DLQ

Exponential backoff on outbound vendor calls inside jobs.
Max attempts capped (3-10 depending on cost).
DLQ configured (SQS redrive or Bull failed set) with alarm.
Runbook: inspect DLQ, fix, redrive documented in on-call wiki.
Poison message identification - log job name + payload hash, not secrets.

D - Operations

Metrics: depth, age of oldest message, processing rate, fail rate.
Autoscale workers on depth (k8s HPA custom metric or ECS step scaling).
Graceful shutdown - worker.close() with grace period > p99 job time.
Redis/ElastiCache memory monitored when using BullMQ.
Visibility timeout / lockDuration > p99 handler duration.

E - Payload and Security

Payload < 64KB ideally; S3 reference for large files.
No raw PII unless necessary - pass ids and load in worker.
Worker IAM least privilege for S3, SQS, Stripe, etc.
Encrypt sensitive job fields if queue is multi-tenant admin accessible.
Admin retry UI (Bull Board) behind auth and audit log.

F - Scheduling

No duplicate cron across replicas without lock or k8s CronJob.
BullMQ repeatable jobs use stable jobId.
Missed run policy documented (catch-up vs skip).
Heavy schedules enqueue worker jobs instead of running inline in API.
UTC timezone explicit in cron definitions.

FAQs

How deep is too deep?

Sustained growth over 15 minutes is incident. Steady state near zero with spikes OK.

One worker process or many?

Many pods with moderate concurrency beats one monster pod for fault tolerance.

Sync fallback if queue down?

Rare. Return 503 and alert. Do not block HTTP 30s on Redis outage.

BullMQ vs SQS?

BullMQ if Redis exists. SQS for AWS-managed with less Redis ops.

Priority queues?

Use separate queues for premium tier instead of starving bulk work.

Job ordering required?

SQS FIFO or single concurrency per MessageGroupId. BullMQ single consumer queue.

Testing retries?

Mock vendor 503; assert attempts count and final DLQ placement.

NestJS same rules?

Yes. Processors must be idempotent; Bull Board still recommended.

Biggest outage pattern?

Queue depth silent growth until Redis OOM or SLA miss. Alert early.

Job versioning?

Include schemaVersion in payload; workers handle N and N-1 during deploy.

Queues Basics - fundamentals
BullMQ - Redis queues
AWS SQS Consumers - managed queues
Idempotency Keys - duplicate safety
Graceful Shutdown - worker drain

Node.js Best Practices (GitHub) - Curated production best practices.
DigitalOcean Node.js Tutorials - Production-oriented how-tos.
Stack Overflow – Node.js Tag - Q&A for real-world problems.
Distributed Systems with Node.js - Enterprise-ready distributed systems.
Node.js Design Patterns, 4th Edition - Production-grade application patterns.

Stack versions: This page was written for Node.js 24.18.0 (Active LTS), npm 10+, TypeScript 5.6+, Express 5, Fastify 5, and NestJS 11.

Queues Best Practices

Monitor queue depth; alert before consumers drown. Queues decouple HTTP from slow work only when ops discipline matches.

How to Use This List

Dashboard waiting/active/failed counts per queue before launch
Load test producer burst with fixed worker count
Pair with Idempotency Keys for payment paths
Run quarterly DLQ redrive drill

A - Producer (API) Rules

HTTP returns 202 with job id for work >500ms expected duration.
Enqueue is fast - no heavy serialization or DB work in enqueue path.
Stable jobId / idempotency key on client-retried mutations.
Correlation id in job payload for tracing across API and worker logs.
Separate queues by domain and blast radius (email, billing, media).

B - Worker Rules

Workers idempotent - safe if job runs twice (stalled SQS/BullMQ delivery).
Concurrency tuned per job type - I/O vs CPU bound profiles differ.
Throw on transient errors to trigger retry; catch permanent errors to fail fast.
Side effects after local idempotency check and DB commit.
Progress updates for jobs >30s user-visible wait.

C - Retries and DLQ

Exponential backoff on outbound vendor calls inside jobs.
Max attempts capped (3-10 depending on cost).
DLQ configured (SQS redrive or Bull failed set) with alarm.
Runbook: inspect DLQ, fix, redrive documented in on-call wiki.
Poison message identification - log job name + payload hash, not secrets.

D - Operations

Metrics: depth, age of oldest message, processing rate, fail rate.
Autoscale workers on depth (k8s HPA custom metric or ECS step scaling).
Graceful shutdown - worker.close() with grace period > p99 job time.
Redis/ElastiCache memory monitored when using BullMQ.
Visibility timeout / lockDuration > p99 handler duration.

E - Payload and Security

Payload < 64KB ideally; S3 reference for large files.
No raw PII unless necessary - pass ids and load in worker.
Worker IAM least privilege for S3, SQS, Stripe, etc.
Encrypt sensitive job fields if queue is multi-tenant admin accessible.
Admin retry UI (Bull Board) behind auth and audit log.

F - Scheduling

No duplicate cron across replicas without lock or k8s CronJob.
BullMQ repeatable jobs use stable jobId.
Missed run policy documented (catch-up vs skip).
Heavy schedules enqueue worker jobs instead of running inline in API.
UTC timezone explicit in cron definitions.

FAQs

How deep is too deep?

Sustained growth over 15 minutes is incident. Steady state near zero with spikes OK.

One worker process or many?

Many pods with moderate concurrency beats one monster pod for fault tolerance.

Sync fallback if queue down?

Rare. Return 503 and alert. Do not block HTTP 30s on Redis outage.

BullMQ vs SQS?

BullMQ if Redis exists. SQS for AWS-managed with less Redis ops.

Priority queues?

Use separate queues for premium tier instead of starving bulk work.

Job ordering required?

SQS FIFO or single concurrency per MessageGroupId. BullMQ single consumer queue.

Testing retries?

Mock vendor 503; assert attempts count and final DLQ placement.

NestJS same rules?

Yes. Processors must be idempotent; Bull Board still recommended.

Biggest outage pattern?

Queue depth silent growth until Redis OOM or SLA miss. Alert early.

Job versioning?

Include schemaVersion in payload; workers handle N and N-1 during deploy.

Queues Basics - fundamentals
BullMQ - Redis queues
AWS SQS Consumers - managed queues
Idempotency Keys - duplicate safety
Graceful Shutdown - worker drain

Node.js Best Practices (GitHub) - Curated production best practices.
DigitalOcean Node.js Tutorials - Production-oriented how-tos.
Stack Overflow – Node.js Tag - Q&A for real-world problems.
Distributed Systems with Node.js - Enterprise-ready distributed systems.
Node.js Design Patterns, 4th Edition - Production-grade application patterns.

Stack versions: This page was written for Node.js 24.18.0 (Active LTS), npm 10+, TypeScript 5.6+, Express 5, Fastify 5, and NestJS 11.

Queues Best Practices

Search Documentation

Queues Best Practices