Queues & Workers Highlights Summary

Every highlight bullet from the 7 pages in this section, gathered on one page and grouped by the page it came from.

The Producer-Consumer Model

• A queue's whole job is decoupling how fast work is produced from how fast it's consumed
• Most brokers guarantee at-least-once delivery, not exactly-once - your worker has to handle duplicates
• Queue depth is a backpressure signal, not just a number on a dashboard
• Scheduling and cron are producers too - they just trigger on time instead of on an event

Queues Basics

• HTTP handlers enqueue work; workers process asynchronously
• Return 202 Accepted with job id for long operations
• Retries with backoff handle transient failures
• Dead-letter queue preserves poison messages for inspection
• Workers scale independently from API replicas

BullMQ

• Separate Redis connections for Queue and Worker
• Set maxRetriesPerRequest: null on ioredis for workers
• Monitor stalled jobs and queue depth in production
• Graceful worker close waits for in-flight jobs
• Use FlowProducer for multi-step job graphs

AWS SQS Consumers

• Long polling reduces empty ReceiveMessage calls
• Visibility timeout must exceed p99 handler duration
• DLQ captures poison messages after maxReceiveCount
• Delete message only after successful processing
• Use AWS SDK v3 modular clients

Idempotency Keys

• Clients send Idempotency-Key header on mutating requests
• Store key + response hash before performing side effects
• Unique DB constraint is the final safety net
• Queue jobId can mirror idempotency key
• TTL old keys to control table size

Scheduling & Cron

• Cron in every API pod duplicates work without leader election
• BullMQ repeatable jobs need stable jobId across deploys
• k8s CronJob is one pod per schedule - simple ops model
• Timezone explicit: UTC in code or document tenant TZ
• Missed runs during outage may need catch-up policy

Queues Best Practices

• Alert on queue depth growth, not only worker CPU
• Workers must be idempotent under at-least-once delivery
• DLQ and failed job review runbook required
• Graceful shutdown drains in-flight jobs
• Job payloads stay small; blobs live in object storage