Dead Letter Queues & Delivery Guarantees
You'll learn to
- -Explain what a poison pill is and how a DLQ handles it
The Poison Pill Problem
What happens when a message is malformed, or triggers a bug, and a worker can never successfully process it? Left alone, that single message gets retried forever, burning worker capacity and (in ordered queues) potentially blocking every message behind it. This is called a "poison pill."
A Dead Letter Queue (DLQ) is the 'Return to Sender' pile at the post office. Imagine a letter that the postal service tries to deliver 3 times, but the address is wrong or the recipient refuses it every time. Instead of trying forever (blocking all the other mail behind it) or throwing it away (losing it permanently), the post office puts it in a special bin for undeliverable mail where someone can look at it later. In software, when a message from your queue fails processing 3 times (maybe the data is malformed, maybe a downstream service is broken), the queue moves that message to the DLQ. This is CRITICAL because without a DLQ, a single bad message (called a 'poison pill') can block your entire queue forever. Your consumer keeps trying to process it, keeps failing, and every message behind it is stuck waiting. With a DLQ, the bad message gets safely set aside, the rest of the queue keeps flowing, and an engineer can investigate the DLQ later to fix the problem and reprocess the messages.
Examples: AWS SQS DLQ (automatic after maxReceiveCount), Kafka dead letter topics, RabbitMQ dead-letter exchanges, Azure Service Bus dead-letter sub-queue, Celery task_reject_on_worker_lost
After N failed attempts, a poison-pill message is moved out of the main queue and into a DLQ instead of retrying forever.
A Dead Letter Queue is the safety valve: after N failed attempts, a message is moved out of the main queue into the DLQ for manual inspection, instead of being retried indefinitely. Production message queues without a DLQ are a common source of silent, hard-to-diagnose outages.
Delivery Guarantees
- -At-most-once: a message is delivered zero or one times. Simple, but messages can be silently lost.
- -At-least-once: a message is delivered one or more times. The safe default for most queues, but your consumer might see the same message twice.
- -Exactly-once: each message is processed exactly one time, no duplicates, no loss. Genuinely achieving this across a network is notoriously difficult; the Idempotency chapter later in this course explains why "exactly-once" in practice usually means "at-least-once delivery plus an idempotent consumer."
Interview Signal is part of Pro
See a real weak answer next to a real strong one for this exact topic.
Quiz is part of Pro
Test what you just read with a short quiz, and bank the XP.