Exactly-Once & the Outbox Pattern
You'll learn to
- -Explain why "exactly-once" delivery is effectively unachievable at the transport level, and how idempotent consumers achieve the same practical outcome
- -Implement the transactional outbox pattern to atomically couple a database write with publishing the corresponding event
Message queues and event streams almost universally advertise "at-least-once" delivery, not "exactly-once" - and understanding why or the difference matters, because building a system that assumes true exactly-once delivery at the transport level is building on a guarantee that doesn't actually exist in practice.
Why True Exactly-Once Is Effectively Unachievable
A publisher sends a message; the network or the broker fails before acknowledging receipt; the publisher, unsure whether the message actually arrived, retries. If the original message did arrive and only the acknowledgment was lost, the retry creates a duplicate - there is no way for the publisher to distinguish "the message was lost" from "the message arrived but the ack was lost" without more machinery, which is exactly what idempotent consumers (not transport-level magic) are built to handle.
Idempotent Consumers: Exactly-Once Effect, Not Exactly-Once Delivery
def handle_order_shipped_event(event):
if processed_events.exists(event["event_id"]):
return # already handled this exact event - safely skip
process_shipment_notification(event)
processed_events.mark_processed(event["event_id"]) # atomically with the aboveThe practical target is "at-least-once delivery, exactly-once effect" - the transport might redeliver the same event, but the consumer detects and ignores the duplicate using the event's unique ID (the same `event_id` field designed in the domain-events chapter), so the net observable effect on the system is as if it had only been processed once, even though delivery itself wasn't exactly-once.
The Outbox Pattern: Atomically Coupling a Write With Publishing
A separate problem: how do you guarantee an event actually gets published if and only if the database write it describes actually succeeded? Publishing to a message broker and writing to a database are two separate systems - if the database write succeeds but the service crashes before the publish call, the event is lost; if the publish succeeds but the database write then fails, a phantom event describes something that never actually happened.
BEGIN;
INSERT INTO orders (id, status) VALUES (501, 'shipped');
INSERT INTO outbox_events (event_type, payload, published)
VALUES ('order.shipped', '{"order_id": 501}', false);
COMMIT;
-- both writes succeed or both roll back TOGETHER - a single local
-- transaction, no distributed transaction across two separate systemsThe outbox table lives in the same database as the business data it describes, so writing the business change and writing the "event to publish" record happens in one ordinary local transaction - either both succeed or both roll back, with no possibility of one succeeding without the other. A separate background process then reads unpublished rows from the outbox table and actually publishes them to the message broker, marking each as published once the broker confirms receipt - decoupling "did the business write and its corresponding event get recorded atomically" (guaranteed by the local transaction) from "has that event actually reached the broker yet" (handled separately, with its own retry logic).
The outbox pattern and idempotent consumers solve complementary halves of the same underlying problem: the outbox guarantees an event is never silently lost on the publishing side; idempotent consumers guarantee a redelivered event is never double-processed on the receiving side. Robust exactly-once-effect systems need both.
Interview Signal is part of Pro
See a real weak answer next to a real strong one for this exact topic.
Quiz is part of Pro
Test what you just read with a short quiz, and bank the XP.
Design Exactly Once in the API Design Lab's gRPC & Event-Driven act.