Skip to content
Exactly-Once Processing: The Hardest Problem in Distributed Streaming 5 min

Exactly-Once Processing: The Hardest Problem in Distributed Streaming

SD
ScaleDojo
May 11, 2026
5 min read1,175 words
Exactly-Once Processing: The Hardest Problem in Distributed Streaming

The Bank Transfer That Happened Twice

A stream processor reads a bank transfer event: 'Move $500 from Account A to Account B.' It processes the transfer, writes to the database, and then crashes before it can acknowledge the message to Kafka. Kafka re-delivers the event. The processor runs again. Account A just lost $1,000 instead of $500. This is why exactly-once processing is treated as the holy grail of distributed stream processing, and why it's genuinely hard to get right.

The Three Delivery Guarantees

Before you can reason about exactly-once semantics, you need to be honest about what your system is actually promising today. Most production pipelines fall into one of three buckets, and picking the wrong one for the job is where outages start.

Delivery Guarantee Comparison:

  At-Most-Once (fire and forget):
  [Producer] --msg--> [Broker]  (no ACK waited)
  If msg lost in transit -> gone forever
  Use for: unimportant metrics, logs you can lose

  At-Least-Once (retry until ACK):
  [Producer] --msg--> [Broker] --ACK--> [Producer]
  If ACK lost -> producer retries -> DUPLICATE
  [Consumer] --process--> [DB] --crash before offset commit--
  Kafka re-delivers -> processed TWICE
  Use for: most things, IF operations are idempotent

  Exactly-Once (the holy grail):
  Each message processed PRECISELY ONCE
  Even across crashes, retries, network failures
  Use for: financial transactions, billing, inventory

Notice that at-least-once isn't a lesser guarantee, it's often the more honest one. Once you accept that duplicates can happen, the design question shifts to making duplicates harmless rather than pretending they won't occur.

Why Exactly-Once Is So Hard

The difficulty isn't in any single step of the pipeline, it's in the gaps between steps. Every hop from reading a message to committing its result is a place where a crash can leave the system in an ambiguous state.

The Crash Window Problem:

  [Read from Kafka] -> [Process] -> [Write to DB] -> [Commit offset]
       1                  2              3                4

  Crash after step 3, before step 4:
  - DB has the result (step 3 committed)
  - Kafka offset NOT advanced (step 4 never happened)
  - Kafka re-delivers the message
  - Processor writes to DB AGAIN
  - Result: DUPLICATE processing

  Crash after step 2, before step 3:
  - DB does NOT have the result
  - Kafka re-delivers (good!)
  - Processor writes to DB (now correct)
  - But what if step 2 had side effects? (sent an email?)

  The gap between ANY two steps is a crash window.
  Every gap is a potential duplicate or loss.

This is the same underlying tension you run into with cache invalidation: two systems (Kafka's offset and your database) need to agree on the truth, and there's no free way to update both atomically unless the infrastructure does it for you. If you've read the piece on caching strategies, the trade-offs here will feel familiar, it's the same problem of keeping two sources of state in sync without a shared transaction.

How Kafka Achieves Exactly-Once

Kafka solves this with two separate mechanisms that work together rather than one silver bullet.

Kafka's Two Mechanisms:

1. Idempotent Producer (dedup at broker):
   Producer assigns sequence numbers to messages.
   Broker tracks: (producer_id, partition, sequence).
   Duplicate sequence? Broker silently drops it.
   Result: no duplicate WRITES to Kafka.

2. Kafka Transactions (atomic read-process-write):
   BEGIN TRANSACTION
     Read from input-topic (offsets 100-105)
     Process events
     Write results to output-topic
     Commit consumer offsets (100-105)
   END TRANSACTION

   Either ALL of these happen, or NONE do.
   Crash mid-transaction? Kafka aborts everything.
   Consumer offsets + output writes are ATOMIC.
   This is how Kafka Streams achieves exactly-once.

The idempotent producer handles duplicate writes at the broker level, while transactions bundle the consumer offset commit and the downstream write into a single atomic unit. If you want the fuller picture of how Kafka fits into an event-driven architecture, including where event sourcing and AMQP-style brokers diverge, the AMQP, Kafka, and Event Sourcing post covers that ground well.

Flink takes a different approach, one built around distributed snapshots rather than transactional writes to a single broker.

Flink's Checkpoint Mechanism (Chandy-Lamport):

  [Source] -> [Map] -> [Aggregate] -> [Sink]
                 |          |
     barrier ----+----------+--- checkpoint barrier
                 |          |
             [snapshot]  [snapshot]
                 |          |
             [S3 / HDFS checkpoint storage]

  Every N seconds, Flink inserts a barrier into the stream.
  When barrier reaches each operator:
    1. Operator snapshots its state to S3
    2. Passes barrier downstream
  When ALL operators have checkpointed:
    Checkpoint is COMPLETE.

  On failure:
    1. Restore all operators from last checkpoint
    2. Replay input from that checkpoint's offset
    3. Combined with idempotent sinks = exactly-once

The checkpoint barrier is really just a coordination signal, it makes sure every operator agrees on a consistent point in the stream before anyone's state is considered durable. On its own, though, a checkpoint only guarantees that Flink's internal state is consistent. Getting exactly-once semantics all the way to an external system still depends on the sink behaving idempotently on replay, which is the same principle you'll see if you look at outbox-based delivery patterns.

The Practical Fallback: Idempotent Sinks

Most teams don't need Kafka Streams transactions or Flink checkpoints. What they need is a way to make at-least-once delivery safe, and that usually comes down to a unique key and a database constraint.

When exactly-once infra is too heavy, use:
  at-least-once delivery + idempotent operations

  Example (payment processing):
  INSERT INTO payments (transaction_id, amount, status)
  VALUES ('txn-abc-123', 500, 'completed')
  ON CONFLICT (transaction_id) DO NOTHING;

  Even if processed 3 times, only 1 row exists.
  The transaction_id is the idempotency key.

  This is what Stripe does: every API call has
  an Idempotency-Key header. Retries are safe.

This is basically the same idea behind the transactional outbox pattern: instead of trying to make two separate systems commit atomically, you record the event and its unique identifier in a place that naturally rejects duplicates. It's less elegant than true exactly-once delivery, but it's far easier to reason about during an incident, and it doesn't require every operator in your pipeline to support checkpointing.

Interview Tip

When discussing stream processing guarantees, say: 'True exactly-once requires atomic coordination between reading, processing, and writing. Kafka achieves this with idempotent producers (sequence-based dedup) and transactions (atomic offset commits and writes). Flink uses Chandy-Lamport checkpoints to snapshot state and replay from checkpoints on failure. For most practical systems, I would use at-least-once delivery with idempotent sinks, using a unique transaction ID as a dedup key so processing the same event twice is a no-op. This gives exactly-once SEMANTICS without the complexity of exactly-once DELIVERY.'

Exactly-Once Processing: The Hardest Problem in Distributed Streaming - Architecture Diagram

Architecture overview

Key Takeaway

Exactly-once is achieved by combining idempotent producers, transactional commits, and recoverable state. Kafka and Flink both provide this but at a performance cost. When exactly-once infrastructure is too heavy, design idempotent sinks as a simpler alternative.

If you're building out a broader event-driven system and want the surrounding context, these go well with this post:

Enjoyed this article?

Share it with your network to help others level up their system design skills.

Discussion0

Join the Discussion

Sign in to leave comments, reply to others, or like insights.

Sign In to ScaleDojo

No comments yet. Be the first to start the thread!

Related Articles

Enjoyed this content?

Your support keeps us creating free resources

We put a lot of hours into researching and writing these guides. If it helped you, consider buying us a coffee. Every bit goes toward keeping ScaleDojo's content free and growing.

$

One-time payment via Stripe. ScaleDojo account required.

ScaleDojo Logo
Initializing ScaleDojo