Skip to content
API Design Learn/Platform & Resilience
Browsing as a guest. Sign in to save your progress and earn XP as you complete chapters.

Circuit Breakers & Fallbacks

4 min read

You'll learn to

  • -Trace a circuit breaker through closed, open and half-open, and say what each state protects
  • -Return a useful fallback while the circuit is open, instead of failing the whole page
  • -Explain what a bulkhead isolates that a circuit breaker does not, and use both

When a downstream dependency starts failing, continuing to call it on every request, and waiting for each call to time out, wastes resources and adds delay for every caller. This often turns a bad situation into a worse one. A circuit breaker stops calling a failing dependency for a while. It fails fast instead of failing slowly.

An Analogy: The Fuse in Your House

A fuse cuts the power when a circuit draws too much, instead of letting the wires overheat. It protects the house. You check the fault, then reset it, and normal service resumes. A circuit breaker in software is the same, with one improvement: it can reset itself by testing carefully whether the fault has gone.

The Three States

  • -Closed: normal operation. Requests pass through to the dependency, and failures are counted.
  • -Open: too many recent failures. Requests fail at once, without trying the dependency, giving it room to recover instead of continuing to hit it.
  • -Half-open: after a recovery timeout, a limited number of requests are let through as a test. If they succeed, the circuit closes. If they fail, it opens again and waits again.

The half-open state is what makes the breaker self-healing. Without it you would have to reset the circuit by hand, or you would flip back to closed after a timer and hammer a dependency that has not actually recovered. The demo runs one through the states with a fake clock, a recommendations service that is down, and a fallback of an empty list.

Press Run: three failures open the circuit, calls then fail fast, and a test call closes it again

Fallbacks: What to Return When the Circuit Is Open

Failing fast is better than failing slowly, but failing is often not the best available response. A product-recommendation service being down does not have to break checkout. Returning a generic, cached or empty set of recommendations, as in the demo, lets the rest of the page work. A well-designed fallback gives a degraded but working experience for the failing dependency's part of the system, instead of treating every failure as equally catastrophic.

The threshold and the recovery timeout are tuning parameters for your workload, not universal constants. Too sensitive a threshold trips the breaker on normal brief blips. Too lenient a threshold keeps hammering a struggling dependency for too long before giving it room to recover.

Bulkhead: Isolating One Failure So It Doesn't Sink Everything Else

A circuit breaker decides whether to keep calling a failing dependency. It does not address a different problem: one slow dependency quietly using every worker thread or connection in a shared pool, so requests to all the other healthy dependencies that share that pool starve. A bulkhead, named after the watertight compartments in a ship's hull, caps how many concurrent calls any one dependency may use. The last lines of the demo show a shared pool being emptied by one slow service, and separate pools containing the damage.

Without a bulkhead, one slow dependency can starve every caller sharing its thread pool
# recommendations-service is slow today, but it's just ONE of five
# downstream calls this handler makes - without isolation, every
# thread in the shared pool can end up blocked waiting on it,
# leaving zero threads free to serve calls to the other four
# (healthy) dependencies.
pool = ThreadPoolExecutor(max_workers=50)   # shared by all downstream calls

# A bulkhead gives each dependency its own bounded slice instead:
pools = {
    "recommendations": ThreadPoolExecutor(max_workers=10),
    "inventory":        ThreadPoolExecutor(max_workers=10),
    "pricing":          ThreadPoolExecutor(max_workers=10),
}
# recommendations-service being slow now only exhausts its own
# 10 workers - inventory and pricing keep serving normally.

A bulkhead and a circuit breaker solve adjacent problems, and production systems normally use both. The bulkhead limits the blast radius of one dependency's slowness on shared resources. The breaker decides when to stop calling that dependency altogether once it is clearly failing and not merely slow.

Interview Signal is part of Pro

See a real weak answer next to a real strong one for this exact topic.

Quiz is part of Pro

Test what you just read with a short quiz, and bank the XP.

Ready to Build This?

Design Circuit Breaker API in the API Design Lab's Full System Design act.

Discussion0

Join the Discussion

Sign in to leave comments, reply to others, or like insights.

Sign In to ScaleDojo

No comments yet. Be the first to start the thread!