Circuit Breakers & Fallbacks
You'll learn to
- -Implement a circuit breaker's three states (closed, open, half-open) and explain what each one protects against
- -Design a meaningful fallback response for when a circuit is open, instead of just failing every request outright
When a downstream dependency starts failing, continuing to call it on every request - and waiting for each call to time out - wastes resources and adds latency to every caller, often making an already-bad situation worse. A circuit breaker stops calling a failing dependency entirely for a while, failing fast instead of failing slow.
The Three States
class CircuitBreaker:
def __init__(self, failure_threshold=5, recovery_timeout=30):
self.state = "closed" # normal operation
self.failure_count = 0
self.failure_threshold = failure_threshold
self.recovery_timeout = recovery_timeout
self.opened_at = None
def call(self, downstream_fn):
if self.state == "open":
if time.time() - self.opened_at > self.recovery_timeout:
self.state = "half_open" # time to test if it recovered
else:
raise CircuitOpenError() # fail fast, don't even try
try:
result = downstream_fn()
if self.state == "half_open":
self.state = "closed" # recovered - resume normal operation
self.failure_count = 0
return result
except Exception:
self.failure_count += 1
if self.failure_count >= self.failure_threshold:
self.state = "open"
self.opened_at = time.time()
raise- -Closed: normal operation - requests pass through to the downstream dependency, failures are counted.
- -Open: too many recent failures - requests fail immediately without even attempting the downstream call, giving the struggling dependency room to recover instead of continuing to hit it.
- -Half-open: after a recovery timeout, a limited number of requests are allowed through as a test - if they succeed, the circuit closes (resume normal operation); if they fail, it reopens and waits again.
The half-open state is what makes this self-healing rather than requiring manual intervention to reset - the breaker periodically re-tests the dependency on its own, rather than staying open forever or (the opposite mistake) immediately flipping back to closed and hammering a dependency that hasn't actually recovered yet.
Fallbacks: What to Return When the Circuit Is Open
Failing fast is better than failing slow, but "fail" is still often not the best available response - a product recommendation service that's down doesn't need to break checkout entirely; returning a generic, cached, or empty set of recommendations lets the rest of the page function normally. A well-designed fallback provides a degraded-but-functional experience specifically for the failing dependency's role in the larger system, rather than treating every failure as equally catastrophic to the caller.
The threshold and recovery timeout are real, workload-specific tuning parameters, not universal constants - too sensitive a threshold trips the breaker on normal, brief blips; too lenient a threshold keeps hammering a genuinely struggling dependency for too long before finally giving it room to recover.
Interview Signal is part of Pro
See a real weak answer next to a real strong one for this exact topic.
Quiz is part of Pro
Test what you just read with a short quiz, and bank the XP.
Design Circuit Breaker API in the API Design Lab's Full System Design act.