Skip to content
The Chubby Lock Service for Loosely-Coupled Distributed Systems 5 min

Mike Burrows · OSDI 2006

The Chubby Lock Service for Loosely-Coupled Distributed Systems

Google's lock service in plain terms: sessions kept alive by KeepAlive calls, grace periods, and the fencing tokens that stop a frozen old leader from corrupting data.

Read the original paper

Systems & Distributed · Yellow Belt · Pro · 5 min read

The Chubby Lock Service for Loosely-Coupled Distributed Systems

Imagine ten identical servers that all want to be "the leader". If two of them think they are in charge at once, they can corrupt the data. Something has to decide, and that something must keep working even when machines fail.

Google's answer, described by Mike Burrows in 2006, was Chubby: a small, highly available service that hands out locks and stores tiny files. GFS uses it to pick its master, and Bigtable uses it in several ways. Chubby is a good paper for beginners because it admits its mistakes. Half of it explains how real engineers used the service differently than the designers expected.

Chubby is a lock service that looks like a tiny file system. Clients keep a session alive with heartbeats called KeepAlives. A lock is held only while the session is alive. Fencing numbers called sequencers stop a slow, confused old lock holder from doing damage.

A lock service, not a library

Why run a separate service for locks? Because almost every team needs the same two things: a way to elect one leader, and a place to publish small facts such as "the primary is on host 17." Writing a correct consensus protocol inside each product is hard. Calling one shared service is easy.

Chubby gives each team a file-system-like tree of names. A file can hold a few bytes of data, and any file can also act as a lock. A leader election then takes four steps. Every candidate tries to lock the same file. One wins. The winner writes its own address into the file. Everyone else reads the file to find the leader.

Clients talk only to the master, mostly with KeepAlive calls that say 'I am still here'. The other replicas keep a copy of the data so a new master can take over.

The paper says Chubby is meant for coarse-grained locks, ones held for hours or days, like "who is the primary right now?" Rare, long-lived locks put little load on the server. Teams that need quick, fine-grained locks are expected to build those on top.

Inside, each Chubby "cell" is five replicas, and three must be running for the cell to work. One replica is the master and answers all requests. The others keep copies. If the master dies, the replicas elect a new one. Clients find the master through DNS and by asking any replica.

Keep reading with Pro

This paper is part of Pro. Pro and Architect open the full breakdown, with animated diagrams, interactive elements you can play with and a quiz. A few papers stay free to read in full.

Still to read

3 more sections, 1 more diagram and 2 interactive elements to play with

  • Sessions and leases: how Chubby knows you are alive
  • Fencing: stopping the zombie leader
  • Lessons from running it for years

Found this breakdown useful?

Share it with someone else wrestling with this paper.

Discussion0

Join the Discussion

Sign in to leave comments, reply to others, or like insights.

Sign In to ScaleDojo

No comments yet. Be the first to start the thread!

Enjoyed this? Get more like it.

New paper breakdowns, levels, and one concept worth knowing, straight to your inbox.

No spam, ever. Unsubscribe in one click.

Enjoyed this content?

Your support keeps us creating free resources

We put a lot of hours into researching and writing these guides. If it helped you, consider buying us a coffee. Every bit goes toward keeping ScaleDojo's content free and growing.

$

One-time payment via Stripe. ScaleDojo account required.