Geo-Sharding & Data Residency

Everything so far in this module has been about speed and availability. Data residency is a different kind of reason to split a system across the world, and it has nothing to do with performance. Rules about where personal data may be stored and sent can turn the question where does this data live from a technical choice into a compliance requirement, and a design that ignores it can be forced to change after launch, at great cost.

Regulations such as the GDPR in Europe restrict sending personal data about people in Europe outside the region unless specific safeguards are in place, and some countries go further, with data localization laws that require certain data to be stored inside the country. The details differ by country, sector and year, and your company's legal team decides what applies. For the architect the practical effect is the same: a simple and defensible way to comply is to keep a user's personal data in their home region and never replicate it outside. That is what geo-sharding does.

A bank keeps each customer's valuables in a vault in the customer's home country. A customer who travels abroad does not have the valuables shipped after them. Instead, the foreign branch phones the home vault when something is needed. And when head office wants a global total, it asks each country for a summary figure, rather than collecting everyone's valuables in one place.

• -A country's vault is a regional shard: it holds that region's users' data.
• -A customer's home country is the user's home region.
• -The foreign branch phoning home is a request routed to the home region.
• -Asking each country for a summary is aggregating per region and combining only the totals.

Instead of sharding by a hash or key range, as the earlier chapter described, geo-sharding partitions data by the user's region. A user who signs up from Germany is assigned the EU region as their home. Their records live on European infrastructure and are never replicated to another region. A US user's data lives in the US. The diagram shows the idea:

The system needs to know a user's home region on every request. A neat trick ties back to the ID chapter: put a region code inside the user's ID, in the same way a Snowflake ID packs in a machine number, so any server in the world can read where a user belongs without a lookup. The gateway then forwards the request to that region.

• -A traveling user: a German user opens the app in New York. Their data stays in the EU. The request is routed to the EU region, adding the extra delay of crossing the Atlantic, which is acceptable for a visitor.
• -Global analytics: a company-wide dashboard needs totals across every region. Run the aggregation inside each region and combine only the anonymized totals, instead of shipping raw personal records to one place.
• -Leaks into other places: personal data does not only sit in the main database. It appears in logs, caches, search indexes, backups and analytics copies, and each of those that crosses the border can break the rule. Check every place data flows, not just the primary store.
• -Shared data: a product catalog has no personal data and can be replicated everywhere. Split what is personal from what is not.

There is a benefit as well. When a user asks for their data to be deleted, a right most privacy laws grant, you know where all of it is: one home region, with its caches, search index and backups, rather than copies scattered around the world.