Every time you open a Google service, your request lands on one of a few addresses and somehow reaches one of thousands of servers. The machine that decides where to send each connection is a load balancer. For years the standard answer was a specialized hardware box. Hardware load balancers are expensive, hard to upgrade, and limited by what one box can handle. If traffic doubles, you need a bigger box, not just another one.
Maglev, described in 2016, is Google's replacement: a load balancer written in software and run on ordinary servers. It has been a critical part of Google's front end since 2008 and serves almost all of Google's incoming user traffic. The paper is a lesson in scale-out design, and it contains an elegant new hashing trick.
Maglev spreads traffic over many commodity machines with the router's equal-cost multipath, sends each connection to a backend using connection tracking plus a lookup table built by Maglev hashing, and lets replies bypass the balancer. Adding capacity means adding machines.
How a packet travels
Every Google service has one or more virtual IP addresses (VIPs). A VIP does not belong to one machine. It is served by many backends. Here is the path of a request:
- The user's packet reaches Google's edge router. All the Maglev machines announce the same VIP with the same cost, so the router uses equal-cost multipath (ECMP) to spread packets over them.
- A Maglev machine picks a backend for the packet, then wraps it in a GRE tunnel header and sends it on. The backend unwraps it and sees the original packet.
- The backend answers the user directly through the router. This is direct server return. Responses are usually larger than requests, so Maglev does not need to carry them.
