Load Balancers: L4 vs L7
You'll learn to
- -Explain what a load balancer does once you have multiple servers
- -Distinguish Layer-4 from Layer-7 load balancing
Horizontal scaling gives you multiple interchangeable servers, but something now has to decide which server handles each incoming request. That something is a load balancer, and it becomes the new front door to your logic tier.
Every request depends on this one machine. It crashes, everyone is down; it saturates, everyone slows down.
The load balancer distributes requests and stops sending traffic to any server that fails its health check.
The two diagrams above show the shape of the fix, but "distributes requests" is doing a lot of work in that sentence. Play through the simulation below to see one specific algorithm, round robin, actually make that decision request by request.
Round Robin, Load Balancing
The balancer walks the server list in fixed order - each request goes to the next server.
Clients
Client A
10.0.0.42
Client B
10.0.0.108
Client C
192.168.1.7
Client D
203.0.113.55
Load Balancer
Round Robin
pick: web-01
Servers
web-01
routed 110.0.1.11:8080
web-02
routed 010.0.1.12:8080
web-03
routed 010.0.1.13:8080
web-04
routed 010.0.1.14:8080
Dispatch Order
Picture a popular restaurant with one door but 10 chefs in the kitchen. Without a host at the front, every customer would crowd around Chef #1 while Chefs #2 through #10 stand around doing nothing. The load balancer IS that host. It stands at the entrance and says 'Table for you with Chef #3, table for you with Chef #7, table for you with Chef #1...' distributing customers evenly so no single chef gets overwhelmed. In tech terms, a load balancer sits in front of your servers and distributes incoming network requests across them. If you have 5 web servers, the load balancer makes sure each one gets roughly 20% of the traffic. If one server crashes, the load balancer detects it (via health checks) and stops sending traffic there. Users never even notice.
Examples: AWS ALB (Application Load Balancer), AWS NLB (Network Load Balancer), Nginx, HAProxy, GCP Load Balancer, Traefik
Layer 4 vs Layer 7
A Layer-4 (L4) load balancer works at the transport level: it sees IP addresses and ports, and routes based purely on that, without looking inside the request. It's fast and protocol-agnostic, but it can't make decisions based on the content of a request. A Layer-7 (L7) load balancer operates at the application level: it can read the HTTP path, headers, and even the body, and route "/api/orders" to one set of servers and "/api/search" to another, or send 5% of traffic to a canary version for testing. L7 costs a bit more latency and CPU in exchange for that intelligence.
How a load balancer DECIDES which backend server gets each incoming request.
A hotel front desk has 5 receptionists. How do you decide which receptionist handles the next guest? Round Robin (next in line), Least Connections (whoever is currently least busy), or IP Hash (the same guest always goes to the same receptionist so they remember their preferences)?
Round Robin: Requests are distributed sequentially across servers: server1, server2, server3, server1, server2... Simple, predictable, and even - but ignores the fact that some requests take 1ms while others take 5 seconds. A slow server accumulates a pile of long-running requests while still receiving new ones.
Weighted Round Robin: Each server gets a weight. Server with weight=3 gets 3 requests for every 1 request the weight=1 server gets. Use this when servers have different hardware capacities.
Least Connections: Route to the server with the fewest active connections. Works well when request processing times vary significantly (e.g., some requests are fast API calls, others are slow image processing jobs). Requires the load balancer to track connection counts.
Least Response Time: A sophistication of Least Connections - route to the server with the lowest combination of active connections AND response time. AWS ALB's Least Outstanding Requests uses this principle.
IP Hash: `server = hash(client_ip) % num_servers`. Same client always hits the same server. Enables "sticky sessions" without requiring a session token. Breaks badly when clients are behind NAT (many clients sharing one IP all go to the same server).
- -Round Robin is almost always the default and fine for homogeneous request patterns.
- -Least Connections is better for heterogeneous workloads (mixed fast and slow requests).
- -Sticky sessions (IP Hash) should be avoided - use server-side sessions (Redis) instead.
- -Layer-4 LBs (TCP level) can only use IP/port - Round Robin or IP Hash.
- -Layer-7 LBs (HTTP level) can inspect URLs/headers and route by content (A/B testing, microservice routing).
One of your five servers behind the load balancer silently starts failing every request. What happens?
"The load balancer spreads requests evenly, so only 20% of users are affected."
"Only if there's no health check. Without one, that broken server keeps receiving its full share of traffic forever, and 20% of all requests fail indefinitely. The load balancer needs an active health check (hit a /health endpoint every few seconds) so it can pull the failing server out of rotation automatically, not just balance load blindly."