Load Balancing Algorithms: Round-Robin, Weighted, and Least Connections

The Slow Server Problem

Choosing the right load balancing algorithm is what separates a system that gracefully handles failure from one that doesn’t .An e-commerce site has 5 servers behind a load balancer using round-robin. Server 3 has a failing disk, making its queries 10x slower. But round-robin does not know this - it keeps sending every 5th request to the slow server. 20% of your users experience a terrible experience while the other 4 servers sit partially idle. The right algorithm would have detected this and routed around it.

Round-Robin: Equal Turns

The simplest and most widely used algorithm. Instead of checking server load, it simply sends requests one by one in order.

Round-Robin distribution:

  Request 1  -->  Server A
  Request 2  -->  Server B  
  Request 3  -->  Server C
  Request 4  -->  Server A  (back to start)
  Request 5  -->  Server B
  Request 6  -->  Server C
  ...

  Pros: Simple, no state to track, perfectly even distribution
  Cons: Ignores server health, capacity, and current load
  Best when: All servers are identical AND all requests
             take roughly the same time

Weighted Round-Robin: Capacity-Aware

What if servers have different capacities? Instead of giving each server equal traffic, assign weights.

Weighted Round-Robin (server capacity differs):

  Server A: weight 5  (16 CPUs, 64GB RAM)
  Server B: weight 3  (8 CPUs, 32GB RAM)
  Server C: weight 1  (4 CPUs, 16GB RAM)

  9 requests distributed as:
  A, A, A, A, A, B, B, B, C

  Use cases:
  - Mixed hardware (big + small servers)
  - Canary deployments (new version gets weight 1,
    stable gets weight 9 = 10% canary traffic)
  - Gradual migration between server pools

Least Connections: Route to Least Busy

Unlike the previous algorithms, Least Connections looks at the current number of active connections on each server. It always chooses the server with the fewest ongoing requests.

Least Connections (self-balancing):

  Active connections:
    Server A: 23 active
    Server B: 47 active  (processing slow report queries)
    Server C: 12 active  <-- next request goes here!

  Why this is better:
  - Server B is slow? It accumulates connections,
    so LB naturally sends fewer new requests to it.
  - Fast server finishes quickly, connections drop,
    so LB sends MORE requests to it.
  - Self-healing: no manual tuning needed.

  This is the best default algorithm for most applications.

All Algorithms Compared

Algorithm           State Needed   Handles Slow  Handles Mixed  Best For
                                   Servers?      Hardware?
------------------  ------------   -----------   ------------   --------
Round-Robin         None           No            No             Identical servers
Weighted RR         Weights only   No            Yes            Mixed capacity
Least Connections   Connection ct  Yes           Partially      Variable requests
Least Response Time Timing data    Yes           Yes            Latency-sensitive
IP Hash             None           No            No             Session affinity
Random              None           No            No             Stateless LBs
Power of 2 Choices  2 samples      Yes           Yes            Large clusters

Which Algorithm Should You Choose?

Use Round-Robin when

• All servers have similar hardware

• Requests are short-lived

• Simplicity is important

Use Weighted Round-Robin when

• Servers have different CPU or memory capacities

• Traffic distribution should match server power

• The environment is relatively stable

Use Least Connections when

• Request durations vary significantly

• Some requests remain open for a long time

• You need dynamic balancing based on real-time server load

Real-World Examples

• NGINX supports Round-Robin (default), Weighted Round-Robin, Least Connections, IP Hash, and more.

• HAProxy offers Least Connections, Round-Robin, Source Hash, Random, and advanced balancing strategies.

• Cloud load balancers like AWS Application Load Balancer, Google Cloud Load Balancer, and Azure Load Balancer use different algorithms depending on the service and configuration.

Health Checks: Don't Route to Dead Servers

Health check configuration (Nginx):

upstream api_servers {
    least_conn;  # algorithm: least connections
    
    server 10.0.1.1:8080 max_fails=3 fail_timeout=30s;
    server 10.0.1.2:8080 max_fails=3 fail_timeout=30s;
    server 10.0.1.3:8080 max_fails=3 fail_timeout=30s;
    # If a server fails 3 health checks, remove it for 30s
    # After 30s, try again. If it passes, add it back.
}

# Active health check (checks even without traffic)
location /health {
    return 200 'OK';
}

# The LB hits GET /health every 5 seconds.
# 3 consecutive failures = server marked unhealthy.
# Healthy servers absorb the dead server's traffic.

Interview Tip

When an interviewer asks which load balancing algorithm to use, say: 'My default is least connections because it naturally adapts to slow servers and variable request durations. If we have mixed hardware, I would add weights. For session-sticky applications, I would use cookie-based routing at L7 rather than IP hash, because IP hash breaks when users are behind NAT or change networks. I would always configure health checks with max_fails=3 to automatically remove unhealthy servers.'

Key Takeaway

Round-robin is simple but naive. Weighted round-robin handles heterogeneous servers. Least connections adapts to real-world load variance. When in doubt, least connections is the best general-purpose choice because it automatically handles slow servers and variable request durations.

Reading about load balancing algorithms is a good start -actually configuring one and watching it route around a failing server is where it clicks. Start Building Free on ScaleDojo → and practice this exact system design pattern with instant AI feedbac