Stateless Services & Session Handling

Everything in this module has been building toward one practical question: if a user logs in on server A, and their next request is routed by the load balancer to server B, how does server B know who they are? Get this wrong and horizontal scaling breaks in a way that is easy to miss in a demo and painful in production.

State is anything a server has to remember between one request and the next. Whether you are logged in, what is in your basket, the half-finished upload of a large file, how many requests you have made this minute: all of it is state. A stateless server is one that keeps none of it itself. Each request carries (or points to) everything the server needs, so any server can handle it, and a server can be added, removed or restarted without anyone losing anything.

Suppose you run three servers and each keeps login sessions in its own memory. Sam logs in and the load balancer sends that request to server A, which stores that Sam is logged in. Sam clicks to another page. With requests spread evenly, the chance it lands on A again is one in three. Two times in three it lands on B or C, which have never heard of Sam, so they answer: please log in. From Sam's side the site randomly logs them out on about two clicks in three. With one server this bug cannot exist; it appears the day you scale out.

Think of a theatre cloakroom. In the badly designed version, each attendant hangs coats on their own private rack, and you can only collect your coat from the attendant who took it. If that attendant is on a break, you wait; if they go home, your coat goes with them. In a better design there is one central cloakroom that every attendant can reach, and you are given a numbered ticket. Any attendant can read your ticket and fetch the coat. And in some situations the best design is that you keep your coat with you, so nobody has to store anything at all.

• -Each attendant's private rack is session data held in one server's memory.
• -Insisting on the same attendant is a sticky session.
• -The central cloakroom is a shared session store, usually Redis, a very fast in-memory database that every server can reach.
• -The numbered ticket is the session ID, which the browser holds in a cookie.
• -Carrying your coat yourself is a token: the session data itself, signed, held by the client.

One tempting fix is to configure the load balancer so that a given user always reaches the same server, for example by hashing the IP address. It works, but it quietly brings back the problem you were solving, in three ways:

• -A crash loses sessions: if server A fails, every user pinned to it is logged out at once.
• -Load becomes uneven: if heavy users happen to be pinned to the same server, it saturates while the others sit idle.
• -Deploys and scaling hurt: restarting a server drops its users, and a newly added server starts empty while existing users stay pinned to the old ones.

• -Shared session store: session data lives in a fast external store (usually Redis) that every server can read. Any server can serve any user, and losing a server loses nothing. The browser holds only a session ID.
• -Tokens (for example JWTs): the session data is encoded into a signed token that the client holds and sends with every request. No server-side lookup is needed; the signature proves the token is genuine and has not been altered.

In a diagram, the servers become interchangeable because they all read the same store:

Each approach has a price. A shared store adds one extra network call per request (often under a millisecond to a couple of milliseconds) and becomes a critical dependency, so it must itself be replicated. A token needs no lookup, but the server cannot easily cancel it before it expires: if a token leaks, it keeps working until its expiry time unless you keep a deny-list, which is state again. Many systems use short-lived tokens plus a way to refresh them, to keep that window small.

Sessions are the famous case, but state sneaks in elsewhere, and each instance breaks scaling out in the same way:

• -Files on local disk: a user uploads a photo to server A's disk, and server B cannot find it. Store uploads in shared object storage instead.
• -In-memory counters and caches: a limit of 100 requests a minute enforced separately on each of five servers quietly becomes 500. Keep such counters in a shared store.
• -Scheduled jobs: a job meant to run once a night runs five times if it is on every server. Run it from one place, or take a lock first.