Worked Example: Designing a Chat System (like WhatsApp)

The URL shortener was almost entirely stateless request/response. A chat system breaks that assumption immediately: it needs a persistent, long-lived connection per user, which directly conflicts with the "any server can handle any request" principle from Module 2. This chapter is about resolving that tension.

• -Functional: 1:1 and group messaging, delivery status (sent/delivered/read), online presence, message history that syncs across a user's devices.
• -Out of scope: media/file transfer internals (assume object storage handles that, covered in Module 11), voice/video calls.
• -Non-functional: messages must arrive in the right order within a conversation, delivery should feel near-instant when both users are online, and no message should be silently lost if a user is briefly offline.

The message store is deliberately a wide-column NoSQL database, not a relational one: the access pattern is almost always "give me messages for conversation X, ordered by time," a simple, massive-write-volume pattern that plays directly to a wide-column store's strengths, exactly as the Databases 101 module framed the choice.

A user's WebSocket connection is pinned to one specific gateway server, which is unavoidable, since a TCP connection cannot be load-balanced mid-flight to a different machine. This means any OTHER server that needs to deliver a message to that user must first discover which gateway currently holds their connection. The fix: a presence registry (a fast key-value store, mapping user_id -> gateway_server_id) that every gateway updates on connect/disconnect, and that the message service consults before routing every message.

• -User comes online: their gateway writes {user_id: gateway_id} to the presence registry.
• -A message arrives for them: the message service looks up their current gateway, then forwards the message there specifically.
• -User goes offline or the gateway crashes: the registry entry is removed (or expires via TTL), and the message service instead writes the message to a durable per-user inbox for delivery on reconnect.

Two different servers' clocks can disagree, so a raw timestamp is not a safe way to order messages within a conversation. Instead, assign a per-conversation monotonic sequence number at write time: every client can then reliably reconstruct "what came after what," independent of clock skew. For delivery, the system guarantees at-least-once (the same guarantee the Async Systems module named as the practical default for queues); a client may occasionally receive a duplicate after a reconnect, and de-duplicates using the message's unique ID, exactly the idempotency pattern from Module 12.

• -Group chat fan-out: for a group of N members, the message service pushes to N recipients' gateways per message. Unlike a social feed, N here is small (tens to low hundreds), so simple fan-out-on-write is fine: no celebrity problem at this scale (the next chapter is the case where that stops being true).
• -Consistency: "delivered" and "read" receipts propagate asynchronously and can briefly disagree across a user's own devices, which is acceptable, since nobody expects perfectly synchronized read receipts within milliseconds.
• -Failure mode: a gateway server crashes. Every connection it held drops. Clients detect the dropped socket, reconnect (likely to a different gateway via the load balancer), re-register their presence, and request "everything since sequence number N" for each conversation to fill the gap: the sequence number from the deep-dive is what makes this resync possible at all.