Building Chat & Presence Systems

Chat is the textbook use of WebSockets: it is genuinely two-way and genuinely sensitive to delay. But a subtlety appears the moment you have more than one WebSocket server. Client A might be connected to server 1 while Client B is connected to server 2, and the two servers know nothing about each other's clients. This chapter shows how a message crosses that gap, how it avoids being lost, and how a system knows who is online.

Picture an office building with a receptionist on every floor. Alice is on floor 1 and wants to send a note to Bob on floor 3. Her receptionist has no idea where Bob sits. So she pins the note on the building's announcement board, and any receptionist who has someone interested in that note picks it up. Floor 3's receptionist knows Bob is there and hands it to him. Meanwhile each person wears a badge that must be stamped every 30 seconds, and a badge that has not been stamped for a minute is taken to mean the person has left.

• -A receptionist is a WebSocket server, holding the connections of the users on its floor.
• -The announcement board is the publish/subscribe layer that links the servers.
• -The stamped badge is a presence heartbeat, and the minute after which a badge counts as expired is its TTL.

The diagram shows the shape: a message from Client A is broadcast through a pub/sub layer to reach Client B, who is connected to a different server.

Here is the full path of one message, including the part that stops it being lost:

• -Client A sends the message over its WebSocket to server 1, with an id the client generated and the conversation it belongs to.
• -Server 1 saves the message to the database first, so it is durable, and tells Client A it was sent.
• -Server 1 publishes the message on that conversation's channel in the pub/sub layer.
• -Every server that has a member of the conversation connected receives it, and server 2 pushes it down Client B's connection.
• -Client B acknowledges receipt. If Client B was offline, nothing is lost: the message is already stored, and when B reconnects it asks for every message after the last one it saw.

The order matters. Pub/sub is for speed and is not the source of truth; the database is. Saving first and then publishing means a message is never delivered to someone while being missing from history.

Presence is usually tracked with a short-lived key per user in a fast store such as Redis. The client's open connection sends a heartbeat every 30 seconds that refreshes the key with a 60-second TTL. If the heartbeat stops, because the connection dropped or the tab closed, the key simply expires and the user counts as offline. No goodbye message is needed, which is exactly what you want, since a crashed app cannot send one. The price is a delay of up to a minute before someone shows as offline.

At scale the heartbeats themselves are the load. With 10 million users online and one heartbeat each every 30 seconds, that is 10,000,000 / 30, about 333,000 writes a second, which needs a sharded store and is a good reason not to heartbeat more often than you must. The second cost is telling people. If someone with 500 contacts goes offline, you do not want to push 500 updates, so systems tell only the contacts who currently have that person on screen, and let everyone else look presence up when they open the conversation.

• -Message ids: the client generates a unique id for each message, so a retry after a dropped connection is recognized as the same message and not shown twice. This is the idempotency idea again.
• -Acknowledgements: sent, delivered and read are separate acknowledgements, and each can be retried until confirmed, which gives at-least-once delivery.
• -Order: the server assigns each message in a conversation an increasing sequence number (or a time-ordered id), and clients display by that number, so everyone sees the same order even if messages arrive out of order.