Service Discovery & Service Mesh

Once services are split apart and each can scale, deploy and restart independently, a new problem appears. How does the Orders service know the current network address of the Users service, when that address can change every time an instance restarts, crashes, or is added during a traffic spike? A hard-coded address works on the day you write it and goes stale the day after.

In a large company, people move desks and change extensions all the time. Nobody memorizes phone numbers. Instead there is a company directory, and when you need someone you look them up and call the extension it shows. The directory only works if it is kept current: people register when they arrive, and someone who has left is removed. A good directory also notices when a person has stopped answering their desk phone.

• -The people are service instances, which come and go.
• -The directory is the service registry.
• -Registering on arrival is registration, and someone proving they are still at their desk is a heartbeat.
• -Looking someone up before calling is service discovery.

Each service instance registers itself with a shared registry when it starts, and removes itself, or is detected as gone, when it stops. Callers ask the registry for the current addresses instead of using fixed IP addresses. The card below describes the registry component as it appears in the lab:

• -A Users instance starts at 10.0.3.17:8080 and registers: the name users maps to that address. It then sends a heartbeat every 10 seconds.
• -The Orders service asks the registry where users is, and receives two addresses, 10.0.3.17 and 10.0.3.22. It picks one in turn and caches the list for a short time, say 30 seconds, so it does not ask on every call.
• -The instance at 10.0.3.17 crashes. Its heartbeats stop, and after a timeout the registry drops it.
• -Until callers refresh their cache, a few calls may go to the dead address and fail. A well-behaved client retries on another instance, and soon everyone sees only the surviving one.

The diagram shows the pattern: every instance registers, and the caller asks the registry for a current address.

Consul, etcd and ZooKeeper are common registries. In Kubernetes, discovery is built in: a service's name resolves to whichever of its instances are currently healthy.

• -Client-side discovery: the caller queries the registry and chooses an instance itself. There is one hop fewer, but every client needs the discovery and load-balancing logic.
• -Server-side discovery: the caller sends its request to a load balancer or router, which consults the registry. Clients stay simple, but there is another component to run and keep available.

As the number of services grows, every one of them needs the same things around its calls: find the other service, retry on failure, time out, encrypt the traffic, and report metrics. Putting that logic into a library in each service means writing it once per language and updating every service to change it. A service mesh moves it out of the code. Next to every service instance runs a small proxy, a sidecar (Envoy is the common one), and all traffic in and out goes through it. A control plane configures every sidecar from one place. It is as if every employee had an assistant who makes their calls: placing the call, retrying if it does not connect, checking the other person's identity and keeping a log.

• -Security: mutual TLS between all services, so each proves who it is and the traffic is encrypted, with no application changes.
• -Reliability: retries, timeouts and circuit breaking configured centrally.
• -Traffic control: send 5% of requests to a new version to test it, or shift traffic gradually.
• -Visibility: metrics and traces for every call, for free.

The costs are real: each call now passes through two extra proxies, which adds a little latency, every instance uses extra memory and CPU for its sidecar, and the mesh itself is a complex system to operate. It earns its place with dozens of services, strict security requirements, or many languages. With three services it is overkill. The Wiki card below goes further: