Service Discovery and Registration: How Services Find Each Other

The vanishing IP problem

In a monolith, calling another module is a function call — the address is compiled in, and it never moves. In a microservices architecture running on Kubernetes or cloud VMs, that assumption breaks completely. Services get dynamically assigned IP addresses. They auto-scale, so new instances appear with new IPs. They crash, so instances disappear without warning. They get redeployed on every release, so even a "stable" service's IP changes constantly. Hardcoding an IP address into a config file is a strategy that fails within hours of going to production. What you need instead is something closer to a phone book that updates itself in real time, and that's what service discovery is — the mechanism by which a service finds the current, correct address of another service without either side ever hardcoding it.

Client-side discovery

Client-Side Discovery:

  [Service A]                    [Service Registry]
       |                          (Consul, Eureka, etcd)
       |  1. Query: "Where is     |
       |     service-B?"          |
       |------------------------->|
       |                          |
       |  2. Response:            |
       |     [10.0.1.5:8080,      |
       |      10.0.1.6:8080,      |
       |      10.0.1.7:8080]      |
       |<-------------------------|
       |                          |
       |  3. Client picks one     |
       |     (round-robin, random,|
       |      least-connections)  |
       |                          |
       +---call--> [10.0.1.6:8080]

In this model, the calling service queries a registry directly, gets back the full current list of healthy instances, and decides which one to call itself — using round-robin, random selection, least-connections, or something smarter like latency-aware routing. The advantage is that the client has full visibility and control: it can route around a slow instance or prefer one in the same availability zone. The cost is that every service in every language needs a registry client library that knows how to query the registry and apply the load-balancing logic — that's real code duplicated across your entire service fleet, and it has to be kept in sync as the registry's API evolves.

Server-side discovery

Server-Side Discovery:

  [Service A]    [Load Balancer / API Gateway]
       |              |
       |  1. Call      |          [Service Registry]
       |  service-b    |                |
       |-------------->|  2. Lookup     |
       |              |--------------->|
       |              |                |
       |              |  3. Instances   |
       |              |<---------------|
       |              |
       |              |  4. Route to healthy instance
       |              +----> [10.0.1.6:8080]
       |              |
       |  5. Response  |
       |<--------------|

Here, the calling service just makes a normal request to a fixed address — a load balancer or gateway — and that intermediary is the one that queries the registry and picks an instance. The calling service needs zero registry-awareness at all; it's a plain HTTP call to a stable endpoint. The trade-off is an added network hop on every call, and the load balancer itself becomes infrastructure that needs to be highly available, since every request now depends on it being up. This is what AWS ELB with Target Groups does, and it's also effectively what a Kubernetes Service provides.

A second axis: who does the registering?

Client-side versus server-side answers "who looks up the address." A separate, often-missed question is "who tells the registry an instance exists in the first place" — and that's its own pattern choice with real trade-offs.

Self-registration means each service instance registers itself with the registry on startup and sends periodic heartbeats to stay listed, deregistering on graceful shutdown. It's simple and requires no extra infrastructure — but it couples every service to the registry's client library, in every language your organization uses. A Python service and a Go service both need to correctly implement registration and heartbeating against the same registry API.

Third-party registration removes that coupling: an external component — often called a registrar, frequently implemented as a sidecar process running alongside the service — watches the deployment environment (polling it or subscribing to its events) and handles registration and deregistration on the service's behalf. The service itself never talks to the registry directly. Netflix's own Prana project is a concrete historical example of exactly this pattern: a sidecar whose entire job was registering and deregistering non-JVM services with Eureka, so those services didn't need a native Eureka client. This is also the direction most modern platforms have moved: Kubernetes itself is a third-party registration system — a pod doesn't register itself anywhere, the platform observes pod lifecycle events and updates service endpoints automatically.

DNS-based discovery: what Kubernetes actually does

Kubernetes' built-in discovery is the simplest version of server-side, third-party-registered discovery available today, and it's worth understanding what's actually happening under the "just call the service name" abstraction:

Kubernetes DNS Discovery:

  Service A code:
    response = http.get("http://user-service:8080/api/users/123")
    # That is it. No registry client. No config.

  Under the hood:
  1. CoreDNS resolves "user-service" to a stable ClusterIP
  2. kube-proxy translates that ClusterIP to a live pod IP
     via iptables, IPVS, or nftables rules on the node
  3. Pod dies? The API server notifies kube-proxy, which
     removes it from the routing rules
  4. New pod? Same notification path adds it automatically

Concretely: CoreDNS (which replaced kube-dns as Kubernetes' default DNS provider starting in version 1.13) watches the Kubernetes API for Services and automatically maintains DNS records for them. When your code resolves user-service, CoreDNS returns a stable virtual IP — the ClusterIP — that doesn't change even as the pods behind it come and go. The actual translation from that stable ClusterIP down to a specific, currently-healthy pod IP happens at the networking layer via kube-proxy, which continuously updates iptables, IPVS, or (on newer clusters) nftables rules on every node in response to pod lifecycle events from the API server. That's the real mechanism behind "it just works" — CoreDNS handles the name-to-virtual-IP mapping, kube-proxy handles the virtual-IP-to-real-pod-IP mapping, and both update automatically as pods are scheduled, rescheduled, or removed.

The one catch worth knowing: DNS resolution itself involves caching with a TTL, so a client that cached a resolved address can briefly keep routing to a now-dead target for a few seconds after a change. This mostly matters for clients doing their own DNS caching outside the Kubernetes networking layer; the ClusterIP-to-pod-IP translation via kube-proxy is what actually removes dead pods from rotation quickly, independent of DNS TTLs.

Health checks and deregistration

Health Check Flow:

  [Service Instance] --heartbeat (every ~30s)--> [Registry]
  [Service Instance] --heartbeat (every ~30s)--> [Registry]
  [Service Instance] --X (missed heartbeats)

  Registry removes instance after ~3 missed heartbeats.
  Traffic stops flowing to the dead instance shortly after.

Registration alone isn't enough — a registry also needs to know when an instance has quietly died without deregistering itself (a crash, not a graceful shutdown, never gets the chance to say "I'm leaving"). This is handled with periodic heartbeats: an instance that stops sending them for a set number of consecutive intervals gets removed automatically. In Kubernetes specifically, this responsibility is split across three distinct probe types, which are easy to conflate but answer different questions:

• Startup probe — has the application finished starting up yet? Useful for slow-starting apps so the other probes don't kill a container that just needs more time to boot.

• Liveness probe — is the app still functioning, or has it deadlocked/hung? A failing liveness probe causes Kubernetes to restart the container.

• Readiness probe — can the app handle traffic right now? A failing readiness probe doesn't restart anything — it just pulls the instance out of the load-balancing rotation until it passes again, which is exactly what you want during something like a temporary downstream dependency outage.

Getting these three confused is a common source of bad incident behavior: using a liveness probe where a readiness probe belongs, for instance, can cause Kubernetes to restart a perfectly healthy container that's just temporarily overloaded, when the correct response was simply to stop routing new traffic to it.

The CAP trade-off hiding inside every registry

This is the part most service discovery explanations skip, and it's a genuinely important design decision: a service registry is itself a distributed system, and it has to make the same CAP theorem trade-off every distributed data store makes during a network partition.

Netflix's Eureka is explicitly designed as an AP system — it favors availability over strict consistency. During a network partition, Eureka would rather serve a client a slightly stale list of instances than refuse to answer at all, on the reasoning that for service discovery specifically, a stale-but-nonempty list (where a caller might occasionally hit a dead instance and simply retry) is a far better failure mode than an empty or failed lookup that breaks every downstream call outright.

Registries built on Raft-style consensus — etcd (which also backs Kubernetes' own cluster state) and Consul — lean more toward CP: strong consistency about the current state, at the cost of potentially refusing writes (new registrations) during a partition until quorum is restored. ZooKeeper, the oldest of this group and still common in Hadoop/Kafka ecosystems, makes a similar CP-leaning choice via the ZAB protocol.

Neither choice is universally "correct" — it's the same CAP trade-off you'd reason about for any distributed data store, just applied to the specific question of "who currently exists." For pure service discovery, most engineers lean toward valuing availability (Eureka's choice): a discovery system that goes down during a network blip, at the exact moment services most need to reroute around failures, defeats its own purpose.

Comparison of discovery tools

Where this is headed: service mesh

Basic service discovery answers "what's the current address of a healthy instance." A service mesh (Istio, Linkerd, both typically built on the Envoy proxy) is best understood as service discovery plus a lot more, pushed down into a sidecar proxy that runs alongside every service instance: mutual TLS between services, automatic retries and circuit breaking, fine-grained traffic splitting for canary releases, and detailed per-request observability — all without the application code needing to know any of it's happening. If your platform is Kubernetes and your traffic patterns are simple, built-in DNS-based discovery is often genuinely sufficient. Once you need consistent retry/timeout policy, mutual TLS, and fine-grained traffic control across many services, that's usually the signal it's time to look at a service mesh rather than building those concerns into every service individually.

Common pitfalls

• Confusing liveness and readiness. Restarting a container that just needs to shed load temporarily (a readiness problem) instead of leaving it running and simply routing around it makes incidents worse, not better.

• No graceful deregistration on shutdown. A service that gets killed without deregistering first relies entirely on heartbeat timeout to get removed from rotation — which usually means a window of failed requests routed to an instance that's already gone. Handling shutdown signals to deregister proactively closes that window.

• Thundering herd on the registry itself during mass restarts. A rolling deployment or cluster-wide restart can cause a burst of near-simultaneous registrations and heartbeats that stress the registry at exactly the moment it's most needed — worth load-testing deliberately rather than discovering during an actual deploy.

• Trusting DNS caching to reflect real-time state. A client library that aggressively caches DNS resolutions outside the platform's own networking layer can keep routing to dead instances well past the point Kubernetes itself has already removed them from rotation.

• Choosing a CP registry without accounting for its availability trade-off. If new service registrations can be blocked during a partition because a CP registry can't reach quorum, that's a deliberate trade-off worth making consciously, not an accident discovered during an actual network event.

Interview framing

A strong answer sounds like: "If I'm on Kubernetes, I'd use its built-in DNS-based discovery — CoreDNS resolves a service name to a stable ClusterIP, and kube-proxy handles routing that to a currently healthy pod via iptables or IPVS rules, all updated automatically as pods come and go. That's server-side discovery with third-party registration: my services never touch a registry client directly, the platform handles it. If I weren't on Kubernetes, I'd reach for Consul, with services self-registering on startup and heartbeating to stay listed. I'd also think about the CAP trade-off in the registry itself — Eureka deliberately favors availability over consistency, which is usually the right call for discovery specifically, since a stale instance list that's still answerable beats a strongly-consistent registry that goes unavailable during a partition, right when services most need to reroute around a failure. And I'd distinguish liveness from readiness probes carefully, since restarting a healthy-but-overloaded instance is a worse outcome than just pulling it from rotation temporarily." Naming the CAP trade-off and the registration-pattern axis (not just client vs. server-side) is what typically separates a strong answer from a memorized diagram.

Key takeaway

Service discovery replaces hardcoded addresses with a live, self-updating record of where healthy instances actually are. Client-side discovery gives the caller smart routing control at the cost of a registry client in every service; server-side discovery keeps services simple at the cost of an extra hop through a load balancer. Orthogonally, registration can be self-managed (simple, but couples every service to the registry) or handled by a third party like a sidecar or the platform itself (Kubernetes' own model). Kubernetes' built-in DNS-based discovery — CoreDNS plus kube-proxy — is the simplest option and is often sufficient on its own; a service mesh is the right next step once you need consistent mTLS, retries, and traffic control across many services, not a replacement for basic discovery from day one. And underneath all of it, every registry makes its own CAP trade-off — Eureka favors availability deliberately, while etcd, Consul, and ZooKeeper lean toward consistency via Raft or ZAB — which is worth choosing on purpose rather than discovering during an outage.

FAQ

Is Kubernetes' built-in discovery enough, or do I need a service mesh? For most applications, Kubernetes' DNS-based discovery is sufficient on its own — it's automatic, free, and handles the core problem of routing to healthy instances. A service mesh becomes worth the added operational complexity once you need mutual TLS between services, fine-grained traffic splitting for canaries, or consistent retry/circuit-breaking policy enforced outside application code.

Why would a registry ever choose consistency over availability? If stale discovery data could cause a genuinely dangerous outcome — routing to an instance that's supposed to be strictly exclusive, for example — a CP registry's willingness to refuse rather than serve stale data can be the safer choice. For general-purpose service discovery, most systems lean the other way, since an unavailable registry breaks every downstream call at once.

What happens if a service crashes without deregistering? The registry relies on missed heartbeats to detect it — after a set number of consecutive missed heartbeats (commonly around three), the instance is removed automatically. This means there's a window, typically tens of seconds, where traffic can still be routed to the dead instance before it's cleared.

Do I need both client-side and server-side discovery? No, they're alternative approaches to the same lookup problem, not complementary layers — though a real architecture often mixes them across different parts of the system (server-side at the external edge via an API gateway, for instance, alongside client-side or mesh-based discovery for internal service-to-service calls).

What's the difference between a service registry and a service mesh's control plane? A service registry answers one question: what are the current healthy addresses for this service name. A service mesh's control plane typically includes a registry as one component, but adds policy distribution (retry rules, mTLS certificates, traffic-split rules) that gets pushed out to sidecar proxies — it's a superset of what a basic registry does, not a different mechanism entirely.