API Gateways & Reverse Proxies

By now a request has several things to get through before it reaches your code: it must be checked for identity, counted against a rate limit, perhaps decrypted, and sent to the right server. If every service did all of that itself, the same logic would be copied into each one and would drift out of step. The usual answer is to put one component at the front that handles the shared concerns. That component is a proxy, and this chapter explains the two kinds you will hear about, and the specialized one called an API gateway.

A company has a mailroom for outgoing letters and a front desk for visitors. Outgoing letters leave under the company's address, so the people who receive them do not know which employee wrote them. Visitors arriving at the building do not know which floor or desk anyone sits at; the front desk checks who they are, sends them to the right department, and can pick up a message if the person is out.

• -The mailroom is a forward proxy: it sits in front of the clients and speaks for them, hiding who they are from the outside world.
• -The front desk is a reverse proxy: it sits in front of the servers and speaks for them, hiding how they are arranged from the people calling.
• -The departments behind the front desk are your backend servers.
• -The security badge check at the front desk is authentication, and a per-visitor limit on how often someone may come in is rate limiting.

These two terms get mixed up constantly, and the direction matters. A forward proxy sits in front of clients: it hides the client from the server (a corporate network routing all employee traffic through one exit point, or a VPN). A reverse proxy sits in front of servers; it hides the servers from the client, and every request you've made in this course to a "web server" has almost certainly gone through one first.

In practice you will meet reverse proxies far more often when designing systems, and they give you several things for free:

• -TLS termination: the proxy handles the encryption work, so the servers behind it speak plain traffic on a private network.
• -Load balancing: it spreads requests over the servers behind it. A load balancer is really a reverse proxy that specializes in this one job.
• -Hiding the layout: clients see one address, and you can change what is behind it freely.
• -Caching and compression: it can store common responses and shrink them before sending.
• -Protection: it can absorb slow or malformed requests before they reach the application.

Nginx, HAProxy and Envoy are common examples. The card below describes the component as it appears in the lab:

An API gateway is a reverse proxy that understands your API. It is the single front door for every client, and it centralizes the concerns you do not want duplicated across every backend service: authentication, rate limiting, TLS termination, and routing each request to whichever service owns that resource. Kong, Apigee and the gateways built into the big cloud providers are well-known examples.

Follow one request, GET /orders/17, through it:

• -The gateway checks the token on the request and rejects it with a 401 if it is missing or invalid.
• -It checks the caller's rate limit and rejects the request with a 429 if the limit is used up.
• -It looks at the path: anything under /orders goes to the Orders service, anything under /users goes to the Users service.
• -It forwards the request, adding a trusted header with the caller's id, so the service does not have to re-check the token.
• -It logs the request and the response time, giving you one place to see all traffic.

The diagram shows the shape: one gateway, many services behind it, each free to focus on its own logic.

The card below describes the gateway component as it appears in the lab:

• -A single point of failure: if the one gateway fails, nothing gets through, so run several behind a load balancer.
• -An extra hop: every request pays a little added latency for passing through it.
• -A shared dependency: every team's traffic flows through it, so a bad change to it can break every service.
• -A magnet for business logic: it is tempting to put just one more rule in the gateway. Keep it to policies that apply to every request (identity, limits, routing), and keep rules about the business, such as how an order's discount is computed, inside the services.