API Gateways in a Microservices World

The Protecting Your System module introduced the API gateway as a good idea even for a single service. With microservices it goes from a good idea to close to mandatory. Dozens of independently deployed services cannot each re-implement authentication, rate limiting and TLS termination without a great deal of duplicated effort and inconsistency, and clients cannot be expected to know the address of every service. This chapter shows what the gateway does for the outside world, and the pattern that tailors it to different clients.

A hotel has dozens of departments: restaurant, spa, taxis, housekeeping, tours. Guests do not learn who runs what. They talk to the concierge, who knows which department to ask, checks that the guest is entitled to the service, and can arrange several things in a single conversation: a taxi, a dinner table and a late checkout. A business traveler and a family on holiday both use the concierge, but the help they want is different.

• -The concierge is the API gateway: one front door that knows where everything lives.
• -The departments are the backend services.
• -Checking that a guest is entitled to a service is authentication and authorization.
• -Arranging several things in a single conversation is aggregation.
• -A separate desk for business travelers and for families is a Backend for Frontend (BFF).

• -Routing: requests for /users go to the Users service and /orders to the Orders service. Clients know one address, and you are free to move or split services behind it.
• -Shared policies: authentication, rate limiting, TLS termination and logging done once, in one place, for every service.
• -Aggregation: gather the data of several services into one response, so the client makes one call instead of several.
• -Translation: speak REST to the outside world while services speak gRPC or events internally.

Suppose a mobile home screen needs the user's profile, their three latest orders and some recommendations, each from a different service. Without a gateway the phone makes three calls over a mobile network. If each round trip takes about 100 ms, that is around 300 ms of waiting on the slowest link in the whole system. With a gateway that aggregates, the phone makes one call of about 100 ms, and the gateway makes the three calls inside the data center, in parallel, at around 10 ms each. The total is about 110 ms. The slow network is crossed once, and the fast one is used for the rest.

Some architectures go one step further and run several gateways, each tailored to one kind of client. A mobile BFF aggregates and trims responses for a bandwidth-limited app, returning small images and only the fields the screen shows. A separate web BFF is shaped around what the browser needs. The same backend services sit underneath, but each front door is designed for its consumer, and is owned by the team that builds that client, so the mobile team can change its BFF without waiting for anyone.

It helps to separate two kinds of traffic. Traffic coming in from outside the system, from clients, is called north-south, and the gateway handles it. Traffic between services inside is east-west, and that is what a service mesh handles. They are complementary: a gateway is the front door, and a mesh is the internal telephone network.

• -A bottleneck and a single point of failure: every request passes through it, so run several instances and watch its capacity.
• -A growing monolith: it is tempting to add business logic to the gateway. Keep it to policies and aggregation, not business rules.
• -A shared dependency: a faulty release of the gateway affects every client, so deploy it carefully.
• -Too many BFFs: each is something to build and keep in step. Create one only when a client has genuinely different needs.