API Gateway Patterns: The Front Door to Your Microservices

The chatty client problem

A mobile app rendering one product page needs data from five different places: the product catalog, reviews, inventory, recommendations, and pricing. Without a gateway, that's five separate HTTP round trips from the client — five TLS handshakes, five chances for one to time out on a bad cellular connection, and a mobile app that now has to know the internal shape of your microservice fleet just to render a single screen.

Without API Gateway (the chatty client problem):

  [Mobile App]
   |  |  |  |  |
   |  |  |  |  +---> GET /api/pricing/product/123
   |  |  |  +------> GET /api/recommendations/product/123
   |  |  +---------> GET /api/inventory/product/123
   |  +------------> GET /api/reviews/product/123
   +---------------> GET /api/products/123

   5 separate HTTP calls over cellular = SLOW
   Client knows internal service boundaries = COUPLING

With API Gateway:

  [Mobile App]
       |
   GET /api/product-page/123    <-- ONE call
       |
  [API Gateway]  <-- fans out internally
   |   |   |   |   |
  [Products] [Reviews] [Inventory] [Recs] [Pricing]
       |
  { unified response }  <-- ONE response back

An API gateway collapses those five calls into one. The client asks for /api/product-page/123; the gateway fans that request out to five internal services over your fast, low-latency internal network, waits for (or times out) each one, stitches the results into a single response shape, and sends it back. This is called API composition, and it's the core value proposition: the client gets simplicity, and your internal service topology stays private and free to change without breaking every app in the field.

What a gateway actually does

The list below looks simple in a diagram, but each line represents a real architectural decision, not a checkbox:

What an API Gateway Does:

  Client Request
       |
       v
  +--------------------------------------------+
  |              API GATEWAY                    |
  |                                            |
  |  1. SSL/TLS Termination                    |
  |  2. Authentication (verify JWT/API key)    |
  |  3. Rate Limiting (429 if exceeded)        |
  |  4. Request Validation                     |
  |  5. Routing (/users -> user-svc)           |
  |  6. Protocol Translation (REST -> gRPC)    |
  |  7. Response Aggregation                   |
  |  8. Caching (frequent responses)           |
  |  9. Logging & Metrics                      |
  +--------------------------------------------+
       |
  [Service A]  [Service B]  [Service C]

• TLS termination — the gateway holds the public certificate and decrypts HTTPS once, so internal services can talk over plain HTTP (or mTLS if you're stricter) without every one of them managing certs.

• Authentication — the gateway verifies a JWT or API key once, centrally, instead of every downstream service re-implementing token validation. It typically attaches the verified identity as a header (X-User-Id) for services to trust.

• Rate limiting — protects downstream services from being overwhelmed by a single noisy client, and enforces API plans/tiers (free vs. paid) at the edge before traffic even reaches your services.

• Routing — path-, header-, or version-based rules decide which service handles a request. This is also where canary releases and blue-green routing typically live.

• Protocol translation — clients speak REST or GraphQL; internal services might speak gRPC. The gateway is a natural place to translate between them so internal services can choose the most efficient protocol without exposing it externally.

• Response aggregation — the composition behavior described above: fan out to N services, merge N responses into one.

• Caching — frequently requested, slow-changing responses (a product catalog page, for instance) can be cached at the edge, sparing backend services from redundant load.

Rate limiting, specifically: two algorithms worth knowing

Rate limiting is usually the first thing that breaks under real traffic, so it's worth being specific about how it's implemented rather than treating it as a single feature.

Token bucket is the more common default because real client traffic is bursty by nature (a user opening an app fires several requests at once), and a hard per-second cap would reject legitimate usage patterns that sliding window or fixed window would also reject unnecessarily.

Backend-for-Frontend (BFF): one gateway doesn't fit every client

A single gateway serving every client with an identical response shape works fine until your clients stop looking alike. A smart TV wants a handful of large images and almost no metadata. A phone on a spotty connection wants compressed payloads and the bare minimum fields. A web browser wants rich, interactive data and SEO-friendly metadata. Forcing all three through one generic gateway means either over-fetching for the constrained clients or under-fetching for the rich ones — nobody gets served well.

The Backend-for-Frontend (BFF) pattern fixes this by giving each client type its own thin gateway layer, owned by the team that owns that client, sitting in front of the same shared downstream services.

BFF Pattern:

  [iOS App]    [Android App]    [Smart TV]    [Web Browser]
      |              |              |              |
  [Mobile BFF]  [Mobile BFF]   [TV BFF]      [Web BFF]
      |              |              |              |
      +-------+------+-------+-----+------+-------+
              |              |             |
         [User Svc]    [Content Svc]  [Rec Svc]

  Mobile BFF:  Small payloads, compressed images, offline hints
  TV BFF:      Large thumbnails, simplified navigation data
  Web BFF:     Rich metadata, interactive features, SEO data

  Each BFF speaks the language of its client.
  Owned by the client team, not the backend team.

Worth getting right for the history here: the term was coined by Sam Newman around 2015, based on work he did with teams at SoundCloud, and it's documented in his book Building Microservices. SoundCloud used per-client BFFs specifically so frontend teams could ship independently of the shared platform team's release cycle — that organizational benefit is arguably as important as the technical one. Netflix later became the pattern's best-known large-scale adopter, running dedicated BFFs per device family across thousands of distinct device types, each translating the same underlying catalog and playback services into a shape suited to that device's constraints.

The key ownership rule that makes BFF work: a BFF is a translation and aggregation layer, not a place for business logic, and it's typically owned by the frontend team consuming it, not the backend team. That's what keeps three BFFs from silently drifting into three different, conflicting sources of truth for the same business rules.

Gateway vs. service mesh: don't confuse the two

This distinction trips up a lot of system design answers. An API gateway and a service mesh solve adjacent but different problems, and modern architectures typically use both, not one instead of the other.

Envoy is a useful example of how blurry this line has gotten: it's the industry-standard L7 proxy, but you rarely deploy it standalone — it runs as the data plane inside Istio-based service meshes, and it also powers dedicated gateway products (Envoy Gateway, AWS App Mesh). So "Envoy" isn't itself a gateway-or-mesh answer; it's infrastructure that gets used for both roles depending on how it's deployed.

Gateway product comparison

Roughly: choose a managed option (AWS API Gateway) when you want the least operational overhead and you're already committed to that cloud, accepting some lock-in in exchange. Choose a self-hosted, plugin-rich option (Kong, APISIX) when you need portability across clouds or on-prem, and are willing to operate it yourself. Choose a mesh-integrated option (Envoy-based) when your internal traffic patterns are complex enough that you're running a service mesh anyway, and want your edge gateway built on the same proxy technology.

When the gateway itself goes down: the Canva case study

The single-point-of-failure risk isn't theoretical. Canva's API Gateway suffered a real outage where canva.com was fully unavailable for close to an hour. The root cause was a performance regression introduced in a telemetry library change: a metric was being re-registered on every recorded value instead of once, which caused off-heap memory to grow steadily inside the gateway's containers. That memory growth eventually triggered Linux's Out-Of-Memory killer, which terminated gateway containers en masse within about two minutes — faster than autoscaling could react — and the resulting capacity loss cascaded into every request to canva.com failing at once.

The lesson generalizes beyond this one incident: because every request flows through the gateway, a subtle bug that would be a contained, service-level problem anywhere else becomes a full-outage problem at the gateway layer. That asymmetry is exactly why gateway deployments get held to a higher operational bar than a typical backend service — more aggressive resource limits and alerting, more conservative rollout practices, and capacity headroom sized for a fast failure, not just average load.

Dangers to design around

• Single point of failure. If the gateway is down, the whole system is unreachable from the outside, even if every backend service is healthy — as Canva's incident shows. Run multiple gateway instances across availability zones behind a load balancer, with no shared failure domain between them.

• Bottleneck risk. Every request, from every client, crosses this one layer. It has to be provisioned for peak load, not average load — Netflix's Zuul, for comparison, is built to handle millions of requests per second precisely because it sits in that exact chokepoint.

• Logic creep. It's tempting to add a small if statement for a business rule directly in the gateway because it's convenient and touches every request. Resist this. The moment the gateway starts making business decisions instead of routing and transforming, you've created a second, hidden implementation of logic that's supposed to live in your services — and now two places can disagree about what the rule actually is.

• Composition failure handling. When a gateway aggregates five downstream calls into one response, it needs an explicit policy for what happens when one of the five is slow or errors: fail the whole request, or return a partial response with the failed section omitted? Silently blocking on the slowest of five calls turns one flaky service into a latency problem for every client.

Interview framing

A strong system design answer sounds like: "I'll place an API Gateway as the single entry point for external traffic. It handles authentication, rate limiting, and routing, and composes responses from multiple backend services so clients don't need to know my internal service boundaries. Since different clients — mobile, web, third-party — need different response shapes, I'd use the Backend-for-Frontend pattern: a thin, client-specific gateway layer owned by each client team, sitting in front of the same shared services. The gateway itself is stateless, so it scales horizontally behind a load balancer, and I'd run multiple instances across availability zones since it's a single point of failure by design — if it goes down, external access to the whole system goes down with it." That covers composition, BFF, statelessness, and the failure-mode awareness interviewers are specifically listening for.

Key takeaway

An API gateway is a single, unified entry point that centralizes cross-cutting concerns — auth, rate limiting, routing, composition — so clients stay simple and your internal service topology stays free to evolve. Different client types often need different response shapes, which is what the Backend-for-Frontend pattern (from Sam Newman's work at SoundCloud, later scaled by Netflix) solves. It's distinct from a service mesh, which handles internal service-to-service traffic rather than the external edge. And because every request passes through it, the gateway has to be treated as critical infrastructure — highly available, provisioned for peak load, and kept free of business logic — not just another service in the fleet.

FAQ

Is an API gateway the same as a load balancer? No. A load balancer distributes traffic across instances of a single service based on network-level rules. An API gateway operates at the application layer and does far more: authentication, rate limiting, routing to different services by path, and response composition. In practice, a gateway typically sits behind a load balancer that distributes traffic across multiple gateway instances.

Does every microservices architecture need an API gateway? Not necessarily at small scale — a handful of services with one client type can sometimes call each other directly. It becomes valuable once you have multiple client types, need centralized auth/rate-limiting, or want to hide internal service boundaries from external consumers.

What's the difference between an API gateway and GraphQL? They can overlap but aren't the same thing. A GraphQL layer lets clients request exactly the fields they need in one query, which solves over-fetching, but it doesn't inherently handle auth, rate limiting, or protocol translation the way a gateway does. Some architectures run GraphQL as a layer behind or alongside an API gateway rather than as a full replacement for one.

Where should business logic live if not in the gateway? In the services themselves. The gateway's job is routing, authentication, and transformation — not decisions about what a valid order looks like or how pricing is calculated. Keeping that boundary clean is what prevents the gateway from becoming an undocumented second copy of your business rules.

How is a BFF different from a regular API gateway? A standard gateway is usually one shared layer serving all clients uniformly. A BFF is a dedicated gateway per client type (mobile, web, TV), each shaped around that client's specific data and performance needs, and typically owned by the frontend team rather than a central platform team.