The API Gateway Pattern
As microservices became the default architecture, every service had its own API. Clients needed to call dozens of services directly, handle service discovery, implement per-service authentication, and aggregate responses manually. The API gateway pattern solved this by placing a single entry point in front of all services: route requests to the right service, aggregate responses when needed, enforce authentication, handle rate limiting, and transform protocols.
API gateways became standard infrastructure. AWS API Gateway, Kong, Nginx Plus, Apigee, and Traefik all occupy this space. The gateway handles cross-cutting concerns so individual services do not have to. A service behind a gateway can expose its full internal API. The gateway makes the external-facing contract.
Backend-for-Frontend: Client-Specific APIs
Sam Newman introduced the Backend-for-Frontend (BFF) pattern in 2015. The problem it solves: a web frontend and a mobile app have different data needs. The web app can afford larger payloads and multiple requests. The mobile app needs minimal data on a single round trip. A single general API serves neither well.
BFF creates a separate API server for each client type: one BFF for web, one for mobile, one for third-party integrations. Each BFF is owned by the team that owns the frontend. It can aggregate downstream services, reshape data, and optimize for its specific client. The BFF is not a monolith - it is a thin orchestration layer that owns the client-specific API contract.
tRPC: Eliminating the Client-Server Gap
tRPC (2021, Alex Johansson) took a different approach to the API contract problem. Rather than generating clients from OpenAPI specs or .proto files, tRPC shares TypeScript types directly between server and client. You define a router on the server with typed procedures. The client imports that type. TypeScript enforces that every client call matches what the server expects.
tRPC means zero code generation, zero manual type synchronization, and immediate TypeScript errors when server and client drift out of sync. The limitation is the ecosystem: it only works when both client and server are TypeScript. For JavaScript/TypeScript monorepos - particularly Next.js applications - it eliminates an entire class of API integration bugs.
OpenAI and the AI Reshaping of API Design
OpenAI's API introduced patterns that are becoming standard in the AI era. Streaming responses as Server-Sent Events so the user sees output as it generates rather than waiting for completion. Function calling (tool use) as a first-class API primitive where the model can invoke defined functions with structured arguments. Token-based pricing that makes usage economics explicit in the API contract.
The OpenAI API design influenced every AI provider that followed. The chat completions format - messages array, role/content structure, model parameter - became an informal standard. Anthropic, Google Gemini, and Mistral all built broadly compatible APIs. This is API design convergence in real time: the market picked a pattern and everyone standardized on it.
The Multi-Protocol Reality
Modern systems do not use one protocol. An architecture might have REST for external APIs (wide compatibility), gRPC for internal service communication (performance), GraphQL for the frontend data layer (flexibility), Kafka for event streaming (async processing), and WebSocket for real-time features. The API gateway manages the external surface. The BFF shapes it per client. The protocols are chosen per problem, not per ideology.
API Gateway Patterns: What the Gateway Should and Should Not Do
API gateways handle cross-cutting concerns correctly: authentication (verify tokens before requests reach services), rate limiting (enforce quotas at the edge), TLS termination (handle certificates in one place), request routing (dispatch to correct service based on path/host), and observability (log, trace, and monitor all traffic). They handle service-specific concerns poorly: business logic (do not put if-then logic in your gateway configuration), data transformation (transforms belong in BFF layers), and service orchestration (gateways should route, not coordinate multi-step workflows).
Gateways that grow into orchestrators become the ESB antipattern in disguise: a central, hard-to-maintain component that every service depends on. The tell-tale sign is a gateway configuration that has service-specific routing rules, per-endpoint authentication exceptions, or data transformation logic for specific services. Each of these should live in the service or its BFF, not in the gateway. The gateway should be a generic routing and security layer that knows nothing about your business domain.
BFF Evolution: From Pattern to Platform
The BFF pattern evolved from 'a separate server per client type' to 'a platform capability that client teams own.' Modern BFF implementations use GraphQL federation (each client team owns a subgraph that shapes their specific data needs), or tRPC routers (TypeScript monorepos where the BFF is a typed adapter layer), or gateway aggregation rules (declarative BFF logic in gateway config). The common thread: each client team controls how they see the data, without requiring changes to downstream services.
Netflix, Spotify, and SoundCloud all developed BFF patterns independently before Sam Newman named it. Their shared motivation was identical: mobile apps needed different data shapes than web apps, and neither the generic microservices API nor a fat client that assembled data in the browser was acceptable. The BFF solved the aggregation and shaping problem by placing the responsibility with the team that understood the client's needs best.
- API gateway rate limits should operate at multiple levels: per IP, per API key, per user, and per endpoint. A burst limit (10 requests per second) plus a sustained limit (1000 requests per hour) prevents both spikes and gradual overconsumption.
- Zero-downtime API deployments using gateways: route 1% of traffic to the new version, monitor error rates, increase gradually. The gateway makes canary deployments declarative.
- tRPC's type inference means you never write type definitions for API calls. The types are inferred from the router definition. This is not magic - it is TypeScript's type inference working on the router's function signatures.
- OpenAI-style streaming responses (Server-Sent Events with data: {json} chunks) are becoming the standard for AI APIs. The pattern works with any API where results are computed incrementally.
- Multi-protocol architectures need protocol translation documented in your API governance: which protocols are approved for external APIs, which for internal, and which for event-driven flows.
The AI API Pattern: Streaming, Tools, and Structured Outputs
OpenAI's API design choices are becoming API design patterns for the industry. Streaming responses via Server-Sent Events (the client receives data: {partial response} chunks as the model generates output) is now expected for any AI endpoint. The alternative - waiting for the full response - is unusable when responses take 5-30 seconds. SSE streaming is simple enough to implement in any web framework in 10 lines of code and eliminates the need for WebSockets for unidirectional streaming from server to client.
Tool calling (function calling) is the pattern where the model can invoke application-defined functions with structured JSON arguments. Your API defines available tools as JSON Schema specifications. The model decides when to call them, generates the arguments, and your application executes the function and returns the result. This turns an AI model from a text generator into an agent that can interact with your systems. OpenAI, Anthropic Claude, and Google Gemini all implement this pattern with similar (but not identical) API shapes. If you are building AI-powered features, you will implement tool calling - learn the pattern now.
API Gateway Patterns: What the Gateway Should and Should Not Do
API gateways handle cross-cutting concerns correctly: authentication (verify tokens before requests reach services), rate limiting (enforce quotas at the edge), TLS termination (handle certificates in one place), request routing (dispatch to correct service based on path/host), and observability (log, trace, and monitor all traffic). They handle service-specific concerns poorly: business logic (do not put if-then logic in your gateway configuration), data transformation (transforms belong in BFF layers), and service orchestration (gateways should route, not coordinate multi-step workflows).
Gateways that grow into orchestrators become the ESB antipattern in disguise: a central, hard-to-maintain component that every service depends on. The tell-tale sign is a gateway configuration that has service-specific routing rules, per-endpoint authentication exceptions, or data transformation logic for specific services. Each of these should live in the service or its BFF, not in the gateway. The gateway should be a generic routing and security layer that knows nothing about your business domain.
BFF Evolution: From Pattern to Platform
The BFF pattern evolved from 'a separate server per client type' to 'a platform capability that client teams own.' Modern BFF implementations use GraphQL federation (each client team owns a subgraph that shapes their specific data needs), or tRPC routers (TypeScript monorepos where the BFF is a typed adapter layer), or gateway aggregation rules (declarative BFF logic in gateway config). The common thread: each client team controls how they see the data, without requiring changes to downstream services.
Netflix, Spotify, and SoundCloud all developed BFF patterns independently before Sam Newman named it. Their shared motivation was identical: mobile apps needed different data shapes than web apps, and neither the generic microservices API nor a fat client that assembled data in the browser was acceptable. The BFF solved the aggregation and shaping problem by placing the responsibility with the team that understood the client's needs best.
- API gateway rate limits should operate at multiple levels: per IP, per API key, per user, and per endpoint. A burst limit (10 requests per second) plus a sustained limit (1000 requests per hour) prevents both spikes and gradual overconsumption.
- Zero-downtime API deployments using gateways: route 1% of traffic to the new version, monitor error rates, increase gradually. The gateway makes canary deployments declarative.
- tRPC's type inference means you never write type definitions for API calls. The types are inferred from the router definition. This is not magic - it is TypeScript's type inference working on the router's function signatures.
- OpenAI-style streaming responses (Server-Sent Events with data: {json} chunks) are becoming the standard for AI APIs. The pattern works with any API where results are computed incrementally.
- Multi-protocol architectures need protocol translation documented in your API governance: which protocols are approved for external APIs, which for internal, and which for event-driven flows.
The AI API Pattern: Streaming, Tools, and Structured Outputs
OpenAI's API design choices are becoming API design patterns for the industry. Streaming responses via Server-Sent Events (the client receives data: {partial response} chunks as the model generates output) is now expected for any AI endpoint. The alternative - waiting for the full response - is unusable when responses take 5-30 seconds. SSE streaming is simple enough to implement in any web framework in 10 lines of code and eliminates the need for WebSockets for unidirectional streaming from server to client.
Tool calling (function calling) is the pattern where the model can invoke application-defined functions with structured JSON arguments. Your API defines available tools as JSON Schema specifications. The model decides when to call them, generates the arguments, and your application executes the function and returns the result. This turns an AI model from a text generator into an agent that can interact with your systems. OpenAI, Anthropic Claude, and Google Gemini all implement this pattern with similar (but not identical) API shapes. If you are building AI-powered features, you will implement tool calling - learn the pattern now.
API Gateway Patterns: What the Gateway Should and Should Not Do
API gateways handle cross-cutting concerns correctly: authentication (verify tokens before requests reach services), rate limiting (enforce quotas at the edge), TLS termination (handle certificates in one place), request routing (dispatch to correct service based on path/host), and observability (log, trace, and monitor all traffic). They handle service-specific concerns poorly: business logic (do not put if-then logic in your gateway configuration), data transformation (transforms belong in BFF layers), and service orchestration (gateways should route, not coordinate multi-step workflows).
Gateways that grow into orchestrators become the ESB antipattern in disguise: a central, hard-to-maintain component that every service depends on. The tell-tale sign is a gateway configuration that has service-specific routing rules, per-endpoint authentication exceptions, or data transformation logic for specific services. Each of these should live in the service or its BFF, not in the gateway. The gateway should be a generic routing and security layer that knows nothing about your business domain.
BFF Evolution: From Pattern to Platform
The BFF pattern evolved from 'a separate server per client type' to 'a platform capability that client teams own.' Modern BFF implementations use GraphQL federation (each client team owns a subgraph that shapes their specific data needs), or tRPC routers (TypeScript monorepos where the BFF is a typed adapter layer), or gateway aggregation rules (declarative BFF logic in gateway config). The common thread: each client team controls how they see the data, without requiring changes to downstream services.
Netflix, Spotify, and SoundCloud all developed BFF patterns independently before Sam Newman named it. Their shared motivation was identical: mobile apps needed different data shapes than web apps, and neither the generic microservices API nor a fat client that assembled data in the browser was acceptable. The BFF solved the aggregation and shaping problem by placing the responsibility with the team that understood the client's needs best.
- API gateway rate limits should operate at multiple levels: per IP, per API key, per user, and per endpoint. A burst limit (10 requests per second) plus a sustained limit (1000 requests per hour) prevents both spikes and gradual overconsumption.
- Zero-downtime API deployments using gateways: route 1% of traffic to the new version, monitor error rates, increase gradually. The gateway makes canary deployments declarative.
- tRPC's type inference means you never write type definitions for API calls. The types are inferred from the router definition. This is not magic - it is TypeScript's type inference working on the router's function signatures.
- OpenAI-style streaming responses (Server-Sent Events with data: {json} chunks) are becoming the standard for AI APIs. The pattern works with any API where results are computed incrementally.
- Multi-protocol architectures need protocol translation documented in your API governance: which protocols are approved for external APIs, which for internal, and which for event-driven flows.
The AI API Pattern: Streaming, Tools, and Structured Outputs
OpenAI's API design choices are becoming API design patterns for the industry. Streaming responses via Server-Sent Events (the client receives data: {partial response} chunks as the model generates output) is now expected for any AI endpoint. The alternative - waiting for the full response - is unusable when responses take 5-30 seconds. SSE streaming is simple enough to implement in any web framework in 10 lines of code and eliminates the need for WebSockets for unidirectional streaming from server to client.
Tool calling (function calling) is the pattern where the model can invoke application-defined functions with structured JSON arguments. Your API defines available tools as JSON Schema specifications. The model decides when to call them, generates the arguments, and your application executes the function and returns the result. This turns an AI model from a text generator into an agent that can interact with your systems. OpenAI, Anthropic Claude, and Google Gemini all implement this pattern with similar (but not identical) API shapes. If you are building AI-powered features, you will implement tool calling - learn the pattern now.