HLD Learn/APIs, Done Right

Browsing as a guest. Sign in to save your progress and earn XP as you complete chapters.

Pagination, Versioning & Error Handling

6 min read

You'll learn to

-Explain why lists must be paginated, and compare offset and cursor pagination with a worked example
-Explain how to version an API, what counts as a breaking change, and how to retire a version safely
-Design one consistent error shape and choose the right status code for each kind of failure
-Say why an error should be useful to the client without leaking internals

An API that works in a demo can still fail in the real world in three predictable ways. A request for a list returns a million rows. A change that seemed harmless breaks an app that someone built last year. And when something goes wrong, the client receives an answer it cannot make sense of. This chapter covers the habits that prevent each: pagination, versioning and error handling.

Pagination: Never Return an Unbounded List

A list endpoint with no limit is a time bomb: it works with 50 rows and takes the server down at 5 million. So every list is returned in pages. There are two ways to say where the next page starts, and the difference is easiest to see with a bookmark.

An Analogy: Page Numbers or a Bookmark

Imagine reading a long list by page number. You read page 1, and meanwhile someone adds a new entry at the top. When you ask for page 2, everything has shifted down by one, and the last item you read on page 1 is now the first one on page 2. You see it twice. A bookmark does not have this problem: it marks the last item you read, and you ask for whatever comes after it, wherever it now is.

-Page numbers are offset pagination: ?page=2&size=20, or equivalently skip the first 20.
-The bookmark is cursor pagination: ?after=<the last item you saw>.

Here is a worked case. The list is newest first: E, D, C, B, A, with a page size of 2. Page 1 is E and D. Then a new item F is added at the top, so the list is F, E, D, C, B, A. Offset pagination now fetches page 2 by skipping two items and returns D and C, so D appears twice. Cursor pagination asks for the items after D and returns C and B, with nothing repeated or skipped. Deleting an item while someone pages through causes the opposite problem for offsets: an item is skipped.

There is a performance difference too. Offset 1,000,000 makes the database find and throw away a million rows before it returns twenty. A cursor lets it jump straight to the right place using an index:

Cursor pagination: continue after the last id seen. The database seeks straight to it with the index, however deep the page

SELECT * FROM posts
WHERE id < 8421
ORDER BY id DESC
LIMIT 20;

The response hands back an opaque cursor. Clients pass it back unchanged and never need to know what is inside

{
  "items": [ ... 20 posts ... ],
  "next_cursor": "eyJpZCI6ODQwMX0"
}

Offset pagination is still fine for small, stable lists and when users need to jump to page 7. For feeds, logs and anything large or fast-changing, use cursors.

Versioning

An API you can never change eventually cannot evolve, and an API you change carelessly breaks the people who depend on it. The skill is knowing which changes are safe. Adding a new optional field or a new endpoint is non-breaking: old clients ignore what they do not know. Removing or renaming a field, changing a field's type, or making a new field required is breaking, because an app that worked yesterday stops working.

-Version in the URL path (/api/v2/users): easy to see and to route, the most common choice.
-Version in a header or media type: keeps URLs clean, but is easy to forget and harder to test in a browser.
-Version in a query parameter: simple, but it tends to be treated as optional.

Whichever you pick, the discipline that matters is the deprecation window. Announce the retirement of a version well ahead (six to twelve months is common), mark responses from it with a Sunset header giving the date, measure how many clients still call it, and switch it off only when that number is small or zero. Never break every existing client overnight.

Error Handling

Use HTTP status codes as they were meant to be used, and return errors in one consistent shape across every endpoint, so client code can handle failure generically instead of special-casing each one.

A stable machine-readable code, a human message, the field at fault, and a request id the user can quote to support

{
  "error": {
    "code": "insufficient_stock",
    "message": "Only 2 of item 7 are in stock.",
    "details": [{ "field": "quantity", "issue": "must be 2 or less" }],
    "request_id": "req_8f2a"
  }
}

-400 Bad Request or 422: the request was malformed or failed validation. The client should fix it before retrying.
-401 Unauthorized: not authenticated. 403 Forbidden: authenticated but not allowed. Do not mix them up.
-404 Not Found: no such resource. 409 Conflict: the request clashes with the current state, such as a duplicate.
-429 Too Many Requests: slow down, ideally with a Retry-After header.
-500 Internal Server Error and 503 Service Unavailable: the fault is on the server's side, and the client may retry later.

The 4xx and 5xx split tells a client who is at fault, and therefore whether retrying can help. Make the message useful, but never leak internals: a stack trace or a database error in a response gives attackers a map of your system. Log the detail on the server and give the client only a request id to quote.

In one line: paginate every list (use cursors for anything large or changing), never break existing clients without a deprecation window, and return errors in one consistent shape with the right status code and no internals.

Interview Signal is part of Pro

See a real weak answer next to a real strong one for this exact topic.

Quiz is part of Pro

Test what you just read with a short quiz, and bank the XP.

REST & Resource Modeling GraphQL & gRPC: When to Reach Beyond REST

Discussion0

Join the Discussion

No comments yet. Be the first to start the thread!