Skip to content
API Design Learn/Production REST II: Correctness & Scale
Browsing as a guest. Sign in to save your progress and earn XP as you complete chapters.

File Upload APIs

6 min read

You'll learn to

  • -Design a presigned-URL upload flow so large files bypass the API server entirely
  • -Know when multipart or resumable upload is needed instead of a single presigned PUT
  • -Finish the handshake: confirm the upload happened before trusting it
  • -Scope a presigned URL narrowly, and validate what the client claims

Routing a large file upload through the same API server that handles ordinary JSON requests works for small files. For large ones it wastes server resources, holding open connections and buffering bytes it doesn't actually need to process. The presigned-URL pattern lets the client upload directly to storage, with the API server only ever handling the metadata.

An Analogy: The Parcel Locker

Parcel lockers do not make the courier hand each parcel to the shop assistant, who then walks it to a shelf. The shop gives you a one-time code for one specific locker that works for the next 15 minutes, and you put the parcel in yourself. The shop only has to issue the code and later check the locker has something in it.

  • -The one-time code is the presigned URL.
  • -Its validity for a single locker and 15 minutes is the scope and expiry in the signature.
  • -The locker bank is object storage, which receives the bytes directly.
  • -The shop issuing the code and checking afterwards is your API: metadata in, bytes never.

The Presigned URL Flow

Step 1: ask the API for permission to upload, not the upload itself
POST /uploads
{"filename": "report.pdf", "content_type": "application/pdf"}

HTTP/1.1 201 Created
{
  "upload_url": "https://storage.example.com/bucket/abc123?signature=...",
  "upload_id": "upl_501",
  "expires_at": "2026-08-07T10:15:00Z"
}
Step 2: the client uploads directly to storage - the API server is never in this path
PUT https://storage.example.com/bucket/abc123?signature=...
Content-Type: application/pdf

[binary file data]

HTTP/1.1 200 OK

The upload_url is a time-limited, cryptographically signed URL granting temporary write access to one specific storage location - the signature encodes exactly what's permitted (this bucket path, this content type, until this expiry) without requiring the client to have any broader storage credentials. The API server's only job is issuing that signed URL and recording the upload's metadata; the actual bytes never pass through it.

The saving is easy to put numbers on. Suppose users upload 1,000 files an hour at 50 MB each. If they go through your API servers, that is 1,000 x 50 MB = 50 GB an hour passing through machines that should be answering small JSON requests, and every slow upload holds a connection and some memory for as long as it lasts. With presigned URLs, that traffic goes straight to storage, and your servers handle only the 1,000 small requests that ask for permission.

Finish the Handshake

The upload happens out of your sight, so the API cannot assume it succeeded. A robust flow has a third step:

  • -The client tells the API it has finished: POST /uploads/upl_501/complete.
  • -The API checks that the object really exists in storage and that its size and checksum match what was declared. It never trusts the client's word.
  • -Only then does it mark the upload ready. Until then it is pending, and unfinished uploads are cleaned up after a day or so.
  • -Slow work such as virus scanning or image resizing runs afterwards, in the background, moving the upload from uploaded to processed.

Multipart and Resumable Uploads for Large Files

A single PUT works fine up to some reasonable size. For very large files (video, large datasets), one failed connection partway through means restarting the entire upload from byte zero. Multipart upload splits the file into chunks, each uploaded (and retried) independently, with a final step that tells the storage layer to assemble the chunks into the complete file. This makes a transient failure on one chunk cheap to retry, instead of expensive to restart from scratch.

Here is why chunking matters, with numbers. A 5 GB video split into 100 MB parts is 50 parts. If the connection drops while part 37 is in flight, the client re-sends that one part, 100 MB, instead of starting the whole 5 GB again. For a user on a flaky connection, that is the difference between an upload that finishes and one that never does.

A presigned URL with too generous an expiry, or scoped too broadly (write access to an entire bucket instead of one specific object), is a real security exposure - the whole value of the pattern depends on each URL being narrowly scoped and genuinely time-limited.

What to Put in the Signature

  • -The exact key (object path), never a whole bucket or a wildcard.
  • -The content type and a maximum size, so the URL cannot be used to store something else, or something huge.
  • -A short expiry, minutes and not days, since anyone who obtains the URL can use it until it expires.
  • -A key you generate, not one the client chooses, so uploads cannot overwrite each other or other people's files.

Interview Signal is part of Pro

See a real weak answer next to a real strong one for this exact topic.

Quiz is part of Pro

Test what you just read with a short quiz, and bank the XP.

Ready to Build This?

Design The File Vault in the API Design Lab's Production Patterns act.

Discussion0

Join the Discussion

Sign in to leave comments, reply to others, or like insights.

Sign In to ScaleDojo

No comments yet. Be the first to start the thread!