File Upload APIs
You'll learn to
- -Design a presigned-URL upload flow so large files bypass the API server entirely
- -Know when multipart or resumable upload is needed instead of a single presigned PUT
- -Finish the handshake: confirm the upload happened before trusting it
- -Scope a presigned URL narrowly, and validate what the client claims
Routing a large file upload through the same API server that handles ordinary JSON requests works for small files. For large ones it wastes server resources, holding open connections and buffering bytes it doesn't actually need to process. The presigned-URL pattern lets the client upload directly to storage, with the API server only ever handling the metadata.
An Analogy: The Parcel Locker
Parcel lockers do not make the courier hand each parcel to the shop assistant, who then walks it to a shelf. The shop gives you a one-time code for one specific locker that works for the next 15 minutes, and you put the parcel in yourself. The shop only has to issue the code and later check the locker has something in it.
- -The one-time code is the presigned URL.
- -Its validity for a single locker and 15 minutes is the scope and expiry in the signature.
- -The locker bank is object storage, which receives the bytes directly.
- -The shop issuing the code and checking afterwards is your API: metadata in, bytes never.
The Presigned URL Flow
POST /uploads
{"filename": "report.pdf", "content_type": "application/pdf"}
HTTP/1.1 201 Created
{
"upload_url": "https://storage.example.com/bucket/abc123?signature=...",
"upload_id": "upl_501",
"expires_at": "2026-08-07T10:15:00Z"
}PUT https://storage.example.com/bucket/abc123?signature=...
Content-Type: application/pdf
[binary file data]
HTTP/1.1 200 OKThe upload_url is a time-limited, cryptographically signed URL granting temporary write access to one specific storage location - the signature encodes exactly what's permitted (this bucket path, this content type, until this expiry) without requiring the client to have any broader storage credentials. The API server's only job is issuing that signed URL and recording the upload's metadata; the actual bytes never pass through it.
The saving is easy to put numbers on. Suppose users upload 1,000 files an hour at 50 MB each. If they go through your API servers, that is 1,000 x 50 MB = 50 GB an hour passing through machines that should be answering small JSON requests, and every slow upload holds a connection and some memory for as long as it lasts. With presigned URLs, that traffic goes straight to storage, and your servers handle only the 1,000 small requests that ask for permission.
Finish the Handshake
The upload happens out of your sight, so the API cannot assume it succeeded. A robust flow has a third step:
- -The client tells the API it has finished: POST /uploads/upl_501/complete.
- -The API checks that the object really exists in storage and that its size and checksum match what was declared. It never trusts the client's word.
- -Only then does it mark the upload ready. Until then it is pending, and unfinished uploads are cleaned up after a day or so.
- -Slow work such as virus scanning or image resizing runs afterwards, in the background, moving the upload from uploaded to processed.
Multipart and Resumable Uploads for Large Files
A single PUT works fine up to some reasonable size. For very large files (video, large datasets), one failed connection partway through means restarting the entire upload from byte zero. Multipart upload splits the file into chunks, each uploaded (and retried) independently, with a final step that tells the storage layer to assemble the chunks into the complete file. This makes a transient failure on one chunk cheap to retry, instead of expensive to restart from scratch.
Here is why chunking matters, with numbers. A 5 GB video split into 100 MB parts is 50 parts. If the connection drops while part 37 is in flight, the client re-sends that one part, 100 MB, instead of starting the whole 5 GB again. For a user on a flaky connection, that is the difference between an upload that finishes and one that never does.
A presigned URL with too generous an expiry, or scoped too broadly (write access to an entire bucket instead of one specific object), is a real security exposure - the whole value of the pattern depends on each URL being narrowly scoped and genuinely time-limited.
What to Put in the Signature
- -The exact key (object path), never a whole bucket or a wildcard.
- -The content type and a maximum size, so the URL cannot be used to store something else, or something huge.
- -A short expiry, minutes and not days, since anyone who obtains the URL can use it until it expires.
- -A key you generate, not one the client chooses, so uploads cannot overwrite each other or other people's files.
Interview Signal is part of Pro
See a real weak answer next to a real strong one for this exact topic.
Quiz is part of Pro
Test what you just read with a short quiz, and bank the XP.
Design The File Vault in the API Design Lab's Production Patterns act.
Discussion0
Join the Discussion
Sign in to leave comments, reply to others, or like insights.
No comments yet. Be the first to start the thread!