How GitHub Actions Runs 100 Million Jobs Per Day
When you push code to GitHub, your CI/CD pipeline might run tests, build Docker images, deploy to staging, and run integration tests, all taking 5 to 60 minutes. Long-Running Operations follow an asynchronous pattern for this kind of work: your push triggers a workflow, GitHub immediately returns a job ID, and you can poll for status or receive webhook notifications. The API returns 202 Accepted (not 200 OK) to signal that work has been ACCEPTED for processing, not COMPLETED. This same pattern powers video transcoding (YouTube), ML training (AWS SageMaker), data exports (Salesforce), and any operation too slow for synchronous HTTP.
The Async Operations Pattern
Long-Running Operation Flow:
Step 1: Submit the work
POST /api/reports/generate
{ "date_range": "2024-01-01/2024-12-31", "format": "pdf" }
Response: 202 Accepted (NOT 200 OK!)
{
"job_id": "job_8f14e45f",
"status": "queued",
"status_url": "/api/jobs/job_8f14e45f",
"cancel_url": "/api/jobs/job_8f14e45f/cancel",
"estimated_duration": "5-10 minutes",
"created_at": "2024-01-15T10:00:00Z"
}
Step 2: Poll for status (or receive webhook)
GET /api/jobs/job_8f14e45f
While running:
{
"job_id": "job_8f14e45f",
"status": "processing",
"progress": {
"percent": 45,
"current_step": "Aggregating Q3 data",
"steps_completed": 3,
"steps_total": 7
},
"started_at": "2024-01-15T10:00:05Z"
}
When complete:
{
"job_id": "job_8f14e45f",
"status": "completed",
"result": {
"download_url": "/api/reports/abc.pdf",
"file_size": 15234567,
"expires_at": "2024-01-22T10:00:00Z"
},
"duration": 342,
"completed_at": "2024-01-15T10:05:42Z"
}
On failure:
{
"job_id": "job_8f14e45f",
"status": "failed",
"error": {
"code": "DATA_TOO_LARGE",
"message": "Report exceeds 100MB limit. Try a smaller date range."
}
}Notification Strategies
How the Client Learns the Job is Done:
1. POLLING (simplest):
Client: GET /jobs/abc every 5 seconds
Pros: simple, stateless, works everywhere
Cons: wastes bandwidth, 5s delay on average
Tip: use exponential backoff (2s, 4s, 8s, 16s, max 30s)
2. WEBHOOK (most efficient):
Client specifies callback_url when submitting job.
Server POSTs result to callback_url when done.
Pros: instant notification, no wasted requests
Cons: client needs a public endpoint
3. WEBSOCKET (real-time UI):
Client connects: WS /jobs/abc/stream
Server pushes progress updates: { percent: 45, step: 'Q3...' }
Pros: real-time progress bar in UI
Cons: needs persistent connection
4. SERVER-SENT EVENTS (simpler real-time):
Client: GET /jobs/abc/events (text/event-stream)
Server: data: {"percent": 45}
Pros: simple, built into browsers, auto-reconnect
Cons: one-directional (server to client only)
Recommendation:
Support polling (required - simple clients) + webhook (efficient).
Add SSE/WebSocket only if you need real-time progress in a UI.Implementation Architecture
Async Job Infrastructure:
Client --> API Server --> [Job Queue (Redis/SQS)] --> Workers
| |
|--Store job metadata in DB |--Update progress
| (status, progress, result_url) | in DB/Redis
| |
|<--------Poll status from DB-----------|
Job States:
queued -> processing -> completed
-> failed
-> cancelled
Database Schema:
CREATE TABLE jobs (
id UUID PRIMARY KEY,
user_id UUID NOT NULL,
type VARCHAR(50), -- 'report', 'export', 'transcode'
status VARCHAR(20), -- 'queued', 'processing', etc.
progress JSONB, -- { percent, step, detail }
result JSONB, -- { download_url, file_size }
error JSONB,
created_at TIMESTAMP,
started_at TIMESTAMP,
completed_at TIMESTAMP,
expires_at TIMESTAMP -- auto-cleanup old results
);Best Practices
Return 202 Accepted (not 200 OK) to clearly signal async processing.
Include status_url in the initial response so clients know where to check progress.
Support cancellation: DELETE /jobs/{id} should stop in-progress work and clean up resources.
Set result expiration: completed job results should expire after 7 days to avoid unbounded storage.
Provide progress information: percentage, current step name, and estimated time remaining when possible.
Rate limit polling: if clients poll too frequently, return 429 with Retry-After header.
Idempotent submission: use idempotency keys so retrying the initial POST does not create duplicate jobs.
Interview Tip
Long-running operations appear in video platforms (transcoding), data platforms (exports/reports), ML systems (training), and CI/CD (build pipelines). The pattern is always the same: 202 Accepted with job ID, then polling or webhook for completion. Draw the architecture: API server -> job queue (SQS/Redis) -> worker pool. Mention job states (queued/processing/completed/failed/cancelled) and the database schema. The key details that impress: (1) exponential backoff for polling, (2) result expiration to prevent unbounded storage, (3) cancellation support, (4) idempotent submission to prevent duplicate jobs. Connect this to the bulk operations pattern - large bulk imports often become long-running jobs.
Key Takeaway
Return 202 Accepted immediately for long-running operations with a job ID and status URL. Provide status polling with progress updates and optionally webhook notifications for instant completion alerts. Support cancellation, expire old results, and use idempotency keys to prevent duplicate job submission. This pattern handles any operation exceeding a reasonable HTTP timeout.