Long-Running Operations: Handling Async Work in Synchronous APIs

How GitHub Actions Runs 100 Million Jobs Per Day

When you push code to GitHub, your CI/CD pipeline might run tests, build Docker images, deploy to staging, and run integration tests - all taking 5 to 60 minutes. GitHub cannot keep an HTTP connection open for an hour. Instead, GitHub Actions follows the async operations pattern: your push triggers a workflow, GitHub immediately returns a job ID, and you can poll for status or receive webhook notifications. The API returns 202 Accepted (not 200 OK) to signal that work has been ACCEPTED for processing, not COMPLETED. This same pattern powers video transcoding (YouTube), ML training (AWS SageMaker), data exports (Salesforce), and any operation too slow for synchronous HTTP.

The Async Operations Pattern

Long-Running Operation Flow:

  Step 1: Submit the work
  POST /api/reports/generate
  { "date_range": "2024-01-01/2024-12-31", "format": "pdf" }

  Response: 202 Accepted  (NOT 200 OK!)
  {
    "job_id": "job_8f14e45f",
    "status": "queued",
    "status_url": "/api/jobs/job_8f14e45f",
    "cancel_url": "/api/jobs/job_8f14e45f/cancel",
    "estimated_duration": "5-10 minutes",
    "created_at": "2024-01-15T10:00:00Z"
  }

  Step 2: Poll for status (or receive webhook)
  GET /api/jobs/job_8f14e45f

  While running:
  {
    "job_id": "job_8f14e45f",
    "status": "processing",
    "progress": {
      "percent": 45,
      "current_step": "Aggregating Q3 data",
      "steps_completed": 3,
      "steps_total": 7
    },
    "started_at": "2024-01-15T10:00:05Z"
  }

  When complete:
  {
    "job_id": "job_8f14e45f",
    "status": "completed",
    "result": {
      "download_url": "/api/reports/abc.pdf",
      "file_size": 15234567,
      "expires_at": "2024-01-22T10:00:00Z"
    },
    "duration": 342,
    "completed_at": "2024-01-15T10:05:42Z"
  }

  On failure:
  {
    "job_id": "job_8f14e45f",
    "status": "failed",
    "error": {
      "code": "DATA_TOO_LARGE",
      "message": "Report exceeds 100MB limit. Try a smaller date range."
    }
  }

Notification Strategies

How the Client Learns the Job is Done:

  1. POLLING (simplest):
  Client: GET /jobs/abc  every 5 seconds
  Pros: simple, stateless, works everywhere
  Cons: wastes bandwidth, 5s delay on average
  Tip: use exponential backoff (2s, 4s, 8s, 16s, max 30s)

  2. WEBHOOK (most efficient):
  Client specifies callback_url when submitting job.
  Server POSTs result to callback_url when done.
  Pros: instant notification, no wasted requests
  Cons: client needs a public endpoint

  3. WEBSOCKET (real-time UI):
  Client connects: WS /jobs/abc/stream
  Server pushes progress updates: { percent: 45, step: 'Q3...' }
  Pros: real-time progress bar in UI
  Cons: needs persistent connection

  4. SERVER-SENT EVENTS (simpler real-time):
  Client: GET /jobs/abc/events  (text/event-stream)
  Server: data: {"percent": 45}
  Pros: simple, built into browsers, auto-reconnect
  Cons: one-directional (server to client only)

  Recommendation:
  Support polling (required - simple clients) + webhook (efficient).
  Add SSE/WebSocket only if you need real-time progress in a UI.

Implementation Architecture

Async Job Infrastructure:

  Client --> API Server --> [Job Queue (Redis/SQS)] --> Workers
                 |                                       |
                 |--Store job metadata in DB             |--Update progress
                 |  (status, progress, result_url)       |  in DB/Redis
                 |                                       |
                 |<--------Poll status from DB-----------|

  Job States:
  queued -> processing -> completed
                       -> failed
                       -> cancelled

  Database Schema:
  CREATE TABLE jobs (
    id          UUID PRIMARY KEY,
    user_id     UUID NOT NULL,
    type        VARCHAR(50),       -- 'report', 'export', 'transcode'
    status      VARCHAR(20),       -- 'queued', 'processing', etc.
    progress    JSONB,             -- { percent, step, detail }
    result      JSONB,             -- { download_url, file_size }
    error       JSONB,
    created_at  TIMESTAMP,
    started_at  TIMESTAMP,
    completed_at TIMESTAMP,
    expires_at  TIMESTAMP          -- auto-cleanup old results
  );

Best Practices

• Return 202 Accepted (not 200 OK) to clearly signal async processing.
• Include status_url in the initial response so clients know where to check progress.
• Support cancellation: DELETE /jobs/{id} should stop in-progress work and clean up resources.
• Set result expiration: completed job results should expire after 7 days to avoid unbounded storage.
• Provide progress information: percentage, current step name, and estimated time remaining when possible.
• Rate limit polling: if clients poll too frequently, return 429 with Retry-After header.
• Idempotent submission: use idempotency keys so retrying the initial POST does not create duplicate jobs.

Interview Tip

Long-running operations appear in video platforms (transcoding), data platforms (exports/reports), ML systems (training), and CI/CD (build pipelines). The pattern is always the same: 202 Accepted with job ID, then polling or webhook for completion. Draw the architecture: API server -> job queue (SQS/Redis) -> worker pool. Mention job states (queued/processing/completed/failed/cancelled) and the database schema. The key details that impress: (1) exponential backoff for polling, (2) result expiration to prevent unbounded storage, (3) cancellation support, (4) idempotent submission to prevent duplicate jobs. Connect this to the bulk operations pattern - large bulk imports often become long-running jobs.

<

/blockquote>

Key Takeaway

Return 202 Accepted immediately for long-running operations with a job ID and status URL. Provide status polling with progress updates and optionally webhook notifications for instant completion alerts. Support cancellation, expire old results, and use idempotency keys to prevent duplicate job submission. This pattern handles any operation exceeding a reasonable HTTP timeout.