Why Go Async? Background Jobs & Task Queues

Recall from Module 2 that a web server thread handling a request should stay fast and stateless. A request that triggers something slow (encoding a video, generating a PDF report, sending a batch of emails) blocks that thread for seconds while the user (and every other request queued behind it) waits. The fix is to stop doing slow work inline.

The web server's job becomes: accept the request, hand the slow work off to a queue, and immediately respond "got it, we're on it." A separate pool of workers pulls tasks off that queue and processes them independently, at their own pace, without holding up any user-facing request.

Not every background job is triggered by a user request. Some work needs to run on the clock regardless of what anyone does: regenerating a recommendation cache overnight, sending a weekly billing summary, purging sessions that expired hours ago. A task scheduler (cron, or a managed equivalent like AWS EventBridge) fires these jobs at fixed intervals and drops them onto the same worker queue as any request-triggered job, so a worker never needs to know or care whether a task exists because a user asked for something or because the clock struck midnight.

Handing work off to a queue does not make capacity problems disappear, it just moves them. If producers enqueue work faster than the worker pool can process it, the queue keeps growing. An unbounded queue turns that growth into an unbounded memory leak and, since work is now sitting for longer and longer before a worker ever reaches it, a slowly climbing effective latency for anyone waiting on the result. Backpressure is the general term for a system pushing back on producers once it cannot keep up, instead of silently absorbing unlimited work and hoping capacity catches up later.

A bounded queue is the first piece: capping how much work can sit waiting turns an unbounded memory problem into a bounded, predictable one. Once that cap is reached, the system has to decide what to do with the next arrival, and that decision is called load shedding.

• -Reject new work (fail fast): return an explicit error, commonly a 429 Too Many Requests, so the caller knows immediately to retry later or back off, rather than waiting on a request that was going to time out anyway.
• -Drop the oldest queued item: useful when only the freshest data actually matters, like a live metrics feed where a stale reading is worse than a missing one.
• -Signal the producer to slow down directly: TCP itself does this at the transport layer, and some queue protocols expose an explicit "pause sending" signal so a well-behaved producer throttles itself instead of being rejected.