Context Engineering for Agents & Long Tasks

Prompt engineering is about writing good instructions. Context engineering is the larger job of deciding everything the model sees on each step: instructions, tool results, history, retrieved documents, notes. For an agent that works for dozens of steps, this is the main design problem, because the context fills up faster than the task finishes.

Imagine working at a desk. If every document you have ever touched stays on it, you soon cannot find what you need, and you make mistakes. A good worker keeps a few things in front of them, files the rest where they can be fetched, keeps notes on what matters, and sometimes sends a colleague off to read a long report and come back with a one-page summary. Those habits are the same four techniques agent builders use.

Anthropic's engineering post on context engineering, published in September 2025, makes a point beyond the hard limit of the context window. As the context grows, a model's ability to recall what is in it tends to get worse, a decline the post calls context rot. It describes context as a finite resource with diminishing returns, with an attention budget that every new token uses up. One reason it gives is that every token can attend to every other, so n tokens create n² pairwise relationships. The decline is gradual, not a cliff. So a smaller, well-chosen context beats a large, cluttered one even when it all fits.

• -Just-in-time retrieval: keep light references, like file paths, saved queries, and links, and load the full content with a tool only when needed. The agent explores like a person with a file system, instead of reading everything up front. It is slower and needs good tool guidance, so many systems preload some context and fetch the rest on demand.
• -Clearing old tool results: once a result has been used, its raw text rarely needs to stay in the context. Dropping it is the lightest form of compaction.
• -Compaction: when the context nears its limit, summarize the conversation and start a fresh window from the summary. Tune the summary first for recall, so nothing important is lost, and then trim for precision. Compacting too hard can drop details that matter later.
• -Structured note-taking: the agent writes notes, such as a to-do list or a NOTES file, to storage outside the context, and reads them back later. It lets work continue across context resets.
• -Sub-agents: a main agent hands a focused task to a sub-agent with a clean window. The sub-agent may use tens of thousands of tokens exploring and returns only a condensed summary, often 1,000 to 2,000 tokens.

The code compares how the context grows over 20 steps of an agent that, at every step, calls a tool returning 6,000 tokens of text, such as file contents. The window is 100,000 tokens, and the starting prompt is 4,000. Four strategies are compared. The numbers are invented but the arithmetic shows the shape. Press Run and look at the last column.

Keeping everything reaches 100,000 tokens at step 15 and overflows at step 16, ending at 132,000. Clearing old tool results keeps it near 25,000 at step 20, because only the two latest results stay in full. Compaction lets the context swell to 68,000, then resets it to a 5,000-token summary at step 11, and ends at 62,600. Delegating each lookup to a sub-agent that returns 1,200 tokens grows the context by 1,600 per step and ends at 36,000. Each has a cost: cleared results cannot be re-read without calling the tool again, a summary can lose a detail, and a sub-agent summary can omit what the main agent would have noticed.

• -Long back-and-forth conversations: compaction, with a careful summary prompt.
• -Iterative work with clear milestones, like a long coding task: notes outside the context, and cleared tool results.
• -Research that explores many sources in parallel: sub-agents, each with a clean window, reporting back summaries.
• -Large knowledge bases: just-in-time retrieval, with metadata such as file names and dates to guide where to look.