From Prompt Engineering to Context Engineering: Write, Select, Compress, Isolate

Anyone who has built an agent has probably seen the same pattern. For the first handful of steps it behaves well: it picks sensible tools, reasons clearly and sticks to the task. Somewhere around step 15 or 20 it gets sloppy. It forgets what was asked, calls tools that make no sense for the job or starts turning out weak results.

The usual first suspicion is the model. More often the culprit is what the model is being shown. Deciding what goes in front of the model, in what shape and at which moment is called context engineering, and for agent builders it's turning into one of the most valuable skills around.

The sections below cover what context engineering is and why agents need it, the four strategies that organize nearly every technique, the four ways context goes wrong, concrete advice for system prompts and tool definitions, a phased workflow that puts the strategies together, and how Claude Code, Manus, ChatGPT Agent and Google's Agent Development Kit each handle it.

From prompt engineering to context engineering in the context window

Prompt engineering is the craft of writing good instructions for a language model: phrasing things clearly, supplying examples, assigning a role. For a chat with a single question and answer, that's often enough.

Agents change the picture. An agent doesn't answer once. It browses, calls APIs, writes code and runs commands, step after step and sometimes for dozens of steps, without a human in the loop. Each step produces output, and all of it lands in the model's context, which has a fixed size.

Context engineering is therefore the design of the whole information system around the model. That covers the system prompt, the tool definitions, the results of earlier tool calls, the conversation history and more, at every single step. Anthropic's engineering team describes context as the set of tokens included when sampling from a model, and context engineering as optimizing how useful those tokens are for reaching the intended result. Put simply, the agent should see the right information, in the right format, at the right time.

Anthropic also frames it as the natural next stage after prompt engineering. Everything from prompting still applies (clear instructions, good examples, structured formatting), and a further layer sits on top: tools, external data, message history, memory and dynamic state. Prompt engineering becomes a subset.

The timing matters. Gartner projects that 40% of enterprise applications will include task-specific AI agents by the end of 2026, up from under 5% in 2025. Agents replace static prompts and fixed retrieval pipelines with a dynamic system in which every tool call, retrieved document and decision gets packed into a window that fills with material the user never directly asked for. Teams that learn to manage that tend to ship agents that stay reliable, because they stop blaming the model for problems that live in its inputs.

The context window as working memory

LangChain offers a handy analogy. Treat the LLM as a new kind of operating system: the model is the CPU that does the thinking, and the context window is RAM, the working memory holding everything the model can currently see. Just as a computer slows when RAM fills up, an agent's reasoning gets worse as its window gets crowded.

This effect has a name, context rot. Chroma tested eighteen frontier models, among them GPT-4.1, Claude 4, Gemini 2.5 and Qwen3, and found that every one of them performs worse as input length grows, even far below the advertised limit. A model with a 200K-token window can show real degradation at 50K tokens. There's no sudden cliff, only a steady slope. Anthropic's own engineering writing reaches the same conclusion and treats degradation as a gradient.

The reason lies in how transformers work. Each token attends to every other token, which produces n-squared pairwise relationships. As the window grows, the model's capacity to track all of them is spread thinner, a bit like asking someone to hold an ever-growing pile of items in their head. At some point things get dropped.

A second well-studied effect is "lost in the middle." A research group led by Liu found that attention follows a U shape. Material near the start of the context is recalled well, material near the end is recalled well, and material in the middle gets missed, with accuracy falling by more than thirty percentage points when the relevant passage moves from the beginning to the middle. Picture an agent whose original instructions sit under fifty thousand tokens of tool output. Those instructions have effectively vanished.

Practitioners have come up with rules of thumb. Claude Code users report that output quality starts to slip at roughly 40 to 60 percent of the window, long before any hard limit.

What competes for space

Seven kinds of information fight for room in an agent's context:

1. The system prompt, which is the agent's identity: behavioral rules, control flow and guidance for different task types. In an agent it's much more than a "helpful assistant" line and can define how the whole thing operates.
2. Tool definitions. Each tool the agent might call needs a schema describing what it does, which parameters it takes and when to use it.
3. Tool results. Every call adds its output. A fetched web page can run five to ten thousand tokens, and a file read is similar.
4. Retrieved knowledge from RAG: vector database hits, search results, API responses, anything pulled in to inform decisions.
5. Conversation history, the full transcript of user messages, replies, reasoning and past decisions, which grows with every turn.
6. Memory, both short-term from the current session and long-term from earlier ones, such as preferences, task outcomes and learned patterns.
7. Agent state: the current plan, to-do list, progress markers and scratchpad notes that tell the agent where it stands in a multi-step job.

The four core strategies, with RAG and Claude Code examples

With a finite, degrading window, the questions become what goes in, what stays out and what gets squeezed. LangChain's widely cited framework sorts every context engineering technique into four buckets: write, select, compress and isolate. Once these are familiar, nearly any technique you meet will land in one of them.

Write

Agents forget. When a full context gets compacted, information disappears, and if the agent hadn't saved it elsewhere, it's gone. Writing means giving the agent ways to keep information outside the window. It takes three common forms.

Scratchpads are a note-taking tool the agent can use mid-task to record interim findings, decisions and things it'll need later. Anthropic built a "think" tool that gives Claude a dedicated place to work through a problem, and on the tau-bench benchmark it improved performance by as much as 54% on certain tasks.

Rules files act as persistent procedural memory. Claude Code users know CLAUDE.md. These files load at the start of every session as standing orders, covering the project's architecture, conventions, how to run tests and what to watch out for, so the basics are never forgotten.

Memory extraction lets the agent store facts, preferences and learned patterns in a file-based system and consult them in later sessions, entirely outside the window.

Select

Notes are only useful if the right ones come back at the right moment. Selection means giving the agent what it needs for this step and not everything it could possibly use. An agent with dozens of tools, a large knowledge base and several sessions of history can't load all of it at once, so something has to decide what's relevant, and the real question is what does the deciding.

In classic RAG the system decides. A question comes in, documents are retrieved and stuffed into the prompt, and the model has no say. In agentic RAG the agent itself chooses what to search, which tools to use, how to refine queries and when it has enough. Retrieval turns into an iterative process, which fits because relevance shifts at each step of a long task and only the agent knows what it needs next.

Both LangChain and Pinecone distinguish three kinds of memory to draw from. Episodic memory is essentially few-shot examples of how a similar situation was handled before. Semantic memory is a store of facts the agent has learned or been told. Procedural memory is the standing behavioral instructions, such as the rules files above. A well-built agent pulls from all three depending on the step.

Tools cause the worst selection trouble. With forty-odd tools, around ten thousand tokens of definitions can be sitting in context before any work starts. And as the failure section below shows, too many tools doesn't just waste space, it actively confuses the model.

The remedy is to apply RAG to the tool descriptions themselves, running a semantic search that surfaces just the handful of tools the current step could use. A paper named RAG-MCP put this to the test and saw tool-selection accuracy rise from about 14% to 43%, more than triple, while prompt tokens fell by roughly half.

Anthropic's general advice is a hybrid. Load a small amount of essential material up front for speed, like the CLAUDE.md file, and let the agent fetch everything else just in time. In practice that means the agent holds light references such as file paths, queries and links, then opens the heavy content only when a step calls for it. Front-loading the basics and retrieving the rest on demand keeps the window lean without leaving the agent blind.

Compress

Even with good selection, the window keeps filling. Picture an agent twenty tool calls in. It carries eighty thousand tokens of outputs, history and reasoning traces, most of which it already acted on. That old material still occupies space, dulls attention and raises cost and latency. This is the context rot problem again, and compression addresses it by cutting tokens while keeping the information that matters. It can be done at three points.

Before information enters the context. Chunking splits large documents into smaller coherent pieces, and reranking lets only the best chunks reach the window. Tool outputs can also be summarized on the fly, before they join the main context.

While the agent works. Summarizing the conversation is the common technique: a running summary refreshed after each exchange keeps a compact record of everything so far. A popular hybrid keeps the last ten messages word for word, since exact details may still be needed, and summarizes everything older. Plain trimming with hard-coded rules, dropping old messages once a size threshold passes, is another option. Claude Code builds in auto-compaction, summarizing the full trajectory when the window reaches 95% capacity. That's a safety net. Ideally compression happens ahead of time and not at the emergency trigger.

After the agent has used something. The easy win is tool result clearing. If a tool ran fifteen steps back and the agent already acted on the output, the raw text can go. A full web page fetched long ago can be replaced with a single-line summary or removed outright.

Isolate

Isolation is what makes multi-agent systems workable. Suppose one agent handles research, planning, coding, testing and debugging inside a single long conversation. Its context will fill up, but the deeper trouble is contamination. File searches from the research phase are still lying around when implementation starts, and that old material is now noise, distracting the model just when it needs to write clean code.

The answer is to give separate parts of the work separate context windows.

The obvious way is sub-agents. A parent hands a focused task, say finding every file related to authentication, to a sub-agent that works in its own clean window. When done, it returns only a condensed summary, and all the messy searching stays inside the sub-agent without polluting the parent.

Another variant is state schema isolation, as implemented in LangGraph. The agent's state is designed so that different fields hold different kinds of context, and the model sees only the fields relevant to the present step. Tool results can sit in a backstage field the model never sees until something deliberately brings them forward. That's fine-grained control without spinning up extra agents.

In short, the strategies are write, select, compress and isolate, and everything that follows applies one or more of them.

How agents fail

Failure modes teach what to avoid. In mid-2025 Drew Breunig wrote an influential two-part series naming four separate ways agents break down as their context grows.

Context poisoning. A hallucination or mistake gets into the context and is then referenced again and again. Maybe a tool returned bad data, or the agent drew a faulty inference five steps ago. The error now sits in the window and the agent builds on it, and because agents iterate on their own output, every bad step compounds. The fix is pruning: actively remove outdated or conflicting information as new details arrive, validate tool outputs (especially from external sources) before they enter the context, and after the agent recovers from an error, compress the failed attempts so that ten steps of dead-end debugging don't linger when only the resolution matters. Left in place, those abandoned attempts keep tempting the model back toward approaches that already failed.

Context distraction. With a very long context, the model begins to lean too heavily on recent history and too little on what it learned in training. It stops synthesizing something new and repeats patterns it recently saw, rehashing past actions instead of forming a fresh plan. The remedy is the same general advice: summarize and prune aggressively, even if the window is huge.

Context confusion. Superfluous content gets picked up by the model and drags response quality down. The classic case is tool confusion. A benchmark study ran a quantized Llama 3.1 8B model on the GeoEngine benchmark. Given all forty-six available tools, it failed, though the context sat well inside its limit. Given just nineteen tools, it worked. The tools fit in the window, but there were too many for the model to reason about cleanly. The cure is dynamic tool management. Trim the available tools to those that fit the phase the agent is in, with a RAG-MCP style retriever picking out the pertinent ones for each step.

Context clash. New information gathered during a run contradicts something already in context. The system prompt may say one thing while a retrieved document says another. The agent can't reconcile them and behaves inconsistently, following one source at times, the other at times and sometimes doing something that matches neither. The fix is a clear authority ordering: system prompt over retrieved facts, retrieved facts over conversation history. Validate new information against what's already there before injecting it, and use structured sections such as XML tags or clear headers so the model can tell where each piece came from and how far to trust it.

The good news is that naming a failure usually points to a strategy. Poisoning is handled by compression and pruning, distraction by compression, confusion by selection, and clash by structured writing and selection. It all returns to write, select, compress, isolate.

System prompts for agents

Prompting an agent differs sharply from prompting a chatbot. A chatbot prompt mostly sets a tone: be friendly, keep it short. An agent prompt defines architecture. It spells out control flow, meaning how to handle different kinds of tasks, which tools fit which situations, what to do after an error and which safety limits apply. It reads more like a job description for an autonomous employee.

Anthropic talks about writing at the right altitude, and there's a Goldilocks zone. Being too prescriptive is brittle. Imagine a rule that says to call tool X whenever the user mentions billing, brings up a refund and the sum tops a hundred dollars. It snaps on every edge case nobody foresaw. Being too vague is just as bad. "Be helpful and use the appropriate tools" gives the agent nothing, since it doesn't know which tools are appropriate for what and can't make decisions without concrete signals. The sweet spot is specific enough to steer autonomous behavior and loose enough that the model can use judgment in new situations. Think strong heuristics, not rigid rules.

A few practical habits help:

• Give the prompt structure, with XML tags or markdown headers that separate background, instructions and tool guidance.
• Start minimal and iterate on failures. Run the agent on real tasks, see where it breaks and add instructions aimed at those specific breakages, instead of trying to predict every edge case in advance.
• Minimal isn't the same as short. A prompt for a complex workflow can run to thousands of tokens and that's fine, as long as every token earns its place.
• Use few-shot examples. Showing diverse, canonical cases of correct tool choice, sound reasoning and proper multi-step execution beats describing every rule in words.

Tool definitions

Each callable tool needs a schema covering its purpose, parameters and when to use it. A good definition is self-contained, tolerant of errors and clear about instructions, and the cost is context space.

In production, tools are increasingly wired up through MCP, the Model Context Protocol, a standard way for agents to reach external tools and data. One MCP server for GitHub, another for the database, a third for the file system, all exposing tools through a uniform interface. The catch is that plugging in tools becomes so easy that you overdo it. With four or five servers connected, definitions can swallow thousands of tokens before the first step.

The default advice is to curate a minimal viable set. When an agent really does need many tools, two approaches scale.

The first comes from Manus, a production agent platform. They advise against adding and removing tools in the middle of a conversation, for a reason many builders don't know about: the KV-cache. When tokens go to an LLM, the model computes key-value representations for each of them, which is expensive, so providers cache them. If the start of the context (the prefix) is identical between calls, the provider reuses the cached work and processes only the new tokens at the end, which is faster and cheaper.

Rearrange or edit the early part of the context between calls, and the cache is invalidated, so everything gets recomputed. With Claude Sonnet, cached input tokens cost thirty cents per million against three dollars per million for uncached ones, a tenfold gap. An agent making thirty or forty calls per task feels that quickly.

Hence Manus prefers tool masking over tool removal. Every definition stays put in the cached prefix, but certain tools are flagged as unavailable for the current phase. The definitions remain for cache stability while the agent knows not to use them, and consistent naming prefixes help it reason about categories of tool.

The second approach is the RAG-based selection covered earlier: retrieve only the tools relevant to the current step, which the RAG-MCP paper showed raised accuracy more than threefold while halving token use. The two methods complement each other. Masking suits a moderate tool set, and retrieval suits a very large one.

Underneath both sits one broad principle. Stable content goes at the top of the context: the system prompt, tool definitions and anything that doesn't change between turns. Dynamic content (history, the current step, agent state) goes at the bottom.

A phased workflow that combines everything

The principles hold with an off-the-shelf coding assistant and with a custom-built agent alike. The only difference is whether you structure the workflow yourself or build the agent to do it.

HumanLayer's CEO, Dex Horthy, gave a talk at the AI Engineer Code Summit describing a method that illustrates all of this neatly. His team reportedly used it to land around thirty-five thousand lines of code in a large Rust codebase in a single seven-hour session. He calls it Frequent Intentional Compaction. The work is divided into phases on purpose. Each phase produces a compact artifact, such as a structured markdown summary, and each new phase begins in a fresh window holding only that artifact. The effect is to stay under roughly 40 to 60 percent of the window deliberately.

Phase one, research. Before any code is written, the agent explores the codebase: reading files, tracing data flows and mapping the architecture relevant to the task. Sub-agents do the raw searching and analysis, so all those grep results and file contents stay in their windows and never touch the parent's. That's isolation. The phase's output is a research file, a compact markdown document of file paths, function signatures, existing patterns and gotchas. That's writing, since the findings are persisted outside the window. Then the context resets. The raw research might have used 60 to 80 percent of the window, while the artifact shrinks it to maybe 15 to 20 percent. That's compression.

Phase two, planning. A brand-new window opens containing only the research file and the problem statement, and the agent uses that clean slate to draft a detailed implementation plan. This is also the best place for human review. A logic error caught here is cheap to fix, so anyone building their own agent should put a human-in-the-loop checkpoint at this point.

Phase three, implementation. Another fresh window holds only the plan, and the agent works through it step by step. For jobs needing several compaction cycles, a progress.md file records what's finished and what remains, which is writing once more.

Some version of this shows up in most production systems.

How real products handle context

Claude Code. Anthropic's command-line coding agent is among the best-documented cases. It uses hybrid retrieval: CLAUDE.md files load up front at the start of each session for foundational context, while tools like glob and grep handle just-in-time exploration of the codebase. It doesn't pre-index everything, it explores on demand. Auto-compaction fires at 95% utilization and preserves architectural decisions plus the five most recently accessed files. It can spawn sub-agents with clean windows for complex subtasks, and a memory tool persists information across sessions. The guiding philosophy is to do the simplest thing that works, trusting the model to be smart about what it needs and giving it tools to find it.

Manus. This is the more infrastructure-heavy approach, which makes sense for a general-purpose agent serving hundreds of thousands of users, where efficiency at scale counts. Its signature move is KV-cache-aware ordering, keeping the prefix stable across turns for maximum cache reuse and masking tools instead of removing them. An observation compression pipeline processes every tool output before it reaches the agent's context. A persistent to-do list tracks state, and the file system serves as overflow memory for evicted context.

ChatGPT Agent. OpenAI's agent, which grew out of Operator, is architecturally different from both. Powered by a Computer-Using Agent model, it takes a GUI-first route. Instead of text-based tool calls, it drives a visual browser, and screenshots enter the context as snapshots that the model reasons over, together with its history of earlier screenshots and actions. The challenge changes accordingly. Visual tokens are costly, so the agent must be choosy about how many screenshots it keeps. OpenAI uses reinforcement learning to discover good tool-use strategies, training across thousands of virtual machines instead of hard-coding when to use which tool. Deep Research and Operator share state, which allows smooth transitions between browsing, text analysis and code execution. Compared with the code-centric, text-tool, large-window Claude Code, this is visual and general-purpose. Both need context engineering, with different problems and answers.

Google's Agent Development Kit. ADK takes the most principled architectural stance, with three design principles for agent context. Separate storage from presentation, because durable state isn't the same as what appears in a given API call. Use explicit transformations, meaning named, ordered processors that reshape context in testable, composable steps instead of ad hoc string concatenation. Scope context by default, so each model call sees only the minimum required and nothing lands in context unless it's explicitly included. The result treats context assembly as a pipeline with clear stages, not a prompt you keep appending to.

The common context pipeline

Looking across these platforms, the same shape emerges on every agent turn:

1. Collect candidates: user input, history, tool results, retrieved documents and agent state.
2. Select what's relevant to this step within the remaining token budget.
3. Compress by summarizing, truncating or restructuring the selection to fit.
4. Order everything for KV-cache reuse, with stable content first and dynamic content last.
5. Assemble the final context and make the call.

The field is moving fast and there's plenty still to learn, so the best way to improve is to experiment: build something, watch where it breaks and adjust.