Context Engineering for AI Agents: Curating What the Model Sees

An AI agent that performs well on its first few steps and then starts ignoring instructions, repeating work or trusting stale facts usually does not have a wording problem. It has a context problem: the model is seeing too much, the wrong things, or the right things in the wrong place. Context engineering is the discipline of deciding precisely what a model receives right before it has to reason, meaning not just the prompt but the entire bundle of instructions, history, retrieved data and tool definitions. This guide explains why that bundle matters more as agents grow, the four components you control, a minimal assembly pipeline, and the failure patterns to watch for.

The core idea: a small, relevant slice

The goal is never to give the model everything that might conceivably help. It is to give it the narrow slice of information that actually lets it answer correctly, and to leave the rest out.

A human analogy makes the point. Ask a new engineer to fix a bug in a million-line codebase by saying "read the code and find it," and they will be overwhelmed; the one relevant file is lost among thousands. Tell the same engineer "the problem is most likely in the payment module, these three files changed last week, and here is what the previous person tried," and the fix may take minutes. Nothing about the engineer changed. Only the information they started with did.

Models behave the same way. Context engineering is the job of handing over those three files instead of the entire repository.

Why wording alone stopped being enough

Early advice about working with language models focused on phrasing: how to word instructions and which formulations produced better answers. That is prompt engineering, and for a single question it still matters.

An agent, however, does not answer a single question. It runs a loop: read something, call a tool, receive a result, choose the next action, and repeat, sometimes dozens of times. Each iteration appends more material to what the model is carrying. Eventually the accumulated context is so large that important details are overlooked, much like someone who has sat through meetings all day and can no longer recall what was decided in the first one.

Anthropic frames context engineering as the problem of finding the best possible set of information for the model at each individual moment, rather than composing a good instruction once. That captures the shift: prompt engineering is about the words; context engineering is about everything that is present when the model responds.

Prompt engineering versus context engineering

The two overlap but differ in scope, typical use and failure mode:

• Scope. Prompt engineering shapes the wording of one instruction. Context engineering governs the whole input: instructions, prior conversation, retrieved facts and available tools.
• Where it shines. Prompt engineering fits one-off questions and tasks. Context engineering matters most for multi-step agents, where what is carried between steps is as important as the current request.
• Typical mistake. In prompt engineering, it is an ambiguous or muddled instruction. In context engineering, it is supplying too much or the wrong information even though the instruction is perfectly clear.
• Typical fix. A weak prompt is fixed by rewording. A context problem is not fixed by rewording at all; you change what gets retrieved, what gets retained and what gets discarded.

A quick diagnostic: if your fix consists of changing words, you are doing prompt engineering. If your fix changes which information the model receives in the first place, you are doing context engineering. For a structured way to decide which layer an agent failure belongs to, see debugging AI agents by layer.

The four components you manage

Every agent's context is assembled from four sources. Each has its own way of going wrong.

Instructions: the job description

This is the system prompt, describing what the agent should do and how. Make it too rigid and the agent cannot handle anything outside the script. Make it too loose and the agent improvises freely. Aim for guidance specific enough to shape behavior without trying to enumerate every situation in advance.

Retrieval: the research assistant

Retrieval brings in outside information by searching documents, querying a database or reading files. Poor retrieval is the leading cause of AI systems stating wrong things with confidence. Often the model is not inventing anything; it was given incorrect or irrelevant facts and reasonably relied on them. Improving what gets retrieved frequently does more for accuracy than any change to the prompt.

Memory: the notebook

Memory covers what the agent retains, both within a single conversation and across sessions. Without a mechanism for summarizing and pruning older material, memory does not stay useful. It grows steadily until most of it is noise that competes with the information that matters now.

Tools: the toolbox

Tools define the actions the agent may take, such as web search, code execution or sending email. Each tool's name and description also occupies context. A set of ten overlapping, nearly identical tools confuses a model in the same way ten indistinguishable screwdrivers would confuse a new hire. Fewer tools with clearly separated purposes make selection easier and cheaper.

A minimal context assembly pipeline

The following pseudocode shows how the four components come together for one model call. The helper functions are placeholders for whatever search, ranking and summarization you use, but the structure reflects how real systems approach the problem.

It works in four stages. First it retrieves broadly, accepting some noise, with a generous limit of 50 candidates. Second it ranks those candidates and keeps only the top five. Third it compresses the conversation history into a summary capped at 500 units rather than replaying it in full. Finally it orders the pieces deliberately, with instructions first and the current question last, and trims the result to the token budget.

def get_context_for_the_model(question, past_conversation, budget):
    # Step 1: Cast a wide net — search broadly, don't worry about noise yet
    possible_facts = search_everywhere(question, limit=50)
    # Step 2: Narrow it down - keep only the genuinely relevant ones
    best_facts = keep_most_relevant(possible_facts, top=5)
    # Step 3: Summarize old conversation instead of keeping all of it
    short_memory = summarize(past_conversation, max_length=500)
    # Step 4: Put the most important things first and last, not buried in the middle
    final_context = [
        job_description,      # instructions
        short_memory,         # memory
        *best_facts,          # retrieval
        question,             # what's being asked right now, last
    ]
    return trim_to_fit(final_context, budget)

Two habits in this sketch are worth adopting.

Search wide, then narrow hard. Broad retrieval reduces the chance of missing the relevant document; aggressive ranking keeps the final context small. Doing only the first floods the model, and doing only the second risks never finding the right material.

Put what matters at the edges. Place the most important content at the beginning or end of the input rather than in the middle. This is not a stylistic preference. Research on long inputs has repeatedly observed that models attend less reliably to information buried in the middle of a long context, an effect often called "lost in the middle." How strong the effect is varies by model, so test with your own setup, but ordering is a cheap lever either way.

A few practical refinements follow from the same logic. Decide how the budget is split among components ahead of time, so a large retrieval result cannot silently crowd out the instructions. When trimming, drop the least relevant retrieved items before touching the instructions or the current question. And log the final assembled context for each call; when an agent misbehaves, that record usually shows why.

What goes wrong when context is not curated

Including everything to be safe

Adding every possibly relevant item feels responsible and tends to backfire. The more unrelated material the model must wade through, the worse it gets at locating the one fact that matters. It is the equivalent of reading a hundred-page report before a five-minute decision.

Stale information

If nothing checks whether retrieved content is still accurate, the model will build a confident answer on something that stopped being true months ago. Freshness metadata, expiry rules or re-validation for time-sensitive sources all help.

Burying the key fact

Even when retrieval finds exactly the right fact, placing it in the middle of a long block makes it statistically more likely to be missed. The fact is identical; only its position changed, and the outcome got worse.

Too many similar options

If a person on your team could not say with confidence which tool or document applies in a given situation, the model will not do better. Consolidate overlapping tools and deduplicate near-identical documents before they reach the context.

Common questions

Is context engineering replacing prompt engineering?

No. Clear instructions remain part of the job. Context engineering is the larger task that surrounds them: deciding what else the model sees besides the instruction itself.

Why can more information make results worse?

Attention is a limited resource, for models as for people. The more material the model has to sift through, the higher the chance it overlooks the single detail that mattered, just as a person struggles to spot one important line in a long document.

Does a larger context window solve the problem?

It helps by providing room, but it does not remove the underlying issue. Information in the middle of long inputs is still used less reliably, and latency and cost both rise as input grows. Curating what goes in stays worthwhile even with very large windows.

Key takeaways

• Treat the model's input as a designed artifact assembled for each call, not as an append-only log.
• Manage all four components deliberately: instructions, retrieval, memory and tools each fail in their own way.
• Retrieve broadly, rank aggressively, summarize history, and place critical content at the start or end.
• When an agent degrades over many steps, inspect what it was given before rewriting what it was told.
• A bigger window buys space, not immunity; the discipline still comes down to handing over the right three files instead of the whole codebase.