Prompt Engineering Isn't Enough: Context Engineering for Multi-Agent Systems
Multi-agent systems rarely fail inside one step. They fail between steps, when the wrong context arrives. How to design state and handoffs in LangGraph.
This article is published in English.
Sources: Prompt Engineering Is Dead for AI Agents — Here Is What Actually Works — Context Engineering
The phrase context engineering has been defined as the craft of filling the context window with exactly the right information for the next step. For anyone who has watched multi-agent systems fall apart in production, it names something that previously had no label.
The typical failure profile is the same everywhere. Prompts are fine, models are capable and the logic looks clean in a notebook. Then the system meets production and breaks, not inside individual steps but between them. Picture a healthcare summarization agent that retrieves exactly the right documents, followed in the graph by an agent that receives a stripped-down state object and invents patient history that was sitting three nodes upstream. The model didn't fail. The handoff of context did. Weeks of system-prompt tuning won't fix something that was never a prompt problem.
Put simply, prompt engineering tunes the question, and context engineering tunes the conditions the question gets answered under. Agents mostly don't fail at their steps. They fail between them, because each node is holding the wrong information.
A problem that has no name until it bites
Most teams build a first multi-agent system the same way. They spend their effort on the system prompt, with clear task descriptions, output formatting rules and a few examples, and the lone agent performs well. Then three agents get chained and quality drops. Agent two has no idea what agent one decided, agent three fills in context that agent two never produced, and by step four the answer is delivered with total confidence and total wrongness, since nothing reached it about what really happened upstream.
It resembles passing a relay baton without telling the next runner where the race has been or which lane to hold. The runner is capable, and the handoff is the issue.
Nor is this simply a model-capability problem. A study from Chroma published in July 2025 tested 18 LLMs, among them Claude 4, GPT-4.1 and Gemini 2.5, and found that performance degrades unevenly as context grows, even on trivially simple retrieval tasks. Attention has a cost. What enters the window, and what is left out, shapes almost everything that comes back. Prompt engineering never addressed this and wasn't built to. A well-crafted system prompt says what to do, while context engineering supplies what the model needs to know at this moment to do it well.
What the discipline actually is
A clear definition: designing dynamic systems that give the model the right information, in the right format, at the right time, so it has everything needed for the next step. It's infrastructure work and not linguistic work. The aim isn't prettier sentences but deliberate control over what flows through the agent graph at each transition.
In a LangGraph project, the biggest improvement often comes from the AgentState TypedDict, not any prompt. Stop treating state as a scratchpad and treat it as a contract in which every node declares what it reads and what it is responsible for writing. Downstream agents stop hallucinating because they no longer guess what happened earlier. It's in the state, structured and retrievable.
Three layers matter in production.
Persistent context is what the agent should always know: the user's goal, their constraints, decisions from earlier in the session and domain rules. In a healthcare pipeline that would be patient context, regulatory limits and risk thresholds, injected once when the graph starts and never dropped.
Time-sensitive context is what is true now but will change: retrieved documents, current tool outputs, API responses and the result of the latest reasoning step. RAG belongs here, as one layer of a larger architecture and not the whole answer. Retrieval is the easy part. The hard part is deciding which retrieved content moves forward, how compressed and in what format.
Transient context is what the current step needs and the next doesn't: raw API payloads, verbose intermediate reasoning and tool-call logs. Drop it actively instead of hoarding it, because every needless token competes for attention with the tokens that matter.
Production bugs usually fall into one of three buckets: the persistent layer was never supplied, the time-sensitive layer aged out between steps, or the transient layer wasn't cleared and fouled later reasoning.
Context in a LangGraph pipeline
Here is a stripped-down way to structure state for a multi-agent research pipeline. The code matters less than the discipline behind it.
from typing import TypedDict, Optional, List
from langgraph.graph import StateGraph
class ResearchAgentState(TypedDict):
# Persistent context — set once, never overwritten
user_goal: str
domain_constraints: List[str]
session_id: str
# Time-sensitive context — updated by retrieval/tool nodes
retrieved_docs: List[str]
current_findings: str
last_tool_result: Optional[str]
# Transient context — cleared after use
raw_api_payload: Optional[str] # cleared after parsing
intermediate_reasoning: Optional[str] # cleared after synthesis
Each node receives this state and updates only its own declared slice, while an orchestrator at the top decides which sub-agents fire and which subset of state they receive. That's context routing. A coding sub-agent doesn't need medical literature fetched by a domain specialist, and a summarizer doesn't need a data fetcher's raw payload. Giving every agent everything is like briefing the whole company before each meeting: technically complete and operationally disastrous. Routing context by role cuts token use and, more importantly, reduces hallucination in specialist agents, which are no longer distracted by irrelevant material competing for a finite window.
The Model Context Protocol is speeding this pattern up. By late 2025 there were reportedly more than 10,000 public MCP servers, a standard interface through which agents query tools, databases and outside systems in structured form. Architecturally it moves context sources outside the agent so it can pull what it needs instead of receiving everything preloaded. Think of a library versus a briefing document: the agent goes to the shelf for a specific book and doesn't carry the library around.
Quality matters more than volume
A 2026 survey reported that 82% of IT and data leaders agree prompt engineering alone no longer suffices for production AI, and 95% of data teams plan to invest in context engineering capability that year. Yet most discussion fixates on volume: bigger windows, more retrieved chunks, more tools. The real bottleneck is quality.
Stale context is worse than none. An agent acting confidently on yesterday's retrieved document, unaware that it has been superseded, produces errors that look like good reasoning. In natural-language-to-SQL pipelines, schema drift is a frequent culprit: the generated SQL is correct for a schema that no longer exists. The prompt was fine, and the context was wrong.
Ungoverned context is also a security problem. If agents can reach everything in state, including data from other users, earlier sessions and sensitive tool output, a context leak is one prompt injection away. Least-privilege context design is a necessity, and with 86% of CISOs reportedly lacking an enforcement policy for agent access, context governance is where the next wave of enterprise AI incidents is likely to start.
Limits
Context engineering doesn't remove the underlying attention limit. The Chroma results showed reliability falling as input length grows even when context is well structured, so better context buys more dependable behavior and not unlimited reliable context. When context is truly massive, such as autonomous agents running for hours or synthesis across hundreds of documents, pair it with aggressive summarization, hierarchical retrieval and explicit memory management. Context engineering decides what goes in the window, and memory architecture handles what happens when the window isn't enough.
It also can't replace evaluation. A beautifully structured pipeline that keeps retrieving the wrong documents will fail with confidence, so the retrieval layer, memory layer and routing logic each need their own evaluation, separate from output quality. And it suits well-scoped tasks with stable context needs. For open-ended exploratory agents, where relevant context can't be known in advance, dynamic retrieval that signals uncertainty to the model fits better.
Where to begin
Open the AgentState definition and ask three things. What does each node assume is already in state? What does it actually receive? What gets dropped before it is needed? The distance between assumption and reality is where the agent is failing right now, and it isn't the system prompt.
The best upgrade to a multi-agent system already running is rarely a newer model. It's an audit of what the window holds at each step: remove the misplaced items, and make sure what remains is up to date, well structured and limited to what the receiving agent needs. Start with one node, audit its inputs, and the reliability gains begin and compound quickly.