Why AI Agents Fail in Production: Harness Engineering Explained

Picture a developer at 2 a.m. who has spent three weeks rewriting prompts, trying the newest flagship model and fiddling with RAG chunk sizes. In a sandbox the agent looks great. In production it falls apart: it forgets a decision made two steps earlier, announces success just before handing back something broken, and its task success rate sits at 68% no matter which model is plugged in. Pushing past 70% seems impossible.

Plenty of teams recognise that scene. The usual diagnosis is that the model isn't good enough. The better diagnosis is that the problem lives in the system built around the model, and until that's accepted, no model swap will change much.

Three stages of working with LLMs

Thinking about LLM applications has moved through three stages, and many teams are still stuck at one or two while hitting stage-three problems.

Stage one: prompt engineering

Everyone begins here, asking whether the model understood the request. LLMs are extremely sensitive to how they're steered. A well-chosen role, a few examples and a firm output format can transform results, and for contained, single-turn tasks good prompting is indispensable.

It also hits a wall. However elegantly instructions are phrased, they can't give the model facts it lacks, make it remember what happened three tool calls back or stop it from inventing data when reality disappoints. The common trap is believing a more articulate prompt can stand in for missing grounding or live context. It can't.

Stage two: context engineering

The next question is whether the model has the facts. Teams add RAG pipelines, dynamic retrieval, tool outputs fed back in and carefully injected conversation history, and start caring about what the model can see when it decides. This is a real step forward, and agents get noticeably smarter when the right information arrives at the right time.

What context work doesn't fix is execution drift. The agent drafts an excellent plan, carries out step one correctly, misreads a tool's return value at step two and drifts off course for a dozen more steps. Nothing in the system notices. It keeps executing a plan that turned wrong long ago.

One frequent error here is treating context engineering as the same thing as vector-database RAG. Real context management also covers injecting state dynamically, summarising tool responses and truncating history on purpose. RAG is the beginning.

Stage three: harness engineering

The question now is whether the model can keep acting correctly over time, and the answer is a bit humbling for anyone who hoped a better model would do it. Harness engineering is the practice of building the scaffolding around a model: deterministic systems that watch what it does, catch its mistakes, enforce limits and pull it back when it wanders. The word comes from reins and safety tethers, equipment for control.

A mental model: agent equals model plus harness

A framing popularised by LangChain puts it plainly: an agent is a model plus a harness. Nearly everything in a codebase that makes an agent work in production, other than the call to the foundation-model API, belongs to the harness.

Think of sending a junior colleague to run an important client meeting.

• Prompting is the agenda: say hello, pitch the product, ask about requirements.
• Context is the folder of background: client history, the price list and the goals for the meeting.
• The harness is the rest: the checklist they carry, a check-in with you halfway through, a recorded transcript, a way to correct them if they stray and clear acceptance criteria for the meeting report.

Better briefing can't make up for a missing accountability structure. The first two stages help a model think better, and the harness makes it act reliably. One team's account, which is an anecdote and not a benchmark, says that reworking task decomposition, state handling, validation of critical steps and failure recovery took the same model with the same prompts from about 70% to over 95% task success.

Six layers of a mature harness

A harness isn't one file or a clever wrapper. It's layered, and each layer covers a different kind of failure.

Layer 1: information boundaries

What the model sees in its immediate context shapes performance more than almost anything. Extra data doesn't make it smarter, it makes it lose focus. Mixing system rules, current task state and outside evidence into an unstructured lump is worse, because the model starts dropping constraints and critical rules become background noise. The harness should explicitly decide and label what the model gets: its role, current objective, success criteria and a clean separation between types of information.

Layer 2: the tool system

Without tools an LLM only predicts text. With a good tool system it can act in the real world. A typical early mistake is handing it too many. Fifteen thoroughly documented tools sounds powerful, but attention gets spread thin and the model starts making up parameters that don't exist or misusing APIs it barely understands.

The harness should govern when tools get used, not just which ones exist. It should stop the model from guessing when it ought to search, and from searching when it already knows. And one rule has no exceptions: don't pipe raw tool output straight back to the LLM. A 50-item JSON reply from an API poisons the context. Filter, parse and summarise results before the model sees them.

Layer 3: execution orchestration

Models often fail not for lack of individual skills but because they can't chain them in order. They run in a stream of consciousness, hopping between steps, skipping checks and writing output before they have what they need. The harness sets fixed rails: understand the goal, assess what information is held, fetch what's missing, analyse, generate, verify, then output.

That moves project management from a probabilistic model to a deterministic system. The model shouldn't be deciding the order of work, and that structure belongs in the harness.

Layer 4: memory and state

A stateless agent has amnesia on every turn, as if each step of a multi-step job were a new conversation. A workable approach keeps three kinds of memory strictly apart:

1. Current task state, meaning which step is active, what's pending and what's confirmed.
2. Intermediate conclusions from this session.
3. Long-term memory and user profiles, such as preferences and context that last across sessions.

The mistake that hurts most is merging task state with conversation history. You end up with an ever-growing, shapeless context window that makes the model worse as the task goes on.

Layer 5: evaluation and observability

Primitive agents collapse here. They produce output, announce success and have no way to know whether it's right. An agent grading its own work has a built-in optimistic bias: it calls broken code working and rates a reply as satisfactory when it never answered the question.

The harness needs independent, automated checks. Human review after the fact is too slow and doesn't scale. This layer holds automated output validation, test environments, detailed logging, metrics and attribution of errors. The system has to keep demonstrating that its actions are right and never simply assume it.

Layer 6: constraints, validation and recovery

Failure is the normal state in production. APIs time out, JSON breaks and search results are off. An agent with no recovery path needs a full human restart after every error. Three elements are needed:

• Constraints: hard-coded rules for what the agent must never do.
• Validation: gating checks before and after output, such as schema validation, format checks and verification of constraints.
• Recovery: retries, fallback paths and rollback to the last known stable state.

Context anxiety

On tasks that run many dozens of steps, something odd shows up, and researchers at Anthropic have given it a name: context anxiety. As the context window nears its limit, models drop fine detail, lose sight of core goals and seem to hurry to finish. They invent conclusions they never reached and skip verification, urgent in a way that makes them sloppy.

The obvious remedy is compression: summarise the history, inject the summary and carry on. It cuts token count but doesn't reset the model's cognitive state or attention dilution. What works is more drastic, restarting the agent completely. When context grows too large, the compressed summary goes to a brand-new agent instance with a clean context and none of the accumulated confusion. It's like fixing a memory leak by restarting the process and not by frantic garbage collection. The label used for this approach is context reflect.

Tool exposure benefits from a similar idea called progressive disclosure. Instead of loading the whole tool library at the start, the model sees only minimal stubs. When it signals that it wants a particular capability, the harness injects the detailed documentation and parameter schema then. Optimising context isn't about giving the model more. It's about giving it the right material on demand at the moment it's needed.

Splitting generation from evaluation

One of the most useful ideas comes from how Anthropic builds autonomous agents that produce complete working products for hours with no human review. The core is a strict three-way division:

• The planner turns loose human requests into precise engineering specifications.
• The generator takes those specifications and carries them out step by step.
• The evaluator works as an independent QA role, functionally separate from the generator.

An evaluator doesn't just read the generator's code. It interacts with the result. For UI work it clicks through the interface, examines layout and tests interactive states. It checks against the real world and not against the generator's description of the real world.

OpenAI went further. Once agents were writing code faster than humans could review pull requests, they gave the agent its own automated CI/CD pipeline. The agent executes its work inside isolated sandboxes, takes screenshots through headless browsers, goes through its own logs, and keeps looping until the deployment checks out, again with nobody watching. "Done" stops meaning that the text was generated. It means the code ran, the logs were checked, a bug was found and fixed and the deployment was verified in the sandbox.

Where the ceiling really is

A foundation model's intelligence sets the theoretical limit on a benchmark leaderboard. How well the harness is engineered decides whether that intelligence can survive, recover and deliver value in messy real conditions. Past a certain level, the model isn't the bottleneck. The distance between a 70% success rate and a 95% one, which is also the distance between a demo and a product, sits in the harness.