Agent Harness Engineering: State, Verification and Control Around the Model

When a strong model still fails

Picture an agent with the best model on the market, a carefully tuned system prompt, dozens of tools and a context window big enough for a whole codebase. It can still fall over on a job any competent engineer would call routine. It picks the right action and carries it out wrongly. It finds the key fact, then loses it when the context resets. It edits ten files correctly and slips a quiet regression into the eleventh. It burns an hour debugging a problem it created. It may even write a convincing report saying the work is finished when nothing was checked.

The usual reaction is to work on the prompt: another instruction, another example, more context, a plea to be careful. That helps for a while and then it doesn't. The deeper flaw lies in the mental model, which treats the model as the agent and everything around it as settings. That view falls apart once an agent has to run continuously in a real environment.

A production agent isn't just a model with a prompt around it. It's a runtime system: it holds state, runs actions, talks to outside systems, meets failures, takes in feedback and needs ways to recover, verify and be controlled. The discipline forming around that runtime is called agent harness engineering. Compare it to prompt work. Prompt engineering asks how to instruct the model to behave, while harness engineering asks how to build an environment where the model can dependably reach a goal. That's no improved prompt. It's a different abstraction, one that covers what the model writes and also what the system can do, what survives a failure, what's permitted and what counts as finished.

A runtime around a probabilistic model

The simplest picture of an agent is a model with a prompt and a few tools. That's fine for understanding a single inference and falls short for a production agent, which is better seen as a runtime built around a probabilistic model.

The model does the reasoning. The harness decides the conditions under which that reasoning turns into execution: what the model sees, what it can reach, which actions are allowed, what state persists and what evidence is needed before success can be declared.

Take a coding agent whose prompt says to inspect the architecture first, run the tests and leave unrelated components alone. Those lines state intent and enforce nothing. A harness can make specific files writable and others not, run validation after each change, block forbidden operations, save progress and refuse to close the task without evidence. So the prompt says what the agent should do, while the harness decides what the system will permit and what counts as done. The model supplies intelligence, and the harness turns it into something that can be operated.

Three layers of engineering

Agent engineering has moved through three widening abstractions: prompts, then context, then the harness.

Prompt engineering asks what the model should do. It improves instructions, examples, tool descriptions and behavioral limits, and for simple tasks it works remarkably well. But a prompt is an instruction and has no power to enforce. An agent told to run tests can still claim success without running them. Telling it to leave production files alone means little if the runtime hands it unrestricted access. Asking it to remember a decision for six hours pits that request against everything else flowing into the context.

Context engineering handles the next problem, which is what the model should know at the moment of decision. Retrieval, skills, memory, tool results, structured state, progressive disclosure and compaction all build a better decision context, and the goal moves from more context to better context. Even so, the model can choose a wrong action, corrupt state, skip validation, get stuck in a loop or declare success too soon.

Harness engineering begins at that point, with the question of what system should surround the model so the task gets done reliably. It supplies what prompts and context can't provide alone: durable state, constrained tools, permissions, isolation, verification, recovery, evaluation and observability. It also changes the engineering loop. A failure no longer just calls for rewording the prompt. It raises the question of which control was missing, and the answer might be a validator, a stricter tool contract, a policy boundary, a checkpoint, a recovery strategy or better visibility. Every failure ends in a change to the system.

The result is a harness ratchet: failure, then a missing control, then an improvement to the harness, then verification, then a stronger baseline. Prompt engineering improves what the model is told, and context engineering improves what it knows. Harness engineering improves the conditions the model works under and makes the system steadily harder to fail the same way twice.

What the runtime is responsible for

If the harness is the runtime, what does it hold? Rather than listing prompts, tools, memory and orchestration code, it's more useful to name runtime responsibilities.

• Context decides what the model needs to know for its next decision: instructions, retrieved material, skills, relevant tool output and current observations.
• State keeps what must outlive context resets and process boundaries: progress, decisions, artifacts, checkpoints and validation results.
• Actions define how reasoning touches the environment through APIs, databases, filesystems, code execution, MCP servers and similar capabilities.
• Control determines which actions are allowed, through authorization, policy, sandboxing, resource limits and other enforced boundaries.
• Feedback reports what actions caused, through tests, validators, runtime signals, health checks and evaluators.
• Recovery defines how work continues after a failure, with retries, replanning, checkpoints, resumability or escalation.
• Observability keeps the trajectory needed to explain what happened: intent, context, actions, policy decisions, state changes, failures, verification and outcome.

These aren't seven independent features, and they don't have to be seven services. Together they form a control loop around a probabilistic model. Once that loop runs for hours instead of minutes, surviving context limits, failures, retries and changes of worker, the architecture starts to look like something engineers already know well, a distributed system.

The context window isn't a state store

One of the easiest mistakes in agent design is treating the context window as memory.

That's a tempting habit. The model can see the whole conversation, earlier tool calls, decisions and intermediate results, so appending more seems harmless. Yet the window is a working set and not a database. Nothing in it is durable: a restart, a truncation or a compaction pass can shrink it or replace it outright. Enlarging it adds capacity and leaves durability exactly where it was.

For a long-running agent, the authoritative task state has to live elsewhere: the goal, finished and pending work, decisions, artifacts, checkpoints, validation results and pointers to outside resources. Context is then rebuilt from that state according to what the model needs right now. The distinction is simple: context is what the model sees, and state is what the system knows.

Compaction makes this obvious. An agent that spent three hours changing a repository may pile up hundreds of tool calls, abandoned approaches and passing observations. Squeezing that history can make the next context usable, but a summary is still an interpretation of what happened, so critical state shouldn't hinge on what the compactor kept. The harness can instead assemble a fresh context from durable state, in this order: task state, relevant artifacts, current observations, skills, and finally the model's context.

Checkpoints apply the same principle across process crashes, and structured handoffs apply it across agents or workers. A successor needs the state required to continue correctly, not its predecessor's whole conversation. The architectural question becomes what information must survive independently of the context, and once that's asked, context management turns into a state-management problem.

Long-running agents are distributed systems

When an agent runs for hours or days, the familiar request-and-response model stops working. A process can die. A tool can time out. A worker can vanish after finishing an operation but before reporting it. Two workers can pick up the same task. A model's context can disappear halfway through. None of this is exotic, because these are classic distributed-systems problems, and a long-running harness needs explicit execution semantics to handle them.

If work can outlive the worker, ownership needs limits. A lease lets a worker claim a task for a bounded time, and heartbeats give an outside observer a liveness signal. When the worker disappears, the lease expires and another can pick up from durable state.

Retries call for equal care. Imagine an agent that submits a deployment and the call times out before any result is seen. A blind retry could run the deployment twice. So agent tools need the guarantees found elsewhere in distributed systems: operation identifiers, idempotency, explicit failure states and a clear owner for retry behavior.

Then there's resuming. A long task should be represented as durable progress and not as an in-memory conversation, following a cycle of checkpoint, execute, observe, persist and continue. If execution stops midway, the harness has to work out whether the current step is safe to retry, already committed, partly done or in need of compensation. Resuming doesn't mean replaying everything. It means rebuilding the minimum state required to continue safely. Handoffs follow the same pattern: a new worker gets the objective, the decisions, the artifacts, the validation state and the outstanding work, and nothing of another worker's chat history.

That sets the central reliability boundary. Once an agent can outlive its process, the process is no longer the unit of reliability. The durable task is.

Verification is the other half of agency

An agent can write code, run a deployment, build a database query or change infrastructure. Generating something isn't the same as getting it right, and the risky pattern is letting the agent be the last word on whether its own work succeeded, as in a message saying the authentication flow is done and all tests pass. The harness shouldn't accept that as evidence. It should run the tests. In the same way, a claim that the service is healthy should trigger health checks and not end the task, and a data migration should be judged against schema and data invariants instead of the model's reading of its own tool calls. An agent's belief that it succeeded isn't a dependable completion signal.

That's why production harnesses separate three roles. The planner decides what has to happen. The executor does bounded work. The evaluator decides whether the resulting system state meets the requirement. One model can fill all three roles, but their responsibilities stay distinct.

Verification should be anchored in things that can be observed, and deterministic evidence is the first choice wherever it's available: compilers, test runners, schema validators, health checks, deployment status, policy checks or plain assertions. Model-based grading is useful when correctness is inherently semantic, but it needs concrete criteria and not a vague question about whether the result looks good. Completion then becomes a property of the system and not something the model says: requirements met, validation passed, required artifacts present, policy checks cleared, and only after that is the task done. Put differently, verification yields evidence and the evaluator reads it to decide whether execution may move forward, so the evaluator acts as the sensor in the harness's feedback loop. The agent drives progress while the harness establishes the evidence.

From instructions to enforcement

An agent can be told not to do something without being unable to do it. Lines like "do not modify production," "only write inside the workspace" and "never expose credentials" are useful, yet none is a security boundary if the runtime still allows the operation. Instructions communicate intent, and enforcement establishes invariants.

Tool design shows the gap clearly. Hand an agent a generic execute_sql() and its action surface is nothing like that of an agent given a scoped, read-only query interface. The scoped one leaves room to check authorization, validate parameters, audit operations and shrink the blast radius. MCP tools, APIs, filesystems, shells and outside services work the same way: the model proposes, and the harness decides whether the action may happen and under which identity, permissions, environment and policy.

A handy rule of thumb sorts constraints by cost:

• If breaking a constraint is tolerable, an instruction may be enough.
• If breaking it is costly, add verification.
• If breaking it is dangerous, enforce it at the boundary.

Sandboxing brings in a second dimension. Authorization settles whether an action is permitted. A sandbox settles what happens when the agent misbehaves regardless. Boundaries on the filesystem and network, resource limits and narrowly scoped credentials keep the fallout from unexpected behavior small. More mature harnesses also enforce workflow transitions, not just single tool calls. A production deployment could stay out of reach until tests pass, staging is healthy and someone has approved it. The model can suggest the move, but the runtime rules on its legality. That's closer to a state machine than to trusting a line of natural language. The point isn't to restrain intelligence. It's to restrain capability and blast radius.

You can't operate what you can't observe

Ordinary observability asks whether a service is healthy, where latency is rising and which dependency is failing. Agent systems need another layer. When an agent fails, the question is what happened and also why its trajectory led there. What was it trying to achieve? What context did it get? Which tools did it call? Which policies allowed or blocked those calls? What state changed? What evidence convinced it that it was done? That's trajectory observability.

Standard LLM metrics such as tokens, latency, cost, model version and inputs and outputs are necessary and insufficient, because one model call is a single event inside a larger chain: intent, context, decision, action, observation, state, verification and outcome. Suppose an agent spends forty minutes migrating a service and then fails. Learning that it consumed 80K tokens says almost nothing. A useful trajectory might show that the harness retrieved an outdated migration guide, the agent chose a deprecated API, the deployment technically succeeded, integration checks failed and recovery retried the very same strategy. That's an actual diagnosis.

That means context needs provenance: a way to see which skills, documents, state and tool results shaped a given decision. Tool calls need causal context as well, covering the task step that triggered them, the identity they used, the policy that was evaluated and the aftermath. Policy decisions and state transitions should be visible too, along with the actions that succeeded. The most telling metrics turn out to be behavioral: task success rate, verification failures, recovery success, premature completion, policy denials, recurring failure patterns, human escalations and steps per successful task.

Observability is what completes the harness ratchet. A failure should leave more than a log entry. Its trajectory ought to make it possible to classify what went wrong and name the control that was absent, be that a context source, an evaluator, a policy, a tool boundary, a checkpoint or a recovery strategy. The target is a system that can't fail in the identical way twice.

A new systems discipline

Asked how to get agents better at work, the answer widens as autonomy grows. Prompt engineering tunes the instruction. Context engineering tunes the information available at decision time. Harness engineering tunes the whole execution trajectory: state, actions, boundaries, feedback, recovery, verification and observability. They don't replace one another and are simply wider and wider scopes.

A helpful split of duties anchors the whole thing: the model handles reasoning under uncertainty, and the runtime handles invariants. A file that must never change gets a filesystem boundary. A production system that must not be written gets authorization checks. A migration that has to succeed gets verified against the resulting state, and task state that must survive a restart gets persisted. Guarantees the runtime can supply more dependably shouldn't be requested from the model.

Production agents are going to fail, and the sign of a mature harness is what it does next. Each failure feeds back into the runtime. If the same mistake keeps happening, it earns its own evaluator. If an action proves risky, the capability behind it gets narrower. If progress was lost, a checkpoint appears where it was missing, and if recovery keeps failing, an escalation policy takes over. Execution produces evidence, the evidence exposes absent controls, and those controls make the following run sturdier.

It also changes how agents get judged. The question stops being how good the model is and becomes how reliably the whole system completes this class of work. Harness engineering isn't one more orchestration framework. It's a systems discipline for making probabilistic intelligence dependable. A demo proves an agent can do something once. A production system has to do it reliably, verifiably, recoverably and repeatedly, with each failure improving the next run. Much of the hardest work now sits beneath the prompt, in the architecture of state, feedback, verification, recovery and control loops that turn agent reasoning into lasting system behavior.