Designing the Agent Harness: Policy, Sandboxing, Memory and Verification
Learn which runtime layers turn a tool-calling LLM into a dependable agent: execution policy, OS sandboxing, persistent state, verification, context control and cost.
This article is published in English.
Sources: Agent Harness Engineering: The Missing Layer Between an LLM and a Real AI Agent
Teams that set out to build an AI agent usually begin by comparing models: context window sizes, benchmark scores, reasoning quality, price per token. Once a model is chosen, a handful of tools gets wired in, and the result is called an agent. What actually exists at that point is a model that can call functions, and the gap between that and a production agent is filled by a runtime layer now commonly called the agent harness. This guide walks through the layers of that harness, what each one protects you from, and how to grow one incrementally without drowning in framework abstractions.
A tool-calling model is not yet an agent
The typical starting kit looks like a short list of capabilities exposed to the model.
search()
read_file()
write_file()
run_command()
With those four functions the model can search, read, write and run commands. That feels like autonomy, but nothing in this setup decides what the model is allowed to touch, which operations need a human to sign off, what information survives a crash or restart, how the output gets checked, or how much of each tool result is fed back into the prompt. All of those responsibilities belong to the harness.
The harness also turns out to be a measurable variable. At the time of writing, a 2026 study that compared coding-agent harnesses reported differences of up to 40x in tokens spent per solved task, while the gaps in pass rate between the same harnesses were much narrower. The takeaway is not that a clever runtime makes a weak model brilliant. It is that the runtime strongly shapes how efficiently, safely and predictably a capable model gets its work done.
The loop at the center and the questions around it
Reduced to its essentials, an agent is a loop: ask the model what to do, execute the requested tool, append the result to the context, and repeat until the model says it is finished.
while True:
response = model(context)
if response.is_final:
break
result = execute_tool(response.tool_call)
context.append(result)
Many real agents have something very close to this at their core. The model makes a decision, the runtime carries it out, the observation flows back, and the model picks the next step. The loop itself is trivial; the engineering effort lives in everything that wraps it. A usable harness has to answer a long list of operational questions on every turn.
Can the agent execute this command?
Can it access this directory?
Does it need human approval?
What happens if the command fails?
What information should return to the model?
What should survive the session?
Should another agent review the result?
How much context should we send?
How much will this turn cost?
What happens if the model loops forever?
Taken together, those answers are the harness. One recent conceptual paper frames it as the runtime that connects a language model to the outside world through tools, context management, safety controls, orchestration and extension points. A convenient mental picture splits the agent into two halves.
AI Agent
│
┌────────┴────────┐
│ │
Model Harness
│ │
reasoning ┌──────┼───────┐
planning │ │ │
generation Tools Memory Safety
│ │ │
Sandbox Context Verification
On one side the model supplies reasoning, planning and generation. On the other side the harness supplies tools, memory and safety, backed by a sandbox, context handling and verification. The model contributes intelligence; the harness governs how that intelligence touches real systems.
Why a prompt cannot be your safety mechanism
Suppose the system prompt contains a clear rule:
Never delete a production file without asking me first.
There is nothing ambiguous about that sentence, yet it remains a request. The model may misread it, lose track of it deep into a long session, weigh it against conflicting instructions elsewhere in the context, convince itself that the current case is an exception, or achieve the same destructive result through a different tool that the rule did not obviously cover. Asking an agent to avoid something is fundamentally different from making that thing impossible without authorization. A prompt expresses a wish:
Please don't do X.
A harness can express a rule that is evaluated in code:
if dangerous_action and not approved:
deny()
The second version works whether or not the model remembers the instruction. That gives the first principle of harness engineering: whenever a rule is deterministic, implement it in deterministic code. Claude Code draws the same line in its design. Hooks run on lifecycle events and tool calls and can, for instance, reject a tool call before it executes, while files such as CLAUDE.md provide guidance that the model reads as context. If you are writing such a file, keep it for guidance and move enforcement into hooks; our playbook for an effective CLAUDE.md covers what belongs there.
Layer 1: an execution boundary between request and action
The first question the harness must settle is which actions the agent may perform at all. Consider a tool that removes a file.
def delete_file(path):
os.remove(path)
The model can invoke it with any argument it likes.
delete_file("important.md")
If that call runs immediately, the model holds unrestricted authority over whatever the function can reach. The fix is to route every call through a gatekeeper that checks policy before dispatching.
def execute(call):
if call.name == "delete_file":
if not call.args.get("human_approved"):
return "DENIED: human approval required"
return dispatch(call)
Now the model can still ask for a deletion, but asking no longer equals doing. The flow becomes a policy decision with two outcomes.
Model decision
↓
Policy check
↓
Permission?
↙ ↘
No Yes
↓ ↓
Deny Execute
This mirrors conventional security design: the model proposes, the runtime disposes. One detail worth tightening in real code is where approval comes from. In the sketch above the approval flag is read from the call's own arguments, which the model generates. A production harness should obtain approval out of band, from a human confirmation step or a signed token the model cannot fabricate, otherwise the model can simply set the flag itself.
Why tool-level checks leak
Tool policies are necessary but not sufficient. Imagine the harness refuses this call:
delete_file("secrets/api_key")
That looks like protection, until the agent runs the equivalent shell command instead:
rm secrets/api_key
The delete_file policy never sees that operation. Or the harness blocks a read of the secret path:
read("secrets/api_key")
but a relative path slips through because nobody normalized it:
read("../secrets/api_key")
Symlinks, helper programs that open files on the agent's behalf, and interpreters that accept arbitrary code all create similar side doors. A policy expressed in terms of tool names and string arguments cannot enumerate every route to a resource, so it is not a complete security boundary. OpenAI's guidance on running coding agents accordingly recommends combining sandboxing, approval policies, constrained execution, network controls and telemetry instead of leaning on prompts. The principle to keep: the harness enforces policy, and the operating system enforces the final boundary.
Layer 2: sandboxing what the agent can physically reach
Policy asks what the agent is permitted to do; a sandbox asks what it can reach in the first place. The weak approach is a denylist that tries to name every sensitive location.
Don't access:
- /etc
- ~/.ssh
- secrets/
- production/
- credentials/
- ...
Such lists are never finished. The stronger approach inverts the default: grant nearly nothing, then add exactly what the task needs, such as a single project workspace.
Agent workspace:
/workspace
├── src/
├── tests/
├── pyproject.toml
└── README.md
The agent receives /workspace and nothing else on the host. Expressed as policy, it is an allowlist.
Allowed:
/workspace
Everything else:
unavailable
Because the rest of the filesystem simply does not exist from the agent's point of view, path tricks, symlinks and helper processes lose most of their power. Containers, dedicated virtual machines and restricted user accounts are common ways to implement this. OpenAI's Agents SDK added native sandbox execution for exactly this purpose, so agents can install dependencies, edit files and run commands inside a controlled environment.
Network and resource limits belong in the sandbox
Isolating the filesystem does not finish the job. An agent that can run something as ordinary as this has a very different threat profile:
curl https://example.com
With outbound network access the agent can:
- download and run arbitrary software
- exfiltrate data to an external host
- call third-party APIs
- reach internal services that trust the local network
- ingest content from untrusted websites, which is also the classic path for prompt injection
A sandbox worth the name therefore covers several dimensions, not just files.
Filesystem
Network
Processes
Environment variables
Secrets
CPU
Memory
Time
Processes inherit environment variables and secrets silently, and CPU, memory and time limits stop runaway builds. That breadth is why modern runtimes design the sandbox as a complete execution environment, not a thin shell wrapper.
Layer 3: persistent state that survives the session
A different failure appears on long tasks. An agent spends an hour exploring a repository and learns an important architectural rule.
The authentication service must never directly access the database.
All database operations go through Repository classes.
Then the context window fills up, the session is restarted, and that knowledge is gone. The next session has no idea why the design looks the way it does and may happily violate it. Conversation history is the wrong home for this kind of fact. A useful harness separates at least three categories of information with different lifetimes.
Conversation state
↓
temporary
Project state
↓
persistent
Harness configuration
↓
persistent + globally enforced
A concrete example of each:
Conversation:
"Let's investigate the login bug."
Project:
"Authentication uses OAuth."
Harness:
"Never modify generated migrations."
The conversation item can safely vanish when the task ends. The project fact should persist across sessions, for example in a versioned notes file or a memory store. The harness rule should be applied on every run, regardless of what the model currently remembers, which usually means it lives in configuration and enforcement code rather than in text. As sessions get longer this separation matters more. OpenAI's Agents API, for instance, includes context-management features aimed at long-running agents, among them automatic compaction as a session nears its context limit. Compaction is exactly the moment when unrecorded project facts get lost, so persisting them before that point is worth doing deliberately. For a deeper treatment of memory tiers, see four-tier agent memory with LangGraph and Bedrock.
Layer 4: verification that the agent does not own
Agents can be wrong with total confidence, so the agent that produced a change should not be the last word on whether it is correct. Take this output:
def calculate_total(items):
return sum(item.price for item in items) + 1
The stray + 1 may look intentional to the model that wrote it. An independent verification step can evaluate the change against several signals.
Code
Tests
Type checking
Linting
Security rules
Expected behavior
What makes this valuable is the separation between producing and judging.
Agent
│
↓
Changes
│
↓
Verification
↙ ↘
PASS FAIL
↓ ↓
Continue Investigate
A pass lets the loop continue; a failure sends the agent back to investigate. One refinement is easy to overlook: the reviewer should report problems rather than quietly repair them. If the verifier silently fixes the bug, you lose the signal that the original agent failed, and your success metrics become fiction.
Dry runs as the default for risky changes
For higher-stakes operations, make previewing the default behavior. Instead of an apply function that acts immediately:
apply_change()
expose one whose safe default is a dry run:
apply_change(dry_run=True)
The agent first receives a description of the effect.
Would modify:
src/auth.py
tests/test_auth.py
Only an explicit, separate step, ideally one that requires approval, moves from preview to execution. This gives both the verifier and any human reviewer something concrete to inspect before anything changes.
Layer 5: context engineering
Every token placed in the context costs money, competes for the model's attention, and on long tasks tends to accumulate faster than it gets cleared. Suppose the agent needs one function out of a 5,000-line file. A naive harness sends the entire file, then the full output of the next search, then the next tool result, and so on. Before long the context contains all of this:
Task
+ entire repository
+ previous searches
+ tool output
+ logs
+ previous reasoning
+ repeated instructions
The model can technically see every token, but visibility is not comprehension. A better design treats context as a pipeline that condenses information before it reaches the main agent.
Main Agent
│
├── Search Agent
│ │
│ └── 50,000 tokens
│
│ ↓
│
└── 1,000-token summary
Here a search agent may burn 50,000 tokens exploring, but the main session receives only a 1,000-token summary of what matters. This is a strong argument for subagents. Claude Code, for example, describes subagents as running in their own isolated context windows and returning only their final result to the main conversation, which keeps large side investigations from crowding the primary context. The trade-off is that a summary can drop a detail the main agent later needs, so summaries should include pointers (file paths, line ranges) that let the agent fetch specifics on demand.
The economics: price per token is not cost per task
Context management is also a budget question. Recall the study that found up to a 40x difference in tokens per solved task across harnesses, with much smaller differences in pass rate. That changes which question is worth asking. Instead of "which model is cheapest?", ask how many tokens the whole agent system needs to finish a task. Compare two setups.
Agent A: cheaper model, wasteful harness
Model:
$X / million tokens
Task:
500k tokens
Agent B: pricier model, lean harness
Model:
2X / million tokens
Task:
100k tokens
Agent B pays twice as much per token but uses a fifth of the tokens, so each completed task costs roughly 40 percent of what Agent A spends, before even counting any difference in success rate. Model pricing is an input; agent economics are determined by how much work the harness makes the model do. Our architecture-first cost model for agentic AI explores this in more depth.
No-op turns and termination policies
Some of the waste is invisible in per-request metrics. Picture an agent stuck announcing its intention without acting:
I should inspect the repository.
and again on the next turn:
I should inspect the repository.
and once more:
I should inspect the repository.
Nothing changes, yet each turn consumes tokens, latency, compute, context space and possibly money. The same harness study characterizes this overhead as a per-task wait tax, not merely a token tax: users wait for turns that accomplish nothing. The harness needs termination rules, and the simplest is a hard cap on turns.
MAX_TURNS = 30
if agent.turns >= MAX_TURNS:
stop()
A cap is a safety net, not a strategy. More useful harnesses detect the signals that indicate a stalled run.
No progress
Repeated tool calls
Repeated identical observations
No state change
Test suite unchanged
Agent oscillation
Budget exhausted
Repeated identical tool calls, unchanged observations, no filesystem changes, a test suite that keeps failing the same way, oscillation between two states, or an exhausted budget all mean continuing is unlikely to help. At that point the harness should stop, summarize, and escalate. Our guide to bounded agentic loops in TypeScript shows these patterns in code.
From tool wrapper to platform
Harness design is emerging as a discipline of its own. A recent study of the source code of eleven production coding-agent systems found recurring architectural patterns across them: tool execution, context management, safety controls, orchestration and extension surfaces. Its authors conclude that coding harnesses have grown from simple tools into platforms. Early agents amounted to this:
LLM + tools
Current systems resemble a stack.
LLM
↓
Agent loop
↓
Tool router
↓
Policy engine
↓
Sandbox
↓
Context manager
↓
Memory
↓
Verification
↓
Observability
↓
Orchestration
The model is still the core, but the runtime around it, from the tool router and policy engine through the sandbox, context manager, memory, verification, observability and orchestration, has become a product in its own right.
A reference layout for a coding agent
A sensible starting architecture for a coding agent places the agent loop directly under the model and fans out into three concerns: a tool router, a context manager and memory. Tool calls pass through a policy engine into the sandbox, results go through verification, and everything is recorded by observability.
┌──────────────┐
│ Model │
└──────┬───────┘
│
↓
┌─────────────────┐
│ Agent Loop │
└────────┬────────┘
│
┌─────────────┼─────────────┐
↓ ↓ ↓
Tool Router Context Memory
│ Manager │
↓ │ ↓
Policy Engine │ Persistent State
│ │
↓ ↓
Sandbox ←────── Tools
│
↓
Verification
│
↓
Observability
Make each box independently testable: unit-test the policy engine with synthetic calls, fuzz path handling, and score the verifier on known-bad patches, all without calling a model.
Grow the harness from failures, not from a framework
You do not need a massive agent framework to get these benefits. The loop at the center can stay small even after policy, sandboxing and verification are added.
while turns < MAX_TURNS:
action = model.next_action(context)
if action.is_final:
return action.answer
if not policy.allows(action):
context.append("Action denied")
continue
result = sandbox.execute(action)
verification = verifier.check(result)
context.append(
summarize(result, verification)
)
Notice the ordering: the policy check comes before execution, a denied action is reported back to the model instead of crashing the loop, execution always goes through the sandbox, and what re-enters the context is a summary of the result and its verification, not the raw output. From there, add complexity only in response to an observed failure.
Agent deleted something?
→ add execution policy
Agent accessed the wrong file?
→ strengthen sandbox
Agent forgets architecture?
→ add persistent project memory
Agent produces bad code?
→ add verification
Agent becomes expensive?
→ optimize context
Agent loops?
→ add termination detection
Each symptom maps to exactly one layer. That is far healthier than introducing fifty abstractions before you know which problem you have. For a broader discussion of whether to build this loop yourself or adopt one, see runtime harnesses vs composed frameworks.
The repository as the agent's knowledge base
In agent-driven development, a repository is more than source code. It can hold everything an agent needs to work on the project: architecture decisions, development and test commands, domain vocabulary, schemas, plans, operational runbooks and agent-specific instructions, all versioned next to the code they describe. OpenAI's own writing on harness engineering makes this point, treating repository-local, versioned artifacts as the system of record for what an agent must know to understand and change a project, and recommending a map that points to focused documents rather than one enormous instruction file. The corollary is blunt: a design decision that lives only in someone's head or a chat thread does not exist for the agent. Making knowledge discoverable has become part of software architecture.
Hooks: behavior the model does not have to remember
Hooks are among the most useful harness primitives. A hook lets the runtime react to a lifecycle event with deterministic code. The most obvious one gates tool calls.
Before tool call
↓
Check policy
↓
Allow / deny
Another keeps code quality consistent after every edit.
After file edit
↓
Run formatter
↓
Run type checker
A third protects knowledge before the context gets compressed.
Before context compaction
↓
Persist important state
A fourth builds an audit trail.
After task completion
↓
Generate audit record
Claude Code documents hooks as lifecycle-triggered commands that run deterministically and can, among other things, block a tool call before it executes. Their power comes from moving behavior out of the model's reasoning entirely. Instead of writing "please remember to run the formatter" in a prompt and hoping, the runtime simply runs the formatter every time.
Never let the model own the boundary
Models are probabilistic; security boundaries must not be. This anti-pattern delegates the decision to the model:
if model_says_safe:
execute()
This pattern delegates it to infrastructure that cannot be talked out of its answer:
if operating_system_allows:
execute()
Use the model for decisions that benefit from judgment, and deterministic infrastructure for decisions that need guarantees. In practice, each layer answers its own question.
Model:
Should I modify this file?
Harness:
Is this file inside the workspace?
OS:
Can this process write to the file?
Human:
Should this high-risk action proceed?
The model decides whether a change is a good idea, the harness checks that the target is inside the workspace, the operating system decides whether the process can write the file, and a human decides whether a high-risk action should go ahead. Layered answers like these are far more robust than expecting any single component to get everything right.
Harnesses amplify models; they do not replace them
It is tempting to conclude that harnesses matter more than models. The evidence does not support that. In the studied setup, harness choice had large effects on token efficiency but much smaller effects on task success, and no amount of runtime engineering turns a weak model into a strong software engineer. A clean way to separate the two:
Model capability
=
What the agent can reason about
Harness quality
=
How effectively that capability is deployed
A brilliant model inside a poor runtime wastes enormous amounts of context and money. A carefully engineered runtime around a weak model still produces weak decisions, just safely and efficiently. You need both.
Five questions to ask before deploying any agent
1. What can it do?
Inventory every tool and flag every dangerous operation.
read
write
delete
execute
network
deploy
send_message
2. What can it reach?
Tool definitions tell only part of the story. Check what the process can actually access:
filesystem
network
environment
secrets
processes
credentials
3. What survives a restart?
Restart the agent and find out:
What does it remember?
What does it forget?
Where is important state stored?
4. Who verifies the result?
For consequential operations, a two-step workflow is not enough:
generate → approve
Insert independent checks between generation and approval:
generate
↓
test
↓
review
↓
approve
5. What does each task really cost?
Per-request token counts are misleading on their own:
tokens per request
Measure outcomes instead:
tokens per successful task
latency per successful task
tool calls per task
retries per task
human interventions
Tokens, latency, tool calls, retries and human interventions per successful task are the figures that reflect whether the system is worth running.
A working definition of a production agent
Putting the layers together yields a more honest definition.
Agent =
Model
+ Tools
+ Execution Environment
+ Policy
+ Memory
+ Context Management
+ Verification
+ Observability
Each term earns its place. Without the model there is no intelligence; without tools there is no action; without an execution environment and policy the agent can be dangerous; without memory long tasks become fragile; without context management costs spiral; without verification errors propagate unnoticed; and without observability you cannot reconstruct what happened when something goes wrong.
Wrapping up
The industry conversation is shifting from which model tops a benchmark to which runtime lets a model do useful work reliably. Vendors are converging on the same building blocks: OpenAI now sells managed agent infrastructure that bundles sandboxes, context handling, tool search, tool calls made from code and long-lived execution, while Anthropic exposes persistent instructions, skills, subagents and deterministic hooks. Research is starting to evaluate the harness as an experimental variable rather than treating the model as the whole system. These features evolve quickly, so check current documentation.
The naive recipe for an agent is short:
Pick LLM
+
Add tools
=
Agent
The realistic one looks like this:
Model
↓
Agent Runtime
↓
┌─────────┼─────────┐
↓ ↓ ↓
Policy Context Memory
↓ ↓ ↓
Sandbox Tools Persistence
└─────────┼─────────┘
↓
Verification
↓
Observability
The model makes decisions; the harness defines the conditions under which those decisions become actions. Before choosing a framework, ask what happens when the model is wrong, and build the harness around that answer.
- Enforce deterministic rules in code and hooks, not prompts.
- Treat tool policies as a first line and the OS sandbox, including network and resource limits, as the real boundary.
- Separate conversation, project and harness state by lifetime, and persist project facts before compaction.
- Keep verification independent and have reviewers report rather than fix.
- Optimize tokens per successful task and stop runs that stop making progress.
The goal is not an agent that never errs, but a system in which a model mistake does not automatically become a production incident.