The Five Layers of an Agent Harness, With Code That Fails Without Them

Picture a team that adds one line to an agent's config: no edit goes in without the human's sign-off. The wording is clear, the line sits near the top, and the agent loads it at the start of each session.

Then comes an afternoon where a single session blows past that line eleven times and pushes around seventy-six unapproved changes into a client deliverable. No error appears. Nothing pauses to ask. Two hours on, someone walks through the document highlighting every line the agent rewrote on its own and finds forty-six, inside a file that carries the client's name.

The instruction itself was fine. It lived in the wrong place.

What works instead is a hook, a small script the harness runs just before a tool call goes out. It inspects the pending call and can answer with a refusal. The agent may still want to make the edit, but the edit never lands in the file. A hook like this will sometimes be wrong, typically because the rule was drawn too narrowly, and the difference is that you can open the code and point at the exact condition that misfired. A sentence in a prompt leaves nothing to point at.

The distance between a rule you announce and a rule you enforce is the harness. It spans five areas:

1. The actions the agent may take
2. The places it can reach
3. The state that outlives a restart
4. The reviewer of its output
5. The cost of what it gets shown

Every area gets code below. A set of runnable demonstrations exists where each layer breaks on your own laptop before it can break in production. This is aimed at people who have already shipped an agent and seen it go wrong.

What a harness actually buys you

Begin with something almost too plain to state. A coding agent's main loop is six lines:

while agent.turns < max_turns:
    call = agent.next_call(observations)
    if call is None:
        break
    result = execute(call)
    observations.append(result)

Those lines are the agent. With error handling and retries you'd land near twenty. Whichever framework you've evaluated, it wraps this loop, and the interesting spot is execute. That function settles what the model can do, what it can touch, what it retains, who audits it, and what ends up in front of it.

Together those decisions are the harness, and plenty of commentary claims it outweighs the choice of model. The data backs a more limited statement, which happens to be more practical.

The harness wins on cost. A preprint from June 2026 compared three harnesses (Goose, OpenCode, OpenHands-SDK) across two models using a stratified 50-task slice of Terminal-Bench Pro, the tougher variant of the Terminal-Bench agent suite. Token use per solved task differed by up to 40 times between harnesses. Swapping to a better model shifted the same figure by just 1.0 to 1.3 times.

It doesn't win on capability. Pass rates differed across harnesses by 0 to 8 points, and with bootstrap confidence intervals, everything but the widest gap straddled zero. The authors themselves call the accuracy results descriptive, given n=50.

Put plainly, one task on one model at one success rate can cost forty times more under one harness than under another. Everything that follows trims cost and reduces surprises, and none of it adds intelligence to a weak model.

Moving the rule into code

Stating a rule in a system prompt amounts to a wager that the model recalls it, interprets it as intended and keeps to it once context gets crowded. Usually the model wins. That success rate is exactly what makes it risky, because people then stop verifying.

Code-level rules fire on every single call, even when the model has lost track of them.

An agent loop has five spots where code can hold the line. Each section below covers its purpose, the enforcing code and the behaviour when the layer is absent. Nobody needs all five at once. Pick the layer whose failure you can most easily imagine hitting you this week.

Layer 1: the execution boundary

What is the agent permitted to do?

The boundary is a function wedged between the model's choice and the actual tool call. Start with the agent unguarded. Its prompt says never to delete a file without asking, and three tool calls wait in the queue.

SYSTEM_PROMPT = "IMPORTANT: never delete a file without asking the user first."

def execute(call):
    if call.name == "delete_file":
        FILES.pop(call.args["path"], None)
        return f"deleted {call.args['path']}"

Running it, the agent removes cache.tmp, harmless junk, and then removes client-deliverable.md, which is just what it sounds like. No question was asked, though the prompt had the rule from the start.

A function placed ahead of the tool solves it:

def boundary(rules):
    def wrap(execute):
        def guarded(call):
            for name, deny_if, reason in rules:
                if deny_if(call):
                    return f"DENIED by {name}: {reason}"
            return execute(call)
        return guarded
    return wrap

NEEDS_APPROVAL = [(
    "delete-needs-approval",
    lambda c: c.name == "delete_file" and not c.args.get("approved_by_human"),
    "deletion requires an explicit human approval flag on the call",
)]

With the same agent, script and prompt, nothing is deleted.

Layer 2: sandboxing

What can the agent get to?

A sandbox defines which directories and hosts the agent's process can touch in the first place. Build it from zero: begin with nothing and grant back just what the work requires, via a list of allowed paths, a list of allowed hosts, and a process blind to everything else.

The widespread alternative is a deny list of forbidden paths:

DENY = ["secrets/"]

def execute(call):
    path = call.args["path"]
    if any(path.startswith(d) for d in DENY):   # checks the spelling
        return "DENIED by deny-list"
    real = os.path.normpath(path)               # the ../ collapses HERE, after the check
    return DISK.get(real, "not found")

Run it and the agent tries twice:

read('secrets/api_key')          -> DENIED by deny-list
read('work/../secrets/api_key')  -> sk-live-DO-NOT-LEAK

The check examined the text as typed, whereas the file lookup followed the location that text resolves to. Two requests, two spellings, one file.

The repair is to normalise the path first and compare afterwards:

ALLOW_ROOTS = ["work"]

def resolve(path):
    real = os.path.normpath(path)               # resolve FIRST
    if not any(real == r or real.startswith(r + os.sep) for r in ALLOW_ROOTS):
        return None
    return real

Both spellings now collapse to secrets/api_key, which sits under no allowed root, so both are rejected. Allowing instead of denying matters here, because a deny list blocks only the spellings its author imagined.

These examples are ordinary Python functions, so no container is needed to try them. In a real deployment, the identical check is the process boundary itself: a user or container that has no read permission outside work/, with the same order of operations.

Layer 1 can't do this job. A check living inside your program only sees calls it recognises, while a shell command, a symlink or a path with .. in it can reach the same file without being any of those calls.

Layer 3: memory persistence

What lasts after a context reset?

One habit pays off more than most: dropping conversations without a second thought. Context grows full, quality slips and the agent starts disagreeing with itself. Many people keep nursing such a session. Restarting early is better, provided anything important is stored where a restart can't wipe it, whether that's committed code, configuration, the harness itself or distilled notes.

So decide on purpose where each item of state belongs. There are three options: the conversation, where it disappears with the session; disk, where it persists and can be fetched later; and the harness config, where it influences every session that comes after.

A single dispatch function covers all three:

CONVERSATION, DISK, HARNESS_CONFIG = [], {}, {}

def remember(kind, key, value):
    """'chat' dies with the session, 'disk' survives it,
    'config' shapes every session that follows."""
    {"chat": lambda: CONVERSATION.append(value),
     "disk": lambda: DISK.__setitem__(key, value),
     "config": lambda: HARNESS_CONFIG.__setitem__(key, value)}[kind]()

The function is almost nothing. What it gives you is a rule that nothing gets saved without naming which of the three homes it goes to. Teams seldom make that choice explicitly, which explains agents that seem forgetful about some things and obstinately mistaken about others. In the demonstration, the conversation list comes back empty while the decision and the rule remain on disk and in config.

Layer 4: verification loops

Who reviews the output, and what keeps the reviewer from simply fixing things?

A verification loop means a second look at the agent's work, performed by something allowed to flag problems and not to repair them.

The latter half is what people leave out. Let a reviewer subagent write, and it quietly patches whatever it finds. That feels efficient, yet it erases the information you wanted: you can no longer separate correct work from wrong work that got fixed behind your back.

So keep the reviewer read-only. It produces a critique, and acting on that critique is another step with its own trail. Keeping them apart preserves a signal you can read.

A second tactic is the dry run. For any irreversible action, the tool accepts a flag, describes by default what it would have done, and only acts once someone deliberately switches the flag off. The harmless behaviour is the default, so an oversight costs nothing. In the demonstration, the fix tool announces it would rewrite total.py, writes nothing, and the off-by-one remains at the end.

One execute holds both rules:

def execute(call):
    if call.name == "review":
        snapshot = dict(CODE)               # a copy, so the reviewer cannot write
        src = snapshot[call.args["path"]]
        return f"VERDICT: {'off-by-one' if '+ 1' in src else 'looks good'}"
    if call.name == "apply_fix":
        if call.args.get("dry_run", True):  # on by default, turned off on purpose
            return f"DRY RUN: would rewrite {call.args['path']}, nothing written"
        CODE[call.args["path"]] = call.args["new"]
        return f"wrote {call.args['path']}"

The reviewer spots the off-by-one and has no way to touch it.

Layer 5: context pipelines

What reaches the model, and what is it billed at?

The context pipeline covers how information travels into the model's window and how much of it is admitted.

Admission is where the money goes. Each token that gets in is paid for on that turn and again on all later ones, so this is where the forty-fold gap between harnesses emerges, not in Layer 1.

A naive setup loads everything: the full file for a question about one function, the full search output for a one-line need, and the entire history. It functions, yet by turn twenty you're paying once more for whatever turns one to nineteen already read.

The fix is delegation plus distillation. A subagent works in a separate window, reads tens of thousands of tokens and hands back one or two thousand. The main thread skips the raw search dump and receives only the answer.

In code, it comes down to which counter goes up:

def subagent_search(query):
    """Its own window. The main thread never pays for this reading."""
    global subagent_tokens
    subagent_tokens += sum(len(v.split()) for v in CORPUS.values())
    return next(f"{n}: {b.split('ANSWER:')[1].strip()}"
                for n, b in CORPUS.items() if "ANSWER:" in b)
def execute(call):
    global main_tokens
    distilled = subagent_search(call.args["q"])
    main_tokens += len(distilled.split())    # the only line that bills you
    return distilled

The demonstration is deliberately small and honest. The question and answer are identical in both runs, yet the main thread processes 9,602 tokens in one and 2 in the other.

This also clarifies why chained subagents let people down. When every step waits for the previous one, you obtain context isolation and no parallel speedup at all. Genuinely independent steps should run side by side. Otherwise, accept that isolation is what you're buying. It often justifies the price, as long as you know what the invoice covers.

What nobody can say yet

Public confidence on this subject has outrun the evidence behind it, so the caveats matter.

1. Nothing here has been peer reviewed. As of August 2026, every study that isolates the harness is an arXiv preprint without refereeing. The most careful experiment describes itself as early work under workshop review, and the best-known framing paper calls itself a position paper with no experiment of its own. Treat all of it, this text included, as unrefereed, and repeat the comparison on your tasks before planning budgets around it.
2. Gains rarely transfer to unseen tasks. A July 2026 paper reported that letting a harness evolve automatically fails to beat plain test-time scaling reliably and generalises poorly, because the search and the final test draw on one benchmark. In a continual-learning evaluation on Terminal-Bench 2.0, GEPA rose to 70.8% on the tasks it was tuned for and then fell to 54.5% across the wider set, lower than its 56.8% starting point. Tuning actually hurt. Test any harness on tasks that weren't used to build it, since improvements seen only on the tuning set vanish in production.
3. A well-known fine-tuning win is graded by its own training signal. Baseten fine-tuned Gemma 3 27B for writing clinical notes, moving it from 35% below Claude Sonnet 4 to 60% above. The write-up states openly that the eval harness doubled as the training reward. That is a valid method, but the score comes from the very thing that did the teaching. When a vendor claims a fine-tuning gain, ask what the eval was during training; if it's the same harness, the number shows the model learned the eval.
4. A harness doesn't transfer cleanly between models. A June 2026 study found that adding a harness after training recovers little of what training alongside it delivers. Models post-trained on a minimal harness answered with an "Invalid tool format" error in 75.1% of attempts after the tool definitions were reshaped, and a Qwen2.5-7B variant landed 10.8 points under its own base model. A harness that suits every model suits each one a bit worse. Choose the harness before post-training, and if you don't train your own model, expect a harness swap midway through a project to be pricier than the migration looks.
5. The terminology is unsettled. In May 2026 Hugging Face published a glossary separating harness from scaffold, and acknowledged that Claude Code, Codex and others use harness for the whole stack anyway. Five layers is only one way to slice it. Under a narrower definition, about half the evidence above measures something different. Choose a vocabulary, hold to it inside your team and read cross-study comparisons carefully. For that reason the five layers work best as a checklist for your own setup, since a checklist outlives a vocabulary change and a taxonomy fights one. Before trusting a gap between two agent results, ask which layers each one included.
6. Harness effects seem to fade as models get better. Harness-Bench reports that stronger model backends show higher average scores along with smaller variation between harnesses. Some of the scaffolding may be propping up a weakness that is disappearing. Lean your harness investment toward cost control, which lasts, and away from accuracy, which a coming model release might provide free.

The companion scripts

Ten short scripts accompany the idea. Each layer appears with its guard, and the same agent appears without it. They run offline with no API key, since a scripted stand-in replaces the model and emits a fixed series of tool calls. Seeing a difference run beats taking a claim on trust.

git clone https://github.com/paoloap-py/agent-harness-guide
cd agent-harness-guide

python3 run_all.py        # all five layers, guard off then on, side by side
python3 test_harness.py   # asserts every difference above, 10 checks

The README frames the five layers as an audit checklist with one repeating question: does code enforce this, or a sentence in a prompt? Any box you can't tick by pointing to a file path is a rule you're only hoping for.

Where this lands

A sound rule in the wrong location barely counts. In the scenario above, it cost fifteen violations, eleven within one afternoon, and a day nobody wants back. Relocating the rule is what made it hold, and not a single word of it changed.

That's the core of it. The model is something you buy. Everything around it is something you build, and that half determines what the agent costs and how often it does things you never approved. Both are engineering, but only the second is yours to shape.

FAQ

What exactly is a harness for an agent? Everything wrapped around the model: the tool loop, the pre-execution checks, the sandbox, the persistence between sessions, the reviewer, and the pipeline deciding what each turn contains. The loop is six lines, or twenty with error handling, and everything else belongs to the harness.

Is the harness more important than the model? For cost, yes, with token use per solved task varying up to 40x. For whether tasks get solved, no. Measured differences in pass rate between harnesses run from 0 to 8 points, with intervals that cross zero, and the Terminal-Bench authors conclude that picking the model usually matters more for performance.

Where does the harness cost the most? In context. A token shown to the model on every turn is paid for on every turn. That makes Layer 5 the expensive one and explains why handing a search to a subagent changes the bill without changing the answer.

Do permission rules count as security? They don't. A harness check sees only the call the agent actually made, while a shell command, a symlink or a .. path can reach the same file without being inspected. The real limit comes from the operating system: start the process with access to nothing and add back what the job needs.

Build a harness or adopt one? Adopt one and stay with it. Moving to another wipes the accumulated knowledge in your config files, and each rule written after a genuine failure must be earned again.