The Harness Is the Product: A Practical Guide to Harnesses in Agentic AI

Imagine giving one and the same language model to two different teams. Team A builds an agent that repairs bugs in a huge repository, runs the test suite and submits a neat pull request. Team B builds a chat assistant that loses track of the conversation within a few messages and happily makes up company policy. The weights are identical, yet the results are worlds apart.

The gap is the harness: everything built around the model. It includes the loop that lets the model act, the tools it may call, the way its context window is managed, the guardrails that stop it from doing something foolish and the feedback that tells it whether it is succeeding. What follows starts from zero, covers what an agent is, and works up through the parts of a harness, production patterns such as sub-agents and compaction, a few well-known systems and the frameworks available today. No prior agent experience is needed.

The basics

What agentic AI means

Without the hype, an agent is simple:

An agent is a language model running in a loop, using tools, until a goal is reached.

That takes three ingredients: a model that can reason and choose the next step, tools the model is allowed to trigger, such as web search, file access, code execution, database lookups or outgoing mail and a loop that feeds each tool result back so the model can decide what to do next.

A chatbot replies one time and stops. An agent continues: it takes an action, looks at the outcome, then takes another. This cycle is what makes a text predictor capable of getting real work done, and the rest of this piece expands on it.

What a harness is

Think of the harness as the runtime for that loop. The model chooses what should happen next, while the harness carries it out and keeps the whole process safe, affordable, visible and controllable. The name echoes a test harness in software, or the tack on a horse: raw power needs structure to be pointed at useful work.

In concrete terms the harness executes the tool calls the model asks for, controls what the model sees at each step, enforces permissions, handles failures, retries and endless loops, tracks progress and knows when the job is finished or when to stop and ask a human.

One useful way to picture it: the model has no memory between turns. Each time it is called, it starts blank, reads the text it is given and emits a single result. Everything else, the script, the props, the stage and the safety net, is the harness. Rearrange those and the identical actor performs completely differently.

Why the harness matters

Three practical reasons stand out.

Model gains converge while harness gains don't. Frontier models are increasingly close in raw ability. On agent benchmarks that measure fixing real software issues, the same model can score noticeably differently depending only on the scaffold around it, through better tools, prompts and feedback loops. At some point the harness becomes the differentiator.

Long tasks magnify small errors. If a model is 99 percent reliable per step, a 50-step task succeeds only about 60 percent of the time, since 0.99 to the power of 50 is roughly 0.605. Verification, retries, checkpoints and course correction are the harness machinery that fights this compounding. They separate a demo from a product.

Freedom needs fences. Any agent able to run shell commands, move money or write to customers must have permission limits, sandboxes and audit logs. These live in the harness, and the model contributes none of them.

Anatomy of a harness

Coding agents, research agents and support agents tend to share seven components. Here they are one by one.

1. The system prompt, the standing orders

Here the agent is told its role, its limits, its manners and the purpose of each tool. In mature products the text spans thousands of words and is maintained with the care given to source code, covering tone, safety, when to ask first and how to handle unclear instructions.

A newcomer might write one line saying the agent is a helpful coding assistant. A production prompt instead states procedures: open a file before editing it, favor small changes, run tests after each edit, halt and report after two failed test runs, and never push straight to the main branch, then continues with many more rules like these.

2. The tool layer, the hands

A tool is a function the model is allowed to call, defined by a name, a description and a typed schema for its arguments. The model requests a call as structured output, then the harness performs it and sends the outcome back.

Tool design is an underrated craft. It amounts to API design for a very literal user:

• Descriptions act as prompts. The model picks tools from their descriptions, so a vague one leads to the wrong tool.
• A few well-shaped tools beat many overlapping ones. Tool sprawl confuses models the way a forty-item menu confuses diners.
• Return errors the model can act on. A message saying the file is read-only and that access must be requested first beats a raw stack trace.
• Keep dangerous tools narrow. A tool that sends only pre-approved drafts is far safer than one that runs arbitrary code.

A major development here is the Model Context Protocol, which Anthropic open-sourced in late 2024 and much of the industry has adopted. It standardizes how a tool provider exposes tools to any agent, roughly a universal connector: write the integration once and plug it into any harness.

3. Context management, the working memory

The context window is the model's entire perceptual world. Windows commonly range from hundreds of thousands to a million tokens, which sounds huge until an agent has read forty files and run sixty commands.

Choosing what the model gets to see on each step is probably the single most valuable lever in the system, and practitioners now call this context engineering, which has mostly replaced prompt engineering as the central skill. Techniques run from simple to sophisticated: truncation (keep only recent messages or a slice of a file), retrieval (index the knowledge base and pull in just what is needed now) and compaction. With compaction, once the window nears full, the model writes a summary of the session, that summary replaces the raw transcript, and work carries on. This is how agents stay on task for hours.

4. Memory, the long-term store

Context lasts one session, and memory persists. Harnesses usually layer it:

• project memory, such as instruction files in a repository that teach a coding agent your conventions and build commands and are read at session start,
• user memory, meaning preferences learned across conversations,
• episodic memory, which keeps records of past runs, often in a vector store, to retrieve when similar situations return.

5. Guardrails, permissions and sandboxing, the brakes

The safety layer continually asks whether an action should really be executed. In practice that means:

• Permission tiers. Reads run freely, writes pass policy checks, and irreversible actions such as deploying, sending, deleting or paying need explicit human approval.
• Sandboxing. Code is executed inside isolated containers with tight limits on files and network, so an agent that is confused or hijacked by injected text cannot harm the host machine.
• Filtering. Tool results are scanned for injection attempts and outputs for leaked secrets.
• Hard budgets. Iteration and spend caps stop runaway runs. Most production harnesses have a story about why the cap exists, because nobody adds it proactively.

6. Verification and feedback, the eyes

Nothing improves an agent more reliably than a way to test its own output. Coding agents compile, lint and run tests after every edit. Research agents compare claims across several sources. Browser agents capture the screen to see what really loaded. Some harnesses add a reviewer, a second model that grades the first model's answer against a rubric before it is accepted.

This attacks the compounding-error problem directly. An agent that can see it failed can retry, while one that can't will confidently hand over garbage.

7. Observability, the flight recorder

In production, every model request, every tool call and every token spent gets recorded. With those traces you can replay a failed run, locate the step that went wrong and correct the prompt, tool or policy at fault. An entire product category grew around this, including LangSmith, Langfuse, Braintrust and the OpenTelemetry conventions for generative AI, since debugging an agent blind is as hard as debugging distributed services without logs.

Running the loop well

Naming the parts is the easy bit. The difference between a harness that survives a demo and one that survives a normal working day comes from the practices below.

Plan, act, verify

Simple agents jump right in. Mature harnesses enforce a cadence: draft a plan by breaking the goal into a task list, take small steps, check every result before continuing, then revise the plan and go again. The written plan serves twice. It keeps the model oriented on long jobs because it is re-read each turn, and it lets a human watching see progress as it happens, which proves more valuable than expected.

Structured outputs

Whenever a program will consume the model's output, constrain it with a schema. Current APIs support forced tool calls and schema-validated generation. Prose is for people; code should get structure.

Cost and latency

Agents burn through tokens, which makes economics a harness concern. Caching the big static prefix (system prompt plus tool definitions) across turns costs far less than resending it, a big win in long sessions. Routing sends routine steps to a small, quick model and reserves a frontier model for steps that need reasoning. Independent subtasks, such as reading ten files or checking five sources, can run side by side.

Multi-agent systems and beyond

Sub-agents and orchestration

Once tasks outgrow one context window, harnesses go multi-agent. An orchestrator splits the goal and spawns sub-agents, each with a fresh context, often a narrower toolset and a focused brief. Each sub-agent does its work and returns only its conclusion, not its whole working history.

The real gain is less obvious than speed: each sub-agent works in its own isolated context, not merely in parallel. Ten sub-agents can each fill an entire window with one subsystem while the orchestrator keeps only ten brief summaries.

Common topologies include orchestrator and workers (by far the most common), pipelines such as draft, critique and revise, and debate panels in which several agents tackle a problem independently and a judge synthesizes the answers. Debate is expensive but suits high-stakes questions.

Teams that shipped this report some hard-won lessons:

• State the expected effort level for each task, otherwise even a trivial question can trigger dozens of searches.
• Write briefs that are detailed and self-contained, because a sub-agent can't see the parent's context.
• Isolate workspaces early, since two agents editing one file will eventually happen.

Checkpointing and durability

Long-running agents fail midway through crashes, rate limits and restarts. Production harnesses checkpoint state, covering the conversation, task list and tool results, so a run resumes from step 37 and doesn't start over. This is the territory of durable workflow engines, and several agent frameworks are built directly on that machinery.

Evals, the test suite of the harness

Agent behavior is probabilistic, so a prompt that works on Monday can fail on Tuesday. Strong teams keep an evaluation suite containing anywhere from dozens to thousands of realistic tasks whose results can be checked automatically. Did the tests pass, was the right answer found, did cost stay within budget? Prompt edits, new tools and model swaps all run through the suite before release. It plays the role CI/CD plays elsewhere, and skipping it means working without instruments.

Computer use, the final exam

At the leading edge, agents are given a screen, a keyboard and a mouse and can drive any application, whether or not it has an API. This raises the difficulty of every earlier problem: screenshots consume huge numbers of tokens, so context control must be strict; a wrong click can do real damage, so permissions tighten; and verification means actually viewing the screen after each step.

How known systems apply these ideas

Claude Code, a coding harness. It is a compact lesson in design. The tool set is small and sharp, with file reading, writing and editing, shell commands and code search, and not hundreds of micro-tools. Verification is central: it runs compilers, linters and tests and iterates when they fail. A project instruction file acts as memory. None of that is a model capability, and that is why the same model feels different inside it.

Deep research products, a research harness. The research tools from the large labs turn a query into an autonomous investigation lasting many minutes that ends in a cited report. Their signature is a research plan drafted up front, sometimes shown for approval, which puts the human in the loop at the cheapest point before the costly work. Search loops read, notice gaps and search again, and citation metadata travels with the facts through the pipeline.

Manus and the context-engineering school. The Manus team published unusually frank notes about harness internals around its 2025 launch, and several points spread widely. Treat the KV cache as a design constraint: keep the start of the prompt stable (no timestamp at the top of the system prompt), because input tokens can outnumber output tokens by a large factor. Avoid adding or removing tools during a session, which invalidates the cache and puzzles the model when earlier calls mention vanished tools; keep the list fixed and mask what can currently be chosen.

Cursor and the IDE harness. The AI-native editor shows a different philosophy of deep environment integration. It uses the editor's semantic index of the codebase for fast retrieval, lands edits as reviewable diffs so that the human approving the merge is the permission model disguised as interface, runs background agents on separate branches and feeds linter and type-checker output back into the loop. The same seven components appear in different clothes.

Support agents in enterprises, where the harness is about compliance. Customer-facing agents flip the priorities: avoiding a wrong action matters more than raw capability. They therefore start from guardrails: scope limited to approved knowledge, actions checked against business-system schemas (refunds capped, identity confirmed first), compulsory handoff to people when confidence is low and complete audit logs. In regulated sectors the harness effectively is the product.

Frameworks and tooling available now

You rarely build a harness from bare API calls any more. Here are common options, grouped loosely by philosophy:

• Claude Agent SDK. The harness behind Claude Code, exposed as a library with loop, tools, sub-agents, permissions and compaction included. Best for building serious agents quickly on Claude.
• OpenAI Agents SDK. Lightweight primitives for agents, handoffs, guardrails, sessions and tracing. Best for multi-agent apps in the OpenAI ecosystem.
• LangGraph. Agents as explicit state machines, with checkpointing, human-in-the-loop interrupts and durable execution. Best for complex, controllable, long-running workflows.
• CrewAI. Role-based teams such as researcher and writer. Best for quick multi-agent prototypes and content pipelines.
• AutoGen, AG2 and Semantic Kernel. Conversation-centric multi-agent work converging into enterprise tooling. Best for .NET and Azure shops and research experiments.
• smolagents. A minimalist library where agents write code as their actions. Best for hackable builds with open models.
• Pydantic AI. Type-safe, schema-first agents with validated outputs. Best for Python teams who want strict typing.
• Vercel AI SDK. TypeScript-first primitives with agent loop control. Best for web and product engineers in the JavaScript ecosystem.

Around these sits connective tissue: MCP for standard tools, tracing and eval platforms, durable execution engines and sandbox providers for safe code execution.

Advice on choosing: when learning, write the raw loop yourself once, using a model API, a while loop and two tools, in roughly a hundred lines. After that no framework feels like magic, which is the point. When shipping, pick the framework closest to your model provider and language, and spend the time you saved on what no framework gives you, namely your tools, your evals and your guardrails.

# The whole idea, in miniature
while not done:
    response = model.generate(context, tools)      # model decides
    if response.tool_calls:
        results = execute(response.tool_calls)     # harness acts (safely!)
        context = manage(context + results)        # harness curates memory
    else:
        done = verify(response)                    # harness checks the work

Failure modes

Here are the typical ways agents fail, each paired with a harness-side remedy:

• Context rot. The window slowly fills with outdated tool output and quality drops until, a couple of hours in, the goal is forgotten. Remedy: compaction, running notes and periodically restating the objective.
• Doom loops. The same failing command repeated forever at a few cents a try. Cure: loop detection, iteration budgets and forced changes of strategy.
• Tool sprawl. With forty similar tools the model selects the wrong one at the worst time. Remedy: a smaller set of sharper tools with clear descriptions.
• Prompt injection. Text on a web page or in an email tells the agent to disregard its instructions and dump a database, and because the agent cannot inherently separate data from commands, it may comply. Remedy: input filtering, least-privilege tools, sandboxes and human approval for consequential actions, layered because no single defense is reliable.
• False finish. The agent announces success while the work is incomplete. Remedy: check results with something independent, and don't let the agent mark its own work.
• Runaway spend. A multi-agent fan-out can silently inflate the bill many times over. Remedy: budgets, routing, caching and a frank look at whether a single capable agent would have sufficed.
• Silent capability drift. Upgrading the model alters behavior, and prompts tuned for the earlier version stop working. Remedy: run the eval suite on every change.

What to remember

The model supplies intelligence, and the harness turns it into work. A harness is a loop plus seven organs: standing orders, tools, context management, memory, guardrails, verification and observability. Around those sit patterns for running long tasks, such as plan, act and verify, compaction, checkpoints, sub-agents and evals. Most agent failures are harness failures, and most agent improvements are harness improvements. So when an agent underperforms, look at the scaffolding first and swap the model afterward.