Harness Engineering: The Runtime That Turns an LLM Into an Agent

When a coding agent goes wrong, the model usually takes the blame. More often the fault sits in the harness, the program wrapped around it.

Anyone who has called a chat-completion API knows the easy half: send messages, receive a reply, perhaps add function calling. The hard half is the distance between that and an agent that can sit inside a repository, edit files, run the tests and recover after it breaks something. The harness covers that distance. It's the runtime around the model: tools, permissions, sandboxing, context handling, memory and error recovery. Text in and text out is all a model does. The harness is the program that repeats the cycle, executes tools, keeps things safe and talks to you in a terminal. If the model is the engine, the harness is the car.

On its own, a language model can't open your files, run pytest, edit code, recall last Tuesday's conversation, or stop itself from typing rm -rf /. All it can do is emit the next message. Everything else is harness work:

• Loop: call the model again and again until it stops requesting tools.
• Tools: the hands (shell, files, edits).
• Permissions: ask before risky actions and block the worst ones.
• Sandbox: make dangerous actions fail at the operating-system level.
• Context: keep the conversation small enough to send.
• Memory: skills, a to-do list, a session file that can be resumed.
• Subagents: explore a codebase without filling the main conversation.

This write-up covers the first three, which are what turn a chatbot into an agent. The rest, sandboxing, context management, memory and subagents, are further layers on the same skeleton.

Chatbot versus agent

A chat assistant makes one round trip: question in, answer out. Tools like Claude Code or the agent mode of Cursor work differently. Say "refactor this module" and they read files, run tests, edit code, run the tests again and patch whatever they broke. A smarter model isn't the reason. A program around the model knows how to loop, call tools and persist when a step fails.

People have begun to call this harness engineering, a rather grand label for a plain idea: Claude Code, Cursor and Codex are better harnesses as much as they are better models.

An example helps more than definitions. You type: "Fix the failing test in test_parser.py." Here's what follows:

Turn 1:  LLM reads the test file         → bash("cat test_parser.py")
Turn 2:  LLM runs the test to see error  → bash("pytest test_parser.py")
Turn 3:  LLM reads the source file       → read_file("parser.py")
Turn 4:  LLM edits the bug               → str_replace("parser.py", old, new)
Turn 5:  LLM re-runs the test            → bash("pytest test_parser.py")
Turn 6:  Tests pass → LLM says "Done! Fixed the off-by-one error in line 42."

Six turns and six decisions, and nobody clicked anything after the first prompt. That's a harness doing its job.

Vocabulary

A handful of terms recur throughout.

• LLM: a function from a message list to one further message. It has no memory of earlier API calls. If you want it to appear to remember, you send the history again.
• Token: a unit of text the model counts, around three quarters of an English word. "Hello world" is about two tokens and a 100-line Python file runs near 500.
• Context window: the number of tokens the model sees at once. Figures of roughly 128K for GPT-4-class models and about 200K for Claude sound enormous until an agent has pasted grep -r output into the chat 40 times.
• Tool call: the model doesn't run your code. It writes a request such as {"name": "bash", "arguments": {"command": "ls"}}, then your Python runs ls and returns the output.
• Stateless: each call_llm(messages) stands alone, and the messages list is the memory.

No machine-learning background is needed. Basic Python (functions, loops, dicts, try/except), a little experience with an OpenAI-style API and the most common shell commands (ls, grep, cat) are enough.

The reference implementation

The example agent is about 600 lines of Python with no framework, using four libraries: openai, rich, prompt-toolkit and pyyaml. Snippets below are the real code, not pseudocode, and by the end you should recognise the same parts inside Claude Code or Cursor. To follow along, clone a copy of the project and install it in editable mode:

git clone https://github.com/shrinidhi-mahishi/coding-harness.git
cd coding-harness
pip install -e .

Any OpenAI-compatible endpoint works: OpenRouter, a local server, OpenAI itself. Put the settings in ~/.agents/env:

BASE_URL=https://openrouter.ai/api/v1
API_KEY=sk-or-...
MODEL=deepseek/deepseek-v4-flash

Genuine environment variables override that file. The coding-harness command, declared in pyproject.toml as an entry point to coding_harness.agent:main, starts a terminal session. You type, and the inner loop runs until the agent has no more tool calls to make.

Seven ideas make up the whole design: the agent loop (the heartbeat), tools (hands), permissions (who may do what), a sandbox (safety enforced by the kernel), context engineering (a clean window), skills, to-dos and sessions (memory), and subagents (spending context somewhere disposable). Only the first three are covered here.

The files that matter:

coding_harness/agent.py         # two nested loops
coding_harness/llm.py           # one API call + the system prompt
coding_harness/tools.py         # tool functions + defensive execute()
coding_harness/permissions.py   # allow / ask / deny
coding_harness/config.py        # BASE_URL, API_KEY, MODEL

There's no database, web server or orchestration library. The "framework" is two while True loops and a dictionary of tool functions. For a first read, go through config.py, llm.py, agent.py, tools.py and permissions.py in that order.

1. The agent loop

Claude Code, Cursor and Codex all share one loop:

1. Send the conversation to the model.
2. If the model asks for a tool, run it.
3. Append the result to the conversation and return to step 1.
4. With no tool calls left, the work is done and the reply is shown.

A chatbot does a single round trip, and an agent continues until the job is finished. That's the entire difference, and most of the remaining engineering exists to stop the loop from falling over.

One point confuses newcomers: models are stateless. They don't remember the previous turn, so every API call carries the full conversation. The messages look like this:

messages = [
    {"role": "system",    "content": "You are a coding agent..."},
    {"role": "user",      "content": "Fix the failing test"},
    {"role": "assistant", "content": "I'll start by reading the test file."},
    {"role": "tool",      "content": "... contents of test file ..."},
]

There are four roles. The system role holds standing instructions, user is you, assistant is the model, and tool carries the output a tool handed back. The model takes in the list and produces the next entry, and that list is its only "memory". No hidden second store exists, so if the list is wrong, the agent is wrong.

Two loops are involved. The inner one covers a single user turn and as many model turns as it takes. The outer one simply waits for input.

This is the inner loop in simplified form. The real file also trims old tool output, appends a fresh reminder to the end of each request, removes temp files and can compact the history.

while True:
    message, usage = call_llm(messages)

    messages.append(message.model_dump(exclude_none=True))
        if message.content:
            ui.agent(message.content)
        # No tool calls? The agent is done thinking.
        if not message.tool_calls:
            break
        # Execute each tool and feed the result back
        for tool_call in message.tool_calls:
            args, result = execute(tool_call)
            ui.tool(tool_call.function.name, args, result)
            messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": result,
            })

The code lives in agent.py. call_llm(messages) sends the full history and the model returns text, tool calls or both. model_dump() merely converts the API object into a dict that can be stored. With no tool calls, the loop breaks. Otherwise it executes them, shows them, appends the results and goes around again, so the model sees what happened and decides whether to carry on.

The outer loop is smaller still:

while True:
    user_input = ui.ask()
    if not user_input:
        break

    messages.append({"role": "user", "content": user_input})
        # ... inner agent loop runs here ...

Your text goes onto the list, the inner loop runs until the agent finishes, and then the program waits again. There's no state machine, just two nested while True statements, the same shape Claude Code and Cursor have. The model talks or asks for tools, tool results feed back into messages, and the cycle repeats.

2. Tools: hands for the model

Without tools the loop is merely a chatbot with extra steps. Tools are how the model gets things done: reading files, running commands, writing code, searching a repo.

Imagine asking a brilliant person to fix a bug while they sit in a locked room with no computer. They can think but can't touch anything. Tools are the door.

Many tutorials gloss over one fact: tools are never executed by the model itself. What it produces is a request along the lines of "run this command", and the surrounding Python is what does the work and sends the output back. The model doesn't get a shell, only a form.

A tool consists of four things: a name (bash, read_file), a description that tells the model when to use it, a JSON Schema for the arguments with required fields and types, and a Python function that does the work.

bash is both the most powerful tool and the most worrying. Here it is from tools.py:

def bash(command: str) -> str:
    """Run a shell command and return its combined stdout and stderr."""
    try:
        result = sandbox.run(command)
    except subprocess.TimeoutExpired as expired:
        return (
            f"Timed out after {expired.timeout}s and was killed. "
            "Narrow it down - search inside the working directory rather than /."
        )
    return (result.stdout + result.stderr) or "(no output)"

What the model sees is the schema:

{
    "type": "function",
    "function": {
        "name": "bash",
        "description": "Run a shell command and return its combined stdout and stderr.",
        "parameters": {
            "type": "object",
            "properties": {
                "command": {
                    "type": "string",
                    "description": "The shell command to run",
                }
            },
            "required": ["command"],
        },
    },
}

When the model wants a directory listing, it emits {"name": "bash", "arguments": {"command": "ls -la src/"}}. The code runs it, returns the output, and the model chooses its next step.

The agent has seven tools, each a function plus a schema:

• bash runs a shell command through the sandbox.
• read_file reads a file.
• write_file creates or overwrites one.
• str_replace makes a surgical edit by swapping one string for another.
• read_skill loads a SKILL.md runbook.
• write_todos replaces the whole to-do list.
• task starts a subagent and returns only its final answer.

The bash function is a thin wrapper: it runs through sandbox.run with a 60-second timeout and returns stdout and stderr, or a timeout message advising a search inside the project directory, not across /.

Treat tool calls as untrusted input

Plenty of "build an agent in 20 lines" posts skip this part. A tool call is text that the model wrote, so the threat model matches a web form. The name may not exist, the arguments may not be valid JSON and the types may be wrong. Any of those can end the session if the code isn't prepared.

Models hallucinate. They make up function names and emit broken JSON. A crash forces the user to start over. If you catch the problem and return a readable error, the model often retries correctly, and it's surprisingly good at that, provided you hand it the error and not a stack trace.

execute(), a defensive dispatcher also in tools.py, works through this order:

1. Parse the JSON, and if it's broken, return an error saying the arguments were not valid JSON.
2. Look up the tool name, and if it's unknown, list the real tools.
3. Consult permissions.check() for a deny, ask or allow verdict.
4. Call the Python function.
5. Catch TypeError and any other exception, and return an "Error: ..." string and never a traceback.

In short, it routes a tool call to a function without ever crashing the session.

def execute(tool_call):
    name = tool_call.function.name

# The model might send broken JSON
    try:
        args = json.loads(tool_call.function.arguments)
    except json.JSONDecodeError as broken:
        return {}, f"Error: arguments were not valid JSON ({broken})."
    # The model might hallucinate a tool name
    if name not in TOOLS:
        return args, f"Error: no tool named '{name}'. Available: {', '.join(TOOLS)}."
    # Run through the permission layer, then call the function
    try:
        action, reason = check(name, args)
        if action == "deny":
            return args, f"Blocked by policy: {reason}"
        if action == "ask" and not ui.approve(reason):
            return args, "The user denied this tool call."
        return args, TOOLS[name](**args)
    except TypeError as mismatch:
        return args, f"Error: wrong arguments for {name} ({mismatch})."
    except Exception as failure:
        return args, f"Error: {name} failed - {type(failure).__name__}: {failure}"

Suppose the model invents a tool called search_code. The JSON is fine but the name isn't in TOOLS, so the dispatcher answers that no such tool exists and lists the available ones: bash, read_file, write_file, str_replace, read_skill, write_todos and task. On the next turn the model uses bash with grep -r "auth" .. Nothing crashed, which is the point: a bad tool call must never take the session down. Errors are just more input.

3. Permissions: who may do what

With bash available the agent is useful, and it's also how you end up running rm -rf / or sending secrets somewhere. If it can run anything, it can delete files, push junk or call home. If you confirm every command, you'll spend the afternoon approving ls. A middle path is needed.

The answer is three levels, like a traffic light:

Rules are glob patterns where * matches anything, and the last matching rule wins. From permissions.py:

BASH_RULES = {
    "*":                   "ask",     # default: ask for everything
    # read-only: let them through
    "ls*":                 "allow",
    "cat *":               "allow",
    "grep *":              "allow",
    "git status*":         "allow",
    "git diff*":           "allow",
    "pytest*":             "allow",
    # risky: never, even if the user says yes
    "rm *":                "deny",
    "sudo *":              "deny",
    "curl *":              "deny",
    "git push*":           "deny",
}

The * entry sets the default to ask. Read-only commands are carved out as allowed, then the dangerous ones are blocked outright, and everything else, such as python script.py or npm install, stays on ask.

The model will try clever tricks. A command like grep foo | rm -rf / starts harmlessly and ends catastrophically. So the checker splits on |, ; and &&, respecting quotes, and applies the strictest verdict from any piece:

def decide(command):
    """Rate every part of a compound command; the strictest verdict wins."""
    verdicts = []
    for part in split_command(command):
        action = "ask"
        for pattern, rule in BASH_RULES.items():
            if fnmatch(part, pattern):
                action = rule
        verdicts.append(action)

    for strictest in ("deny", "ask"):
            if strictest in verdicts:
                return strictest
        return "allow"

In practice, grep foo | rm -rf / is denied because rm is denied. ls && mkdir test gets ask, since mkdir isn't allowlisted. cat file.py | grep TODO is allowed because both halves are read-only.

Permissions and sandboxing do different jobs. Permissions are user experience: is this worth interrupting you for? They aren't security, because a clever command can look like ls and be something else. A sandbox is security, since it answers whether the action is physically allowed at all. A real agent needs both.

Where this leaves you

A loop, defensive tools and permissions already add up to a working coding agent. It can read a repository, run tests, edit files, ask before doing something foolish and recover from a malformed tool call.

It isn't yet safe, because a determined model can still do damage. It isn't efficient either, since the context window will fill up. And it has no memory, so each session starts cold. An OS-level sandbox, so destructive actions fail in the kernel, plus layered context management so long sessions hold together, are the natural next steps. These are the same primitives production agents use, minus the mystery.