Agent Harness, Loop and Graph: Three Layers Behind Reliable AI Agents

Names that feel familiar and aren't understood

The phrases "agent harness engineering" and "loop engineering" turn up so often in social posts and blog articles that they begin to feel understood. Familiarity is deceptive, and a small failure usually exposes it.

Imagine a research agent whose job is fetching arXiv papers, condensing them and compiling a weekly digest. Three test runs go well. Then a rate-limited API call fails without a sound, nothing in the code notices, and the agent begins inventing summaries of papers it never opened. The model isn't at fault, and neither is the prompt. What's missing is any way for the surrounding code to separate a paper that was really read from a plausible fabrication. That gap is real, it has a name, and prompt tweaks don't touch it.

Building the same agent three separate times makes the distinctions concrete: one pass tuned for the environment, another for feedback loops and a third for explicit control of the workflow. The field is only about eighteen months old in its current form, so naming is still contested. The underlying distinctions are real, though, and they map to real engineering decisions. Afterward, a broken agent should point to which of three layers to fix first.

On its own, a language model just maps text to text. Memory between calls exists only if you provide it. File access, shell commands and API calls all require plumbing that someone has to build, and the model can't verify its own answers without an outside check. People building agents in 2024 found this out when a smooth demo fell apart on the first unexpected user action.

The industry has responded by no longer treating "the agent" as one artifact. It's three separable engineering problems stacked on the model.

LAYER            QUESTION IT ANSWERS                         FAILS AS
----------------------------------------------------------------------
Harness           What can the agent see and touch?          Missing tools,
                   (environment, tools, memory,               lost state, blown
                   permissions, execution limits)             budgets, leaks
----------------------------------------------------------------------
Loop               How does one attempt turn into a           Confident wrong
                   correct one? (retry, verify, stop)          answers, infinite
                                                                spinning, silent
                                                                failure
----------------------------------------------------------------------
Graph              What is the actual shape of the work?       Wrong step order,
                   (sequence, branches, parallel paths,         no recovery path,
                   human checkpoints)                           no visibility
----------------------------------------------------------------------

Each layer is easiest to understand by building the smallest version, breaking it and then reading the research and framework docs to see why it broke.

The harness: what the agent can see and touch

The cleanest definition appears in a LangChain engineering note that reduces the idea to an equation: agent equals model plus harness. The model supplies reasoning. The harness supplies everything that reasoning needs to reach the real world, meaning tools, memory, context, permissions and the guardrails that stop it from doing something costly or dangerous.

A common early mental model of "giving an agent tools" is writing a few Python functions and describing them in the system prompt. That's fine for a demo. It doesn't survive a task longer than one context window, or a tool call that fails at 2 a.m. with nobody watching. In practice the harness turns out to own six responsibilities.

• Context injection decides what the model sees before each reasoning step: the user's message, but also retrieved documents, earlier conversation, project state and organizational policy. Over-correcting is easy. Dumping an entire file tree and git log into every call makes accuracy worse because the model drowns in irrelevant tokens, and a smaller curated context consistently beats a larger raw one.
• Action surfaces turn a decision into an effect in the world, whether that's an API request, a shell command, a database write or a browser click. The Model Context Protocol sits here, and it shifted a lot this year. It started in late 2024 as a standard way for agents to find and invoke tools, sparing teams a custom integration per API. By the July 2026 spec it had a stateless core that works behind regular load balancers with no sticky sessions, a Tasks extension from AWS covering jobs that outlast a single request, and stricter authorization built on OAuth and OpenID Connect. The source cites over ten thousand public production servers and SDK downloads above 97 million a month. Building a harness today on a tool layer that isn't MCP or similar needs a good justification.
• Persistence is anything that has to outlive a restart, a crash or a full context window, such as checkpoints, session state, vector memory and git history. The first research agent kept none of it, so one dropped connection wiped forty minutes of progress. Writing a small JSON checkpoint after every finished sub-task did more for reliability than any prompt edit.
• Execution control bundles retry rules, timeouts, token and spending budgets, model routing, limits on spawned sub-agents and the steps where a person must sign off. Without it, an agent stuck in a call-and-fail cycle overnight can inflate a job into a one.
• Safety and governance cover least-privilege access to tools, handling of secrets, sandboxing, audit logs and allowlists. Code written by the model should always run in a sandbox. A coding agent has been caught about to run rm -rf on a folder it wrongly took for a build artifact. No malice was involved, just an error combined with permission to do damage.
• Observability means traces, tool-call logs, token usage, latency and evaluation results. It sounds dull until an agent starts behaving oddly in production and there's no record of what it actually did as opposed to what was assumed.

Below is a minimal but functioning harness in Python. A local Ollama model drives it, so no API key is required, and switching to a hosted model takes one line.

import json
import subprocess
import requests
from datetime import datetime
OLLAMA_URL = "http://localhost:11434/api/chat"
MODEL = "llama3.1"
# --- Action surface: the only two things this agent is allowed to do ---
def read_file(path: str) -> str:
    with open(path, "r") as f:
        return f.read()[:4000]  # cap it, don't blow the context window
def run_tests() -> str:
    result = subprocess.run(
        ["pytest", "-q"], capture_output=True, text=True, timeout=60
    )
    return result.stdout[-2000:] + result.stderr[-2000:]
TOOLS = {"read_file": read_file, "run_tests": run_tests}
TOOL_SCHEMA = [
    {
        "type": "function",
        "function": {
            "name": "read_file",
            "description": "Read a file from the project",
            "parameters": {
                "type": "object",
                "properties": {"path": {"type": "string"}},
                "required": ["path"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "run_tests",
            "description": "Run the project's pytest suite",
            "parameters": {"type": "object", "properties": {}},
        },
    },
]
# --- Persistence: cheap but real ---
def checkpoint(state: dict, path: str = "checkpoint.json"):
    with open(path, "w") as f:
        json.dump({**state, "saved_at": datetime.utcnow().isoformat()}, f)
# --- Execution control: hard ceilings, not suggestions ---
MAX_TOOL_CALLS = 8
def harness_run(user_task: str):
    messages = [{"role": "user", "content": user_task}]
    calls_made = 0
    while calls_made < MAX_TOOL_CALLS:
        response = requests.post(
            OLLAMA_URL,
            json={"model": MODEL, "messages": messages, "tools": TOOL_SCHEMA, "stream": False},
        ).json()
        msg = response["message"]
        messages.append(msg)
        if not msg.get("tool_calls"):
            return msg["content"]  # model is done, no more tools needed
        for call in msg["tool_calls"]:
            name = call["function"]["name"]
            args = call["function"].get("arguments", {})
            if name not in TOOLS:
                result = f"error: tool {name} is not permitted"
            else:
                result = TOOLS[name](**args)
            messages.append({"role": "tool", "content": str(result)})
            calls_made += 1
            checkpoint({"messages": messages, "calls_made": calls_made})
    return "stopped: hit the tool call ceiling before finishing"

Nothing here is fancy, which is the lesson: a harness is careful bookkeeping around a model that doesn't keep books. For a hosted model, replace the requests.post block with an OpenAI or Anthropic SDK call. The tool schema and checkpointing stay untouched because they don't depend on who answers.

Complaints that the model has gotten dumber nearly always trace back to the harness. A tool may have returned stale data, the context may be clogged with irrelevant history, or a permission may be too wide or too narrow, leaving the agent hammering on something it was never allowed to do. Look at the harness before editing prompts or changing models, because that's where the fault usually is.

The loop: how one attempt becomes a good one

If the harness decides what the agent can touch, the loop decides how it gets from a first try to a good result. It's the layer most often underestimated, since "just let it retry" sounds trivial until an agent actually retries.

The classic failure is a self-correcting loop that asks the model, after it answers, whether it's confident the answer is right. The reply is always yes. A model grading its own homework without any external signal passes itself nearly every time, because there's nothing for it to fail against. This is a well-documented problem and not a one-off mistake: a loop with no evidence to fail against will always believe it succeeded.

The way out is to quit asking the model to grade itself and give the loop an outside referee: a test suite with a pass or fail, a schema validator, a compiler, a lint rule, or a second independent retrieval that backs up or contradicts the first answer. Evidence-driven loops look at the state of the world. Confidence-driven ones look at how the model feels.

The Ralph Wiggum technique and why a crude loop works

A pattern called "Ralph Wiggum" gained popularity after engineer Geoffrey Huntley described it in 2025. Boiled down, it's strikingly basic. A bash loop passes the same prompt to a coding agent over and over until the job is finished, and progress lives in files and git commits instead of the model's recollection of the chat.

#!/bin/bash
# ralph.sh - crude but surprisingly effective
while true; do
  claude -p "$(cat PROMPT.md)" --dangerously-skip-permissions
  if git diff --quiet HEAD~1 HEAD -- STATUS.md && grep -q "DONE" STATUS.md; then
    echo "Task complete."
    break
  fi
  sleep 5
done

It looks like it shouldn't work, and on a small refactor its crudeness turns out to be the feature. Discarding the conversational context on every pass and forcing progress into git history and status files sidesteps context rot, and it stops the agent from convincing itself that it already finished something it didn't. It isn't elegant. It's evidence-driven by accident, since git and the filesystem don't lie to flatter the model.

Here's a small evidence-driven loop of the same kind, again runnable locally against Ollama, which treats a failing test as the only acceptable meaning of "not done yet."

def evidence_loop(task_description: str, max_attempts: int = 5):
    attempt = 0
    history = []
    while attempt < max_attempts:
        attempt += 1
        code = generate_patch(task_description, history)   # calls the model
        apply_patch(code)
        test_output = run_tests()                          # external evidence
        passed = "failed" not in test_output.lower()
        history.append({
            "attempt": attempt,
            "patch": code,
            "test_output": test_output,
            "passed": passed,
        })
        if passed:
            return {"status": "success", "attempts": attempt}
        # feed the *actual failure*, not a vague "try again"
        task_description = (
            f"{task_description}\n\nPrevious attempt failed with:\n{test_output}\n"
            "Fix the specific failure above."
        )
    return {"status": "gave_up", "attempts": attempt, "history": history}

Look at the last line before the return, because it carries the idea. The loop doesn't say "try harder". It feeds the model the real test failure, since concrete outside evidence is what makes the second attempt better than the first rather than a random reroll. In most agents, the largest reliability gain comes from swapping "does this look right?" for a genuine external check of whatever kind fits the task.

It's also essential to know when to stop. A loop with no hard ceiling on attempts, cost or wall-clock time isn't a sturdy agent. It's a bug waiting for a bill.

The graph: when the task has real branches

A harness provides the agent's body and a loop lets it improve attempt by attempt, yet neither describes a task shaped like a tree: do this, then pick one of three actions depending on the outcome, wait for a human's approval, then run two checks in parallel and merge them.

That's graph engineering. It tends to be the layer people hold out against, since a loop looks adequate for a long while, but it stops being adequate when the work isn't one straight run toward a single passing test. Loops are good at one thing only: attempt, verify and revise until one condition is met. They're awkward when the next step hinges on which of several results occurred, when two subtasks could proceed simultaneously, or when a person has to approve before continuing. Stuffing all that into a single prompt and hoping the model keeps track worked for some teams, and it's brittle. Once branching is real, the control flow belongs in an explicit structure, not in whatever the model improvises each turn.

Graph engineering makes the workflow explicit. Nodes are steps, such as an agent call, a tool call, a validator or a human checkpoint, and edges are the conditions that move execution from one node to the next. Looking at the graph shows exactly which path a run took, which matters a great deal the first time a production incident hits at midnight and you need to know what happened rather than what you assume happened.

LangGraph is the usual pick here. It represents a workflow as a directed graph, passes typed state from node to node, uses conditional edges for branching, and has checkpointing built in, so a run can stop, restart or revert to an earlier state.

from langgraph.graph import StateGraph, END
from typing import TypedDict

class AgentState(TypedDict):
    task: str
    draft: str
    review_passed: bool
    attempts: int
def draft_node(state: AgentState) -> AgentState:
    state["draft"] = generate_draft(state["task"])
    return state
def review_node(state: AgentState) -> AgentState:
    state["review_passed"] = run_validator(state["draft"])
    state["attempts"] += 1
    return state
def needs_human(state: AgentState) -> str:
    if state["review_passed"]:
        return "approved"
    if state["attempts"] >= 3:
        return "escalate"   # branch: give up automating, ask a human
    return "retry"
graph = StateGraph(AgentState)
graph.add_node("draft", draft_node)
graph.add_node("review", review_node)
graph.set_entry_point("draft")
graph.add_edge("draft", "review")
graph.add_conditional_edges(
    "review",
    needs_human,
    {"approved": END, "retry": "draft", "escalate": "human_approval"},
)

A plain loop struggles to say what this does easily: one check with three results (finished, retry, stop and ask a person), each leading somewhere different. The escalate path matters. It's the difference between an agent that silently gives up after a third failure and one that gives a human the full history of what was attempted.

Microsoft's AutoGen added something similar this year, called GraphFlow, for the same reason. Multiple agents or branching workflows make an implicit loop hard to follow, and a graph can be inspected directly.

To judge where each framework actually shines instead of relying on marketing pages, four were tried on a single small project: a document-review pipeline featuring a validation branch and a human approval gate.

FRAMEWORK          ORCHESTRATION MODEL         STRONGEST AT               STATE / PERSISTENCE
--------------------------------------------------------------------------------------------
LangGraph           Directed graph, typed        Complex branching,        Built-in checkpointing,
                     state, conditional edges     long-running workflows,   time-travel debugging
                                                   human-in-the-loop
--------------------------------------------------------------------------------------------
CrewAI               Role-based "crew" of         Fast prototyping,         Sequential task output
                      agents, sequential or        multi-agent brainstorm   passing, lighter state
                      hierarchical process         style collaboration      model
--------------------------------------------------------------------------------------------
OpenAI Agents SDK     Explicit agent handoffs      Minimal-friction start   Ephemeral context
                                                    inside the OpenAI        variables by default
                                                    ecosystem
--------------------------------------------------------------------------------------------
AutoGen / GraphFlow   Multi-agent conversation     Agent-to-agent           Conversation history
                       plus optional graph layer   negotiation, graph-      plus graph state
                       for explicit control        based orchestration
--------------------------------------------------------------------------------------------

The takeaway is practical. If the workflow fits on a page as a flat list of steps, a graph framework isn't needed, and a well-built loop inside a solid harness does the job with less code to maintain. Once you draw the workflow and it has a fork, a graph framework begins paying for itself right away, mostly in debugging time saved rather than anything it does at run time.

How the three layers stack

These aren't rival approaches. They're stacked vertically, and here's how the stack looks once you've built with all three.

+--------------------------------------------------------------+
|  GRAPH   - decides which node runs next, where branches      |
|            are, where a human has to approve                 |
+--------------------------------------------------------------+
|  LOOP    - inside a given node, drives attempt -> verify ->  |
|            revise until the evidence says stop               |
+--------------------------------------------------------------+
|  HARNESS - underneath both, provides the tools, memory,      |
|            permissions, and limits that make any of it       |
|            possible to execute safely                        |
+--------------------------------------------------------------+
|  MODEL   - the reasoning engine none of the above exists     |
|            without                                           |
+--------------------------------------------------------------+

Inside a LangGraph workflow, one draft node could run an evidence-driven loop for a few tries and only then pass control back to the graph. That inner loop works only because the harness beneath lets the model run tests, read files and recall its earlier attempt. Pull out any layer and the rest break, so prompting a monolithic "agent" harder was always going to plateau.

Thinking in three layers makes debugging noticeably faster. This is the checklist to run through, in order, before touching a prompt.

SYMPTOM                                    LIKELY LAYER TO CHECK FIRST
----------------------------------------------------------------------
Agent doesn't know something it should      Harness -> context injection
Agent "can't" do something it should be     Harness -> action surface /
  able to do                                 permissions
Agent forgets earlier progress after a      Harness -> persistence
  restart or long run
Costs or run time spiral unexpectedly       Harness -> execution control
Agent confidently reports success on        Loop -> swap confidence checks
  work that's actually wrong                 for external evidence
Agent retries the same mistake              Loop -> is it seeing the actual
  repeatedly without improving                failure, or just "try again"?
Agent takes the wrong path when a task      Graph -> is the branching logic
  has more than one possible outcome          explicit, or implicit in a
                                               single prompt?
No human ever gets a chance to catch a      Graph -> is there an approval
  bad decision before it ships                node, or is it all automatic?

A table like this would have saved a full weekend of blaming the prompt for what was really a missing checkpoint.

Open questions

Some things remain unsettled, and it would be wrong to wrap up sounding more certain than the evidence allows.

First, the vocabulary is still moving. "Harness engineering" barely existed at scale before this year, and in another year it may be absorbed into a broader "agent engineering" label, with harness, loop and graph becoming subheadings and not disciplines people specialize in.

Second, the Ralph Wiggum approach feels slightly off even when it succeeds. Giving a model one prompt in a blunt loop and relying on git history as its only memory looks like a patch over a harness lacking real persistence. It fits small, tightly scoped refactors, and for anything with real branches or consequences it shouldn't be trusted yet.

Third, graph frameworks may still be changing shape. LangGraph's checkpointing is excellent, but teams do adopt a whole graph framework for a three-step flow that never forks, just because it's trendy. A workflow that's a straight line can stay one.

One claim survives all of this: an unreliable agent is almost never cured by a better prompt. The gap nearly always sits in one of the three layers, and the questions above locate which.

Starting the research agent over today would mean building the harness first and treating it as boring, load-bearing infrastructure rather than an afterthought. An evidence-driven loop would go in from day one in place of trusting the model's self-assessment, since that change alone fixes more bugs than anything else. A graph framework would wait until the workflow could be drawn on paper with a visible fork.

The model has soaked up attention for three years, but reliability in production comes from the engineering wrapped around it. That work is far less glamorous and far more tractable than the claim that the model isn't smart enough. Most of the time it is. It simply hasn't been handed the environment, feedback or map required to show it.