Debugging AI Agents by Layer: Prompt, Context, Harness or Loop

When an AI agent misbehaves in production, teams tend to argue about vocabulary instead of evidence: one engineer wants a better prompt, another blames the context, a third points at the harness. Prompt, context, harness and loop engineering are not rival schools of thought; they are four stacked layers, and each one fails in its own recognizable way. This guide defines each layer with a small coding-agent example, then gives you a trace-first routine for deciding which layer actually broke before you change any code.

The short version

• Prompt engineering shapes the instruction given to the model.
• Context engineering decides what the model gets to see before it answers.
• Harness engineering builds the environment the model acts in: tools, memory, files, permissions and recovery.
• Loop engineering designs the cycle that keeps the system working, checks its own progress and decides when to stop.

Most expensive agent incidents come from fixing the wrong one of these layers.

When the demo passes and production does not

The pattern is familiar. An agent looks flawless in a demo. A few days after launch it starts looping and burning tokens, loses track of something it was told an hour earlier, or falls over the first time a tool returns an unexpected payload.

The team splits into camps. Someone proposes rewriting the prompt, someone else calls it a context problem, and a third person suspects the harness. Without a shared way to locate the failure, a two-day fix becomes a two-week rewrite, because each patch lands on a layer that was working fine.

That is the practical reason to keep the four terms apart. Each names a distinct place where things go wrong, and once you can tell them apart, agent debugging becomes a process instead of a guessing game.

PACT: a mnemonic for the four layers

A compact way to hold the layers in your head during an incident is PACT: Prompt, Awareness, Control, Trajectory. Each word maps to one question:

• Prompt: was the job specified clearly?
• Awareness: did the model receive the information it needed for this step?
• Control: can the runtime execute what the model asked for, safely and reliably?
• Trajectory: does the repeated process move toward a finish that can be verified?

The goal is not yet another layer of jargon. It is to make the boundaries between failure types easy enough to recall that you actually use them when something is on fire.

How the layers appeared, and why the order matters

The terms arrived roughly in sequence, each as agents gained a new capability:

• Prompt engineering (roughly 2022 to 2024) was the first skill: phrasing, examples, constraints and few-shot patterns aimed at one model call.
• Context engineering (roughly 2024 to 2025) took over once agents juggled retrieved documents, conversation history and tool definitions at the same time. The question shifted from how to phrase a request to what the model actually sees. Anthropic, LangChain and Chroma helped popularize the framing among practitioners, and Andrej Karpathy's commentary helped carry it into wider engineering discussion.
• Harness engineering (early 2026) became a focus once agents had filesystem access, shell commands and runs lasting hours. Attention moved to the reach of the agent and to the behavior after a failed tool call. OpenAI's work on Codex made this a prominent concern for coding agents; its write-up on harness engineering with Codex is a useful primary reference.
• Loop engineering (mid-2026) is the newest label. With a harness in place, something still has to decide what each iteration does and when to quit: observe, act, verify, repeat or stop. IBM maintains an explainer on loop engineering if you want another framing.

These dates describe when the terms gained traction, not formal milestones, and the vocabulary is still settling at the time of writing. The important observation is that nothing was replaced. Every layer was added on top of the previous one, because each increase in autonomy needed its own control surface.

Prompt engineering: the local contract

Prompt engineering is the craft of wording, structuring and illustrating one instruction so that the response is more predictable. Without it you get vague instructions, drifting output formats and a model forced to guess what you meant.

In a coding agent, a code-review prompt might look like this:

SYSTEM: You are a code reviewer.
Given a diff, output ONLY valid JSON:
{"issues": [{"line": int,
"severity": "low|medium|high", "note": str}]}
No prose. No markdown fences.
If there are no issues, return {"issues": []}.

This prompt sets a local contract. It assigns a role, describes the transformation (diff in, list of issues out), fixes the output shape down to the field names and allowed severity values, and even defines the empty case so downstream parsing never has to handle prose or a missing key.

A common claim is that prompt engineering is obsolete. It is not; it is the innermost layer. Every context pipeline eventually hands the model an instruction, and a weak instruction inside an excellent harness still yields weak results, only now wrapped in impressive infrastructure.

Context engineering: curation under a budget

Context engineering is the job of choosing, for each call, which documents, history, tool definitions and memories make it into the window, and just as deliberately, which ones do not. When it is neglected, the model answers from stale information, gets pulled off course by irrelevant retrieved passages, or loses focus because the window is packed with material included just in case.

A context builder for a coding agent might retrieve candidates, rerank them, and compress the history:

def build_context(query, full_history, kb):
    relevant = retrieve(query, kb, top_k=8)
    reranked = rerank(relevant, query)[:3]
    summary = summarize_if_long(
        full_history, max_tokens=800
    )
    return {
        "docs": reranked,
        "history": summary,
        "query": query,
    }

Read it as a funnel. Retrieval casts a wide net of eight candidates, reranking keeps the three most relevant, and the conversation history is summarized to roughly 800 tokens only when it grows long. The function returns a small, structured payload rather than raw material. The skill on display is selection, ranking and compression, not volume.

That is why the most common misunderstanding, that context engineering means giving the model more information, is usually backwards. A bigger context tends to carry more noise: outdated instructions, repeated facts and conflicting evidence. Much of the work is choosing what to leave out.

Context engineering is broader than RAG

Retrieval-augmented generation is one technique inside context engineering, covering the retrieval step. A RAG pipeline might pull eight chunks and rerank them to three, as above. Context engineering also includes compressing history, formatting tool definitions, keeping critical task state alive, feeding verification results back to the model, and deciding what to omit.

A coding agent without any vector store still has a real context problem to solve. Git status, open files, compiler errors, test output, the current plan and the log of previous actions all need to reach the model in a usable form.

Harness engineering: the boundary between intent and effect

The harness is the non-model part of the system: the tools and file access, persisted memory, permission rules, sandboxes, tracing, and whatever happens on failure. Without a good one, you get an agent that reasons well but cannot act on it, or one that acts but has no dependable way to recover when a tool call errors.

A tool-call wrapper illustrates the idea:

def call_tool(tool_name, args, retries=2):
    for attempt in range(retries + 1):
        try:
            result = TOOLS[tool_name](**args)
            log_trace(tool_name, args, result, status="ok")
            return result
        except ToolError as e:
            log_trace(tool_name, args, str(e), status="failed")
            if attempt == retries:
                return {"error": str(e), "recoverable": False}
            args = repair_args(args, e)

The wrapper makes no attempt to solve the task. It governs the boundary between what the model requested and what the real system does. Every attempt is traced with its arguments, result and status. A failure triggers a bounded retry with repaired arguments, and once retries are exhausted it returns a structured error flagged as non-recoverable, so the caller gets data it can reason about instead of an exception that kills the run.

It is tempting to equate the harness with whichever agent framework you installed. A framework provides scaffolding; the harness is the set of concrete decisions you make on top of it: what gets logged, what a failed call does next, what state survives a crash, which commands are allowed, and how execution results are presented back to the model.

Loop engineering: the termination contract

Loop engineering defines the work of every iteration, the way the agent measures whether it is getting anywhere and, most importantly, the conditions for stopping. Its signature failure is the unbounded loop: an agent that keeps invoking tools and spending tokens because nothing ever told it that it had finished, or that it was stuck.

A minimal controller looks like this:

def run_loop(task, harness, max_iters=15, stall_limit=3):
    state = init_state(task)
    stalls = 0
    for i in range(max_iters):
        action = plan_next_step(state)
        result = harness.call_tool(action.tool, action.args)
        state = update_state(state, result)
        if is_goal_met(state):
            return state, "success"
        if made_no_progress(state):
            stalls += 1
            if stalls >= stall_limit:
                return state, "stalled — escalate"
        else:
            stalls = 0
    return state, "max iterations reached"

Several limits are layered here. max_iters caps total work, is_goal_met provides a success exit, and a stall counter tracks consecutive iterations without progress, resetting whenever progress resumes. After stall_limit stalled steps, the loop returns a distinct status that asks for escalation rather than quietly continuing. Each exit path returns an explicit reason, which makes runs easy to classify afterwards.

The central idea is the termination contract: a clear goal, measurable progress signals, capped retries, budgets for time and tokens, a verification step, and a stated policy for what to do when progress stalls. For TypeScript implementations of the same ideas, see bounded agentic loops for LLM tool use.

Loop and harness engineering are often conflated. The harness defines what the agent may touch and how failures are handled at the level of individual tools. The loop sits above it and decides the turn-by-turn behavior, including the moment the run as a whole should end.

The four layers side by side

• Prompt (P): owns the instruction. Typical failure: the model misreads intent or returns the wrong format. First place to look: the output itself.
• Context (A): owns the model-ready state. Typical failure: the model acts on stale, missing or distracting information. First place to look: context snapshots per turn.
• Harness (C): owns execution and recovery. Typical failure: tool calls error, retry blindly or cannot be recovered. First place to look: failed entries in the tool trace.
• Loop (T): owns progress and stopping. Typical failure: the agent never converges or never stops. First place to look: repeated successful calls with near-identical arguments.

Telling a harness bug from a loop bug

The most time-saving observation is that two different bugs look identical from outside. An agent stuck in a loop because of a bad controller and an agent stuck because of broken tool handling both show the same symptom: it keeps running, the bill keeps growing, and nobody knows why. A short checklist separates them and the other layers.

1. Pull the tool-call trace

Calls that all succeed with plausible results, while the agent hammers one tool with almost unchanged arguments, point to the loop.

2. Look for repeated tool failures

If one tool fails again and again and the agent retries without changing its approach, suspect the harness.

3. Inspect what the model could see

If the model appears to forget a fact partway through a run, examine context construction: truncation, replacement, compaction or how state is carried between turns.

4. Check the output last

If tools worked, context was accurate and the loop ended cleanly, but the answer is still wrong, look at the prompt.

A rule of thumb

Loop bugs sit in the decision about what happens next. Harness bugs sit in what happens when something breaks. Context bugs sit in what the model can see. Prompt bugs sit in what you asked for.

Answer which of the four questions applies before you edit anything.

Case study: a pull-request reviewer that would not stop

Picture a team shipping a coding agent that reviews pull requests. Testing goes smoothly. In production, some reviews run for more than 40 minutes on a single pull request and generate an unexpected bill.

The first theory is that the prompt is too vague and the agent is overthinking. The prompt gets rewritten; nothing changes. The second theory is context: perhaps the agent rereads the whole repository on every step. The trace says otherwise. Retrieval is properly scoped, and only the files that matter get pulled in.

Then someone reads the tool-call trace carefully. The agent invokes its test-runner tool repeatedly, and each invocation completes without error. The test suite really is failing, because of a flaky integration test unrelated to the pull request. The agent keeps trying to fix that failure with unrelated code changes and rerunning the tests, since nothing tells it that it has spent enough attempts on this sub-goal and should stop and escalate.

This is not a prompt problem, not a context problem, and not a harness problem; the tools behaved exactly as designed. It is purely a loop bug. Walking through it step by step shows how to reach that conclusion.

Step 1: confirm the tools work

Successful executions in the trace eliminate the simplest harness failures, such as a command that never runs or an adapter that keeps throwing.

Step 2: compare state between iterations

The test output is present in the agent's context, and retrieval is scoped correctly, so the model is not missing the evidence.

Step 3: check for convergence

The repository changes on every iteration, but the verification signal that matters never improves. There is no stall detector and no cap on attempts at the same sub-goal.

Step 4: teach the controller to detect stagnation

The fix is a small piece of controller logic that compares a signature of progress between steps:

if progress_signature == previous_signature:
    stagnant_steps += 1
else:
    stagnant_steps = 0

if stagnant_steps >= 3:
    escalate("no measurable progress")
    stop()

The progress_signature can be anything that captures meaningful progress, for example the set of failing tests plus their error messages. If it does not change, the stagnation counter grows; after three stagnant steps the controller escalates with a reason and stops. Any real change resets the counter.

You can go further and cap attempts per failure signature, not just total iterations. The distinction is important: fifteen productive iterations may be perfectly fine, while fifteen tries against one unfixable test failure are pure waste. The real fix is not to make the model less stubborn but to give the controller a definition of stalled.

Case study: a constraint that disappears from context

Now imagine an agent that establishes, at the start of a migration, that nothing may break existing clients. Forty turns later, the conversation is compacted and the constraint is lost in the summary. The agent then proposes a breaking schema change.

It looks like faulty reasoning, but the cause lies elsewhere. Compare the context the model actually received at different turns. If the constraint was there at turn five and missing at turn forty, the fix belongs to context engineering: store critical invariants separately from the conversation, make summaries carry explicit constraints forward, and stop treating raw history as the only source of truth.

"The model forgot" is never a complete diagnosis. The useful question is what the model was actually given.

Layers, not replacements

Each newer discipline builds on the older ones rather than retiring them. A production agent wraps a clear prompt in carefully chosen context, runs it inside a dependable harness, and drives the whole thing with a deliberate loop. The succession of terms mirrors what teams have had to add over time: clearer instructions, then better-selected information, then an execution environment for the model, and finally a controller that keeps that environment running unattended. If you are deciding whether your own runtime or a framework should own that outer loop, the comparison of runtime harnesses and composed frameworks covers the trade-offs.

Restated through PACT:

• Prompt: define the instruction.
• Awareness: build the state the model will reason over.
• Control: make execution observable, bounded and recoverable.
• Trajectory: convert repetition into progress you can measure, ending at a verifiable finish.

Then match the fix to the failure. A model that misreads a clearly specified task needs prompt work. A model missing facts it depends on needs context work. Sound decisions that fail to turn into dependable actions call for harness work. Actions that succeed while the overall run never converges call for loop work.

Common questions

Is harness engineering just another name for an agent framework?

No. A framework supplies building blocks. The harness consists of the decisions you make while assembling them: behavior on tool failure, persisted state, logging, permissions, and how results return to the model.

Does a single-question assistant need loop engineering?

Not really. Loop engineering starts to matter when an agent takes several actions per task and has to decide, on its own or via controller code, that the work is complete. A single-turn question-and-answer bot has little or no loop to design.

Which layer should you learn first?

Start with prompt and context engineering, then move to the harness and the loop. You need to tell a bad instruction apart from missing state before you can debug the runtime and controller around them.

Can one bug span several layers?

Yes, and those are often the hardest. A context bug can hide a progress signal from the loop, which then looks like a loop bug. Follow the failure through the layers instead of patching the first visible symptom.

Key takeaways

• The four terms describe control surfaces, not competing trends: the instruction, the model-ready state, execution and recovery, and repeated progress with a stopping rule.
• Start every investigation from the trace, not the prompt: ask what the model saw, what the runtime did, what changed between iterations, and why the controller chose to continue.
• Successful tool calls that repeat with near-identical arguments point at the loop; repeated failures with blind retries point at the harness.
• Give loops an explicit definition of stalled, ideally per failure signature, and an escalation path.
• Keep durable constraints outside compressible history so compaction cannot erase them.