Prompt, Context, Harness and Loop Engineering: The Four Layers of an AI Agent

In mid-2025, Shopify's CEO Tobi Lütke wrote that the real skill in working with LLMs wasn't prompt engineering at all, but supplying enough context for the task to be plausibly solvable. Andrej Karpathy agreed, calling it a craft that involves putting precisely the right information into the context window for whatever the model does next. Soon after, "context engineering" had replaced "prompt engineering" in most serious discussions of building on LLMs.

A year on, the teams shipping autonomous coding agents, issue-triage bots and self-healing deploy pipelines keep running into the same wall. Context engineering turned out to be one slice of a four-layer problem. From narrowest to broadest the layers are prompt, context, harness and loop, and each one wraps the layer inside it. The outer two, harness engineering and loop engineering, get the least attention, which is odd, because that's where most of the payoff now sits.

Think of it as four questions. How is the request worded? That's the prompt. What does the model already know when it starts? That's the context. What can it actually do once running? That's the harness. And when, and how often, does anyone call it? That's the loop. Today most of the real payoff sits in the loop, a layer that barely had a name a year ago.

What follows is a working framework for telling the layers apart, plus a build order. It assumes you've already done tool or function calling against an LLM API. If you haven't, start there.

A pattern shows up in practice that explains why the framework is useful. When an agent's output disappoints, the fix is nearly never in the prompt. It's almost always one layer up.

Layer 1: the prompt, or what you say

Prompt engineering is the narrowest layer and deals with a single thing: how one instruction to the model is phrased.

Examples: how a system prompt is phrased, how a few-shot sample is laid out, if you ask for step-by-step reasoning or careful deliberation, and if the answer should come back as JSON or free text. It's the craft of writing one message that gets the model to do what you want on its next turn.

The work is real. Accuracy on one and the same task can differ enormously between a loose instruction and an exact one. Take a triage agent for a GitHub project, which will serve as the running example. A weak prompt looks like this:

Look at this GitHub issue and label it.

And a precise one:

You are a triage agent for project X. For each issue, output JSON with:
  component  (one of the values in the taxonomy provided in context)
  severity   (critical | high | medium | low)
  team       (the owning team from the on-call table provided in context)
Base severity only on user-facing impact stated in the issue, not on your
own guess about difficulty. If the component is ambiguous, set
"component": "unknown" rather than guessing.

The model, the tools and everything else are identical, yet the second prompt yields structured, defensible output. Look at what it takes for granted, though. It runs inside one turn of one call. It presumes the taxonomy and the on-call table are already loaded, that the tools exist, and that something else decides when the model gets invoked.

Most tutorials stop at this point. Most production agent systems are barely getting started here.

Layer 2: the context, or what the model knows

Context engineering covers everything that enters the window before the model reasons about your prompt. It sits upstream of prompt work. A flawless prompt still produces garbage when the context is wrong.

That includes which documents, code files or database records are retrieved and injected, which tool schemas are loaded and which are held back to save tokens, how much conversation history survives versus being summarised or dropped, and what the system instructions say about the model's role and limits.

Drew Breunig catalogued the ways context fails in 2025. Context poisoning is when a hallucination enters the window and keeps getting referenced. Context distraction is when an overlong window makes the model lean on it and neglect what it learned in training. Context confusion is when irrelevant material drags output quality down. Context clash is when newly added tools or facts contradict existing instructions.

Context also costs money, and its encoding changes the bill. TOON, a compact encoding of the JSON data model, reportedly uses around 42.6% fewer tokens than JSON across its own benchmark suite as of mid-2026, with retrieval accuracy about level (72.2% against 71.4%). The saving depends on shape. On flat tabular data the cut climbs to nearly 59%, on mixed nested data it falls closer to 33%, and plain CSV can beat both when the data is purely flat. Those are the project's own figures, so check them on your data. The takeaway is that one set of facts, written out in three formats, differs in price and in how reliably it is read back. Context engineering means choosing not only what to include but how to encode it.

A short illustration shows how this goes wrong. Picture an article-generation agent that keeps producing generic output. Two days go into rewriting the prompt, adding examples and constraints, and tightening the role definition, and nothing moves. The prompt was never the problem. Thirty tool schemas are loaded into context up front and drown out the relevant instructions. Switching to lazy loading, where a definition appears only when the agent asks for it, fixes quality without changing a word of the prompt. Two days were spent on the wrong layer.

If prompt engineering is picking your words, context engineering is picking what the model may see before you say anything.

Layer 3: the harness, or what the model can do

The harness is the executable environment around the model: everything it can reach for and everything that reaches back.

Anthropic's well-known guidance on building effective agents introduced the idea of an agent-computer interface, deliberately parallel to the human-computer interface. Its argument is that you should put as much effort into designing how an agent works with tools as the industry puts into how humans work with software.

Here is a simplified tool definition from the triage example:

{
  "name": "apply_label",
  "description": "Apply a label to a GitHub issue. Only use after reading the issue body and checking the component taxonomy.",
  "parameters": {
    "type": "object",
    "properties": {
      "issue_number": { "type": "integer" },
      "labels": { "type": "array", "items": { "type": "string" } }
    },
    "required": ["issue_number", "labels"]
  }
}

The description field is doing context work. It limits when the model reaches for the tool, not only which arguments it passes. But the schema is half the harness.

The other half lives in the orchestrator, which wraps the tool in things the model never sees. An approval gate decides whether apply_label fires automatically or queues for a person. A rate limit, say 60 label writes an hour, stops a runaway loop from spamming the repository. A sandbox boundary confines what the tool can touch. Such wrappers shape behaviour without any change to the system prompt.

A harness also includes:

• File system. Which files the model may open, edit, create or delete.
• Memory. How information is stored and retrieved across turns, such as correction logs, past decisions and project conventions.
• Feedback. The route by which results, error text and environment state reach the next turn.
• Safety boundaries. Permissions, sandboxing and approval gates.

Give an average prompt great tools and it will usually outperform a brilliant prompt with no tools. An agent that can't open the file it needs gains nothing from better wording. Harness first.

Layer 4: the loop, or what makes it an agent

Loop engineering is the mechanism that makes an agent autonomous. It acts, observes the result, decides what to do next and continues until a goal is met or a stopping condition fires, with no human prompting each step. This layer turns a tool-using chatbot into something that can work through problems alone.

Here is the same idea in pseudocode. It's illustrative and not runnable: helpers like model.decide and execute are left out.

def agent_loop(goal, max_iterations=10):
    context = gather_initial_context(goal)
    consecutive_errors = 0
    for i in range(max_iterations):
        action = model.decide(context, goal)
        if action.type == "done":
            return action.result
        result = execute(action)
        if result.error:
            consecutive_errors += 1
            if consecutive_errors >= 3:
                # stuck: same approach keeps failing, decompose differently
                context.add("Previous approach failed 3 times. Try a different strategy.")
                consecutive_errors = 0
        else:
            consecutive_errors = 0
        context.add(result)
    return escalate_to_human(context)

A few lines carry the decisions that matter: when to stop, when to change tack, when to give up. OpenAI built this pattern into a product when it added a /goal command to Codex CLI in April 2026. You set a goal, and the system loops until it judges the goal complete or runs out of token budget.

A naive loop retries until something works. A well-engineered one notices it's stuck, decomposes the problem differently, escalates to a person, or drops a failing approach. Designer Matt Webb remarked in March 2026 that an agent given a problem and a while loop will eventually solve it. The qualifier "eventually" is where the engineering lives: retry budgets, backoff after repeated failures, and decomposition strategies for when a single pass stalls.

That's why this layer is the biggest shift in power. With prompt engineering, a person writes each instruction. With context engineering, a person decides what the model sees on every call. With harness engineering, a person builds the tools once and the model reuses them indefinitely. Loop engineering differs in kind. A person designs a system that prompts the model, judges the result and calls it again with nobody in the room. Control moves from whoever prompts the agent to whoever designed the system doing the prompting.

Jon Udell offered a framing in mid-2026 worth keeping: he dislikes the phrase "human in the loop" because it hands authority to the machines, when really it's still our loop and we now recruit agents to join the team.

All four layers on one agent: GitHub issue triage

Imagine an agent working through new GitHub issues for a mid-size open-source project: tagging the component, rating severity and sending each to the right team.

• Prompt. The system prompt tells it to find the affected component, grade severity from critical to low, assign the correct team and answer with structured JSON.
• Context. For every decision the harness supplies the title and body, the five latest comments, the component taxonomy (a YAML file revised quarterly), earlier similar issues with their labels (retrieved by vector search as examples), and whoever is on call.
• Harness. The agent can read the issue through the GitHub API, search the codebase to confirm which component is involved, query a vector database of earlier triage decisions, apply labels, post a comment explaining its reasoning, and assign a team member. It also keeps a memory of its own decisions that humans later overrode, so it can learn from corrections.
• Loop. A webhook fires on every new issue and the orchestrator starts a triage run. The agent reads the issue, gathers context, decides and applies the result. When confidence falls below a threshold it flags the issue for human review and doesn't auto-label.

Every 100 issues the system compares its accuracy with human overrides and adjusts its thresholds. In a comparable system, the confidence threshold reportedly had to start at 0.85 and not the assumed 0.7, because wrong labels created more human work than missed ones. That kind of calibration is loop engineering: you find it only by running the system and watching its mistakes over time.

Take away any one layer and it fails differently. With no prompt, the output format is a guess. With no context, component names get invented. With no harness, it can work out what to do but can't label or search. With no loop, you're left with an assistant that only acts when someone types "triage this issue" into a chat window, one at a time.

Where the vocabulary comes from

The term "prompt engineering" dates to 2022. Riley Goodside proposed breaking it into context engineering and prompt programming back in 2023, but the vocabulary only stuck after Lütke and Karpathy used "context engineering" in June 2025 and Simon Willison explained why it was a better fit.

"Harness" and "scaffolding" entered standard use through Anthropic's work on effective agents, which formalised the agent-computer interface and treated tool design as first-class engineering.

Loop engineering is the youngest of the four, and nobody coined it. It grew out of several efforts converging: OpenAI's goal-driven loop in Codex, Anthropic's split between workflows (fixed paths) and agents (loops steered by the model), LangChain's scale from router to state machine to autonomous agent, and a spreading habit of having the human design the loop and stay out of its individual turns.

The four-layer split here is a synthesis of that converging usage and not an industry standard. Still, naming the layers speeds up diagnosis. Asking whether a problem is one of prompt, context, harness or loop narrows the search immediately.

Where to invest first

The layers were listed from narrowest to broadest, but you build them broadest-first, or rather harness-first. For a first autonomous coding agent, the order is as follows.

1. Harness. Provide file reading and writing, shell access, a way to run tests and git commands. A non-coding agent gets API calls, database queries or messaging instead; the idea is identical.
2. Loop. Even a crude one helps: run, see if the tests pass, return the error if not, and quit after three tries or on success. This alone delivers most of the benefit of autonomy.
3. Context. After watching the agent work in its loop, you'll see what it lacks, whether that's code, past errors or project conventions. Guessing beforehand mostly burns time.
4. Prompts. With a working harness and loop, the prompts needed are generally brief and plain. The difficult engineering has happened elsewhere by then.

There's one exception. If the agent has to produce precisely formatted output, such as structured JSON for an API or specific markdown conventions, prompt work matters early, since format compliance is hard to repair inside the loop. That is a special case, not the default.

Newcomers usually flip the sequence. Weeks go into ornate system prompts while the agent still has no tools, no loop and the wrong files in its context.

Moving from prompt to context engineering amounted to admitting that the material supplied to a model counts for more than the phrasing of the request. Moving on to harness and loop engineering concedes more: whoever builds the system surrounding the model holds more power than whoever chats with it.