Loop Engineering: Build Agent Loops with Verifiers and Stop Rules

The mindset behind all this is simple to state. Stop steering an AI one instruction at a time and design the loop it lives in. The claim to carry forward is that the verifier, more than the model, limits what you can trust, and the picture to keep in mind is four loops nested inside one another. Here is how to build it: the parts of a loop, code for each layer, the ways loops fail, and the stop rules that stop one from eating your budget. At the end you'll know enough to let a loop prepare a pull request while you're asleep.

Code samples follow the documented APIs, model IDs are placeholders, and any snippet that's only pseudocode announces it in a leading comment. One example runs throughout, the usual starter for this field: a coding loop that reproduces a failing test, repairs it, gets the repair checked by something other than the agent itself, and opens a draft PR.

The five parts of a loop

Whatever framework you use, a sound loop always comes down to five pieces. Leave one out and the loop drifts, stalls or spends without end.

1. A trigger. The thing that kicks it off, whether a human message, a scheduled tick, a webhook or a fresh table row.
2. A goal a machine can check. "Make the tests pass" qualifies and "improve the code" doesn't. If software can't tell whether you've arrived, the loop can't either.
3. Tools that give real feedback. The agent has to act on the world and see the outcome by running code, reading stderr or calling an API.
4. Lean, managed state. Keep the goal, a scratchpad and a record of what's been tried. Bloated state is how loops slow down, wander and overflow their context window.
5. Explicit stop rules. This is the most neglected piece. A loop with no stop rule is a bug with a billing account.

Plan for three exits and not two. Success: the verifier passed, so ship. Surrender: the budget or iteration cap ran out, so stop and summarize what's left. Hand-off: the agent hit a decision only a person should make, so it escalates. Part of designing a good loop is recognizing the calls it shouldn't make itself.

Loop 1: the agent loop

At the center sits the cycle all agents already run, the 2022 ReAct pattern: think, act by calling a tool, observe the result, and repeat until finished. In 2026 it's designed on purpose and no longer driven by hand. In its simplest form it's one function call:

from langchain.agents import create_agent
def run_tests() -> str:
"""Run the project's test suite and return the raw output."""
…
agent = create_agent(
  model="anthropic:claude-opus-5",
  tools=[run_tests],
  system_prompt="You are a careful Python engineer. Fix failing tests.",
  )
# create_agent returns a graph that calls tools in a loop, repeating
# until the model replies with no tool calls. The loop runs on .invoke()

That short call hides a loop, and you should be able to picture it. Below is the same control structure as pseudocode with no framework at all. Note that every exit is bounded:

# PSEUDOCODE - shows the control structure a harness implements.
# The helper names below are yours to supply, not a real library API.
def run_loop(goal):
  state = init_state(goal) # goal + scratchpad + attempt log
  for step in range(MAX_STEPS): # hard cap: never loop forever
    thought = model.reason(state) # ReAct: reason…
    action = model.choose_action(state) # …then pick a tool call
    result = tools.execute(action) # touch the real environment
    state = update(state, thought, action, result)
    state = compact(state) # keep context under budget
    if verifier.passes(state): # deterministic check = reward
      return success(state)
    if no_progress(state) or budget.exhausted():
      return escalate_to_human(state) # third exit: hand-off
  return escalate_to_human(state)

Of all the shapes here, remember this one. The rest are refinements of it.

Loop 2: add a verifier

A bare agent decides for itself when it's done, and trusting that is how you get hallucinated success. So the agent gets wrapped in a grader that checks its work against a rubric. When the work fails, the critique goes back in and the loop continues, up to a hard cap. The verification middleware lives in the deepagents package.

from deepagents import RubricMiddleware, create_deep_agent
from langchain.messages import HumanMessage
from langgraph.checkpoint.memory import InMemorySaver
RUBRIC = """
- All tests pass in run_tests
- No test is skipped or deleted to force a pass
"""
agent = create_deep_agent(
  model="anthropic:claude-opus-5",
  system_prompt="You are a careful Python engineer.",
  middleware=[
  RubricMiddleware(
  model="anthropic:claude-haiku-4–5", # a cheap grader is fine
  system_prompt="Grade against the rubric. Be strict.",
  max_iterations=5, # the stop rule (default is 3)
  ),
  ],
  checkpointer=InMemorySaver(),
 )
# The rubric is passed at invoke time - not to the constructor.
result = agent.invoke(
  {
  "messages": [HumanMessage("Make the failing tests pass.")],
  "rubric": RUBRIC,
  },
  config={"configurable": {"thread_id": "fix-tests-session"}},
)

Here's how it works. Whenever the agent is about to finish, a separate grader sub-agent reads the transcript against the rubric. A needs_revision verdict puts the feedback back into the conversation as a HumanMessage, and the agent carries on. The only ways out are satisfied, failed, max_iterations_reached and grader_error, so the loop is closed and always terminates. If the grader needs to run real checks of its own, give it tools as BaseTool instances.

There are two kinds of verifier, and picking well is most of the craft.

• Deterministic: run the tests, validate the schema, confirm the score. Cheap, fast and impossible to flatter, so try these first.
• LLM-as-judge: a second model scores against a rubric. It suits fuzzy goals, but position, verbosity and self-preference biases can game it. Never make the agent that did the work the only judge of it.

Make it a graph

Once a task is more than trivial, an explicit graph is better than a hidden loop: the nodes, the edges and the place where stopping is decided are all visible. A research agent fits in three nodes, with every stopping decision made by the router:

from typing import Literal, TypedDict
from pydantic import BaseModel, Field
from langgraph.graph import END, StateGraph
class ResearchState(TypedDict):
question: str
draft: str
verdict: str
feedback: str
iterations: int
class Verdict(BaseModel):
status: str = Field(description='"verified" or "needs_work"')
feedback: str = Field(description="claims lacking support, if any")
def decide(state: ResearchState) -> Literal["research", "__end__"]:
if state["verdict"] == "verified": # success exit
return "__end__"
if state["iterations"] >= MAX_ITERATIONS: # surrender exit
return "__end__"
return "research" # else: loop again with the feedback
builder = StateGraph(ResearchState)
builder.add_node("research", research_node) # generator
builder.add_node("verify", verify_node) # verifier
builder.add_edge("research", "verify")
builder.add_conditional_edges(
"verify", decide, {"research": "research", "__end__": END}
)
builder.set_entry_point("research")
graph = builder.compile()

No clever prompt appears in that code. The smarts are in the shape: one node generates, one verifies, and a router turns the verdict into either another pass with feedback or a stop. LangGraph treats the sentinel __end__ and END as the same thing.

The loop contract

This is where the mindset shift becomes concrete. Instead of writing instructions, you write a specification of "done", meaning a reward function plus a stop rule. That's the artifact you review, diff and commit:

Goal: a green CI build on the payments-refactor branch. Done means every test passes and none was skipped or deleted to cheat. Steps: read the failing test, patch the probable cause, run again. Memory: a log of approaches already attempted, so nothing gets repeated. Stop on green by opening a draft PR, after six rounds by summarizing what's left, or whenever the work touches auth or payments logic by handing off to a person.

Not one instruction went to the model. What got written was the goal, the checks, the memory rule and the three exits, and that contract is loop engineering.

Loops 3 and 4: stacking outward

A single well-built loop already helps. The gains compound when you wrap further loops around it, each running on a slower clock. Loops 1 and 2 get correct work done, and loops 3 and 4 make it continuous and self-improving.

Loop 3: events

An external event changes your loop from a tool you call into a service that runs itself. It's also the natural place for a human approval gate ahead of anything shipping:

# Loop 3: an event turns the loop into a service
from pydantic import BaseModel
class CIEvent(BaseModel):
  branch: str
@app.post("/webhook/ci-failed")
def on_ci_failure(event: CIEvent):
  if not event.branch.startswith("fix/"): # cheap triage first
    return {"skipped": True}
  result = agent.invoke(
    {
    "messages": [HumanMessage(
      f"Fix the failing test on {event.branch}.")],
    "rubric": TEST_GREEN_RUBRIC,
    },
    config={"configurable": {"thread_id": event.branch}},
  )
  open_draft_pr(result) # a human approves the merge, not the loop
  return {"opened_pr": True}

Loop 4: hill climbing

The outermost loop is the slowest, and it works on the loop itself. Each run leaves a trace behind. An analysis agent reads those traces, finds failures that keep recurring and suggests edits to the harness: the prompt, the tools, the rubric that humans review. Imagine the return arrow. Instead of only feeding back to the start, it reaches into the inner loop and changes it. The system is improving the system.

One caution: outer loops multiply whatever sits inside them. Add them only once the inner verifier is strong, because a fourth loop around a weak verifier just automates confident garbage at scale.

Why this counts as engineering

The guardrails can look like red tape. They stop looking that way once you see an agent loop as a closed-loop feedback controller, a mapping that holds point for point. The goal is the setpoint, the agent is the controller, and tools plus environment make up the plant. The verifier acts as the sensor, and its "needs work" verdict is the error signal that triggers the next correction. A finished task means convergence. The controller is driven by the gap between setpoint and sensor reading, whether the plant is a jet engine or a test suite.

The same vocabulary describes two ways a loop breaks. It can oscillate, repeating one fix indefinitely, or diverge, drifting away from the goal while money drains. Guardrails are just the stability conditions that keep it bounded, and the rule above them all is that the harness must guarantee termination, and the agent can't be trusted with that.

A day with all four loops

Watch the four loops work through a morning. At 09:14 a test on payments-refactor fails and a webhook fires, which is loop 3, with nobody typing a thing. For the next seven minutes the agent reads the failure, patches, re-runs, reads the new failure and patches again, keeping a log of attempts (loop 1). After each round a deterministic grader runs the whole suite and checks that nothing was skipped to force a pass (loop 2). Round four comes back green.

At 09:22 the loop ends the right way, with a draft PR that shows what changed and why. A human must approve it because the change touches payments. Overnight, the hill-climbing loop spots that this was the week's third timezone bug and drafts a new rubric check, which goes through your review (loop 4). Tomorrow's run begins smarter than today's.

You slept through most of it. In the morning there's a draft PR and a decision to make, and no keyboard to hammer.

When loops go wrong

Loops amplify, and amplification works in both directions. Each failure mode hits the loop at a specific spot and has a standard guardrail, with one theme repeating: whatever generates must not also grade. The cheapest protection is to run the verifier against inputs known to be good and known to be bad before leaving the loop alone. See that it rejects the bad ones, since an untested verifier is only an opinion with a budget attached.

When not to build a loop

Rushing to loop everything is a failure mode of its own. Keep it as a single call in these situations:

• One pass is enough, as with a summary, a translation or a clean extraction. A loop only adds cost and latency.
• No goal can be verified. Without a way to check "done" there's no error signal, which leaves an open loop spending money.
• Mistakes can't be undone, such as moving money, deleting data or emailing customers. The loop must stop and hand off.
• A person would be quicker. If the task takes ten seconds by hand, the loop is the worse tool.

Loop engineering means knowing which problems deserve a loop as much as knowing how to build one.

Build it like you'll keep the job

Those are the tools: anatomy, agent loop, verifier, graph, contract, outer loops and failure catalog. Equip the loop with a strong verifier, a hard stop and limits it can't cross. Add human checkpoints wherever the stakes are real, and keep in mind that a loop delivering a fix doesn't mean you understand it. Then let it run, the way someone does who plans to remain the engineer and not merely the person who starts it.

The next rung

One question is worth asking before a loop goes unattended: how feeble can a verifier be and still show green? A benchmark was assembled to find out. It holds recorded reward hacks and scores gate setups by how many hacks they let through. A gate that disregards the agent's own report and reruns your test suite, which is the deterministic check suggested earlier as a first resort, stopped none of eight. Rerunning the suite only confirms that the reward signal was satisfied, and that's exactly what a reward hack accomplishes.

So if the verifier limits the system, the scarce skill that comes next is obvious: designing, versioning and improving the checks themselves. People are calling it eval engineering, the job of setting the bar a loop must clear. One verifier is craft within a single loop. Eval engineering spans many of them, through the datasets and rubrics that reveal whether the system as a whole is getting better or worse.

Hardening a gate is a ladder, and where it tops out is unexpected. The best check turns out to be no clever code at all, only a few assertions the agent was never allowed to see. So the limit on verification sits somewhere awkward. It depends less on how well a gate is built and more on the quality of your held-out data and on whether you keep it from leaking back into the loop.