Home / Articles

The Free Verifier Problem: Why Agent Loops Stall Once the Work Isn't Code

Coding agent loops work because compilers and tests verify for free. Outside code there's no free verifier: four subsidies, failure data and an oracle test.

Publisher: REACTAPP.TOP6 min read1331 words

This article is published in English.

Sources: Loop Engineering for Enterprise Agents: The Free Verifier Problem

Plenty of vendors now present loop engineering as what comes after prompt engineering, and the demos are real. Instead of prompting a coding agent by hand, you build a loop around it. It fires on a schedule, works on an isolated branch, gets graded by a second agent and records its state in a file so each run resumes from the last. Boris Cherny, who leads Claude Code at Anthropic, has said his job is writing loops. Addy Osmani at Google named the pattern, and both Claude Code and OpenAI Codex now offer the same six primitives as built-in features.

It works, and it works for a reason that the people selling it rarely say aloud. That reason doesn't transfer to other kinds of work.

Coding loops work because the codebase ships a verifier of its own. Compiler, type checker and test suite grade every attempt immediately for almost nothing. The maker/checker split, widely seen as the trust mechanism behind loop engineering, costs very little in software, since half of it lived in the repository before any agent showed up.

Move the same loop to a claims workflow, a procurement approval or a customer remediation, and no verifier comes along. You have to construct it, and hardly anyone puts that in the budget. It's the best predictor available of whether an agent program stalls at the pilot stage.

Why code is easy and everything else isn't

Of the six loop primitives, not all travel equally well. Scheduled runs, connectors, skills and memory carry over to other domains with little friction. Two don't.

Isolation comes first. Per-session git worktrees let Spotify run dozens of agent sessions in parallel on one codebase without the agents corrupting each other. Its internal agentic development environment, Xirp, has recorded over 36,000 sessions across thousands of engineers, and the worktree is what keeps that concurrency safe. A general ledger has no git worktree. Enterprise agents write to shared systems of record, and two agents working one entity at the same moment is a data integrity incident, not a merge conflict.

Verification is the second, and it's the costly one. In code, "did this work" is answered by a machine in seconds. In an enterprise workflow, it's a judgment that often can't be known until a person looks, sometimes not until month-end close, occasionally not until a regulator asks. Claude Code grades results with a separate model because self-grading is too generous. That's the right design, and outside code it's the piece you build from scratch.

What production numbers say about failure

Here the evidence stops being anecdotal.

Foundra telemetry, covering 6,259 deployed agents and about 4.5 million runs, puts real-world task success at 56.6%, roughly 37 points below benchmark performance. Repetition makes it worse, and repetition is exactly what a loop does. In tau-bench, agents that score around 60% on one attempt drop to roughly 25% when they must succeed eight times in a row.

The funnel figures agree: 46% of proofs of concept reach production, 14% scale across the organization and about 5% show measurable profit-and-loss impact. Gartner expects more than 40% of agentic AI projects to be cancelled by the end of 2027.

The attrition itself isn't the interesting part. Where failures cluster is. They gather at the seams: handoffs between steps, messy inputs and blind spots in monitoring, and not in raw model capability. Those three are the very places a compiler would have flagged the problem in a coding loop, and the very places an enterprise loop has no automatic grader.

One more number comes from a LangChain survey of over 1,300 practitioners: 89% of teams running agents have observability, yet only 52% run offline evaluations and 37% run online ones. Almost everybody watches. Far fewer grade. A loop that is watched but ungraded doesn't correct itself. It produces more of whatever it already produced, faster, with a dashboard proving it's still running.

What a codebase gives a loop for nothing

Anyone moving a loop out of the IDE and onto a business process should count four things that git and the compiler hand over free, and what each costs when the work isn't code.

1. The oracle. Tests and the compiler return pass or fail on every run, in seconds, at no charge. Outside code there's no default oracle. You pay for one by building a labeled evaluation set, which means a person deciding what "correct" means across a few hundred real cases before the loop may run unattended. This line item tends to get cut, and it decides whether the loop works.

2. Rollback. With git revert, undoing a bad action costs nothing, and that's the property that makes an autonomous coding agent acceptable. A sent email stays sent. So does a posted journal entry or a filed claim. When mistakes carry a price, the loop needs a checkpoint before the action instead of a repair after it, and deciding where that checkpoint sits is an architecture call.

3. Isolation. A worktree lets many agents work in one repository without colliding. Systems of record have no such thing, so isolation must be engineered through scoped leases, per-entity locking or a staging layer where the agent writes before anything reaches the real system.

4. Provenance. Each code change is a diff with an author and a history, reviewable months later by someone who wasn't there. Enterprise loops often write through a shared service account, which shrinks the whole trail to "the integration did it." With EU AI Act enforcement live since 2 August 2026, that is more than an operational nuisance.

Call this the free verifier problem: the economics of loop engineering depend on four subsidies that only a codebase pays.

The oracle test

One step comes before any architecture work.

Identify what will declare the work done, and who foots the bill for that. If both can't be answered in a single sentence, what you're building isn't a loop. It's a process that runs forever under the label of autonomy.

A person can be the answer. Having a senior claims adjuster review a 30-case sample each week counts as a valid oracle, and it comes with a visible price. Saying the agent checks itself doesn't qualify. The labs ran into that already, which explains why Claude Code uses a separate model as the grader.

A sequence that holds up

For teams building agentic frameworks in large enterprises, this order tends to survive contact with reality.

Pick the workflow closest to shipping. Before adding any loop primitive, write the oracle answer on one line. If that takes more than a sentence, the workflow isn't ready, and no orchestration will change that.

Next, set your evaluation coverage against the 89/37 split. If you have dashboards but no online evals, what you own is a cron job plus a monitoring bill, not a loop. Add one grader to the release path of one workflow and give it the power to block. A grader without that power is merely a report.

Finally, treat the four subsidies as a budget line and not a design footnote. Each one the codebase supplied for free becomes real engineering once the work isn't code, and ignoring that accounts for most of the gap between the 46% that reach production and the 5% that show P&L impact.

The loop isn't where the difficulty lies. The difficulty is whatever gets to declare the work finished, which code never made you build.