From Evals to Self-Improving Agents: Five Eras of Agent Evaluation
Agent evaluation now means auditing traces. See how deterministic checks, LLM and agent judges, diagnosis and a meta-harness build a safe improvement loop.
This article is published in English.
Sources: From Evals to Self-Improving Agents
The short version
- Judging an agent used to mean reading its last answer. Now it means reading the whole run.
- Hand-written deterministic tests stay valuable, though they can only flag problems somebody anticipated.
- A language-model judge fits rubric grading of outputs. A judge that is itself an agent fits failures in the path, in tool use and in the workflow.
- What comes next is diagnosis across traces: spotting failure patterns that repeat over thousands of live runs.
- An agent that improves itself requires a meta-harness, meaning telemetry, diagnosis, proposed fixes, sandbox trials, validation, staged rollout and a way to keep the evaluators themselves honest.
Why outputs alone stopped being enough
Early LLM apps were short and mostly linear: a prompt went in, an answer came out and someone checked it. Agents broke that shape. They plan across several steps, invoke tools, read and write state, edit files, bounce back from failed actions, delegate to subagents and stay busy for long stretches.
What such a system leaves behind is a trajectory. It holds the request, the system context, whatever was retrieved, interim reasoning, each tool call and its observation, retries, errors, memory writes, spend, timing and the final output. For evaluation purposes, that record has become the real artifact of execution.
So grading has to shift toward auditing. A correct-looking answer can sit on top of a wrong tool call, an overlooked constraint, an unrelated retrieval hit, a retry loop nobody saw, poorly chosen context, a risky recovery or an assumption with no grounding. Those flaws are the costly ones, because they pile up over many runs and quietly erode reliability.
In that picture, evaluation acts as the control layer. It combines live traces, rubric judges, agent judges, root-cause analysis, sandbox replays, regression suites and deployment gates. The target isn't a person reviewing each exchange. It's a meta-harness that notices failure patterns, drafts fixes, tests them on past and adversarial cases and sends to people only the situations where their judgment matters.
Layer one: deterministic checks
Rule-based checks remain the foundation, since they catch anything that can be written down beforehand. Code gets unit tests, type checking, linters, security scanners, contract tests and snapshots. Structured output gets schema validation, mandatory fields, permitted values, date formatting, valid JSON and policy limits.
Reach for them wherever the expected result is explicit. They run fast, cost little, give the same answer every time and are easy to debug. The catch is coverage: only what someone remembered to encode gets tested.
What to measure with them includes pass rates, how much of the behavior assertions cover, schema conformance, runtime, repeatability and how precisely a failure can be located. They suit APIs, code changes, extraction from data, forms, regulated flows and tasks bound by hard rules.
Layer two: a language model as judge
Judges built on LLMs spread because so many AI failures are matters of quality. A reply may read smoothly and still leave things out. A summary may be easy to read and still miss the point. A support message may sound sure of itself while breaking policy.
The pattern is a model scoring an output against a written rubric, perhaps on factual accuracy, following instructions, completeness, tone, safety, quality of citations and overall usefulness, and answering with a label, a score, a justification and supporting evidence. It's a good match when the output has a familiar form but still needs a judgment call, and it allows review to scale without a human reading everything.
Vague rubrics are where it goes wrong. Tell a judge to look for a "good answer" and polish gets rewarded. Ask it to check evidence, constraints and failure modes and it becomes much more useful.
Judges deserve their own measurement: how consistent scores are, how often different judges agree, how sound the stated reasoning is, how well evidence is grounded, how frequently they wrongly pass or wrongly fail an output and how closely they track human labels. They work for summaries, support replies, writing quality, policy review, extraction quality and reasoning-heavy answers that have clear criteria.
Layer three: an agent as judge
Agents leave trajectories behind, so a judge that only sees the end result misses the strongest signals. An agent-judge reads the entire run, including prompts, how context was chosen, tool calls, observations along the way, retries, edited files, tests that ran and the end product. It asks pointed questions. Was the tool well chosen? Was a failing test brushed aside? Was the context out of date? Did unrelated files change? Did the agent stop before finishing?
This matters because agent failures are frequently subtle. An output can clear a shallow check while the route that produced it reveals the actual defect.
Useful measures include whether the trajectory was valid, how good tool routing was, how the agent recovered, how it used evidence, whether the task finished, how well hidden failures were detected and whether escalations were correct. Coding, research, data, browser and workflow agents all benefit, as does anything where the route counts as much as the destination.
Layer four: diagnosing traces
The fourth layer steps back from single runs. A diagnosis agent reads thousands of traces and asks what shifted. Maybe failures grew after a prompt edit. Maybe the wrong tool gets picked for spreadsheet analysis. Maybe latency jumps once retrieval returns too many files. Maybe a QA agent flags hallucinations but lets skipped requirements through.
Its job is to pin a performance change on a cause: context selection, prompt wording, tool routing, workflow setup, model behavior, retrieval quality or plain code. Reporting that a score fell isn't enough. The output should be a specific candidate remedy, such as a prompt edit, an added eval case, a tweaked routing threshold, an updated skill, a code change or a pull request.
Quality here depends on how much of the traces get covered, how precisely root causes are named, how well drift is detected, how failures cluster, how confident the attribution is, how good the proposed fix is and how risky it is for regressions. It suits production agent platforms, busy support agents, coding assistants, enterprise workflow agents and anything updated often.
Layer five: closing the loop
At the top, the loop closes. Telemetry gets captured, recurring failures get diagnosed, a candidate fix gets generated and tried in a sandbox, then checked against golden datasets and past traces and finally promoted through a controlled rollout. Self-improvement becomes engineering at that point, not vibes.
An agent mustn't ship a change because one metric ticked up once. Gates are required: regression tests, adversarial scenarios, counterfactual prompts, canary traffic, rollback plans and thresholds for bringing in a person.
Reward hacking is the largest danger. An agent can learn to satisfy its judge without doing the real job. A coding agent could conceal an error, an image agent could invent details that lift a visual score and a writing agent could chase rubric phrases and lose the author's voice. Evaluators therefore need evaluating, using independent checks, human-derived benchmarks, audits of disagreement, layered QA and regular recalibration of rubrics.
Treating eval as the control system
The deepest change is operational. Teams should run eval as the control system of the agent platform, and a solid program answers five questions:
- Was the task completed? The answer counts, and so do instruction following, factual accuracy, policy compliance and the success criteria specific to the task.
- Was the route sound? With tools involved, look at which ones ran, what evidence backed them, which retries happened, what broke without notice and whether escalation came at the right moment.
- Why did performance move? A production dip may trace back to prompts, model behavior, retrieval, routing thresholds, workflow setup, tool errors or code, and diagnosis must separate symptoms from causes.
- Can a fix be proposed safely? Candidates go through sandbox trials, replay of old traces, adversarial cases and regression suites before release.
- Who audits the auditor? Judges drift as well, turning lenient, harsh or easy to fool, so calibration sets, human-checked benchmarks, disagreement checks and layered QA are needed.
Deterministic checks handle known failures, LLM judges grade quality, agent judges inspect routes, diagnosis agents surface repeating patterns and the meta-harness rules on whether an improvement is safe to ship. People stay in the loop, but higher up: less repetitive review, more guardrails, exception handling, calibration and policy design.
Feeding production back in
With an eval system running, the next move is to learn from live usage. Telemetry feeds diagnosis, diagnosis refines evals, evals steer optimization and validated changes flow back into staged deployment. Two threads run side by side.
One thread is qualitative. Production feedback, user traces, analytics, failed tasks, escalations to support and tool errors shouldn't just sit on dashboards. They should flow into evals. When users keep fixing the same answer, that's a new test case. When traces show a wrong tool choice, that's a trajectory check. When analytics reveal drop-off at one step, that's a scenario worth replaying.
The other is systematic optimization. Once specs, code and evals are all clear, tuning gets far more disciplined. Specs describe the intended behavior, code holds the logic, constraints, tools and routing, and evals supply the metrics deciding whether things improved. Given those three, tooling can suggest prompt edits, threshold changes, routing tweaks, extra evals or code changes.
The cycle, step by step:
Live usage signals feed eval refinement, which drives optimization of the agent program, which gets replayed in a sandbox, passes regression tests, rolls out under control and then generates fresh live signals.
Under this view, evals aren't a grade handed out once. They're the operating loop that drives agent improvement: learn from real usage, convert it into sharper evals, tune the agent against them and release only what survives validation.