A Stable Agent Name Is Not the Artifact You Reviewed: Pin the Candidate

Two agents can share a name and differ in every way that counts for a review. Suppose a "Research Agent" passes evaluation on Monday. By Tuesday the label is unchanged, yet one skill has been swapped out, a tool has gained write access and a routing rule sends some requests down another path. On an org chart it's still the same agent. As evidence for what was tested, that answer isn't precise enough.

The label points at a role. What it can't point at is the particular mix of behavior-shaping parts that earned the pass. The gap grows as agent behavior leaves the single model call and becomes a composition: model, instructions, skills, tools, permissions, memory policy, routing, hooks, harness and anything the host app adds.

Names coordinate, candidates carry evidence

No one wants each internal revision to become a new public persona with a serial number. People need a name that lasts. A team with a Research Agent, a Dependency Agent and a Support Agent benefits from labels that say what each is for and where it sits.

A review asks something else: which specific behavior candidate does this verdict cover? Imagine an evaluation run against a single model, prompt bundle, skill set, tool boundary, permission policy and routing setup. Wherever those pieces mattered to the measurement, they are part of what the result describes. Swap one and the old result may still hint at the new candidate, but it no longer vouches for it.

A friendly name can hide exactly this. The product identity stays steady while the executable composition under it moves. Identity for people gives continuity, and identity for candidates gives precision.

Configuration is turning into inspectable material

Current agent products already expose bits of this pattern, though no single architecture is shared across them. Some tools let you define agents in files that bundle scoped instructions, tools, constraints, chosen skills and MCP servers, with model and permission settings alongside, and project-specific definitions can live in the repository. That's one design and not a standard, but it shows that configuration no longer has to be invisible setup around a model call. It can be an artifact you read and review.

Another pattern keeps prompts, tool lists, schedules, credentials and memory settings in Git, so every change flows through a pull request and leaves review history, rollback and an audit trail. One organization doing this proves nothing about what every team should do, but it demonstrates that agent behavior can be handled as changeable configuration with software-style history.

Smaller signals lean the same direction. Certain provider platforms already offer reusable, versioned agent configurations, although those interfaces are in beta and keep moving. Protocol roadmaps name agent identity as a topic for future work, which is a plan and not shipped behavior. None of it means the problem is solved. Still, the examples justify a modest claim: when behavior comes from independently changeable parts, the assembled whole needs an identifier of its own.

Versioned parts can still add up to an ambiguous whole

This is where the difficulty becomes deceptively annoying. A model has a version. A skill has a revision. A repository has a commit. An MCP server has a deployment, a memory policy may have its own version, and the agent may have a provider-visible one too. Every piece can be named while the combined candidate stays uncertain.

Consider the ways pinning goes partial:

• The prompt is pinned, but the model alias moves.
• The model is pinned, but a skill tracks latest.
• The skill is fixed, but the host application changes which tools it exposes.
• A permission policy changes while the public name stays the same.

The system is versioned in fragments, and the thing that actually ran is left unclear. Software release practice offers a useful analogy, with one limit: an agent configuration isn't necessarily a single binary or one giant manifest. The analogy concerns what a decision applies to. A software team doesn't stop at "we tested Product X" when several builds exist. The product name helps coordination, and the exact build or commit says which candidate the claim covers. Agent review needs the same separation whenever configuration can change behavior materially.

Three identities beat one

Forcing a single identifier to carry every meaning creates confusion quickly. At least three layers are worth separating.

Human-facing identity

The lasting role: its name, purpose, owner, responsibilities and position in the system. It should hold still enough for people and other agents to coordinate around.

Behavior-candidate identity

This is the exact set of behavior-shaping dependencies that a team evaluates, approves, compares or releases. Depending on the architecture, it can reference:

• which model runs, with any settings that change its behavior,
• the instruction text or prompt bundle,
• skills, pinned to exact revisions,
• the tools and MCP servers it can reach,
• what it may do and what needs sign-off,
• how memory is kept and used,
• rules for routing and handoffs,
• hooks that intercept the lifecycle,
• the harness or runtime version, if it alters behavior,
• settings of the host surface, if the host controls context, tools, approvals or execution.

The key word is material. The aim isn't to serialize every setting simply because it exists. It's to capture the dependencies whose change could make a claim about review, reproduction or rollback false.

Execution identity

Even a perfectly pinned candidate doesn't capture everything affecting one run. A run also depends on user input, mutable memory contents, retrieved context, external system state, live data, credentials, human decisions and the model's own nondeterminism. Where they matter, those belong to the lineage of the execution.

Keeping the layers apart avoids two opposite mistakes: treating a friendly name as exact technical identity, and treating a configuration snapshot as though it recorded every fact about a live run.

What deserves pinning

How much to pin depends on what is at stake. A low-risk personal helper may be fine with a repository commit or a provider version. A production workflow holding write access warrants more. For an evaluation, record whatever bore on the measurement: model, prompt, skills, tools, permissions, routing and harness state.

Skip the schema at first and use the list above as a checklist. For each item, ask whether you'd still call it the reviewed candidate after it changed. If the honest answer is no, that item belongs in the candidate's identity or its lineage.

Sensitive values needn't be copied into the record. A configuration can point to a secret reference, a credential class or a permission boundary without turning the manifest into a fresh place to leak secrets. Exact identity should improve auditability and shouldn't create a worse security problem.

Evaluation, debugging and rollback share one need

These look like three separate operational worries, but each needs a recoverable object.

Evaluation. Any score is a claim about some object. When that object can't be recovered closely enough, the score can outlast the thing it described. Not every edit voids every evaluation: fixing a typo in an unrelated skill hardly touches a reasoning benchmark, whereas a permissions change may not affect prose quality yet decide a safety review. What matters is what was being measured, and the call has to be made openly so a steady name can't mask material drift.

Debugging. With no configuration identity, investigating an incident turns into digging through layers. A prompt file, console history, tool registry, deploy log, memory snapshot, branch and transcript may each be right about their own layer while the whole remains unclear. An exact candidate reference simplifies the opening question: was this failing run really using the candidate everyone assumed? When it was, look at execution state and behavior next. When it wasn't, the mismatch is itself the finding. A surprise from the accepted candidate and a run that never used it are separate classes of failure.

Rollback. Suppose the approved candidate combined one prompt, one skill set, one permission policy, one routing rule and one runtime. Putting back the prompt alone doesn't show the old candidate is back. Teams should ask whether they can rebuild the whole last approved candidate or only undo whichever component caught their eye. The former is real rollback, and the latter is hopeful editing.

A last piece is easy to overlook. Finding "the last accepted candidate" is a lookup, and it only works if candidates carry a succession record. Exact identity says what each candidate was, but not which one replaced which or what changed in between. If rollback matters, each candidate should name its predecessor and list the material differences, so restoring is never a hunt.

Don't let configuration swallow the run

The tempting overreaction is a giant snapshot holding everything. That brings new trouble. Mutable memory, retrieved documents, user input, external API state, the clock and credentials all shape a run. Some are sensitive, some are meant to be dynamic and some can't be replayed exactly, so they don't all sit inside the candidate.

A cleaner boundary looks like this:

• The configuration layer records which behavior-shaping dependencies were chosen.
• The run layer records which execution used that candidate, plus any mutable inputs that mattered.
• The approval layer records the exact candidate a sign-off applied to.

The layers can point at each other without fusing into one artifact. That boundary also stops "exact identity" from swelling into a claim the evidence can't carry. Pinning a candidate doesn't make a model deterministic, freeze outside systems or guarantee identical output later. It only makes the subject of the claim recoverable, and that already helps a lot.

A five-question test

When a named agent matters enough to review or release, ask:

1. What can change without changing the name? In most systems, more than people expect.
2. Which of those changes could invalidate this particular review claim? It depends on whether the decision concerns quality, safety, permissions, reliability, cost or readiness.
3. Can the accepted composition be recovered later? A version only helps if its dependencies resolve to something concrete.
4. Can a failing run be tied to that composition? If not, debugging can't separate candidate failure from candidate drift.
5. Can rollback restore the composition and not just one part? If not, the system has versioned parts without a reliable release identity.

None of this calls for ceremony around every edit. Precision is needed when a claim starts to carry consequences. As agent behavior is assembled above and around the model, and that assembly becomes modular, portable, versioned or swappable, configuration is no longer incidental setup. It is part of the executable candidate. The name may stay put, but the evidence has to know what changed.