When a Classifier Quietly Swaps Your Model: The Case for Served-Model Headers

When Anthropic posted on 7 August about relaxing the biology safety classifier on its flagship model, most commentary filed it under the old over-refusal debate, a topic that felt exhausted. The more interesting word in the announcement is fallback, not refusal.

A substitution with no signal

According to the announcement, a classifier that fires doesn't stop the request. It routes the request to Opus 5, a strong model that lacks the biological capability of Fable 5. Nothing gets blocked or filtered, and no warning shows. A user asks for the top model, a classifier decides the prompt resembles biology, and a different model replies.

The models differ in cost. On the public price list Fable 5 charges $10 per million input tokens and $50 per million output, while Opus 5 charges $5 and $25, half in each direction. The post is silent on how a rerouted request gets billed and that wasn't tested, so billing is left out of the claims. What's established is a swap with a documented target and a published price difference.

Specs matter more than price. In the docs, the comparison runs like this:

• Positioning in the docs: Fable 5 targets agents that run for a long time, while Opus 5 is aimed at intricate agentic coding and business workloads.
• Price per million tokens, input and output: $10 and $50 against $5 and $25.
• Context window: 1M for both.
• Maximum output: 128K for both.
• Reliable knowledge cutoff: January 2026 for Fable 5, May 2026 for Opus 5.

Context size matches, the output ceiling matches, and the stand-in model's knowledge runs four months newer. Its own docs even recommend it first for demanding agentic coding. The announcement spells out the cost of the swap, namely biological capability, and that's a loss no code diff would reveal.

So the substitution doesn't exhaust a context budget, doesn't trip a token limit, and on a coding task might not look worse at all. That isn't a mitigation. It's exactly what makes the swap invisible.

Reading the numbers carefully

The headline figure is a drop of about 85% in biology-related fallbacks across Anthropic's product surfaces: one number, limited to biology, aggregated over everything.

Surface-level numbers measure something else and appear in a footnote about all fallbacks, biology or not. Total fallbacks are projected to fall by about two thirds on Claude.ai (67%), a bit over half on Cowork (55%), 17% in Claude Code and just 7% on the Claude Platform. They don't add up to portions of the 85, and blending the two yields a sentence that's tidy, quotable and wrong.

They're reductions too, not rates. A 7% cut on the Platform reveals nothing about the absolute frequency there, and fallbacks might always have been uncommon. The spread does show where the retuning had its effect: consumer chat shifted roughly ten times more than the developer platform, and coding sat in between at 17%.

A classifier whose relaxation shifts a chat box ten times more than an API clearly wasn't calibrated against API traffic. Anyone running a fleet of subagents sat downstream of a guardrail tuned for a person typing a question into a text field.

Fairness requires one note, and it matters for trust: nothing was leaked. Anthropic published the details itself while shipping the fix, with a number attached. The complaint isn't concealment. It's the shape of the mechanism.

Why a downgrade is worse than a refusal

A refusal is a good kind of failure. It's typed, carries a status code, appears in a dashboard, triggers an alert, takes a retry path and is legibly someone else's fault.

A downgrade offers none of those hooks. No error arrives, no status code, no log entry. What comes back is a valid, well-formed reply that's a little worse: a patch missing one edge case, a test written off as flaky, a summary missing the caveat that mattered.

A weak subagent result doesn't usually make anyone blame the vendor. The prompt gets the blame, because it's the part under direct control and the part most often at fault. So it's reworded, results improve, and the theory seems confirmed. But new wording also reaches the classifier as different input. The usual cure for a bad answer disturbs whatever may have caused the swap, so the repair appears to work and the mistaken diagnosis sticks. If a single edit fixes both a vague prompt and one that tripped a filter, nobody can say which problem existed.

Months of mediocre subagent output could have been blamed on prompting when some of it was a substitution, and there's no way to tell which, because nobody recorded which model answered.

The data was always there

Nothing here was missing. The Messages API response object has a model field next to id, content, stop_reason, stop_details and usage. Every answer ever returned has named the model that produced it. Hardly anyone reads the field, because it looks like an echo of the request.

The SDK still frames it as a request echo: the doc comment on model in the response type describes it as the model that will complete the prompt. Future tense, on a field inside a completed response. That wording was correct for years and no longer is. Which model served a call is determined at runtime, and the one authoritative record of it is the response most code throws away.

Claude Code users already have this on disk. Every assistant turn is appended to a JSONL transcript under ~/.claude/projects/, holding the full API message object. The obvious one-liner counts model strings:

grep -ho '"model":"[^"]*"' *.jsonl | sort | uniq -c | sort -rn

Counting over the twelve latest transcripts in one project folder gave these tallies: claude-opus-5 answered 4,691 turns, claude-opus-4-8 handled 1,231, six turns were marked synthetic, and a final group of 133 lines said only sonnet, haiku or opus.

Those last 133 aren't served models. They sit at message.content.input.model, which holds subagent spawn parameters: the model requested for a subagent, recorded inside a tool call. The served model lives at message.model. The key name is the same, the path differs and the meaning is opposite. Selecting by path makes the tail vanish:

jq -r 'select(.type=="assistant") | .message.model' *.jsonl | sort | uniq -c | sort -rn

The first attempt got this wrong, and the manner of the mistake is the lesson. One file stores both the requested and the served model under an identical key, with the path as the sole distinguishing feature. A plain grep yields a count, but not the distinction that counts.

Both full ids were picked on purpose, in sessions that span a version bump, so those totals don't show any classifier substitution; none has turned up in these logs. What exists is a measuring tool, not a finding. Records also hold isSidechain, marking subagent traffic, plus a requestId, which means comparing requested and served models for each subagent is an afternoon's job over files already on disk. Plumbing this simple shouldn't be something users have to write.

The harness already treats model changes as events

Claude Code has treated fallback as a first-class idea for months, release after release. Version 2.1.166 introduced a fallbackModel setting that lists as many as three models to try in sequence when the main one is overloaded or down. In 2.1.76 those alerts became permanently visible, no longer buried in verbose mode, and gained readable names. Release 2.1.183 prints a stderr warning in print mode if the requested model is deprecated or silently upgraded. In 2.1.233 an unknown model id produces an unrecognized-model line on stderr. The most recent build adds PreModelSwitch and PostModelSwitch hook events so code can veto, confirm or annotate a switch.

Five decisions point to one conclusion: a change of model is something the operator should hear about.

All of that governs switches the harness itself performs. A swap performed upstream by a classifier comes with no notice, no hook event, no stderr output and no typed error. A team that decided an availability downgrade merits a permanent banner ships a safety downgrade with nothing. Malice is an unlikely explanation. A seam fits better: the harness reports its own actions, the model reports its own, and the substitution happens in the space between.

The same pattern elsewhere

On 19 August OpenAI previewed Private Safety Processing, a monitoring layer that looks at inputs and outputs from several conversations together, so it can spot abuse that only emerges across sessions. The company says it keeps none of the customer's data and produces just a narrowly scoped signal describing the kind of activity. It responds to the 30-day retention that Anthropic announced in July for covered models, a policy that worried enterprises handling sensitive material.

Calling that false would be unfair. It simply can't be verified. Correlating behavior across sessions while retaining nothing are claims in visible tension, and resolving it requires knowing what the signal contains and how long the correlation window lasts, neither of which has been published.

The shape matches the fallback case: a property of the service that materially affects customers, observable only by the vendor and disclosed at whatever granularity the vendor picks. Safety behavior has become a competitive surface, and the competition runs on attributes customers have no instrument for.

What would fix it

Nobody here is asking for a rollback. The biology classifier can stay, and so can the fallbacks the post lists for virology, toxicology and molecular design. The request is small: it costs a single string.

The substitution belongs in the response envelope. Any of these would do: a header, an extra field in usage, or a telemetry attribute that separates the model asked for from the model that answered. Nothing about that is unusual. Requests from subagents already go out carrying agent-id and parent-agent-id headers, so attaching model details to a request is established practice in one direction. Meanwhile OpenTelemetry export tags only a single metric (lines of code) with a model attribute, and no signal anywhere shows that the answering model differed from the requested one.

The reason needn't be disclosed. A category is enough, or even nothing beyond the fact that it happened, delivered at the moment it happens, inside the response.

The rule that keeps recurring outlives this vendor and this month: whatever a provider declines to put in the response becomes the customer's bug. It lands in their error budget, their code review, their postmortem and their estimate of their own competence. The 85% was measured on the vendor's side of that line. A header would put the next one on the customer's side.