When Quantization Passes Perplexity and Silently Breaks Model Safety

One measurement should unsettle anyone who ships compressed models. In an evaluation covering eleven instruction-tuned models from 3.8B to 72B parameters, run over five benchmarks and 1,894 prompts, compressing the KV cache to a low bit-width cost Mistral-7B 15.2% of its refusals. The perplexity change across that same compression came out to a factor of 1.03. A three-percent shift is the kind of movement most teams would file under measurement noise and release without a second thought. The usual sign-off for a quantized build, perplexity steady, task accuracy steady, latency down, would have approved this one. What follows is the study that recorded the effect, the mechanism that explains it, and a short set of changes to your quantization gate so that a capability cannot disappear while every dashboard stays comfortably green.

What the research documented

The work is a preprint, "Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation" (arXiv:2606.09864). Its thesis is uncomfortable and direct: aggressive KV cache compression can quietly dismantle a model's safety alignment; no single bit-width is safe across models; and the breakdown arrives as an abrupt, per-model transition that the metrics teams normally trust simply cannot register. Since it has not yet passed peer review, the sensible stance is to read it as a compelling case rather than a closed one, a point worth revisiting at the end.

The word carrying the weight of that thesis is invisible. Not merely faint or awkward to spot, but undetectable by the instruments in common use, because those instruments track a quantity other than the one that fails. The most vivid demonstration uses speculative decoding, exactly the acceleration teams lean on to make on-device inference bearable. With a 4-bit model acting as the verifier, the refusal rate fell from 63.2% all the way to 0.0%, even as the acceptance rate stayed at 23.5% and throughput held at 17.0 tokens per second, both comfortably within normal operating ranges. One capability bottomed out at zero while every figure under observation looked ordinary.

The mechanism, which is the part worth keeping

Scary numbers alone make a headline; the reasoning behind them is what you can build a process around, and the reasoning is geometric. The features responsible for safe behavior sit in a narrow slice of the activation space, and the study estimates that slice is between 100 and 1,000 times more sensitive to compression noise than the broad space that perplexity summarizes.

That estimate is worth sitting with, because it is the crux. Perplexity is a single number rolled up across every dimension the model uses. Spread a small amount of error over a thousand of those dimensions and the rolled-up figure notices. Obliterate three dimensions completely while leaving the remaining thousand intact and the figure hardly stirs, since a summary dominated by the untouched majority stays healthy.

The next step explains why one model breaks and another does not, and it comes down to structure. Quantizing a group of values forces a choice of scale large enough to fit the biggest value present. Activations carry outliers, individual channels whose magnitudes tower an order of magnitude over their neighbors, and those outliers dictate the range. With the levels stretched to reach them, any value far below the first level collapses to zero. So the fate of a given model hinges on a single question: are the channels that encode the behavior you depend on clustered near those outliers, or are they marooned in the low-magnitude region that compression erases?

The paper rests this on earlier results showing that a model's tendency to refuse is governed by only a handful of directions in activation space (Arditi and colleagues in 2024, and Pan and colleagues in 2025), and that alignment is front-loaded into the first output tokens (Qi and colleagues in 2025). How compression noise lands on those directions depends entirely on whether the channels involved overlap with the dominant outliers the quantizer has to make room for. That structural coincidence, rather than parameter count or training method, is what determines survival.

Why a house standard for bit-width cannot exist

The contrast that settles the argument is jarring. Qwen-2.5-7B falls apart at 6-bit. Gemma-2-9B is still intact at 3-bit. These are both instruction-tuned models in the 7-to-9B range, produced by broadly similar pipelines, yet a four-bit gap separates two models you would have assumed were swappable. A policy that reads "our standard is 4-bit KV cache" therefore says nothing useful: it is perfectly safe for one of these and disastrous for the other, and no artifact shipped with the model reveals which case you are in.

To turn that into something you can test before rollout, the study offers a diagnostic named Per-Channel Reduction that sorts a model into one of three distinct failure categories in advance. That is its practical payload, and it is inexpensive enough to fold into routine pre-deployment work.

A generalization the study stops short of making

All of the above is about safety refusals, and precision matters: refusals are the only behavior the paper actually measured. Yet look again at what the mechanism requires. It needs a behavior concentrated in a small subspace, paired with a metric that averages over a large one. Refusal qualifies on both counts. There is no obvious reason it is the sole behavior that does, and stretching the finding this far is an extrapolation rather than a documented result.

Take document AI, where several behaviors share exactly that profile. Schema adherence is one: whether the output is JSON that conforms to the intended structure, or slides into repeating the structure back at you. The empty-versus-invented decision is another: whether a field the model cannot read is reported as blank or backfilled with something plausible. Extraction evaluations keep omission and fabrication apart on purpose, because their consequences diverge sharply. A fabricated quantity on an inbound document seeds a phantom inventory record, while a dropped one merely opens a support ticket, and only the former hides itself. Well-formed tool calls are a third: whether the model produces a valid call or output that merely reads as text.

Each of those is a behavior, not a probability distribution over tokens. A model can post entirely respectable perplexity on the document body while silently choosing to invent a value it had no basis to read, and quantization sweeps aimed at field-level grounding are practically nonexistent. Whether these behaviors give way in the same manner is an open question, apparently unpublished, which is exactly the argument for measuring it instead of guessing.

Three cheap changes to make now

None of these costs much. Add a behavioral assertion to your quantization gate that is not a benchmark score, but a raw count of one concrete behavior over a fixed prompt set, taken at full precision and again at the target bit-width and compared side by side; choose the behavior your product would genuinely wake someone up for. Stop treating bit-width as a setting the whole team shares, because it belongs to the individual model and perhaps the individual model version, so the check has to run again at every upgrade. And guard against compounding, since the speculative-decoding case is the lesson: two optimizations each looked fine alone, and together they erased a capability, so validate the full stack of quantization, pruning, and speculative decoding rather than its parts.

Weigh the caveats before citing the figures

Being candid about the evidence is part of using it well. This is a preprint authored by three researchers, still under review, roughly 61 pages long with nine figures. That is substantial, but peer review has not signed off, so it deserves to be treated as a forceful argument rather than a verdict. The reported figures have also appeared in slightly different form between the arXiv posting and the version in review, the latter under a different title and adding a result about FP8 formats in vLLM. Before quoting any specific number, retrieve the current PDF and confirm it directly rather than relying on a secondhand summary. The extension to extraction behaviors is likewise an inference from the mechanism and not something the paper asserts; the logic supports it, but logic is not evidence.

What is safe to carry away is narrower and sturdier than the headline figures: measure only likelihoods and you will be blind to the moment a behavior drops out. That was already true before this work appeared. The study's contribution is to put a number on it.

References

• "Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation" (preprint). Identifier arXiv:2606.09864v2; posted 8 August 2026.
• A revision in review, retitled around a 35-minute deployment audit, on OpenReview under BqKhzrtkGe; it adds a result on FP8 formats in vLLM.
• Arditi and colleagues (2024) and Pan and colleagues (2025) on refusal being controlled by a few activation-space directions.
• Qi and colleagues (2025) on alignment concentrating in the earliest output tokens.
• Ashkboos and colleagues, "QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs," arXiv:2404.00456.