Copy Alone, Fail in Company: Contextual Fragility Among Tokens That Pass an Isolated Copying Test

Repeating a token correctly when it stands alone says little about whether the same token will survive inside a longer passage. This piece examines vocabulary entries that pass an isolated copying probe yet slip when placed in context. Eight open-weight models from the Qwen and OLMo families were tested. Fixed token IDs went into shared banks of random common-word sequences, the probability of the expected token at its copy position was recorded, and selected decoding failures were inspected by hand. A probability-based screen flagged between 0.4% and 10.9% of isolation-passing tokens in at least a tenth of the tested contexts. Those figures show how many tokens the screen catches and say nothing about how many cause verified decoding errors.

Per-token screening scores agree strongly between the two halves of the context set, with correlations from 0.82 to 0.95. Stored examples show deletion, substitution, truncation and translation, some of them produced while the output distribution was sharply concentrated. A second question is whether closeness in embedding space to known or context-screened glitch tokens picks out this population. The better of input-embedding and output-embedding proximity reaches an AUROC between 0.744 and 0.881 across seven models, although static features beat surface-feature baselines on only three of them. In several instruction and simulated tool-use tasks, selected candidates are kept less often than stable controls. A separate test on natural entity names found no extra predictive value beyond the controls already in place. What results is a controlled description of contextual susceptibility among tokens that pass an isolation probe, with evidence on its reproducibility, its geometric associations and its task-level effects. It argues for contextual verification and leaves open both prospective discovery and the prediction of which context will trigger a given failure.

A token that copies alone and vanishes in company

A model can reproduce a token when asked to copy it alone, then replace that very token when it appears inside a longer sequence to copy. The mismatch matters whenever literal identity is part of the instruction: quoting a title, keeping an identifier intact, handing back a list item, or putting a name into a tool query. A fluent answer can satisfy the task on the surface while changing the exact item the task is about.

A concrete case comes from OLMo-2-7B and the token according, ID 4184. Decoding and re-encoding the string gives the same ID back. In an isolated repetition probe, the correct continuation has a log-probability of -0.001. In the sequence deliver rel behind according sent floor wrong, though, the model emits sent at the position meant for according and carries on from the word after it. The wrong emission has a probability that rounds to 1.000 and an entropy that rounds to 0.00 bits. Over 48 sampled contexts, the recorded continuations drop according in 44 and reproduce it, with probability near 0.999, in the other four. Model, token ID and copying instruction stay the same throughout, and only the surroundings differ.

Earlier glitch-token work covers inability to reproduce or manipulate certain vocabulary entries, using detection from embeddings, classifiers over activations, prompt variation and multi-task evaluation (for instance Land and Bartolo, 2024, then Li et al., 2024 and Zhang et al., 2024, and more recently Wu et al., 2026 along with Tang et al., 2026). A narrower question is asked here. Once a token clears a stated isolation probe, does it stay vulnerable when it is copied inside a longer text, and can that weakness be measured consistently?

The term "fragile token" is used for this conditional behavioural class. Membership depends on the model, the copying protocol and the distribution of contexts. The label makes no claim that vulnerability is context-free, and it does not assume that isolated-probe failures and contextual failures share a single cause. The main experiments use a probability screen to pick out candidates for the class, and throughout, that screen is kept separate from verified decoding errors.

Three contributions follow. First, the passing isolated-token population is measured with shared context banks and fixed token IDs, so token susceptibility can be compared across contexts, independent runs and model configurations. Second, geometric signals tied to established glitch tokens are tested to see whether they also separate candidates within this population, against surface features and random reference sets. Third, the study asks whether selected candidates survive later instructions and simulated tool interactions, and reports a plain negative result for natural entity-name prediction.

The outcome extends glitch-token assessment empirically. It does not claim to invent embedding-based glitch detection, nor that earlier work ignored context altogether. What it shows is why an isolation result should count as one measurement condition and not as a general certificate that a token will be copied faithfully.

Where this sits among earlier work

Early reports described odd model behaviour on particular tokens under varied prompts and tied some of those tokens to unusual regions of the embedding space (Rumbelow and Watkins, 2023). Later studies built systematic detectors and evaluation procedures, and they supply both the method and the comparison point for fragile tokens.

The Magikarp framework of Land and Bartolo (2024) uses indicators derived from embeddings to find candidates for under-trained tokens, then checks them with behavioural probes. Its analysis includes alternative prompts and examples with copied code lists. Li et al. (2024) define glitch behaviour broadly and test repetition, spelling and length, and their GlitchHunter finds problem regions with an embedding graph and iterative clustering. So neither geometric candidate generation nor use of surrounding prompt text is new here.

GlitchProber classifies tokens from attention and MLP features and studies mitigation by altering intermediate activations (Zhang et al., 2024). GlitchMiner applies entropy-guided discrete optimisation with local embedding neighbourhoods and verifies candidates under three repetition templates (Wu et al., 2026). The experiments in this study differ by freezing model weights and judging contextual susceptibility only after conditioning on success in an isolation probe. Wrong emissions with low entropy bear on that judgement, but their mere existence does not show that an entropy-guided detector would miss the token under every prompt.

The nearest behavioural comparison is the work of Tang et al. (2026). Their GlitchQuiz covers eight tasks, among them sentence and paragraph construction, repetition and multi-turn recall. They show a token that passes simple tests and fails another task, and they acknowledge that failures beyond their full assessment may exist. The contribution here is repeated measurement of a single fixed copying operation across shared context banks inside an isolation-passing population, followed by analyses of reproducibility and prediction. Passing isolation in these experiments does not mean passing GlitchQuiz, and no priority is claimed for the idea that token failure depends on the task.

The experiments directly compare available Magikarp labels for three model configurations. They are not a head-to-head rerun of all five systems under one shared contextual oracle, so no claims about comparative recall or efficiency are made for those systems.

Rarity is a rival explanation. Yu et al. (2022) examine how gradients acting on rare-token embeddings can feed representation degeneration. That motivates comparing against rarity-related surface features, yet it does not settle the training history or causal mechanism of the tokens at hand. The token-ID feature used here is only a proxy for vocabulary order and does not measure pretraining frequency.

Experimental setup

Models and samples

The main copying experiments span eight open-weight models in two families: one OLMo-2 model and seven Qwen models from three generations. A separate representation study uses Pythia-1.4B and is treated as an auxiliary experiment, so conclusions from the eight-model copying study should not be assumed to carry over to Pythia or to other families.

Sampling is stratified by token-ID range within each vocabulary. The Qwen models use between 4,200 and 8,400 IDs apiece, while OLMo-2-7B has an initial sample of 2,098 and a replication sample of 6,273. Every sampled token goes into every context of its run. Contexts come from banks of random common-word sequences, each word passing the copying check alone. Sequence lengths are 8, 16, 32 and 64, and the target takes a random interior slot. Such sequences are controlled word lists and not grammatical prose.

Vocabulary sizes are counted as embedding rows, and the two largest models, Flash-Next and 235B, run from FP8 checkpoints. The OLMo Team (2025) and the Qwen Team (2025a, 2025b, 2026a, 2026b) document the families. Tokenizer, architecture, training, framing and precision all vary between configurations, so the ladder of models is a set of replications and does not isolate the effect of parameter count.

Copying protocol and token identity

Each copying prompt carries two demonstrations and then a sequence holding the target. The isolation probe reuses that template with the target as the only text. Prompts are assembled from token IDs, which keeps the target ID constant while everything around it changes.

That choice separates model behaviour from re-tokenisation. Adding a space to a decoded token string can change how it is segmented before the model ever sees it. Inserting IDs directly avoids the confound, but it also allows sequences that a tokenizer would not ordinarily produce from a normal string. Exact-ID copying is therefore distinguished from string preservation, and claims about everyday text use are limited accordingly. The canonical according example shows the effect is not confined to a noncanonical string.

Teacher forcing supplies the expected copied prefix before the distribution at the target's output position is read. Call q0(t) the probability the model gives target ID t in the isolation probe, and q(t,c) the probability at the matching position in context c. A small value shows weak support for the required token even after a correct prefix. It cannot show what free continuation would do once an earlier slip has happened.

The Qwen3.8 configurations place instruction, demonstrations and final text in the user turn of the chat template, open the assistant turn with Copy: and keep thinking off. All other runs use a raw few-shot sequence, and framing never varies within one model. It matters: under the raw cue, Qwen3.8 models often begin a new line and shift how leading spaces are tokenised. Of 2,000 common-word checks, 3 succeed with raw framing, whereas chat framing succeeds 1,921 times on the 27B model and 1,925 times on Flash-Next.

Behavioural fragility and the probability screen

Define the model's greedy prediction at the target's copy position, given the teacher-forced prefix. The empirical copying-error rate over a context bank of m contexts is then the fraction in which that prediction is not the target. A behavioural fragile token clears the stated isolated test and still makes contextual errors, and a quantitative class also fixes a threshold on the error rate. Failure of a whole copied sequence is a different outcome, since errors can occur before or after the target position.

Aggregate experiments run on a related probability screen instead. The primary isolation gate lets through tokens whose isolation probability q0(t) exceeds 0.5, which makes t the unique greedy output alone. That condition is sufficient for an isolated greedy pass, though some passes fall outside it. A stricter gate, applied in the OLMo sensitivity analysis, requires ln q0(t) above -0.1, or q0(t) above roughly 0.905.

A cell is flagged by the contextual screen when ln q(t,c) is below -0.5, meaning q(t,c) under about 0.607. A token's susceptibility score is the fraction of its contexts flagged. Candidates pass the gate and score 0.10 or higher, stable controls pass and score exactly zero, and tokens in between form an intermediate group. For banks of 24 contexts a candidate needs at least 3 flagged cells, and for banks of 48 at least 5.

Screen and error rate should not be merged. Imagine a token at probability 0.55 both alone and in context: it clears the main gate and is also flagged in context, yet its probability did not fall and its greedy output is right. Reported screening prevalence is therefore neither an estimate of behavioural prevalence nor an upper bound on it. The strict gate removes that particular overlap, but a contextual probability under 0.607 still does not prove the argmax is wrong. Decoded specimens serve to demonstrate genuine errors, and the aggregate probability results stay labelled as screening measurements.

Reference classes and stored specimens

For OLMo-2-7B an independent reference of 442 Magikarp-verified glitch tokens exists. None passes the main isolation gate here, and their mean contextual score in the initial bank is 0.993. Other models lack such labels, so their reference sets pair poor isolation with a contextual score of 0.90 or more. The contextual condition is needed because weak isolation performance may come from the prompt format and not from lasting copying trouble.

Each inspected token contributes three of its worst contextual cells plus one control cell to the specimen store. A record holds the context IDs, target slot, probability of the expected token, the five likeliest output IDs with probabilities, entropy and, if generation ran, a greedy continuation. Provenance fields name the model and checkpoint revision, precision, architecture metadata and software versions. Such records allow case studies. Because they pick worst cells, they cannot show how confidence is distributed across all contextual errors.

Failures are sorted descriptively into deletion, substitution, truncation and translation, with script conversion noted where it occurs. Calling something a deletion requires evidence that the model moves on to the next text, since a top-token reading in isolation cannot reveal how a whole continuation unfolds. An entropy under 2 bits serves only as a label for a concentrated stored distribution. It is no calibrated gauge of task correctness or of whether a model will admit doubt.

How susceptible are isolation passes?

Screening prevalence

With the primary gate, candidate rates run from 0.4% on Flash-Next, where 15 of 4,047 gated tokens qualify, to 10.9% on Qwen3-235B, with 684 of 6,277. Sampled vocabulary, gate, context distribution and threshold all shape that range, and it estimates neither prevalence in natural text nor the share of all greedily repeatable tokens.

The strict gate leaves OLMo-2-7B with 14 candidates out of 873 tokens in the first bank (1.6%) and 29 out of 1,888 in the replication bank (1.5%). Their isolation probability sits far above the contextual cutoff, so they serve as a check on overlap between the two thresholds. The rate of verified argmax errors still has to be measured directly.

Limiting the comparison to IDs shared across models keeps part of the gap between configurations. Take the 1,592 IDs that clear both gates on Qwen3-1.7B and Qwen3-235B: candidates make up 2.0% of them on the former and 6.2% on the latter. For 1,569 IDs shared by Qwen3-32B and Qwen3-235B, they are 0.8% and 4.3%. That cuts sampling differences but cannot tell scale apart from architecture, precision or training.

Reproducibility and the role of context

Per-token scores from the two context halves correlate at 0.82 to 0.95 across all eight models. A tougher replication on Qwen3-32B doubles the sample and draws a fresh context bank. Scores over the 4,200 shared IDs then correlate at 0.984, and 12 of the original 15 candidates still qualify. Re-scoring stored matrices for Qwen3-32B and Qwen2.5-72B on other hardware reproduces 99.9% and 99.6% of cells, with token-level score correlations of 0.9999 and 0.9996.

Variance decomposition of the first OLMo matrix puts 95.2% on the token, 0.1% on the context and 4.7% on the residual. For the strict-gate group the split becomes 71.7%, about zero and 28.2%. That large fall in the token share matters: over the full sample much of the variation merely separates broadly broken tokens from reliable ones, while a good deal of unexplained variation survives inside the gated group.

All this supports a reproducible token-level susceptibility under the sampled distribution, and it does not show that the triggering context is irrelevant. A small context main effect means no carrier is hard for every token, which is compatible with strong token-by-context variation. Consistent with that, predicting held-out OLMo cells from a token's score elsewhere gives an AUROC of 0.94, against 0.56 from how hard the carrier proved for other tokens. Token-context similarity features add nothing.

Length matters for certain groups. In the strict-gate OLMo sample, flagged-cell rates are 0.005, 0.008, 0.008 and 0.008 at lengths 8, 16, 32 and 64. For Qwen2.5-72B candidates, the shortest carriers flag 33% of cells and the longest 52%. The ranking of susceptibility is thus clearer than the conditions behind any single error.

Against released Magikarp labels

Set against available Magikarp records, none of the candidates in three model comparisons is a verified glitch token. Most count as acceptable under that procedure, and a few are classed as undecodable, which must not be read as certified healthy. Some had entered Magikarp's geometric candidate set before its behavioural check rejected them. The Qwen2.5-7B comparison uses 294 candidates and the screening summary 296, because the two analyses draw on different populations.

So the contextual screen picks up behaviour outside the available verified-glitch sets. It is not proof that every contextual candidate is a real copying error, nor that other detectors would miss them under their own assessment tasks. Quantifying that broader claim needs a comparison against a common oracle.

Demonstrated errors and concentration

The according case already proves an actual copy error despite near-certain isolated success, and other stored cases show similar behaviour across languages and configurations. A specimen's score summarises a token over its entire bank, while the displayed emission is one chosen case, so the two carry different weight as evidence. The specimens are illustrative and give no frequency for any failure mode.

Across the seven Qwen configurations, the share of selected stored failing specimens with entropy under 2 bits ranges from 14% to 74%. Some errors happen under highly concentrated distributions and others under diffuse ones. The records support the existence of confident errors, but they support neither the claim that contextual errors are usually confident nor the claim that they never come with uncertainty.

Embedding geometry and prediction

Scores and what they cost

The geometric analysis asks whether candidates relate measurably to a model's glitch-reference set. For a token t, take its row e(t) in either the input or output embedding matrix and find the five reference tokens with the highest cosine similarity in that space. The proximity score summarises similarity to those five. A second score projects the input embedding onto the line from the vocabulary mean to the mean of the glitch-reference embeddings, so proximity is local and the projection global. After the reference set and embeddings exist, neither needs a forward pass for a new token. Constructing a behaviourally screened reference set does need model runs, and that cost belongs in any discovery comparison run from start to finish.

The fitted prediction analysis compares four feature tiers:

• Surface baseline: token ID, character length of the decoded string, and whether it has a leading space, is alphabetic or is ASCII.
• Static: row norms and centroid distances for input and output embeddings, nearest-neighbour similarities, a direct input-output self-score with competitor margin, and the glitch-direction projection. The five-neighbour proximity score is not part of this tier as evaluated.
• Dynamic: summaries of the target-position representation over one half of the contexts, such as its spread, norm and alignment with the target's output row.
• Behavioural: copying measurements taken from those same contexts.

The dynamic and behavioural tiers are used to predict the candidate label as it appears in the remaining half of the contexts. The fits are out-of-fold L2-regularised logistic regressions compared with the surface baseline by paired bootstrap. Since both tiers need forward passes, they are not weight-only screens, and half-bank labels give positive counts unlike those of the full-bank screening.

Link to the glitch reference class

Proximity was measured separately in the input and output spaces. Choosing the better of the two spaces for each model gives AUROCs from 0.744 to 0.881 across the seven configurations with untied embeddings. This is a descriptive, model-by-model choice and not the performance of one prespecified score applied the same way everywhere. The weaker values on Qwen3-1.7B are kept in the report.

Which space carries the signal depends on the model. OLMo-2-7B scores 0.846 on input proximity and 0.528 on output proximity, and Qwen2.5-72B shows the reverse, 0.685 against 0.798. A single global direction does not always suffice, scoring only 0.599 on Qwen2.5-72B, yet on Qwen3-32B it reaches 0.898 and beats both proximity scores. Geometric associations therefore depend on the model, and no universal form is supported.

Swapping the glitch reference set for a random stable set of equal size gives mean AUROCs from 0.41 to 0.55 over ten draws. Arbitrary stable reference sets therefore do not reproduce the reported ranking. The comparison does not separate glitch proximity from rival explanations such as rarity, local embedding density or the way the reference set was chosen. A plain token-ID baseline actually outperforms both proximity scores for Qwen2.5-72B and for Qwen3-235B.

Weight tying cannot be judged from the Qwen3-1.7B figures, because that model is the single tied configuration in the study, and its input and output proximity scores differ even though it is described as tied. That discrepancy blocks a clean architectural reading of the row.

Where candidates sit in a projection

Candidates reach past the visible concentrations of the glitch-reference class. UMAP projections of the output-embedding rows under cosine distance (McInnes et al., 2018) show red candidate points scattered over large stretches of the grey background population in Qwen3-1.7B, Qwen3.8-27B and Qwen3-235B, while black reference points stay comparatively bunched. Qwen2.5-7B and OLMo-2-7B show candidates both beside reference clusters and among grey points. The colours encode screening and reference labels and do not verify each plotted point independently.

Li et al. (2024) observe that glitch tokens tend to cluster in embedding space, and that observation motivates the iterative graph clustering in GlitchHunter. The projections here prompt a question about its scope: can the geometry of a glitch-reference class be applied to contextual candidates that clear isolation as well? The scattered red points suggest the visible reference clusters describe this conditional population only in part.

Tang et al. (2026) likewise find clustering, supporting it with Euclidean distances and entropy comparisons run on Llama2-7B-chat and Mistral2-7B-it, and linking it to poor differentiation in the input embedding layer. Cosine-based projections of output embeddings do not test that account directly. Untied models call for separate evaluation of input and output matrices, and a scattered output projection cannot show that input embeddings are free of the reported crowding.

This fits the proximity results above. A population can sprawl over several regions and still sit close to reference tokens locally, so a useful ranking score needs no separate candidate cluster. UMAP also distorts density and may manufacture separations (UMAP developers, n.d.), so the panels do not exclude clustering in the original space. A direct test would compare neighbourhoods and contextual-failure coverage in the original space on the same checkpoints, with matched embedding spaces, matched metrics, independently verified errors and labels from each assessment method.

Value beyond surface features

On held-out context-half labels, static features beat the surface baseline with a positive interval on Qwen2.5-7B, OLMo-2-7B and Flash-Next, give no gain on five other models and a negative interval on Qwen2.5-72B. Dynamic features beat it, with intervals that exclude zero, on five models: OLMo-2-7B, Qwen3.8-27B, Flash-Next, Qwen2.5-72B and Qwen3-235B. That ties contextual representations to susceptibility beyond what the surface baseline captures. It does not make a representation-based procedure cheaper than measuring copying directly under the same contexts.

The behavioural tier reaches an AUROC of 0.945 to 1.000. That agrees with the split-half correlations: watching a token in some contexts predicts its screening label in other contexts built by the same process. This shows repeatability, but labels and features share one copying task and context distribution, so performance on new natural-text distributions remains unmeasured. The targets are probability-screen labels and not independently verified argmax-error labels.

Geometric pre-screening is plausible for deciding what to test first, but nothing here shows high-precision discovery under a fixed budget. AUROC measures ranking and not the precision of a chosen shortlist. Proving practical discovery would take a score and reference set that are frozen first, candidates that do not overlap, a set verification budget, and a comparison against random, token-ID and existing-detector selection judged on the same behavioural outcome.

What survives in generation and tasks

Free generation

Free-generation probes check whether a model keeps a supplied token while describing or answering it. The OLMo run asks for exact identification plus an explanation and then supplies the prefix Let me think about this carefully. Other configurations put the token alone in the user turn of the chat template. Generation stops at 256 tokens. Because framing differs, the counts cannot be read as a controlled scale comparison.

Two reproduction measures are needed. One asks if the seed ID appears anywhere among the generated IDs, and the other if the decoded seed string appears anywhere in the output, even with different segmentation. Neither proves correct identification or faithful reproduction of a full requested sequence. On Qwen2.5-72B, none of the 32 worst-isolation seeds returns under its original ID, yet 12 return as strings, because a seed such as _ghost can be spelled with several output tokens.

With generation available, the candidate string-preservation rate falls below the control rate in four of five comparisons and ties on Qwen3-32B. Exact-ID rates are less uniform. Qwen2.5-7B ties, while on Qwen3-32B candidates come back 8 times out of 14 and controls 7 times out of 14. A candidate can even come back while being misidentified. Vulnerability evidently depends on the task, and there is no blanket inability to emit the candidate token.

Seeds picked as the 32 worst in isolation for the separate generation probes should not be mistaken for context-screened glitch references. Teacher-forced checks give the expected token a probability of at least 0.5 in 71% of 768 Flash-Next cells and in only 0.1% of Qwen3-235B cells. Such disagreement is one more reason to verify a reference set's behaviour in the relevant context.

Free generation, task probes and specimen continuations were not run on the two FP8 mixtures, because the evaluation stack needed about seven seconds per generated token and the compute was paid for out of pocket. Their next-token observations must therefore not be read as generated reasoning traces. Candidate-free generation was also not run on Qwen2.5-72B.

Instructions that demand exact wording

Five instruction templates force the model to keep the target string intact: naming a Python function, quoting a document title, returning a list item, correcting spelling and summarising while retaining proper nouns. Ten selected tokens per class, model and task yield 50 episodes per class, and tokens enter by ID through the chat template.

The main metric checks whether the answer contains the target string, deliberately narrower than task correctness. It ignores whether code is valid, whether a function name is a legal identifier and whether a summary is otherwise faithful. The spelling task also invites normalisation, so its rate depends on its wording.

Candidates survive less often than controls in all five configurations. Episode-bootstrap intervals exclude zero in four of them (OLMo-2-7B, Qwen3.8-27B and both Qwen3-32B modes), and Qwen2.5-7B is the exception. Coarse completion stays at 0.96 to 1.00 while string preservation varies, so a task that looks finished proves nothing. Each token recurs across tasks, which limits independence.

The transcripts make the gap concrete. OLMo-2-7B, asked for a function named according, writes sum_list, rewrites according as correct in the spelling template, and answers LocalizableString when the second list item is NSLocalizedString. Qwen3-32B turns realtà into reality in the spelling task. Task form survives in these answers while the designated content changes.

Simulated tools

The tool experiment poses five assignments around a target term: investigating a configuration key, drafting a background note, recommending whether to adopt a technology, explaining a term and checking a release. Up to six rounds with search, open_page and run_python are allowed, and ten tokens per class again make 50 episodes per configuration.

The environment is wired to the supplied token. Searching for its exact string returns documentation, release notes and an issue page, and any other search yields nothing relevant and generic pages. This tests whether the target reaches a tool call at all and does not model retrieval failure in a real search engine. Qwen models use their native tool templates and OLMo a textual protocol, which weakens cross-model comparison further.

In plain mode Qwen3-32B keeps stable targets in 96% of first queries and of final answers, candidates in 74% of first queries and 68% of final answers, and glitch-reference targets in none of 50 episodes. Some responses go on from generic retrieved material after the term vanishes or changes in the first query.

On Qwen3.8-27B the first-query contrast looks alike, whereas completion behaves differently. In plain mode only 14% of glitch episodes finish inside the round limit, compared with 66% of stable controls and 58% of candidates, and identical tool calls recur in 28%, 10% and 14% respectively. Thinking mode alters those rates and the stable baseline too. Losing the target can thus end in a finished generic answer or in repeated tool use, depending on configuration.

The data do not establish widespread invention of specific retrieved facts. Fabricated version numbers appear at only 0.00 to 0.02 per class and configuration, with scattered nonzero cells in both glitch and control-related groups. The clearer effect is loss of the literal target from a query or an answer. Chosen token sets, repeated templates and a built environment all limit generalisation to autonomous systems doing natural tasks.

A negative result on natural names

Another experiment turns to everyday material: 1,255 genuine code identifiers and proper nouns appear in a natural task. Scoring how fragile their component tokens are brings no gain in predicting a wrong entity name, once rarity, constituent isolation scores and the model's probability for the whole entity are controlled. AUROC changes by -0.001, with a 95% interval from -0.008 to +0.006.

That limits how far the synthetic findings can be read into practice. The evidence does not show that the contextual score explains natural entity-name errors beyond existing predictors, nor that rarity causes either outcome, nor that every possible link between token fragility and hallucination is excluded. The fitting conclusion is that this experiment finds no incremental predictive power.

Side analyses

Pairs of tokens

A factorial copying experiment asks whether pairs of tokens do more damage than their members separately. Carriers have two slots at separations of 2, 4, 8, 16, 32, 64 and 128 with two independent fillers each. Each ordered pair is run four ways: with both targets present, with only one of them next to a control, and with controls alone. The interaction contrast is the joint score minus both single-target scores plus the control score.

The screen fits an additive model within each carrier, standardises residuals, permutes within carriers and applies Benjamini-Hochberg correction at q = 0.05 (Benjamini and Hochberg, 1995). Its count criterion sets the number of rejected pairs against the 95th percentile obtained from twenty simulated datasets in which interactions are purely additive. No pool passes: each OLMo pool rejects 1 of 4,032 pairs, Qwen3-32B rejects 13 of 1,190 and Qwen3-235B 1 of 2,256.

That does not amount to no interaction. Maximum-statistic tests are significant for the OLMo pools and Qwen3-32B, and Qwen3-235B has p = 0.059. The criteria probe different parts of the interaction distribution, so both stay in the report. Repeated-token contrasts average positive, which suggests facilitation in these carriers, at +0.005 for OLMo at separation 2 and +0.018 at 128. None of this shows universal additivity, the absence of rare harmful combinations or an induction-based mechanism, and positive means can hide negative interactions for single pairs.

In the mixed pools, five OLMo tokens cause 47% of failing carriers and five Qwen3-32B tokens cause 57%. Such concentration makes token-level susceptibility a useful summary, while failures that do not repeat across fillers show the summary is not context-free.

Is the identity still there?

In the Pythia-1.4B experiment, 36 verified glitch tokens and 1,088 controls are each put at the end of eight contexts (Biderman et al., 2023, describe the model). At each layer it tries to match a token's contextual state to its state in another context among 1,124 candidates, after subtracting the component shared inside each context. Glitch tokens match perfectly from the embedding layer to layer 12, and controls match at 0.988 to 1.000. At the last layer accuracy drops to 0.880 and 0.874. Plenty of identity information stays recoverable, then. A logit-lens example also favours continuing a glitch token, idepress followed by ant, over reproducing the input token.

Recoverability is not a causal localisation of the failure. The experiment cannot show that copying uses the recoverable information, that identity is preserved perfectly or that the output layer alone is to blame, and it concerns independently labelled Pythia glitch tokens and not the eight-model candidates. Telling poor readout from context-dependent competition would need causal interventions.

What the evidence supports and where it stops

The central finding is a repeatable gap between isolated copying success and contextual susceptibility. Shared banks make it possible to estimate a per-token tendency with no adversarial context hunted per token. Decoded examples establish that some discrepancies are literal copying errors, and task experiments show selected candidates losing their identity outside the copying template. Embedding associations tie this population to earlier glitch-token work, yet their uneven added value rules out treating geometry as a full explanation or a validated detector.

Several limits define the scope.

• A probabilistic class. Candidates are defined through a probability threshold, and the link between that threshold and genuine decoding error is only partial. Results under the strict gate and inspected continuations make the case stronger but do not replace a population-wide argmax analysis, and a better prevalence estimate must keep the isolation outcome and contextual output identity of every cell.
• Artificial carriers. Word-list carriers are contrived, and model-specific framing affects what is measured. Direct ID insertion pins down the target's identity but can build inputs a tokenizer would not produce. String tasks partly help, and the strongest natural-entity test came out negative after existing controls. Nothing licenses extrapolating prevalence to everyday text or treating a score as a context-free token property.
• Narrow coverage. Only Qwen and OLMo are tested, with different precision and architecture, and the Pythia probe asks something else. Generation coverage is incomplete, and some task classes hold only ten tokens. Episode intervals ignore dependence among five tasks sharing a token, so they are descriptive, and firmer inference needs token-clustered or hierarchical uncertainty.
• Associations only. Geometry is evaluated against screen labels, reference construction partly uses contextual behaviour, the better embedding space is picked in hindsight and random stable references are not matched on rarity or density. Static gains appear on three models while token-ID baselines are often strong. UMAP gives qualitative evidence only, and no fixed-budget discovery experiment or common-oracle comparison with other detectors exists.
• Unknown triggers. Which ordinary context provokes a given token's error stays unidentified. Split-half prediction captures average susceptibility under related banks, small context effects do not remove token-context interactions, and the factorial test keeps significant maximum-statistic results. Neither recoverable identity nor proximity explains training. Mechanism and context-specific prediction remain open.

The most direct next step is to fix four things ahead of time (isolation test, candidate score, reference construction and context distribution) and then test on disjoint data. Reporting argmax error at the target position, whole-sequence copying accuracy and exact-string preservation beside the probability scores would connect screening sensitivity to failures people can observe. Claims about discovery would be better served by precision and recall at a fixed verification budget than by retrospective AUROC alone.

What a reproduction needs

A reproduction needs model and tokenizer revisions, the sampled token IDs, complete context IDs with insertion slots, prompt framing, thresholds, numerical precision and scoring code, besides run-specific matrices and environment metadata. Weights are never modified. The projection figure additionally needs plotted IDs, class masks, embedding preprocessing, UMAP parameters and seeds, and generation coverage is limited to configurations explicitly reported.

Candidate counts differ between analyses. The screening summary gives 296, 418 and 684 for Qwen2.5-7B, Qwen2.5-72B and Qwen3-235B, an appendix gives 294, 417 and 682, and each analysis keeps its own numbers. Half-bank prediction counts differ once more, since their labels use other contexts. Closing the remaining gaps exactly needs the token manifests and exclusion masks.

The recorded protocol was amended along the way: the isolation gate, sample size, printable-seed filter and string-level scoring all changed. Sampling follows embedding-row counts, so padding lowers ID overlap between Qwen2.5 and Qwen3 samples, and matched-ID scaling inside Qwen2.5 was never tested. These choices belong to the experimental specification and limit cross-model reading.

Prompt templates and probes

Short labels used above drop instruction and quantisation suffixes. Exact checkpoint and tokenizer revisions, loader changes and environment versions belong in run manifests.

The copying template begins with one short instruction:

Repeat the text exactly.

Two demonstration pairs and the final text slot follow. The final copied prefix is teacher-forced and the target probability is read at its copy position. Angle-bracketed phrases in the raw template describe experimental insertions and are not literal prompt text:

Text: apple pie is goodCopy: apple pie is goodText: the quick brown foxCopy: the quick brown foxText: <context containing the target ID>Copy: <teacher-forced expected prefix>

For isolation, the final text slot holds only the target ID. With Qwen3.8, instruction, demonstrations and final text sit in the user turn, the copy marker opens the assistant turn and thinking is off. The tokenised delimiters are part of the run specification, since a decoded prompt string does not fully pin down the ID sequence.

The OLMo identification prompt asks what the token is, to identify it exactly and then to explain step by step what it means:

What is this token? Identify it exactly, then explain step by step what it means: <target ID>

The assistant prefix is Let me think about this carefully. Separate bare-token probes put the target in the user turn under each model's template, and plain and thinking settings count as distinct configurations. A reproduction is an occurrence anywhere in the output, scored by exact ID and by decoded string separately, and it needs no correct explanation.

Further screening and geometry details

Reference-set sizes differ by model, from 36 tokens for Flash-Next to 442 for OLMo-2-7B, with 100 for most of the Qwen models and 50 for Qwen3.8-27B. A side diagnostic counts how many of the 100 worst isolation tokens also reach a contextual score of 0.90 or more. The tally runs from 36 on Flash-Next up to 78 on Qwen3-32B. Since these counts come from different selection steps, they cannot alone rebuild each reference set, and the full negative-class mask has to travel with any reproduction.

On OLMo, a direction score from an independently labelled reference and one from a context-screened label-free reference both reach an AUROC of 0.879, while the unfiltered worst 100 isolation tokens reach 0.642. Reference construction is thus an empirical part of the method, not a swappable detail.

The strict-gate OLMo prediction analysis has 28 positive half-bank labels. Static features reach an AUROC of 0.911 against 0.711 for surface features, a gain of +0.202 with interval +0.096 to +0.320, though this is a sensitivity analysis on a different gated population. Small mismatches between bootstrap differences and subtracted rounded AUROCs cannot be resolved from the available summaries.

Entropy figures for stored failing specimens come from chosen worst cells, not a random sample of flags or errors, and they describe the distribution at the target position. No matching summary exists for OLMo.

Generation and task protocols in detail

In plain-mode generation, Qwen3-1.7B has 31 candidates and Qwen3-32B has 14 after seed filtering, and the OLMo candidate run uses bare-token framing, distinct from the instructed-identification experiment. In that instructed run no verified glitch token is reproduced by ID, while 17 controls and 11 candidates are, each out of 32, so the prompt change moves the counts and belongs to their interpretation.

Trajectories show no entropy rise against controls: the glitch-minus-control entropy gap starts at +0.27 bits (step 8) and ends at -0.29 bits (step 255). Centred final-state pairwise cosine tends to zero for both groups, so there is no sign of one shared final-state attractor. Worst-isolation seeds are chosen by isolation performance with a printable-string filter and are not necessarily the geometric reference class. Where strings were logged at all, the sample is small: eight seeds and eight controls, on Qwen3-1.7B and in the OLMo run.

In every simple-instruction template the placeholder receives the target token ID and not a re-tokenised interpolated string. The five templates:

• Function naming: write a Python function named exactly after the token that returns the sum of a list of numbers, code only.
• Title quotation: quote a document's title in double quotes and write nothing more, the title being a note on the token and related matters.
• List retrieval: from a numbered list of lantern, token, harbour and compass, name the second item only.
• Spelling correction: fix mistakes in a sentence about filing the token form before Friday, or return it unchanged.
• Summary: summarise a committee meeting in which the token presented an unchanged budget, keeping proper nouns as written.

Some templates demand exact naming and the spelling task permits correction, so task-by-task comparison matters. Each cell rests on ten episodes per class.

The tool assignments ask for:

• Configuration: a diagnosis, in two or three sentences, of why a service ignores a configuration key set through an environment variable, using documentation search, page reading and a quick check.
• Background: a note under 80 words for a newcomer, with a definition, two facts and a cited page.
• Adoption: a recommendation to adopt, trial or avoid the token, with two reasons gathered through tools.
• Meaning: an explanation for an English reader of what the term means and where it comes from, with an example sentence.
• Release: a yes-or-no answer, with one sentence of justification, on whether the latest release changed the default configuration.

In the mock environment, a query containing the target string returns three useful hits, namely documentation, release notes for a version 2.4.1 and an open issue, and other queries get generic pages. Pages open normally and the library check reports a version. These are experimental fixtures and say nothing about real libraries tied to the sampled strings. A run is finished if it gives a final answer within six rounds, repeated if an identical call recurs, and a version fabrication if it states a version without tool support. A round may contain several calls.

Glitch-class targets survive the first query at 0.02 in both Qwen3.8-27B modes and at 0.00 in the four other configurations, and survive in the final answer at 0.02 for Qwen3.8-27B plain and 0.00 elsewhere. The rates measure lexical inclusion and do not grade truth.