模型自身重复令牌的去向

# Where a Model Sends Its Own Repeated TokenA vocabulary-wide destination map, its measured robustness envelope,and the estimand it replaced
Source: [https://arxiv.org/html/2609.31181](https://arxiv.org/html/2609.31181)
###### Abstract

Black\-box model identification is an established problem with established solutions, and they work by scoring a model’s response to natural\-language prompts\. One line of work does feed models a degenerate input — their own token, repeated — but in order to find a failure mode rather than an identity\. We take that same input and ask a different question of it: not whether the model gets stuck, but where it goes when it does not\. For each tokenttin a vocabulary, readargmaxp\(⋅∣t,t\)\\arg\\max p\(\\cdot\\mid t,t\)in a single forward pass\. The resulting object is a map defined on the whole vocabulary, and it has two halves\. The first — which tokens are fixed points of that map — is partially anticipated, and we report it as a*failed*estimand: the natural distance on it is83%83\\%explained by set cardinality alone, it separates a corpus manipulation by two bits in34713471against a precision floor of zero, and it attributes families at0\.58330\.5833\. The second half, where the map sends tokens that are*not*fixed points, is not recorded in prior work; the one paper that had those tokens logged them as a zero\. Pairing the comparison on the source token removes the cardinality confound by construction \(rrfrom0\.91280\.9128to−0\.0932\-0\.0932\) and attributes model families at0\.83330\.8333— twelve models scored against a pool of nineteen — with a chance rate of0\.13890\.1389, across seven tokenizer groups and several corpora\. Two nulls clear it: frequency\-matched destinations agree at0\.14290\.1429, independent marginals at0\.07980\.0798\. Family predicts agreement better than tokenizer \(0\.20310\.2031against0\.12050\.1205\), and recurrent architectures cluster, at balanced accuracy1\.01\.0against a majority rate of0\.78950\.7895and0\.900\.90once each model’s dominant destination is excluded, which is the figure we stand behind\. We measure the robustness envelope rather than assume it:88\-bit weight rounding moves the map less than deduplicating the training corpus does \(0\.90040\.9004against0\.63530\.6353, on one support\),44\-bit destroys it \(0\.00980\.0098, and0\.18120\.1812at the granularity deployment actually uses, so the failure is not an artefact of coarse quantization\), and the numeric precision floor varies by model from0\.2010\.201to0\.97780\.9778, so every robustness statement here is per\-model\. All estimands, thresholds and kill conditions were registered before the data, and the failed one is reported at the same length as the surviving one\.

## 1Introduction

Deciding which model is behind an endpoint is a solved\-enough problem to have a literature\. Model equality testing, provenance testing and single\-shot fingerprinting all work, and they work by sending natural\-language prompts and scoring what comes back\([Nikolić et al\., 2025](https://arxiv.org/html/2609.31181#bib.bib3);[Zhang et al\., 2026](https://arxiv.org/html/2609.31181#bib.bib5);[Bruckner, 2026](https://arxiv.org/html/2609.31181#bib.bib1)\)\. We are not proposing a better version of that\. We are asking a different question: what does a model do when the input is not language at all?

The input we use is the simplest degenerate one available — a token followed by itself — and the reading is a single forward pass\. Two quantities fall out\. Whether the state\(t,t\)\(t,t\)reproduces itself under the model’s ownarg⁡max\\arg\\max, and, when it does not,*which token it produces instead*\. The first is a bit per token; the second is a destination per token\. Neither requires generation, sampling, a temperature, or a prompt\.

### What is already known, and what we therefore do not claim\.

The probe shape is not ours\.[Hammouri et al\. \(2025\)](https://arxiv.org/html/2609.31181#bib.bib2)feed models their own repeated tokens and formalise the temperature\-zero condition under which the state reproduces itself\. The argmax map over token space as a dynamical object is also not ours: our own earlier work\([Vera Zúñiga, 2026](https://arxiv.org/html/2609.31181#bib.bib4)\)identifies it and reports its per\-model structure\. Section[3](https://arxiv.org/html/2609.31181#S3)states both deltas in full, because a novelty claim that has to be reconstructed by the reader is not a novelty claim\.

### What this paper contributes\.

A map defined on the*whole vocabulary*rather than sampled from it, valued in*token identities*rather than counts, read as decoded strings so that models with different tokenizers are comparable, and characterised by a measured robustness envelope rather than an assumed one\.

### And a negative result we report at length\.

We registered the set of fixed points as the feature first\. It fails, for a reason worth printing: the natural distance between two such sets is dominated by how large they are rather than by which tokens they contain\. We report that in §[4](https://arxiv.org/html/2609.31181#S4)at the same length as the result that works, because a paper that shows only its surviving estimand gives the reader no way to judge whether it was the first one tried\.

## 2Setup

### The measurement\.

For a modelMMand a tokentt, one forward pass on the two\-token input\(t,t\)\(t,t\)givesarg⁡maxx​pM​\(x∣t,t\)\\arg\\max\_\{x\}p\_\{M\}\(x\\mid t,t\)\. Define the*self\-continuation bit*as whether that argmax isttitself, and the*destination*as the argmax when it is not\. We also store the logit margin between the winner and the runner\-up\. Nothing is sampled and nothing is generated; there is no temperature and no prompt\. The measurement is deterministic, and we assert that rather than assume it: every cell is re\-measured and required to be bit\-for\-bit identical before it is written\.

### The probe set, frozen before any model was loaded\.

Token identifiers are not comparable across tokenizers, so the index is a list of*strings*: the20002000most frequent case\-sensitive words of20002000Pile documents, bare and space\-prefixed, plus the printable ASCII block both ways and a fixed whitespace list —40904090candidates, hashed and committed before any measurement\. Each model resolves them against its own tokenizer, and the comparison set is the intersection: strings encoding to exactly one token everywhere\.

### A rule that inverts, and confusing the two manufactures findings\.

Within one model, token identifiers are authoritative and decoded strings are not, because two identifiers can print alike\.*Across*models the rule reverses: identifiers are meaningless and the string is the only bridge\. We key sources by the frozen strings and compare destinations as decoded strings, and we report the hazards of that bridge — leading spaces, non\-ASCII, empty decodings — rather than assume them away\.

### Cohort\.

Nineteen models spanning Pythia, GPT\-Neo, RWKV, Mamba, Llama, Gemma, Qwen, OLMo, Falcon, StableLM and SmolLM, with seven distinct tokenizer groups\. Twelve are Pile\-trained; the rest are not, so corpus varies\. It is not*controlled*: this design can say that a result is not confined to one corpus, and cannot attribute anything to corpus\.

## 3Related work, and what separates this from it

### The probe is not new; the index set is\.

The degenerate diagonal input and its fixed\-point reading are published\.[Hammouri et al\. \(2025\)](https://arxiv.org/html/2609.31181#bib.bib2)feed a model its own token repeated, formalise the temperature\-zero condition — atτ=0\\tau=0, a fixed pointxxofffwithf⁡\(x1,x2,x3\)=x1,x2,x3f\(x\_\{1\},x\_\{2\},x\_\{3\}\)=x\_\{1\},x\_\{2\},x\_\{3\}yields an output that never halts — and tabulate, for100100randomly chosen words across five aligned models, how many repetitions each word needs before the model stops emitting its end\-of\-string token\. They further observe that a fixed point found in a base model transfers to models derived from it, which is the same invariance our attribution rests on\. We claim none of this\.

Three things separate the present work from theirs, and only the third is a claim about novelty rather than scope\. First, the measurement: theirs is behavioural and requires generation, since non\-halting is defined by what the model does over many steps, while ours is a single forward pass per token and generates nothing\. Second, the sample: one hundred words against five aligned models — two open\-weight, three served only through an API — against an exhaustive sweep of5050k\-token vocabularies across nineteen models\. Third, and this is the delta: their table enters00wherever the model halts normally, and discards what it emitted instead\. Those cells are where our measurement begins\. We ask which token the model produced in place of the repetition, read that answer across the whole vocabulary, and compare it between models as a decoded string\. Every result below rests on the contents of their zeros\.

### What our own earlier work established, and what it did not\.

The argmax map over token space is not introduced here\. Paper 1\([Vera Zúñiga, 2026](https://arxiv.org/html/2609.31181#bib.bib4)\)identifies it as the mechanism behind the transition it reports and states the per\-model contrast directly: forpythia\-410mthe map sends1818of2424random starts to the newline token, a genuine fixed point, whilegpt2\-mediumhas no such point and wanders to1111distinct endpoints\. It also shows the map’s behaviour is a property of its domain rather than its parameters, one prepended token moving the frozen fraction from74\.4%74\.4\\%to24\.1%24\.1\\%\. The object, the mechanism, and the per\-model contrast are therefore already ours and already in print, and this paper claims none of them\.

What paper 1 measures is a trajectory census: twenty\-four random starts, iterated, with the number of distinct endpoints as the readout\. It reports how many places trajectories land, and never which tokens are fixed points, because twenty\-four starts cannot enumerate a vocabulary\. This paper inverts that\. It evaluates every token of every model’s vocabulary exactly once, keeps identity rather than count, and keeps it for the tokens that are*not*fixed points as much as for those that are\. The difference is not one of scale\. A census of endpoints and a map defined on the whole vocabulary are different objects, and only the second supports the comparison the results turn on — which tokens, agreeing between which models\.

### The claim, stated narrowly\.

What is new here is a vocabulary\-wide, set\-valued destination map: for every token in a model’s vocabulary, where its own two\-token diagonal state is sent by the argmax of the model’s conditional, decoded to a string so that models with different tokenizers are comparable at all\. Not the probe, which is[Hammouri et al\. \(2025\)](https://arxiv.org/html/2609.31181#bib.bib2)’s\. Not the fixed\-point framing, which is theirs and paper 1’s\. Not the agreement statistic, which is the established estimand of the model\-provenance line\([Zhang et al\., 2026](https://arxiv.org/html/2609.31181#bib.bib5);[Nikolić et al\., 2025](https://arxiv.org/html/2609.31181#bib.bib3)\)\. Not the leave\-one\-out family\-attribution protocol, which is already published for a single\-shot probe battery\([Bruckner, 2026](https://arxiv.org/html/2609.31181#bib.bib1)\)\. The index set, and what is recorded at each of its elements\.

## 4E1: the set of fixed points, and why we abandoned it

We registered the self\-continuation set as the feature: the bit vector over the shared probe intersection, compared between models by Hamming distance\. The intersection over twelve Pile\-trained models is34713471strings, of which16841684vary across the cohort, so the estimand had room\.

It does not work, and the numbers are worth printing\.

Table 1:The registered estimand separates families and does not separate a corpus manipulation from numeric noise\. Source: F183,results/selfcont\_verdict\.json\.### Two bits against a floor of zero\.

The pair that differs only in training\-corpus deduplication is separated by22bits out of34713471, and by*zero*at the strictest margin threshold, while the far comparisons hold hundreds\. The kill condition we registered — that the decisive distance must exceed the precision floor — passes, because the floor is00\. Its own registered corollary is what carries: a floor of exactly zero makes the test weak, since any nonzero distance clears it\.

### And the distance is mostly set size\.

Across the6666pairs, the Hamming distance correlates with the sum of the two set sizes atr=0\.9128r=0\.9128\. Some83%83\\%of what looked like an identity comparison is cardinality\. The reason is arithmetic rather than subtle: for sparse sets, Hamming*is*\|A\|\+\|B\|−2​\|A∩B\|\|A\|\+\|B\|\-2\|A\\cap B\|\. The consequence is visible in the errors —rwkv\-4\-169mis nearerpythia\-1bat6969than its own siblingrwkv\-4\-430mat412412, while their overlap coefficients are0\.69230\.6923and0\.72970\.7297\. Family attribution on this estimand is0\.58330\.5833\.

We had registered an anti\-vacuity gate on this estimand, and it did not fire, because it gates a different degeneracy: coordinates that are constant across the cohort, which contribute zero to a Hamming distance\. That reasoning is correct and incomplete, and the gate passing is what made the defect survive inspection\.

## 5E2: where the map sends the tokens that do not stay

For a source token whose bit is00, the argmax is the destination\. Comparing two models by the*agreement rate over shared source tokens*— paired on the source, destinations compared as decoded strings — removes the cardinality confound by construction rather than by adjustment:rrfalls from0\.91280\.9128to−0\.0932\-0\.0932\.

Table 2:The destination map resolves the corpus manipulation above numeric noise, where the fixed\-point set could not\. Source: F185,results/escape\_destinations\.json\.### Both nulls clear, and one of them is the test rather than a control\.

Roughly50,00050\{,\}000of a Pythia’s tokens do not self\-continue, and their destination is whatever that model’s generic high\-probability continuation is\. Two models agreeing about that would be a fact about token frequency, not about the models\. Against destinations redrawn inside their own frequency band the decisive pair agrees at0\.14290\.1429; under independent marginals it would agree at0\.07980\.0798\. The observed0\.63550\.6355is not a frequency effect\.

## 6E3: nineteen models, seven tokenizers, several corpora

The twelve\-model result has two confounds we could only disclose: both attribution errors were Mamba models landing on RWKV, so what was recovered might be architecture class rather than family; and eleven of the twelve shared a GPT\-NeoX or GPT\-2 vocabulary, so family and tokenizer were the same variable\. Widening the cohort makes both testable\. All three questions were registered with kill conditions before the new cells existed\.

Table 3:1919models, intersection33553355strings, seven tokenizer groups, corpus not fixed\. Source: F188,results/escape\_widening\.json\.### Family attribution is not principally a tokenizer effect\.

With seven tokenizer groups rather than a near\-uniform vocabulary, agreement is better predicted by shared family than by shared tokenizer\. This is the question the narrower cohort could not ask\.

### Recurrent architectures cluster, and here is exactly how much that is worth\.

Every model’s nearest neighbour is in its own architecture class\. But the recurrent class is four models from two families — the same four whose errors generated the hypothesis — because the hybrid Mamba/attention model we registered as the informative test case failed to load\. The widening added seven transformer distractors and no new recurrent model\. And when each model’s single most common destination is excluded, the perfect separation becomes0\.900\.90: above the base rate, not perfect\. The honest figure is0\.900\.90\.

### What the widening did not widen\.

No new family reached two members, so the twelve models scored for attribution are the same twelve as before; what grew is the pool of distractors they are scored against\. The chance rate falls accordingly, from0\.22730\.2273to0\.13890\.1389\.

## 7The robustness envelope, measured

A fingerprint that only works at full precision on the machine that measured it is not a fingerprint\. We registered the perturbations before running them\.

### Quantization\.

Weight\-only symmetric per\-channel round\-to\-nearest over every linear layer, with embeddings and norms untouched\. At88bits the destination map agrees with its full\-precision self at0\.90040\.9004—less movement than deduplicating the training corpus produces\(0\.63530\.6353\)\. Both are computed on the same33553355\-string support,33523352and33513351escaping sources respectively; per\-model probe sets differ, and a number from one may not be set beside a number from another\. At44bits it collapses to0\.00980\.0098, and the fingerprint is scoped to full precision\. That negative is located rather than merely bounded\. Per\-output\-channel rounding is the coarsest granularity possible and nothing deployed uses it, so we repeated the measurement over groups of128128input channels — the granularity GPTQ, AWQ and bitsandbytes all quantize at\. Grouped44\-bit reaches0\.18120\.1812: an order of magnitude better, and still short of0\.63530\.6353by a factor of three and a half\. Granularity is not what kills it\. What remains untested is calibration, the half those methods add on top of grouping, and we do not implement it\.

### The fixed\-point set does not survive44bits either, and its natural statistic hides that\.

At44bitspythia\-410m’s bit vector differs from its full\-precision self on88positions out of34713471, which reads as near\-perfect robustness\. The model has88self\-continuing tokens in that index and keeps00of them\. We report kept fractions, never Hamming counts, for the reason §[4](https://arxiv.org/html/2609.31181#S4)gives\.

### Numeric precision, and why it is reported per model\.

Reading the same weights atbfloat16instead offloat32changes the destination on2%2\\%of sources for one model and on80%80\\%for another\. There is no single precision floor for this instrument: they run from0\.2010\.201to0\.97780\.9778across eight models, so every robustness statement here is per\-model\. Withinpythiathe floor rises monotonically with scale,0\.2010\.201at7070M to0\.89890\.8989at11B, but that ordering does not survive leaving the family —gpt\-neo\-125mis the smaller model and the more robust, at0\.95690\.9569\. Infloat32the measurement is invariant to how tokens are grouped into batches \(00of1825418254argmaxes change\); inbfloat16it is not \(163163of36833683\), so part of anybfloat16floor is irreproducibility rather than precision\.

### A confidence threshold fixes the floor and destroys the signal with it\.

Restricting to sources where the winner beats the runner\-up by a margin raises the precision floor toward1\.01\.0, so the flipped destinations are near\-ties\. But it removes exactly the low\-confidence sources that carried the discrimination: at the threshold we registered as primary, the corpus pair sits at0\.99810\.9981against a floor of0\.99830\.9983— indistinguishable from numeric noise\. Signal and noise live in the same place\. An intermediate threshold looks much better, and it is not a result: surveying every model’s floor across the ladder,*all eight*sit at or above0\.990\.99at that threshold\. The ratio has no stable denominator anywhere in the cohort, so the rung is not merely unreplicated — it is unresolvable on this estimand, and we retire it\.

## 8Limits

The causal gap first\. This is an observational comparison of checkpoints\. It cannot say*why*two models disagree about where a token goes, and the design separates neither architecture nor training schedule nor hyperparameters\.

Family is confounded with tokenizer, less than before but not removed\.Seven tokenizer groups over nineteen models is enough to show family predicts agreement better; it is not enough to say tokenizer contributes nothing\.

The architecture result rests on four models from two families, the same four whose errors generated the hypothesis, and its perfect separation is0\.900\.90once the dominant destination is excluded\. A hybrid architecture would be the informative test; ours could not be loaded — a weight\-tying incompatibility between the checkpoint and the modelling code, not a capacity limit — and the test is owed rather than answered\.

Corpus varies but is not controlled, so no claim here is a corpus claim\.

The index set is a shared\-string intersection, and that bounds every cross\-lingual extension\.Common support shrinks as a cohort diversifies, and it shrinks fastest in the direction diversity comes from: adding a model with a3030k non\-Latin vocabulary took our intersection from34713471strings to152152\. Pairwise intersections with per\-pair normalisation, or byte\-level anchors, would remove this; neither is used here\.

Quantization beyond weight rounding is untested\.Activation quantization, real serving stacks and deployed quantized checkpoints are all outside what we measured\.

Instance identification is not attempted and cannot be\.The measurement is deterministic, so repeated measurement of one checkpoint is bit\-identical and the test that would license an instance claim cannot fail\. What is reported is family attribution\.

## 9Conclusion

A model’s response to its own repeated token, read across the whole vocabulary and kept as token identities rather than counts, attributes model families at0\.83330\.8333against0\.13890\.1389chance, with family outweighing tokenizer, and survives88\-bit weight rounding better than it survives a change of training corpus\. The half of that object which was already published — which tokens are fixed points — does not support the same comparison, and we report why at length: the natural distance on it measures how many there are rather than which ones\. The distinction is the paper\. What a model does when it does*not*repeat itself turns out to carry more about which model it is than whether it repeats itself at all\.

## References

- Bruckner \[2026\]Tomáš Bruckner\.One token is enough: Fingerprinting and verifying large language models from single\-token output distributions, 2026\.
- Hammouri et al\. \[2025\]Ghaith Hammouri, Kemal Derya, and Berk Sunar\.Non\-halting queries: Exploiting fixed points in LLMs\.In*IEEE Conference on Secure and Trustworthy Machine Learning \(SaTML\)*, 2025\.
- Nikolić et al\. \[2025\]Ivica Nikolić, Teodora Baluta, and Prateek Saxena\.Model provenance testing for large language models, 2025\.
- Vera Zúñiga \[2026\]Nicolás Vera Zúñiga\.What iterated self\-feeding probes of language models measure, and a test that separates the construction from the model, 2026\.
- Zhang et al\. \[2026\]Keyu Zhang, Vadim Safronov, and Andrew Martin\.Stemma: Induced decision regions reveal LLM provenance, 2026\.