相同文本，不同预测：文本分类器中的服务上下文非确定性

# Serving-Context Nondeterminism in Text Classifiers
Source: [https://arxiv.org/html/2610.09111](https://arxiv.org/html/2610.09111)
\\tl\_set:Ne\\tcboxmath

tcboxmath\\tl\_set:Ne\\tcbhighmathtcbhighmath

## Same Text, Different Prediction: Serving\-Context Nondeterminism in Text ClassifiersThanks:Accepted at the LIGHT, ODI, and TAE workshops at NeurIPS 2026\.

###### Abstract

Deterministic inference is essential for reliable and trustworthy machine learning\. Prior studies of text generation have shown that changing factors such as batch size, batch composition, hardware, or inference engine can alter the generated text, even when the prompt, model parameters, and sampling randomness are fixed\. These differences have been attributed in part to floating\-point non\-associativity, shape\-dependent kernel selection, and other implementation\-level differences in numerical execution\. However, it remains unclear whether, when, and to what extent the same factors affect text classification\. We present a systematic study of serving\-context non\-invariance in text classifiers, which prior work has measured only through generated text\. We train 180 models spanning discriminative, pseudo\-generative, and fully generative classifier formulations and evaluate each across four categories of serving contexts, holding the checkpoint and the text fixed\. Label stability does not imply score stability\. Changing only the batch shape changes no labels across fp32 comparisons, yet under bf16 it moves up to 56\.7 percentage points of predicted probability mass, with label changes concentrated at small margins\. Fully generative classifiers change more labels than their discriminative counterparts under the same serving changes\. We derive sufficient conditions for label stability under each serving change and give a separate mitigation for each mechanism\. Our results identify and quantify the serving conditions that must be fixed for reproducible text classification\.

## 1Introduction

A large language model given the same prompt, the same parameters, and the same sampling seed is widely expected to produce the same output, yet in practice it often does not\. With every user\-visible input fixed, the generated text can still change when the batch size or batch composition changes, when numerical precision changes, when the hardware changes, or when one inference engine is exchanged for another\([Yuan et al\., 2025](https://arxiv.org/html/2610.09111#bib.bib34),[Pape et al\., 2026](https://arxiv.org/html/2610.09111#bib.bib32)\)\. This phenomenon has been termednondeterminism\. One documented cause of nondeterminism is numerical\. Floating\-point addition is not associative\([Goldberg, 1991](https://arxiv.org/html/2610.09111#bib.bib8)\), and serving systems select kernels and reduction orders as a function of batch shape and load, so the same logical computation is executed as a different sequence of rounded operations\([Yuan et al\., 2025](https://arxiv.org/html/2610.09111#bib.bib34),[He, 2025](https://arxiv.org/html/2610.09111#bib.bib46),[Zhao et al\., 2026](https://arxiv.org/html/2610.09111#bib.bib31)\)\. A second cause is the inference backend itself\. Implementation choices such as paged attention, custom CUDA kernels, and optimized memory management introduce divergence large enough that switching backends alone, with model weights, prompts, decoding parameters, and hardware held constant, shifts benchmark scores by several percentage points, and by up to 16\.6 in the extreme case\([Pape et al\., 2026](https://arxiv.org/html/2610.09111#bib.bib32)\)\. In autoregressive generation these perturbations compound, because a single altered token early in the sequence conditions every token that follows\. Nearly all of this evidence, however, concerns text generation\. Whether the same serving\-context changes affect the single forward pass of a text classifier has received almost no direct attention or measurement\.

Text classification is among the most widely deployed applications of neural language models, spanning sentiment analysis\([Socher et al\., 2013](https://arxiv.org/html/2610.09111#bib.bib25),[Maas et al\., 2011](https://arxiv.org/html/2610.09111#bib.bib11)\), topic classification\([Zhang et al\., 2015](https://arxiv.org/html/2610.09111#bib.bib30)\), emotion detection\([Saravia et al\., 2018](https://arxiv.org/html/2610.09111#bib.bib23)\), etc\. Classifiers operate in both online real\-time and offline batched settings, embedded in production pipelines for relevance estimation and ranking\([Nogueira and Cho, 2019](https://arxiv.org/html/2610.09111#bib.bib54)\), filtering\([Sahami et al\., 1998](https://arxiv.org/html/2610.09111#bib.bib53)\), and content moderation\([Wulczyn et al\., 2017](https://arxiv.org/html/2610.09111#bib.bib51),[Lees et al\., 2022](https://arxiv.org/html/2610.09111#bib.bib52)\), where they form part of larger machine learning systems whose behavior must be maintained over time\([Sculley et al\., 2015](https://arxiv.org/html/2610.09111#bib.bib24)\)\. Transformer\-era classifiers also come in distinct paradigms\. Following the terminology of[Kasa et al\. \(2025\)](https://arxiv.org/html/2610.09111#bib.bib12), discriminative classifiers attach a dedicated classification head to a pretrained encoder\([Devlin et al\., 2019](https://arxiv.org/html/2610.09111#bib.bib4)\); pseudo\-generative classifiers are trained with a language\-model objective and predict a class token in a single forward pass; and fully generative classifiers compare the likelihood of the input under each candidate class, a distinction with a long history from classical statistics to neural models\([Efron, 1975](https://arxiv.org/html/2610.09111#bib.bib35),[Ng and Jordan, 2001](https://arxiv.org/html/2610.09111#bib.bib36),[Yogatama et al\., 2017](https://arxiv.org/html/2610.09111#bib.bib38),[Zheng et al\., 2023](https://arxiv.org/html/2610.09111#bib.bib41)\)\. These formulations differ measurably in propensity to shortcut learning\([Li et al\., 2025](https://arxiv.org/html/2610.09111#bib.bib43),[Kasa et al\., 2026](https://arxiv.org/html/2610.09111#bib.bib44)\), sample efficiency\([Ng and Jordan, 2001](https://arxiv.org/html/2610.09111#bib.bib36),[Zheng et al\., 2023](https://arxiv.org/html/2610.09111#bib.bib41)\), calibration\([Guo et al\., 2017](https://arxiv.org/html/2610.09111#bib.bib9)\), and privacy risk\([Makroo et al\., 2026](https://arxiv.org/html/2610.09111#bib.bib45)\), so a serving\-time perturbation need not affect them equally\.

In every one of these deployments, serving\-context nondeterminism can change the final probabilities without any visible change to the model or the text, with direct consequences such as a reclassified review, a rerouted document, or a filtered message\. Yet deployed classifiers are commonly monitored through aggregate metrics such as accuracy or F1, which are invariant to which examples flip\. This motivates our present study\. We ask whether the serving\-context effects documented for generation transfer to classification, and whether the apparent invariance of classification reflects stability or only the absence of measurement\. We present a systematic study of serving\-context sensitivity in neural text classifiers, conducted through large\-scale controlled experimentation\. The contributions of this work are as follows:

1. 1\.First systematic empirical study\.We organize serving\-context factors into a taxonomy and measure their effects\. We train 180 text classifiers from scratch \(four transformer classification formulations, three sizes, five datasets, three seeds\) and evaluate each across 3,960 serving contexts \(padded length, batch size, batch composition, precision, quantization, device, backend\), holding the checkpoint and text fixed so any change is attributable to the setting rather than run\-to\-run randomness\. Two findings follow\. Changing the padded length alone changes about 10% of predicted labels, more than any other factor, and settings that change almost no labels still shift class probabilities by tens of points, so label accuracy does not reveal the instability\.
2. 2\.Mitigations\.We show that each mechanism has a targeted fix and measure its invariance–cost trade\-off\.
3. 3\.Theory\.We formalize the conditions under which logit and probability drift can lead to label flipping\. We also give sufficiency conditions to rule out padding\-based non\-determinism\. Based on the empirical results, we formalize the three\-way tension among inference speed, task accuracy, and serving\-context invariance into an impossibility condition: the optimizations that make serving fast, including dynamic batching, reduced precision, quantization, and shape\-dependent kernel selection, are the mechanisms that threaten invariance, while remedies that restore invariance can cost speed or accuracy\.

## 2Related Work

Discriminative, generative, and pseudo\-generative classifiers\.The comparison between discriminative and generative classifiers has origins in the classical analyses of logistic regression, discriminant analysis, naive Bayes, and related estimators\([Efron, 1975](https://arxiv.org/html/2610.09111#bib.bib35),[Ng and Jordan, 2001](https://arxiv.org/html/2610.09111#bib.bib36),[Xue and Titterington, 2008](https://arxiv.org/html/2610.09111#bib.bib37)\)\.[Yogatama et al\. \(2017\)](https://arxiv.org/html/2610.09111#bib.bib38)extended the comparison to neural text classification,[Rezaee et al\. \(2021\)](https://arxiv.org/html/2610.09111#bib.bib40)studied transformer implementations, and[Zheng et al\. \(2023\)](https://arxiv.org/html/2610.09111#bib.bib41)developed a broader theoretical analysis\. Recent work studies language models used directly for text classification\([Sun et al\., 2023](https://arxiv.org/html/2610.09111#bib.bib42)\)and provides a transformer\-era comparison of encoder, autoregressive, masked\-language\-model, and discrete\-diffusion classifiers\([Kasa et al\., 2025](https://arxiv.org/html/2610.09111#bib.bib12)\)\. The masked\-language\-model formulation is related to pseudo\-likelihood estimation\([Wang and Cho, 2019](https://arxiv.org/html/2610.09111#bib.bib39)\)\.[Kasa et al\. \(2026\)](https://arxiv.org/html/2610.09111#bib.bib44)show that fully generative classifiers are not inherently immune to shortcut learning, while[Makroo et al\. \(2026\)](https://arxiv.org/html/2610.09111#bib.bib45)show that modeling the input distribution can increase membership\-inference risk\. These studies compare training objectives, model formulations, sample efficiency, robustness, calibration, and privacy\. They do not examine whether a fixed trained classifier output changes when padding, batching, numerical precision, quantization, device, or backend changes at inference time\.

Non\-determinism in language\-model inference\.For text generation, prior work documents that outputs change when batch size, precision, hardware, or the inference engine changes, even with the prompt, parameters, and sampling seed fixed\([Yuan et al\., 2025](https://arxiv.org/html/2610.09111#bib.bib34),[Pape et al\., 2026](https://arxiv.org/html/2610.09111#bib.bib32)\)\. Repeated calls to hosted models under nominally deterministic settings show the same variability in the generated answer strings, even when those strings are then parsed into task labels\([Atil et al\., 2024](https://arxiv.org/html/2610.09111#bib.bib55)\)\. The numerical cause is that floating\-point addition is not associative\([Goldberg, 1991](https://arxiv.org/html/2610.09111#bib.bib8)\)and serving systems select kernels and reduction orders as a function of batch shape\([He, 2025](https://arxiv.org/html/2610.09111#bib.bib46),[Zhao et al\., 2026](https://arxiv.org/html/2610.09111#bib.bib31)\)\. Backend implementation choices add further divergence\([Pape et al\., 2026](https://arxiv.org/html/2610.09111#bib.bib32)\)\. Dataset\-level metrics attenuate exactly the per\-example movement these effects produce\([Fang et al\., 2026](https://arxiv.org/html/2610.09111#bib.bib49)\)\. Remedies proposed for generation include batch\-invariant kernels\([He, 2025](https://arxiv.org/html/2610.09111#bib.bib46)\), fixed\-shape scheduling\([Zhao et al\., 2026](https://arxiv.org/html/2610.09111#bib.bib31)\), and verified recomputation of the decoded steps that disagree\([Gond et al\., 2026](https://arxiv.org/html/2610.09111#bib.bib56)\)\. This literature studies generation, where autoregressive feedback amplifies a one\-token change\. Whether and when the same mechanisms appear in the single forward pass of a classifier is the question we study\.

Extended discussion\.Appendixexpands this section: an extended account of the three classifier families and of non\-determinism in language\-model inference, and a discussion that situates the present study against prediction instability and churn, imprecise probability and calibration, the numerical and positional mechanisms the paper relies on, and behavioral and production\-readiness testing\.

## 3Serving Context and Predictive Claims

Letpθ\(⋅∣x,c\)p\_\{\\theta\}\(\\cdot\\mid x,c\)denote the class\-probability vector for targetxx, checkpointθ\\theta, and serving contextcc\. The example whose prediction we track is the target\. A context records every implementation setting that can vary while the trained parameters and the text stay fixed: the software backend, the device, the numerical precision, the quantization configuration, the batch size, the order in which examples are processed, the position an example occupies within its batch, the other examples that share the batch, the padding policy, and the rule that assigns position indices to tokens\. For a declared finite set of contexts𝒞\\mathcal\{C\}, our comparisons observe selected pairs drawn from𝒞\\mathcal\{C\}\. Each pair fixes the checkpoint and the text and changes one recorded setting\. These pairs are controlled witnesses of how far the prediction can move, not a guarantee over every serving configuration, and we place no distribution over the contexts\.

Letfθ​\(x,c\)=arg⁡maxy​pθ​\(y∣x,c\)f\_\{\\theta\}\(x;c\)=\\arg\\max\_\{y\}p\_\{\\theta\}\(y\\mid x,c\)\. For reference contextc0c\_\{0\}, the paired flip indicator isFlipθ\(x;c0,c\)=𝟙\[fθ\(x;c0\)≠fθ\(x;c\)\]\.\\mathrm\{Flip\}\_\{\\theta\}\(x;c\_\{0\},c\)=\\mathbb\{1\}\\\!\\left\[f\_\{\\theta\}\(x;c\_\{0\}\)\\neq f\_\{\\theta\}\(x;c\)\\right\]\.The flip rate averages this indicator over the targets that both contexts scored\. It measures the decision changes a user would observe rather than aggregate quality\.

Labels can remain stable while scores move\. For aligned probability vectorsp0p\_\{0\}andpcp\_\{c\}, we measure total variationTV⁡\(p0,pc\)=12​∑k\|p0,k−pc,k\|\.\\mathrm\{TV\}\(p\_\{0\},p\_\{c\}\)=\\frac\{1\}\{2\}\\sum\_\{k\}\|p\_\{0,k\}\-p\_\{c,k\}\|\.We also report the maximum absolute logit change∥z0−zc∥∞\\lVert z\_\{0\}\-z\_\{c\}\\rVert\_\{\\infty\}\. Score drift is computed both over all aligned targets and over the subset whose top label does not change\.

We separate two paths by which context can reach the predictive distribution:pθ\(⋅∣x,c\)=N\(Ec\(θ,Pc\(x\)\)\)\.p\_\{\\theta\}\(\\cdot\\mid x,c\)=N\\\!\\left\(E\_\{c\}\\\!\\left\(\\theta,P\_\{c\}\(x\)\\right\)\\right\)\.PcP\_\{c\}constructs token IDs, masks, position IDs, padding, and tensor layout;EcE\_\{c\}executes the checkpoint and returns class scores; andNNapplies the fixed class normalization\. A numerical intervention changesEcE\_\{c\}while preserving the model\-relevant output ofPcP\_\{c\}\. A structural intervention changesPc​\(x\)P\_\{c\}\(x\)itself, as when a real token’s position index is read from its tensor column and shifts with the padded width\. This decomposition supplies one of the two axes we use to organize every serving factor\. The first axis is the*mechanism*by which a factor can move the prediction, and it is the split above\. A factor is structural if it changesPcP\_\{c\}, the represented input, and numerical if it changes onlyEcE\_\{c\}, the execution of a fixed represented input\. The second axis is the*locus*at which a factor is set\. A deployment\-level factor is chosen once when the model is put into service and then held fixed across requests, whereas a request\-level factor varies from one request or batch to the next\. Every factor we study occupies one cell of the resulting table \(Table\)\. Padding is the single factor that spans a boundary\. Its policy is deployment\-level, but the width it produces under a dynamic policy is realized per request, and it is the only request\-realized factor whose mechanism is structural, so we report it on its own\. This table organizes both the design below and the results that follow\.

Each comparison changes exactly one recorded setting and holds the checkpoint, the dataset split, the evaluated examples, and every other recorded setting fixed\. A comparison whose reference run is missing is discarded rather than substituted from another batch size or seed, and when the intervention is which examples share the batch, the padded width is held fixed so that changing the other examples cannot change the tensor shape\.

## 4Experimental Design

The experiments address four questions: \(a\) does a prediction measured in one serving context still hold in the others; \(b\) do the predicted label, the class scores, and the decision margin agree about stability; \(c\) which of the factors move the prediction by executing the same input differently and which by changing the input itself; and \(d\) what evidence distinguishes a working mitigation from prediction collapse or from a comparison that applies no effective intervention?

Models and data\.We train 180 classifiers from random initialization: four classifier formulations built on BERT or GPT\-2, three jointly scaled sizes, five datasets \(SST\-2, SST\-5, AG News, Emotion, and IMDb\([Socher et al\., 2013](https://arxiv.org/html/2610.09111#bib.bib25),[Zhang et al\., 2015](https://arxiv.org/html/2610.09111#bib.bib30),[Saravia et al\., 2018](https://arxiv.org/html/2610.09111#bib.bib23),[Maas et al\., 2011](https://arxiv.org/html/2610.09111#bib.bib11)\)\), and three random seeds, reusing public tokenizers but no pretrained weights\. The small, medium, and large sizes scale the feed\-forward width, depth, and attention heads jointly, following the three jointly scaled sizes of[Kasa et al\. \(2025\)](https://arxiv.org/html/2610.09111#bib.bib12)and[Makroo et al\. \(2026\)](https://arxiv.org/html/2610.09111#bib.bib45)\. The exact configurations, objectives, and optimization are in Appendix\.

Serving interventions\.The taxonomy of Tablesorts the comparisons into three groups, each departing from the reference deployment of full precision, GPU execution, and static padding\. Deployment\-level comparisons hold the request context at a single example and change one deployment factor: the numerical precision \(full precision versus the reduced\-precision format bf16, with fp16 reported in the appendix\) and the 8\-bit integer scale computation \(an offline fixed scale versus a scale recomputed from each batch\)\. Request\-level comparisons instead hold the deployment fixed and change one request factor: the batch size, the padded width \(static versus dynamic\), the position the target occupies within its batch, the order in which examples are processed, and the other examples that share the batch at a fixed width\. The order comparison permutes the full test list before fixed\-width batching, so it can change the target’s peers and slot but not the tensor width; the composition comparison changes the peers at a fixed width; a separate random\-peer experiment lets peer lengths change the dynamic width and is a joint peer and shape intervention\. A third comparison changes device and 8\-bit integer quantization together, from GPU full precision to CPU integer execution; because it moves several factors at once it is reported as a whole\-deployment migration rather than as an estimate of any single factor, and it is the change a practitioner makes in a single step\. Padding comparisons keep the target and the other examples in its batch fixed and change dynamic versus static width under each family’s existing padding side\. They do not randomize left versus right padding within a checkpoint\.

Model\-size analysis\.The model\-size analysis compares small, medium, and large models and retains a family, dataset, and seed combination only when all three sizes exceed 1\.15 times chance validation accuracy, leaving 40 matched combinations overall and 16 for the GPT\-2 padding comparison\. We average repeated comparisons within each checkpoint and size, weight each retained combination equally, and resample combinations while preserving the three\-size pairing\.

## 5Results

We report every effect against a matched reference that fixes the checkpoint and the text and changes one recorded setting\. The reference deployment is full precision on GPU with static padding\. The deployment\-level and batch\-size comparisons are measured against a single example, and the within\-batch comparisons against the canonical arrangement at a fixed batch\. Each comparison records three quantities on the same target: whether the predicted class changes, how far the class probabilities move \(the total variation distance, a value between 0 and 1\), and how far the unnormalized class scores move \(the largest absolute change among the logits\)\. Because a classifier that always predicts one class would look perfectly stable while being uninformative, we keep only the 158 of 180 trained models whose validation accuracy exceeds random guessing by a 15% relative margin\. Repeating the identical context reproduces the logits bitwise on every hardware and software configuration we test \(Table\), so each reported change is caused by the setting we vary, not by run\-to\-run variation\.

We present the comparisons in the order of the taxonomy \(Table\): first the deployment\-level changes a model owner chooses once, then the request\-level changes that vary with traffic after a deployment is fixed, and finally the combined migration a practitioner makes in one step\. Two findings hold across all three\. First, a stable predicted label routinely coexists with movement in the class probabilities and scores that downstream thresholds, ranking, and filtering consume, so we report label changes and score movement together throughout\. Second, the two mechanisms of Tablebehave differently\. A change to the represented input \(the structural mechanism\) moves labels across a wide range of confidence, whereas a change to execution alone \(the numerical mechanism\) moves labels only where the two leading classes are nearly tied\.

### 5\.1Deployment\-level changes

Moving a fixed model to a different deployment while holding the request at the reference, a single example at static width, can change the prediction even though there is no batching, no neighboring example, and no change to the input\. Figurereports two such single\-factor changes\. Reducing the numerical format from full precision to bf16 leaves the discriminative BERT classifier’s labels unchanged, but flips about one percent of the fully generative GPT\-2 classifier’s labels and moves one of its scores by up to 0\.56\. Replacing the offline fixed 8\-bit integer scale with a scale recomputed from each batch is the larger change\. It flips 4 of 192 fully generative labels and moves a score by up to 0\.84, again with no observed change for BERT\. Precision and quantization are distinct changes\. Reduced precision \(bf16\) is a floating\-point format with fewer mantissa bits, whereas the 8\-bit integer path represents activations as integers through a scale\. Here it is the choice of scale rather than the format that produces the larger effect\. The gap between the discriminative and the fully generative formulation appears again throughout the paper and is largest under the combined migration of Section\.

Figure 1:Deployment\-level changes at the reference request context \(one example, static width\): reduced precision \(full precision to bf16\) and the 8\-bit integer scale computation \(offline fixed to per\-batch\)\.
Table 1:Two axes for the serving factors\. Rows: deployment\-level \(fixed once per service\) and request\-level \(varied per request\)\. Columns: structural \(the represented input changes\) and numerical \(only execution changes\)\. Padding spans the boundary\.

### 5\.2Request\-level serving context within a fixed deployment

Once a deployment is fixed, the settings that still vary from one request or batch to the next are the batch size, the padded width, the target’s position within its batch, the order of the other examples, and which other examples share the batch\. Figurereports the two that change the tensor shape, batch size and padded width, across three deployments\. The within\-batch rearrangements are treated separately below\.

Figure 2:Request\-level shape changes within a fixed deployment: batch size \(against the single\-example reference\) and padded width \(static to dynamic\), for full\-precision GPU, bf16 GPU, and 8\-bit integer CPU\. Padded width is the dominant effect in every deployment; batch size is near the floor at full precision and grows only under reduced precision and 8\-bit integer execution\.Padding is the dominant request\-level effect, and it is structural\.Changing the padded width is the largest request\-level effect on every quantity in Figure, and it is the only one that changes the represented input rather than its execution\. Every padding\-induced label change occurs in the two GPT\-2 families, about one in five of their comparisons, while the right\-padded BERT and masked\-language\-model classifiers change none\. Because the attention mask already blocks padding tokens in every model, the difference is where the real tokens sit\. Right padding leaves their position indices fixed at every width, while left padding shifts each real token to a higher tensor column\. Two pretrained models separate the padding side from the position representation\. Left\-padded Qwen3 with rotary positions\([Qwen Team, 2025](https://arxiv.org/html/2610.09111#bib.bib14)\)and right\-padded ModernBERT\([Warner et al\., 2025](https://arxiv.org/html/2610.09111#bib.bib27)\)both change no labels, so left padding alone is not the cause \(Table\)\. The exact counts, the pretrained\-model replication, and a no\-position\-encoding boundary case are in Appendix\.

Why width\-dependent absolute positions change the input\.For a real sequence of lengthLLin a left\-padded tensor of widthTT, the positions read from the tensor column are\(T−L,…,T−1\)\(T\-L,\\ldots,T\-1\)\. ChangingTTtherefore changes the learned position embedding added to every real token, so the model receives a different input even though the text is unchanged\. Assigning positions instead by each token’s order among the real tokens,\(0,…,L−1\)\(0,\\ldots,L\-1\), does not depend on the width\. Two matched interventions confirm the cause\. Assigning positions by real\-token order removes the padding effect, and deliberately forcing the width\-dependent positions back restores it, on both GPU and CPU\. Sectionstates the width\-invariance result formally, and Appendixreports the per\-seed rates and the intervention figure \(Figure\)\.

Batch size matters only through the numerical format\.The batch\-shape effect documented in the generation literature transfers to a single classifier forward pass only under a reduced numerical format\. At full precision, changing only the batch shape flips no labels across 226,104 comparisons and moves probabilities by less than10−510^\{\-5\}, whereas under bf16 it moves one prediction’s probability by up to 56\.7 percentage points, and the 8\-bit integer path is larger still \(Figure\)\. Batch size is therefore a rounding effect at full precision and a measurable effect only once the format is reduced\. The margin stratification and the label\-count effect are in Appendix\.

The remaining within\-batch rearrangements move no label, with one exception\.Reordering the other examples is bitwise identical at every precision, because it never touches the target’s own row\. Swapping which other examples share the batch is bitwise identical under floating\-point execution for the same reason: no operation mixes one example’s rows into another’s\. Under dynamic 8\-bit integer quantization, however, it is the one within\-batch change that moves a label\. The activation scale is computed over the whole batch, so the content of the other examples reaches the target through that shared scale\. Figureshows this composition effect\. It grows with batch size, reaching about 3\.7 percent of fully generative labels and a score movement of 1\.3 at a batch of 256\. Appendixdetails the mechanism and its removal under a fixed scale\.

The target slot is invariant only up to the kernel’s reduction order\.Moving the target to a different row at a fixed batch shape leaves the output unchanged in exact arithmetic yet not bitwise on the GPU\. A linear layer computes each output coordinate as a sum over the hidden dimension,yn=∑kak​Wk,ny\_\{n\}=\\sum\_\{k\}a\_\{k\}W\_\{k,n\}, so the target row’s value depends only on its own activations and never on the row it occupies\. The GPU matrix multiplication, however, folds the batch and the sequence into one dimensionMMand evaluates that sum in an order that depends on the row’s offset inMM, so changing the slot changes the summation order and, because floating\-point addition is not associative, moves the logits by about10−510^\{\-5\}with no label change\. The integer CPU path uses one fixed reduction order, so the logits are bitwise identical across slots\. Batch\-slot invariance is therefore a property of the kernel’s reduction order rather than a mathematical guarantee, which is why the batch\-invariant kernels of Sectionremove it\. Appendixgives the full account\.

Model size and score drift under unchanged labels\.Among predictions whose label does not change under padding, the probability movement roughly doubles as the model grows, so a larger model can appear more label\-stable while the scores under those labels move as much or more\. The label\-change rate itself is not monotone in size\. Because size, width, depth, and training quality change together, this is a descriptive association rather than a scaling law, and the per\-size breakdown is in Figuresand\.

### 5\.3The combined effect of a whole\-deployment migration

A practitioner rarely changes one factor at a time\. A common change moves a trained classifier, without retraining, from a full\-precision GPU to a quantized CPU service, changing the device, numerical format, and operator path together\. Figurereports this combined change by classifier formulation\. The effect grows with how generative the classifier is\. The discriminative BERT classifier changes about 0\.3 percent of labels, the two pseudo\-generative formulations about 2% each, and the fully generative GPT\-2 classifier about 5\.6 percent on average and up to 39 percent on its worst checkpoint\. We report this as a whole\-migration profile rather than a single\-factor estimate; Appendixseparates the two factors\.

A second combined change occurs after the deployment is fixed, because real traffic arrives in large, variable\-length batches that the serving system batches and pads dynamically\. Figureshows that this serving load moves the fully generative classifier, about 1\.6 percent of labels at full precision and more under 8\-bit integer execution, while it leaves the discriminative classifier near the floor\. The driver is the realized padded width rather than the batch count\.

\(a\)Whole\-deployment migration \(full\-precision GPU to quantized CPU\), by classifier formulation\. Reported as a whole\-migration profile, not a single\-factor estimate\.\(b\)Serving load within a fixed deployment, per deployment: a large batch with dynamic \(batch\-maximum\) padding against the single\-example static\-width reference\.
Figure 3:Two combined deployment changes a practitioner encounters: a whole\-deployment migration \(left\) and serving load within a fixed deployment \(right\)\.The two combined changes rank differently under labels than under scores, and the reason applies whenever only the label is monitored\. Holding the dataset fixed, the migration and the serving load flip the same fraction of fully generative labels\. However, the serving load moves the raw scores several times more in the mean and far more in the worst example, while the probability movement stays comparable\. A label change is governed by the gap between the two leading classes after the bounded softmax\. The small, near\-uniform perturbations that the quantization introduces therefore convert efficiently into flips near the boundary\. By contrast, the large, heavy\-tailed score changes that the padding mechanism produces often fall on confident examples, so they inflate the score movement without changing the label\. Comparisons across the two changes must therefore hold the dataset and its number of classes fixed, and a label\-only monitor would rank them backward\.

### 5\.4Mitigations and their costs

Each measured mechanism has a targeted mitigation, and none covers the others \(Table\)\. Assigning positions by real\-token order removes the structural padding effect, provided the same rule is used in training, since applying it only at inference degrades accuracy or collapses predictions \(Appendix\)\. Because this rule no longer requires a fixed maximum width, it also permits dynamic padding, which processed about one quarter as many padded tokens for about two to two and a half times the target throughput in our harness\. Batch\-invariant matrix\-multiplication kernels drive the batch\-shape movement to bitwise equality at full precision and bf16, and they remove the residual bf16 flips\. Their precision\-matched cost is roughly neutral at batch size 1 and about1\.31\.3to1\.5×1\.5\\timesthe standard latency at large batch\. Adopting them changes the numerical result, so they are a deployment choice rather than a run\-time switch\. A fixed offline 8\-bit integer scale removes the coupling between co\-batched inputs\. For the small\-margin flips that remain under reduced precision, recomputing only the low\-margin predictions in the reference context detects almost all observed flips while recomputing a small fraction of requests\. All costs are measured in our experimental setup and do not represent production speedups\.

Figure 4:Changing which other examples share the batch, under dynamic 8\-bit integer quantization on CPU, at a fixed batch shape, as a function of batch size\.
Table 2:Mechanism\-matched mitigations\. Costs are measured in our experimental setup and do not represent production speedups\.

### 5\.5Formal results

Three formal statements specify when the effects measured above can occur\. Appendixstates the assumptions and gives the proofs\.

The first statement identifies when padding can change a prediction\. It gives a sufficient condition for a change of padded width to leave the prediction unchanged, together with the converse for the one scheme that violates it\.

Theorem 5\.1:Width invariance is governed by the position rule\.Consider two left\-padded tensors containing the same real\-token sequence, differing only in the number of padding columns, so that every real token’s tensor column shifts by a common offsetss\. Suppose \(i\) every positional contributionΦ\\Phito the computation is invariant to a common shift,Φ⁡\(i\+s,j\+s\)=Φ⁡\(i,j\)\\Phi\(i\+s,j\+s\)=\\Phi\(i,j\); \(ii\) padding is excluded by the mask from every attention and pooling operation; \(iii\) the output position from which the class scores are read follows the same real\-token state; \(iv\) any cache offsets use one consistent coordinate system; and \(v\) no other operation depends on the absolute tensor column\. Then, in exact arithmetic, the classifier logits are identical at both widths\. Conversely, if positions index a learned absolute embedding table by tensor column, the two widths present different represented inputs wheneverp⁡\(πi\+s\)≠p⁡\(πi\)p\(\\pi\_\{i\}\+s\)\\neq p\(\\pi\_\{i\}\)for any real token, and no same\-input numerical guarantee applies\. Proofs: Propositionsand, with the per\-scheme instantiation in Proposition, Appendix\.

With correct masking and a correct output position, whether padding changes the represented input is therefore decided by the position rule, before any numerical consideration\. A rule that depends only on the distance between tokens, or that numbers real tokens from zero, is safe\. A learned embedding indexed by the tensor column is not\. This theorem gives a sufficient condition together with its converse for the learned\-absolute case\. It does not characterize every scheme\. It also covers exact arithmetic only, since removing the structural mechanism does not by itself guarantee bitwise equality under finite\-precision execution\. Next, the following statement limits which predictions are influenced by a numerical change\. It is a one\-sided certificate whose contrapositive is the necessary condition for a flip\.

Proposition 5\.1:Margin certificate for label stability\.Letwwbe the unique predicted class under the reference contextc0c\_\{0\}, with probability marginm⁡\(x\)=pw​\(x,c0\)−maxj≠w⁡pj​\(x,c0\)m\(x\)=p\_\{w\}\(x,c\_\{0\}\)\-\\max\_\{j\\neq w\}p\_\{j\}\(x,c\_\{0\}\), and letTV⁡\(x,c,c0\)=12​∑y\|py​\(x,c\)−py​\(x,c0\)\|\\operatorname\{TV\}\(x,c,c\_\{0\}\)=\\tfrac\{1\}\{2\}\\sum\_\{y\}\\lvert p\_\{y\}\(x,c\)\-p\_\{y\}\(x,c\_\{0\}\)\\rvert\. Ifm⁡\(x\)\>2​TV⁡\(x,c,c0\)m\(x\)\>2\\operatorname\{TV\}\(x,c,c\_\{0\}\), the predicted class is identical underccandc0c\_\{0\}; equivalently, every label change requiresm⁡\(x\)≤2​TV⁡\(x,c,c0\)m\(x\)\\leq 2\\operatorname\{TV\}\(x,c,c\_\{0\}\)\. The analogous logit statement holds with the logit marging⁡\(x\)g\(x\)and threshold2​ϵ2\\epsilonwhen the logit change is bounded byϵ\\epsilonin the infinity norm\. Proofs: Propositions,, andin Appendix\.

A label can change only where the two leading classes are close, which is why reduced precision and quantization move labels only near the decision boundary while a wide margin leaves the label intact\. The certificate is one\-sided\. It preserves the label, not the calibration, ranking, or thresholds that consume the scores, and it is sufficient rather than necessary for stability, so a label can also remain fixed below the threshold\. For a fully generative classifier that sums a score over every token, the admissible movement grows with the input length, which is why that formulation is the most exposed \(Proposition\)\.

The third statement states the single condition under which speed, accuracy, and serving\-context invariance cannot hold together\.

Theorem 5\.2:Conditional infeasibility of joint speed, accuracy, and invariance\.Fix a task distribution, a bounded context set𝒞\\mathcal\{C\}, one execution stack, an implementation familyℋ\\mathcal\{H\}, and predeclared thresholdsα\\alpha,τ\\tau,δ\\deltaon worst\-context accuracyA\(f\)=infcPr\[argmaxypf\(y∣X,c\)=Y\]A\(f\)=\\inf\_\{c\}\\Pr\[\\arg\\max\_\{y\}p\_\{f\}\(y\\mid X,c\)=Y\], worst\-case latencyT⁡\(f\)T\(f\), and score driftV⁡\(f\)=supx,c,c′TV⁡\(x,c,c′\)V\(f\)=\\sup\_\{x,c,c^\{\\prime\}\}\\operatorname\{TV\}\(x,c,c^\{\\prime\}\)\. Suppose a statisticQ=q⁡\(X\)Q=q\(X\)exists such that every implementation withT⁡\(f\)≤τT\(f\)\\leq\\tauandV⁡\(f\)≤δV\(f\)\\leq\\deltapredicts as a deterministic function ofQQin every context\. If the Bayes accuracy available fromQQsatisfiesAQ⋆<αA\_\{Q\}^\{\\star\}<\\alpha, then no implementation inℋ\\mathcal\{H\}meets all three thresholds\. Proof: Theoremin Appendix; Propositionderives the premise from an execution model and Propositiongives a pairwise\-feasible, jointly infeasible instance\.

In the configurations we tested, none of the three had to be given up to bound the others\. Order\-based positions removed the structural mechanism while improving throughput, fixed 8\-bit integer scales preserved quality, and batch\-invariant execution removed the batch\-shape drift at the bounded cost reported above\. The theorem does not assert that such a tradeoff exists\. Its premise holds for execution models whose fast path exposes a fixed representation, and no such model has been identified for these classifiers\. It marks the point at which being fast and invariant would force a classifier to discard information that accuracy needs\. Propositions,, andstate the quantization coupling of Sectionand the length\-proportional generative drift\.

## 6Conclusion

Findings\.For the classifiers we study, a prediction is not fixed by the trained model and the input text alone\. It also depends on serving settings the user never sees\. The largest effect is input\-side\. When a classifier with a learned embedding for each absolute position is padded on the left, changing the padded length moves every real token to a new position and changes the input the classifier receives\. Assigning positions by order among the real tokens, in training and inference alike, removes the effect in our experiments without lowering accuracy\. The numerical causes \(reduced precision, batch shape, quantization\) are smaller and concentrate near the decision boundary, yet a stable label can coexist with large probability movement that thresholds, ranking, and filtering consume, and a wide margin does not protect against the input\-side cause\.

Scope\.The measurements cover transformer text classifiers on unimodal text, trained from initialization in the from\-scratch grid and fine\-tuned in the pretrained controls, on one GPU, one CPU, and two Android phones\. The mechanism statements do not depend on this choice, since width invariance is decided by the position rule and the margin certificate applies to any serving change\. What remains to be measured for in\-context classifiers, multimodal inputs, and production engines is magnitude\. The three model sizes scale depth, width, and head count jointly and describe how the effect moves with capacity\. The GPU\-to\-CPU comparison is the one\-step migration a practitioner performs, with its components separated in Appendix\. Most evaluation examples sit at wide margins\. The five\-class cells confirm the certificate’s prediction that narrower margins convert the same score movement into more label changes \(Table\)\. Appendixstates the boundaries in full\.

Recommendation\.Reproducible classification requires fixing the serving settings we identify, monitoring class probabilities rather than labels, and recomputing inputs on which fixed settings disagree\.

#### AI Use Statement

Generative AI tools were used to assist with manuscript editing, code implementation, and checking mathematical arguments and experiments\. The authors made all scientific decisions, verified the incorporated outputs, and take responsibility for the final content\.

#### Ethics Statement

This work uses only public benchmark datasets \(SST\-2, SST\-5, AG News, Emotion, and IMDb\) and collects no human\-subject data\. The measured instabilities are relevant to deployed classification systems, and the paper provides mitigations rather than mechanisms for exploitation\.

#### Reproducibility Statement

Every experiment is specified by a manifest that records the checkpoint, dataset, random seed, serving settings, and output location, and every run writes to its own output directory, so each reported number can be traced to a stored artifact\. Appendixprovides the proofs, Appendixreports the model, training, and serving settings, and Appendixcontains the complete result tables and figures\.

## References

- Atilet al\.\(2024\)B\. Atil, S\. Aykent, A\. Chittams, L\. Fu, R\. J\. Passonneau, E\. Radcliffe, G\. R\. Rajagopal, A\. Sloan, T\. Tudrej, F\. Ture, Z\. Wu, L\. Xu, and B\. BaldwinNon\-determinism of “deterministic” LLM settings\.External Links:2408\.04667,[Link](https://arxiv.org/abs/2408.04667)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2610.09111#S2.p2.1)\.
- Brecket al\.\(2017\)E\. Breck, S\. Cai, E\. Nielsen, M\. Salib, and D\. SculleyThe ML test score: a rubric for ML production readiness and technical debt reduction\.InIEEE International Conference on Big Data,Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px6.p1.1)\.
- Corani and Zaffalon \(2008\)G\. Corani and M\. ZaffalonLearning reliable classifiers from small or incomplete data sets: the naive credal classifier 2\.Journal of Machine Learning Research9,pp\. 581–621\.Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px4.p1.1)\.
- Dettmerset al\.\(2022\)T\. Dettmers, M\. Lewis, Y\. Belkada, and L\. ZettlemoyerLLM\.int8\(\): 8\-bit matrix multiplication for transformers at scale\.InAdvances in Neural Information Processing Systems,Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px5.p1.1)\.
- Devlinet al\.\(2019\)J\. Devlin, M\. Chang, K\. Lee, and K\. ToutanovaBERT: pre\-training of deep bidirectional transformers for language understanding\.InProceedings of NAACL\-HLT,Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px5.p1.1),[§1](https://arxiv.org/html/2610.09111#S1.p2.1)\.
- Dodgeet al\.\(2020\)J\. Dodge, G\. Ilharco, R\. Schwartz, A\. Farhadi, H\. Hajishirzi, and N\. A\. SmithFine\-tuning pretrained language models: weight initializations, data orders, and early stopping\.arXiv preprint arXiv:2002\.06305\.Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px3.p1.1)\.
- D’Amouret al\.\(2022\)A\. D’Amour, K\. Heller, D\. Moldovan, B\. Adlam, B\. Alipanahi, A\. Beutel, C\. Chen, J\. Deaton, J\. Eisenstein, M\. D\. Hoffman,et al\.Underspecification presents challenges for credibility in modern machine learning\.Journal of Machine Learning Research23\(226\),pp\. 1–61\.Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px3.p1.1)\.
- Efron \(1975\)B\. EfronThe efficiency of logistic regression compared to normal discriminant analysis\.Journal of the American Statistical Association70,pp\. 892–898\.External Links:[Document](https://dx.doi.org/10.1080/01621459.1975.10480319)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.09111#S1.p2.1),[§2](https://arxiv.org/html/2610.09111#S2.p1.1)\.
- Fanget al\.\(2026\)Z\. Fang, Z\. Jiang, H\. Chen, X\. Zhang, T\. Li, K\. Tang, X\. Li, and J\. LiDataset\-level metrics attenuate non\-determinism: a fine\-grained non\-determinism evaluation in diffusion language models\.External Links:2604\.13413,[Link](https://arxiv.org/abs/2604.13413v1)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2610.09111#S2.p2.1)\.
- Fuet al\.\(2026\)T\. Fu, G\. Martínez, J\. Conde, C\. Arriaga, P\. Reviriego, X\. Qi, and S\. LiuBeyond reproducibility: token probabilities expose large language model nondeterminism\.External Links:2601\.06118,[Link](https://arxiv.org/abs/2601.06118)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px2.p1.1)\.
- Goldberg \(1991\)D\. GoldbergWhat every computer scientist should know about floating\-point arithmetic\.ACM Computing Surveys23\(1\),pp\. 5–48\.Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.09111#S1.p1.1),[§2](https://arxiv.org/html/2610.09111#S2.p2.1)\.
- Gondet al\.\(2026\)R\. Gond, A\. K\. Kamath, R\. Ramjee, and A\. PanwarLLM\-42: enabling determinism in LLM inference with verified speculation\.External Links:2601\.17768,[Link](https://arxiv.org/abs/2601.17768)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2610.09111#S2.p2.1)\.
- Guoet al\.\(2017\)C\. Guo, G\. Pleiss, Y\. Sun, and K\. Q\. WeinbergerOn calibration of modern neural networks\.InProceedings of the 34th International Conference on Machine Learning,Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.09111#S1.p2.1)\.
- Havivet al\.\(2022\)A\. Haviv, O\. Ram, O\. Press, P\. Izsak, and O\. LevyTransformer language models without positional encodings still learn positional information\.InFindings of the Association for Computational Linguistics: EMNLP 2022,pp\. 1382–1390\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.findings-emnlp.99)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px5.p1.1)\.
- He \(2025\)H\. HeDefeating nondeterminism in LLM inference\.Note:Thinking Machines Lab technical articleExternal Links:[Link](https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.09111#S1.p1.1),[§2](https://arxiv.org/html/2610.09111#S2.p2.1)\.
- Hsu and Calmon \(2022\)H\. Hsu and F\. CalmonRashomon capacity: a metric for predictive multiplicity in classification\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 28988–29000\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2022/hash/ba4caa85ecdcafbf9102ab8ec384182d-Abstract-Conference.html)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px3.p1.1)\.
- Hüllermeier and Waegeman \(2021\)E\. Hüllermeier and W\. WaegemanAleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods\.Machine Learning110,pp\. 457–506\.Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px4.p1.1)\.
- Kasaet al\.\(2024\)S\. R\. Kasa, A\. Goel, K\. Gupta, S\. Roychowdhury, A\. Bhanushali, N\. P\. Pattisapu, and P\. S\. MurthyExploring ordinality in text classification: a comparative study of explicit and implicit techniques\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 5390–5404\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.320),[Link](https://aclanthology.org/2024.findings-acl.320/)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px1.p1.1)\.
- Kasaet al\.\(2025\)S\. R\. Kasa, K\. Gupta, S\. Roychowdhury, A\. Kumar, Y\. Biruduraju, S\. K\. Kasa, P\. N\. Priyatam, A\. Bhattacharya, S\. Agarwal, and V\. HuddarGenerative or discriminative? revisiting text classification in the era of transformers\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 9604–9626\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.486)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px1.p1.1),[Appendix F](https://arxiv.org/html/2610.09111#A6.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.09111#S1.p2.1),[§2](https://arxiv.org/html/2610.09111#S2.p1.1),[§4](https://arxiv.org/html/2610.09111#S4.p2.1)\.
- Kasaet al\.\(2026\)S\. R\. Kasa, K\. Raavi, N\. P\. Pattisapu, A\. Kumar, Y\. Biruduraju, S\. K\. Kasa, S\. Roychowdhury, Ankith M S, and S\. NegiGenerative vs\. discriminative? revisiting the shortcut learning debate in text classification\.InWorkshop on Combining Theory and Benchmarks: Towards a Virtuous Cycle to Understand and Guarantee Foundation Model Performance at ICML,Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.09111#S1.p2.1),[§2](https://arxiv.org/html/2610.09111#S2.p1.1)\.
- Kazemnejadet al\.\(2023\)A\. Kazemnejad, I\. Padhi, K\. N\. Ramamurthy, P\. Das, and S\. ReddyThe impact of positional encoding on length generalization in transformers\.InAdvances in Neural Information Processing Systems,Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px5.p1.1)\.
- Leeset al\.\(2022\)A\. Lees, V\. Q\. Tran, Y\. Tay, J\. Sorensen, J\. Gupta, D\. Metzler, and L\. VassermanA new generation of perspective API: efficient multilingual character\-level transformers\.InProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,External Links:2202\.11176Cited by:[§1](https://arxiv.org/html/2610.09111#S1.p2.1)\.
- Liet al\.\(2025\)A\. C\. Li, A\. Kumar, and D\. PathakGenerative classifiers avoid shortcut solutions\.InThe Thirteenth International Conference on Learning Representations,Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.09111#S1.p2.1)\.
- Maaset al\.\(2011\)A\. L\. Maas, R\. E\. Daly, P\. T\. Pham, D\. Huang, A\. Y\. Ng, and C\. PottsLearning word vectors for sentiment analysis\.InProceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies,pp\. 142–150\.Cited by:[§1](https://arxiv.org/html/2610.09111#S1.p2.1),[§4](https://arxiv.org/html/2610.09111#S4.p2.1)\.
- Makrooet al\.\(2026\)O\. Makroo, S\. R\. Kasa, S\. Roychowdhury, K\. Gupta, N\. P\. Pattisapu, S\. K\. Kasa, and S\. NegiThe hidden cost of modeling P\(x\): vulnerability to membership inference attacks in generative text classifiers\.Transactions on Machine Learning Research\.External Links:[Link](https://openreview.net/forum?id=SHMC01wdVM),2510\.16122,[Link](https://arxiv.org/abs/2510.16122)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.09111#S1.p2.1),[§2](https://arxiv.org/html/2610.09111#S2.p1.1),[§4](https://arxiv.org/html/2610.09111#S4.p2.1)\.
- Marxet al\.\(2020\)C\. Marx, F\. Calmon, and B\. UstunPredictive multiplicity in classification\.InProceedings of the 37th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.119,pp\. 6765–6774\.External Links:[Link](https://proceedings.mlr.press/v119/marx20a.html)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px3.p1.1)\.
- Micikeviciuset al\.\(2018\)P\. Micikevicius, S\. Narang, J\. Alben, G\. Diamos, E\. Elsen, D\. Garcia, B\. Ginsburg, M\. Houston, O\. Kuchaiev, G\. Venkatesh, and H\. WuMixed precision training\.InInternational Conference on Learning Representations,Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px5.p1.1)\.
- Milani Fardet al\.\(2016\)M\. Milani Fard, Q\. Cormier, K\. Canini, and M\. GuptaLaunch and iterate: reducing prediction churn\.InAdvances in Neural Information Processing Systems,D\. Lee, M\. Sugiyama, U\. Luxburg, I\. Guyon, and R\. Garnett \(Eds\.\),Vol\.29\.Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px3.p1.1)\.
- Ng and Jordan \(2001\)A\. Y\. Ng and M\. I\. JordanOn discriminative vs\. generative classifiers: a comparison of logistic regression and naive bayes\.InAdvances in Neural Information Processing Systems,Vol\.14\.Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.09111#S1.p2.1),[§2](https://arxiv.org/html/2610.09111#S2.p1.1)\.
- Nogueira and Cho \(2019\)R\. Nogueira and K\. ChoPassage re\-ranking with BERT\.arXiv preprint arXiv:1901\.04085\.External Links:1901\.04085Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.09111#S1.p2.1)\.
- Papeet al\.\(2026\)D\. Pape, J\. Evertz, and L\. SchönherrThe silent hyperparameter: quantifying the impact of inference backends on LLM reproducibility\.External Links:2605\.19537,[Link](https://arxiv.org/abs/2605.19537)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.09111#S1.p1.1),[§2](https://arxiv.org/html/2610.09111#S2.p2.1)\.
- Presset al\.\(2022\)O\. Press, N\. A\. Smith, and M\. LewisTrain short, test long: attention with linear biases enables input length extrapolation\.InInternational Conference on Learning Representations,Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px5.p1.1)\.
- Qwen Team \(2025\)Qwen TeamQwen3 technical report\.Technical reportAlibaba Group\.Note:arXiv:2505\.09388Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px5.p1.1),[§5\.2](https://arxiv.org/html/2610.09111#S5.SS2.p2.1)\.
- Radfordet al\.\(2019\)A\. Radford, J\. Wu, R\. Child, D\. Luan, D\. Amodei, and I\. SutskeverLanguage models are unsupervised multitask learners\.Technical reportOpenAI\.Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px5.p1.1)\.
- Rezaeeet al\.\(2021\)M\. Rezaee, K\. Darvish, G\. Y\. Kebe, and F\. FerraroDiscriminative and generative transformer\-based models for situation entity classification\.arXiv preprint arXiv:2109\.07434\.External Links:[Link](https://arxiv.org/abs/2109.07434)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2610.09111#S2.p1.1)\.
- Ribeiroet al\.\(2020\)M\. T\. Ribeiro, T\. Wu, C\. Guestrin, and S\. SinghBeyond accuracy: behavioral testing of NLP models with CheckList\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,pp\. 4902–4912\.Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px6.p1.1)\.
- Sahamiet al\.\(1998\)M\. Sahami, S\. Dumais, D\. Heckerman, and E\. HorvitzA Bayesian approach to filtering junk e\-mail\.InLearning for Text Categorization: Papers from the 1998 AAAI Workshop,Note:AAAI Technical Report WS\-98\-05Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.09111#S1.p2.1)\.
- Saraviaet al\.\(2018\)E\. Saravia, H\. T\. Liu, Y\. Huang, J\. Wu, and Y\. ChenCARER: contextualized affect representations for emotion recognition\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,pp\. 3687–3697\.Cited by:[§1](https://arxiv.org/html/2610.09111#S1.p2.1),[§4](https://arxiv.org/html/2610.09111#S4.p2.1)\.
- Sculleyet al\.\(2015\)D\. Sculley, G\. Holt, D\. Golovin, E\. Davydov, T\. Phillips, D\. Ebner, V\. Chaudhary, M\. Young, J\. Crespo, and D\. DennisonHidden technical debt in machine learning systems\.InAdvances in Neural Information Processing Systems,Vol\.28\.Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px6.p1.1),[§1](https://arxiv.org/html/2610.09111#S1.p2.1)\.
- Shawet al\.\(2018\)P\. Shaw, J\. Uszkoreit, and A\. VaswaniSelf\-attention with relative position representations\.InProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 \(Short Papers\),pp\. 464–468\.External Links:[Document](https://dx.doi.org/10.18653/v1/N18-2074)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px5.p1.1)\.
- Socheret al\.\(2013\)R\. Socher, A\. Perelygin, J\. Wu, J\. Chuang, C\. D\. Manning, A\. Ng, and C\. PottsRecursive deep models for semantic compositionality over a sentiment treebank\.InProceedings of the 2013 Conference on Empirical Methods in Natural Language Processing,pp\. 1631–1642\.Cited by:[§1](https://arxiv.org/html/2610.09111#S1.p2.1),[§4](https://arxiv.org/html/2610.09111#S4.p2.1)\.
- Suet al\.\(2024\)J\. Su, M\. Ahmed, Y\. Lu, S\. Pan, B\. Wen, and Y\. LiuRoFormer: enhanced transformer with rotary position embedding\.Neurocomputing568,pp\. 127063\.External Links:[Document](https://dx.doi.org/10.1016/j.neucom.2023.127063)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px5.p1.1)\.
- Sunet al\.\(2023\)X\. Sun, X\. Li, J\. Li, F\. Wu, S\. Guo, T\. Zhang, and G\. WangText classification via large language models\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 8990–9005\.Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2610.09111#S2.p1.1)\.
- Taoet al\.\(2023\)M\. Tao, Y\. Feng, and D\. ZhaoA frustratingly easy improvement for position embeddings via random padding\.InNatural Language Processing and Chinese Computing,pp\. 301–313\.External Links:[Document](https://dx.doi.org/10.1007/978-3-031-44696-2%5F24)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px5.p1.1)\.
- Walley \(1991\)P\. WalleyStatistical reasoning with imprecise probabilities\.Chapman and Hall,London\.Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px4.p1.1)\.
- Wang and Cho \(2019\)A\. Wang and K\. ChoBERT has a mouth, and it must speak: BERT as a markov random field language model\.InProceedings of the Workshop on Methods for Optimizing and Evaluating Neural Language Generation,pp\. 30–36\.Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2610.09111#S2.p1.1)\.
- Wanget al\.\(2024\)J\. Wang, T\. Ji, Y\. Wu, H\. Yan, T\. Gui, Q\. Zhang, X\. Huang, and X\. WangLength generalization of causal transformers without position encoding\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 14024–14040\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.834)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px5.p1.1)\.
- Warneret al\.\(2025\)B\. Warner, A\. Chaffin, B\. Clavié, O\. Weller, O\. Hallström, S\. Taghadouini, A\. Gallagher, R\. Biswas, F\. Ladhak, T\. Aarsen, G\. T\. Adams, J\. Howard, and I\. PoliSmarter, better, faster, longer: a modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 2526–2547\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.127)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px5.p1.1),[§5\.2](https://arxiv.org/html/2610.09111#S5.SS2.p2.1)\.
- Wulczynet al\.\(2017\)E\. Wulczyn, N\. Thain, and L\. DixonEx machina: personal attacks seen at scale\.InProceedings of the 26th International Conference on World Wide Web,Cited by:[§1](https://arxiv.org/html/2610.09111#S1.p2.1)\.
- Xue and Titterington \(2008\)J\. Xue and D\. M\. TitteringtonComment on “on discriminative vs\. generative classifiers: a comparison of logistic regression and naive bayes”\.Neural Processing Letters28\(3\),pp\. 169–187\.External Links:[Document](https://dx.doi.org/10.1007/s11063-008-9088-7)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2610.09111#S2.p1.1)\.
- Yogatamaet al\.\(2017\)D\. Yogatama, C\. Dyer, W\. Ling, and P\. BlunsomGenerative and discriminative text classification with recurrent neural networks\.arXiv preprint arXiv:1703\.01898\.External Links:[Link](https://arxiv.org/abs/1703.01898)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.09111#S1.p2.1),[§2](https://arxiv.org/html/2610.09111#S2.p1.1)\.
- Yuanet al\.\(2025\)J\. Yuan, H\. Li, X\. Ding, W\. Xie, Y\. Li, W\. Zhao, K\. Wan, J\. Shi, X\. Hu, and Z\. LiuUnderstanding and mitigating numerical sources of nondeterminism in LLM inference\.External Links:2506\.09501,[Link](https://arxiv.org/abs/2506.09501)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.09111#S1.p1.1),[§2](https://arxiv.org/html/2610.09111#S2.p2.1)\.
- Zaffalon \(2002\)M\. ZaffalonThe naive credal classifier\.Journal of Statistical Planning and Inference105\(1\),pp\. 5–21\.External Links:[Document](https://dx.doi.org/10.1016/S0378-3758%2801%2900201-4)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px4.p1.1)\.
- Zhanget al\.\(2015\)X\. Zhang, J\. Zhao, and Y\. LeCunCharacter\-level convolutional networks for text classification\.InAdvances in Neural Information Processing Systems,Vol\.28,pp\. 649–657\.Cited by:[§1](https://arxiv.org/html/2610.09111#S1.p2.1),[§4](https://arxiv.org/html/2610.09111#S4.p2.1)\.
- Zhaoet al\.\(2026\)S\. Zhao, J\. Yang, Q\. Chen, J\. Hu, J\. Zheng, G\. Chen, and X\. ChenCoRun: padding is simple and efficient for deterministic LLM inference\.External Links:2608\.14376,[Link](https://arxiv.org/abs/2608.14376)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.09111#S1.p1.1),[§2](https://arxiv.org/html/2610.09111#S2.p2.1)\.
- Zhenget al\.\(2023\)C\. Zheng, G\. Wu, F\. Bao, Y\. Cao, C\. Li, and J\. ZhuRevisiting discriminative vs\. generative classifiers: theory and implications\.InProceedings of the 40th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.202,pp\. 42420–42477\.External Links:[Link](https://proceedings.mlr.press/v202/zheng23f.html)Cited by:[Appendix D](https://arxiv.org/html/2610.09111#A4.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.09111#S1.p2.1),[§2](https://arxiv.org/html/2610.09111#S2.p1.1)\.

Appendix Contents

## Appendix ANotation

## Appendix BAdditional Results

This appendix collects the measurements that the main results reference, organized as listed in the contents above\.

### B\.1Identical contexts are bitwise repeatable

Every effect reported below is measured against repeated identical runs\. Repeating the identical serving context produces bit\-identical logits on every measured configuration \(Table\)\. Repetition contributes exactly zero, so every nonzero difference reported below is attributable to the declared intervention\. This also separates our subject from run\-to\-run nondeterminism, since the models are deterministic given a serving context but not invariant across serving contexts\.

Table 3:Same\-context repeatability floor\. Identical inputs, identical context, repeated runs; entries are bitwise\-identical comparisons over the total\.
### B\.2Position\-assignment intervention

Figureaccompanies the position\-assignment experiment of Section, in which removing the width\-dependent positions eliminates the effect and deliberately restoring them brings it back\.

Figure 5:Intervening on the position assignment\. Assigning positions by order among real tokens removes the observed padding\-induced class changes; deliberately restoring column positions on models trained with order positions brings the changes back\. Matching the assignment rule between training and inference is separately required to preserve accuracy \(Section\)\.
### B\.3High confidence does not guarantee stability

\(a\)Fraction of predicted classes changed, by initial top\-two logit margin in fixed intervals\[0,0\.01\)\[0,0\.01\),\[0\.01,0\.05\)\[0\.01,0\.05\),\[0\.05,0\.1\)\[0\.05,0\.1\),\[0\.1,0\.25\)\[0\.1,0\.25\), and\[0\.25,∞\)\[0\.25,\\infty\)\(95% intervals clustered by model\)\. Numerical changes concentrate where the two leading scores are nearly tied; padding persists to wider gaps\. Zero\-observed groups sit at the10−510^\{\-5\}floor\.326412825610−410^\{\-4\}10−310^\{\-3\}10−210^\{\-2\}10−110^\{\-1\}10010^\{0\}real\-token length \(width 256\)max abs\. logit differencebf16fp32\(b\)Within\-example length intervention \(IMDB, fully generative GPT\-2, batch size 1 versus 32, 256 targets per point, bitwise floor\)\. bf16 label changes rise 2, 6, 6, 7 of 256 with length; fp32 changes none\.
Figure 6:Two further amplifiers of numerical drift: the decision margin suppresses numerical label changes but not padding \(left\), and a longer real\-token input increases per\-forward drift at fixed padded width \(right\)\.Higher predicted\-class probability is associated with fewer label changes but not with less score movement\. Across the matched sizes of Table, the average probability assigned to the predicted class rises from 0\.554 to 0\.846, while class changes from precision fall from 3\.68% to 0\.568% and from the GPU\-to\-CPU comparison from 3\.30% to 0\.666%; both contrasts exclude zero\. The underlying score movement from precision nevertheless rises by 0\.0118 \(95% CI 0\.0076 to 0\.0160\); across these models, higher confidence coincides with fewer label changes despite larger score movement, a comparison that does not separate confidence from model size or training quality\. Figureshows where the changes occur\. For the two numerical comparisons they concentrate where the two leading scores are nearly tied, whereas padding continues to change classes at score gaps where the numerical comparisons show none\. Where the top\-two logit margin is at least 4, padding still changes 51 of 46,006 binary decisions and 312 of 74,745 multiclass decisions, while precision and the GPU\-to\-CPU comparison change none across 3\.65 million comparisons\. Confidence therefore suppresses label changes from the numerical effects, which perturb the computation of a fixed input, but not the padding effect, which changes the input itself\. These observations do not establish confidence as the cause, and a probability threshold calibrated under one inference setting carries no guarantee under another\.

### B\.4Computational length amplifies numerical drift

A longer target amplifies the numerical mechanism even when nothing structural changes\. Truncating the same 256 IMDB reviews to real\-token lengths 32 to 256 at a fixed padded width of 256 increases drift monotonically with length, and the trend holds within a fixed margin band \(Figure\)\. At bf16 the maximum logit difference grows from 0\.41 to 1\.53 and flips from 2 to 7 of 256, while fp32 stays at the10−410^\{\-4\}scale with zero flips at every length\. A second dataset \(AG News, fixed width, bins of different targets\) reproduces the descriptive contrast between fp32 and bf16 \(2/256, 0/256, and 2/54 flips\)\.

Proposition B\.1:Length\-proportional drift bound for generative scoring\.For classyy, letSy​\(c\)=∑t=1Lℓy,t​\(c\)S\_\{y\}\(c\)=\\sum\_\{t=1\}^\{L\}\\ell\_\{y,t\}\(c\)be the summed token log\-probability score over theLLreal tokens under contextcc, padding excluded\. If every token contribution satisfies\|ℓy,t​\(c\)−ℓy,t​\(c′\)\|≤ε\\lvert\\ell\_\{y,t\}\(c\)\-\\ell\_\{y,t\}\(c^\{\\prime\}\)\\rvert\\leq\\varepsilon, then\|Sy​\(c\)−Sy​\(c′\)\|≤L​ε\\lvert S\_\{y\}\(c\)\-S\_\{y\}\(c^\{\\prime\}\)\\rvert\\leq L\\varepsilonfor each class, and every top\-two score gap changes by at most2​L​ε2L\\varepsilon\(or\(Ly\+Lj\)​ε\(L\_\{y\}\+L\_\{j\}\)\\varepsilonunder class\-dependent lengths\), against2​ε2\\varepsilonfor a classifier that reads a single output position\. If the reference score margin exceeds this bound, the prediction cannot change\. Proofs: Propositionsandin Appendix\.

A classifier that scores every token of the input under each class accumulates a perturbation that grows with the number of tokens it reads, while a classifier that reads a single output position does not\. This is the aggregation mechanism consistent with the measured difference between formulations and with the drift trend in Figure; the bound limits how much drift the aggregation can admit; it does not predict the realized drift or a corresponding change in label stability\.

### B\.5Score movement under padding grows with model size

Tablecompares models of three sizes, restricted to combinations in which all three sizes pass the accuracy threshold\. The fraction of predicted classes changed by padding shows no consistent size trend \(25\.0%, 18\.4%, 26\.4% for small, medium, and large; the large\-minus\-small interval crosses zero\)\. Score movement under unchanged predictions increases with model size\. Among examples whose predicted class does not change, the average total variation distance rises from 0\.0716 to 0\.0880 to 0\.1441, and the largest score change rises from 5\.09 to 10\.06 to 20\.84\. The paired large\-minus\-small differences are 0\.0725 \(95% CI 0\.0039 to 0\.1386\) and 15\.75 \(9\.27 to 23\.41\)\. Larger models therefore do not become more stable under padding\. Their scores move at least as much under equally stable labels\. Because size, width, depth, and training quality change together, these are descriptive associations, not a causal effect of size\.

Table 4:Comparison across model sizes, restricted to models of comparable accuracy\. Padding rows use the 16 GPT\-2 model, dataset, and seed combinations for which all three sizes pass the accuracy threshold; the remaining rows use 40 matched combinations across all four families\. Total variation \(TV\) is computed only over examples whose predicted class does not change\. Paired uncertainty for the principal contrasts is reported in the text\.
### B\.6Batch size moves scores even where predicted classes hold

Class changes from batch size remain rare at every model size, but score movement among examples whose predicted class does not change grows\. Among examples whose predicted class does not change, the average total variation distance rises from2\.26×10−62\.26\\times 10^\{\-6\}in the small models to2\.32×10−42\.32\\times 10^\{\-4\}in the large ones, a paired increase of2\.29×10−42\.29\\times 10^\{\-4\}\(95% CI1\.15×10−41\.15\\times 10^\{\-4\}to3\.71×10−43\.71\\times 10^\{\-4\}\)\. These comparisons pool more than one numerical precision, so they do not measure a pure fp32 trend\. They show that a tolerance on predicted classes and a tolerance on scores are separate requirements\.

### B\.7Matching the position rule between training and inference

When a model trained with column positions is served with order positions, the accuracy cost reaches 9\.88 points, because that model receives position inputs it was never trained on\. When each position rule is instead used consistently in both training and inference, across three seeds and five datasets, the accuracy differences between the two rules are−0\.15\-0\.15and\+0\.32\+0\.32percentage points at the family level, with conservative 95% intervals inside a±2\\pm 2percentage\-point band\. For a learned absolute embedding, order positions under left padding produce the same position representation as right padding, so this consistently trained configuration is the right\-padded baseline\. Order positions additionally allow each batch to be padded only to its own longest sequence rather than to a fixed maximum width\. A pilot on one seed processes 73\.5% fewer padded token slots\. Its latency ratios are measurements of our evaluation setup, not production speedups\.

### B\.8Padding across architectures: further detail

Table 5:Padding\-induced class changes by position mechanism and padding side\. Changes occur only where padding moves real tokens that carry a learned embedding for each absolute position\.Fine\-tuning public pretrained GPT\-2 models reproduces the effect at larger magnitude\. The mean class\-change rate across its 15 dataset and seed combinations is 40\.015% \(95% confidence interval 20\.620 to 53\.790% from a hierarchical bootstrap\), or 11,780 of 29,463 individual evaluations\. All 15 of these models pass the accuracy threshold and predict more than one class, although the SST\-5 models are weak\.

##### Zero disagreement is not sufficient\.

One correction leaves training unchanged and, only at inference, assigns each real token a position index equal to its order among the real tokens, which no longer depends on the padded width\. Under this change the padding effect nearly vanishes \(one changed class in 29,463 evaluations\), but five of the fifteen models begin predicting the same class for every input\. A model with a constant prediction is perfectly stable and uninformative, so its zero disagreement has no evidential weight\. We therefore use this experiment only as further evidence that pretrained GPT\-2 fine\-tunes are vulnerable to padding\. Any proposed correction must pass three checks at once\. The effect must disappear, the predictions must remain accurate and varied, and a deliberately vulnerable control must still show the effect, which confirms that the measurement could have detected it\.

A classifier trained from initialization with no position encoding at all provides the boundary case\. It reaches accuracy 0\.747, its padded width changes on all 256 evaluated examples, and none of its predicted classes change, with residual score movement only at the fp32 numerical scale \(at most7\.6×10−67\.6\\times 10^\{\-6\}in the logits\)\. Removing positional encoding removes the structural mechanism entirely, while the numerical effect of execution shape remains\.

### B\.9Position\-assignment mechanism: further detail

Condition \(i\) sorts the known position\-encoding schemes\. It is satisfied by positions assigned by order among real tokens \(which do not move at all\), and by every scheme whose positional term depends only on the differencej−ij\-i: rotary embeddings, ALiBi’s linear distance bias, T5’s bucketed relative bias, Shaw’s clipped relative representations, and Transformer\-XL relative attention\. A model with no position term satisfies it vacuously\. It is violated by learned absolute embeddings indexed by tensor column, and sinusoidal absolute encodings offer no general end\-to\-end cancellation once the encoding is mixed into content by learned projections\. With correct masking and a correct output position, whether padding changes the represented input is decided by the position rule, before any numerical consideration, and a serving system that wants padding freedom must either use a difference\-based rule or assign positions by order among real tokens\. Of these schemes, we directly measure learned absolute \(affected\), order positions, rotary \(Qwen3 and ModernBERT\), and a native no\-position model \(all unaffected\)\. The ALiBi, T5, Shaw, Transformer\-XL, and sinusoidal rows are literature\-supported mechanism boundaries, not measurements from our experiments\.

The intervention is plotted in Figure\(Appendix\)\. On conventionally trained GPT\-2 models, switching inference to order positions reduces family\-mean class\-change rates from 16\.25% and 17\.37% to zero observed\. We then train models with order positions and repeat the comparison across three seeds, two GPT\-2 families, and five datasets\. Served with the same order positions they were trained with, these models show 0 changed classes in 25,986 comparisons in which the padded width changes\. Deliberately reassigning column positions to the same models restores the effect\. 5,769 of 25,986 classes change, with every family and seed average between 17\.4% and 26\.9%\. This deliberate restoration rules out a false negative\. A broken measurement that made the two padding conditions identical could not reproduce the effect on demand\.

We reproduce the same removal and restoration on CPU\. Under column positions, 23\.4% of predicted classes change on SST\-2 and 7\.8% on AG News when the padded width changes, and under order positions none change on either dataset\. The effect is therefore a property of the position rule, not of one device configuration\.

### B\.10Further detail on the numerical mechanism

The bf16 label changes concentrate at small decision margins \(Figure\)\. The change rate falls monotonically from 30\.6% below margin 0\.01 to 0\.18% at margin 0\.25 or greater, a factor of about 167\. Margin preserves the label under numerical perturbations, but it is not a safety certificate for scores\. The single largest probability movement \(0\.567\) occurs in the widest margin group, where no label changes\. Label count converts drift into flips through the same margin channel \(Table\)\. On five\-class SST\-5 the same bf16 batch intervention flips 6\.2 to 7\.8% of fully generative predictions at roughly half the logit drift of binary SST\-2 \(about 3\.1%\)\. With more classes the probability mass is divided among more alternatives, so the top\-two margin is smaller on average and the same logit movement crosses the decision boundary for a larger fraction of inputs\.

Table 6:Label changes and logit drift for the fully generative GPT\-2 classifier under the same bf16 batch\-shape change, by number of classes \(64 targets per cell\)\.A prediction can only change if its confidence gap is at most twice the probability movement the serving change produces\. Label changes are therefore confined to examples whose top\-two margin is within that bound, which for the small numerical perturbations measured here means nearly tied predictions, and a wide margin preserves the label without limiting how far the scores may move\. All 375 observed bf16 batch\-induced label changes satisfied the necessary condition \(recall 100%, precision 56\.5%\), whereas a margin threshold alone is a weak predictor \(95% recall requires flagging every margin below 0\.46\)\. The certificate is one\-sided\. It bounds labels, never scores\.

### B\.11Quantization lets other examples in the batch change a prediction

We call the one example whose prediction we track the target\. Under floating point, the other examples in its batch cannot reach the target’s prediction\. At fixed batch shape, their content leaves the target’s output unchanged bit\-for\-bit \(1,080 GPU composition pairs and all CPU cells exactly zero\), because no operation mixes one example’s rows into another’s and the kernel is chosen by tensor shape, not by the values in it\. Dynamic int8 quantization breaks this, because the scale that converts activations to integers is computed from the whole batch tensor, so the values of the other examples in the batch enter the target’s own computation \(Table\)\. Replacing the runtime scale with a fixed offline\-calibrated scale restores bitwise equality, and a fake\-quantization control that varies only the scale source reproduces the effect, isolating the mechanism\.

Table 7:Effect of changing the other examples in the target’s batch at fixed batch shape \(CPU, static width 128, 64 targets, SST\-2 and AG News, threads 1 and 32\)\. Only the batch\-global dynamic activation scale lets those other examples reach the target\.Proposition B\.2:Batch\-global scales couple co\-batched inputs; fixed scales do not\.LetQs​\(a\)=s​clip⁡\(round⁡\(a/s\),−M,M\)Q\_\{s\}\(a\)=s\\,\\operatorname\{clip\}\(\\operatorname\{round\}\(a/s\),\-M,M\)be the dequantized activation quantizer with scaless\. \(i\) If the scale is computed from the whole batch,s=s⁡\(B\)s=s\(B\), and two same\-shape batchesBB,B′B^\{\\prime\}with the same target activationaabut different other examples satisfys⁡\(B\)≠s⁡\(B′\)s\(B\)\\neq s\(B^\{\\prime\}\)andQs⁡\(B\)​\(a\)≠Qs⁡\(B′\)​\(a\)Q\_\{s\(B\)\}\(a\)\\neq Q\_\{s\(B^\{\\prime\}\)\}\(a\), then the target’s quantized activation depends on the other examples in the batch, and its logits differ whenever the downstream map separates the two activations\. \(ii\) If instead the scale is fixed before serving and does not depend on the other examples, and the target’s execution otherwise does not depend on them and is deterministic at fixed shape, then the other examples cannot alter the target’s output\. Proofs: Propositionsandin Appendix\.

Under a batch\-global dynamic scale, the content of the requests that share a batch sets the rounding grid applied to the target, and fixing that grid offline removes the dependence\. The result does not extend to dynamic quantizers whose scale depends only on the target request\. Both directions match the measurements\. Dynamic scales moved the target by up to 0\.774 logits with label changes, and fixed scales were bitwise identical under the identical intervention\.

Under dynamic int8 quantization on CPU, changing only the co\-batched examples at a fixed batch shape moves GPT\-2 labels, and the effect grows with the batch\. The GPT\-2 change rate rises from 8 of 384 comparisons \(2\.08%\) at batch size 8 to 10 of 384 \(2\.60%\) at batch size 64 and 14 of 384 \(3\.65%\) at batch size 256, with the largest absolute logit movement of 1\.32 occurring at batch size 256\. Each co\-batched example contributes to the batch\-global activation scale, so a larger batch gives the other examples more ways to change the quantization step applied to the target\. BERT shows almost no movement under the same change, with at most 1 change out of 192 comparisons in any single cell\.

At the reference request context of batch size 1 with static padding there is no batching and no other example in the batch, so switching the int8 pipeline from static to dynamic activation scales changes only the scale computation\. Under this change GPT\-2 labels change on 4 of 192 targets \(2\.08%\), all four in the small checkpoint \(4 of 64\), with a maximum absolute logit movement of 0\.839, while BERT changes on 0 of 192 targets \(Figure\)\. Because no other example is present, none of this movement can come from batch composition\. It measures the scale policy itself, separately from the coupling above\.

### B\.12The target slot and the matrix\-multiplication reduction order

Moving the target to a different row at a fixed batch shape leaves the output unchanged in exact arithmetic\. A linear layer computes each output coordinate as a sum over the hidden dimension,yn=∑kak​Wk,ny\_\{n\}=\\sum\_\{k\}a\_\{k\}W\_\{k,n\}, so the target row depends only on its own activations\. A batched matrix multiplication folds the batch and the sequence into a single dimensionMMof sizebatch×sequence\\text\{batch\}\\times\\text\{sequence\}and tiles theM×NM\\times Noutput into blocks, partitioning the reduction over the hidden dimension across those blocks\. Which tile a row occupies, its position inside that tile, and the partition boundary it falls on are all functions of the row’s offset inMM, so moving the target to a different slot changes the order in which its partial sums are accumulated\. Floating\-point addition is not associative, so the rounded logits differ by about10−510^\{\-5\}for the fully generative GPT\-2 classifier and about10−610^\{\-6\}for BERT, and the gap grows mildly with batch size as a largerMMadmits more tiles\. The integer CPU path uses one fixed reduction order for a given problem shape regardless of the offset, so changing the target slot leaves the output bitwise identical there\. An operator census of the forward pass finds the linear\-layer matrix multiplication is the only computation whose reduction order depends on this offset\. Attention, normalization, and the softmax are computed per token and do not\. The effect never crosses a decision boundary, and the batch\-invariant matrix\-multiplication kernels of Section, which fix the reduction order independently ofMM, remove it\.

### B\.13Autoregressive feedback after the first token disagreement

The batch\-shape perturbation above never flips a high\-margin classification label, yet the generation literature reports large output changes from the same root cause\. The cause is autoregressive feedback, not the per\-step perturbation\. Under greedy decoding with full recomputation and positions assigned by order among the real tokens, with batch size 1 versus 32 as the only difference between contexts, bf16 flips a greedy token within 4 to 14 steps in 3 of 3 margin\-selected prompts, after which the trajectories separate \(91 to 119 of 128 tokens differ\)\. By contrast, fp32 diverges on none of 3 within 128 steps \(Table\)\.

Table 8:Accumulation experiment: greedy decoding under two batch shapes \(GPT\-2, 128 steps, 12 of 12 same\-context floor checks bitwise\)\.One classification forward reads a single, usually high\-margin argmax and never feeds its prediction back into the model\. Greedy generation conditions every later step on the decoded prefix, so the first low\-margin step at which the two contexts select different tokens changes all subsequent inputs\. Initial margin did not order the divergence step, so margin at the prompt is not a schedule for when a generation diverges\.

### B\.14Mitigation costs and backend comparisons

Each measured mechanism admits a mitigation aimed at exactly that mechanism \(Table\)\. None of these mitigations covers the other mechanisms\. Order\-based position assignment, which does not depend on the padded width, eliminates the structural mechanism and simultaneously permits dynamic padding, which processed0\.265×0\.265\\timesas many padded tokens for2\.04×2\.04\\timesto2\.51×2\.51\\timesmeasured target throughput in our evaluation setup\. Batch\-invariant matrix\-multiplication kernels drive batch\-shape drift to bitwise equality \(64 of 64 targets\) at fp32 and bf16 for both formulations, removing the two bf16 label flips\. Their precision\-matched end\-to\-end cost is roughly neutral at batch size 1 \(a dispatch\-bound eager\-mode effect\) and about1\.3×1\.3\\timesto1\.5×1\.5\\timesat large batch in true fp32, up to1\.45×1\.45\\timesTF32\-matched and1\.40×1\.40\\timesat bf16 \(Figure\)\. They cover the linear\-layer GEMMs plus mean and log\-softmax, and adopting them changes the computed values\. Moreover, the released library computes fp32 matrix multiplications at reduced tensor\-core precision, so a comparison against a full\-precision standard baseline is not matched in precision unless the kernel is patched\. Standard versus invariant at a fixed batch size is never bitwise and moves up to 0\.219 logits with 2 of 64 flips at bf16 on the generative head\. Static calibration removes this dependence between co\-batched inputs \(Table\)\. For residual numerical flips, a verifier that recomputes only the small\-margin predictions in a canonical context detects 95\.2, 99\.2, and 100% of the 375 observed flips at 7\.1, 10\.8, and 13\.8% recompute rates when thresholds are fitted on the same comparisons; with thresholds fixed on a disjoint calibration split, the label policy recomputes 22\.45% of held\-out requests and detects all 44 observed disagreements, while a fixed 0\.8 threshold policy leaves 3 of 21 action disagreements undetected \(Section\)\. Backend changes are label\-stable but not score\-stable in our cells\. Eager versus torch\.compile moves logits by up to3\.1×10−53\.1\\times 10^\{\-5\}with 0 of 64 changes, PyTorch versus ONNX Runtime by up to8\.4×10−78\.4\\times 10^\{\-7\}with 0 of 64, CUDA Graph replay is bitwise at each captured shape while distinct shapes retain ordinary batch drift, and semantic\-position\-preserving width buckets remove the structural mechanism but keep numerical shape dependence \(bf16 generative max TV 0\.044, 0 of 64 flips\)\.

Figure 7:Latency of batch\-invariant matrix\-multiplication kernels, as the ratio of the batch\-invariant to the standard whole\-forward median, by batch size, for four classifier formulations\. A ratio below one is faster\. Batch invariance is roughly neutral at batch size 1 \(a dispatch\-bound effect under eager execution\) and costs about 1\.3 to 1\.5 times at large batch under a precision\-matched comparison \(true fp32, TF32\-matched, or bf16\); the default fp32 comparison, in which the invariant kernel runs at reduced tensor\-core precision against a full\-precision baseline, is the only one that appears faster at large batch\.
### B\.15Request\-level factors by deployment

The main\-text figures summarize the request\-level factors on the checkpoints that pass the accuracy threshold for which every request\-level cell completed\. Figurecompares batch sizes and padded widths for each of the three deployments, and Figuregives the within\-batch rearrangements \(target slot, order of the other examples, and which examples share the batch\) for each deployment\. The dynamic\-padding versions of the within\-batch figures are identical by construction, because the padded width is held fixed under these rearrangements, and are omitted\.

\(a\)full precision, GPU\(b\)bf16, GPU\(c\)8\-bit integer, CPU
Figure 8:Batch size \(against the single\-example reference\) and padded width \(static to dynamic\) by deployment; batch size is shown as bar shade\. Rows are label\-change rate, probability total variation, and maximum logit change; BERT solid, fully generative GPT\-2 hatched\.\(a\)full precision, GPU\(b\)bf16, GPU\(c\)8\-bit integer, CPU
Figure 9:Within\-batch rearrangements at a fixed batch shape \(target slot, order of the other examples, composition of the other examples\) by deployment; batch size shown as bar shade\. Reordering is bitwise at every precision; composition is bitwise at full precision and bf16 and moves only under dynamic 8\-bit integer quantization; the target slot is bitwise on the integer CPU path but shows a small nonzero score movement at full precision and bf16 \(a matrix\-tiling effect, with no label change\)\.
### B\.16Serving\-context effects by model size

Tablereports the size trends numerically\. Figuresandgive the same comparisons split by model size \(shown as bar shade\), restricted to combinations in which all three sizes pass the accuracy threshold\. The score movement under unchanged labels grows with size while the label\-change rate does not\. The plots describe an association with nominal model size, in which depth, width, head count, and training quality vary together\.

\(a\)full precision, GPU\(b\)bf16, GPU\(c\)8\-bit integer, CPU
Figure 10:Batch size and padded width by deployment and model size \(small, medium, large shown as bar shade\), pooled over batch size\.\(a\)full precision, GPU\(b\)bf16, GPU\(c\)8\-bit integer, CPU
Figure 11:Within\-batch rearrangements by deployment and model size \(small, medium, large shown as bar shade\), pooled over batch size\.
### B\.17The padding effect shrinks as the batch grows

Dynamic padding pads each batch to its longest member, while static padding pads every batch to the fixed cap\. As the batch grows, the longest member approaches the cap, so the two policies produce increasingly similar padded widths and the disagreement between them shrinks \(Table\)\.

Table 9:GPT\-2 label changes from dynamic versus static padding, by numerical format and batch size\. BERT shows zero changes at every GPU cell and at most 1 of 192 at a single int8 cell\.
### B\.18Precision comparison including fp16

fp16 preserved every fp32 label in the tested cells \(0 of 192 for BERT and 0 of 192 for GPT\-2\), while fp32 to bf16 and bf16 to fp16 each moved 2 of 192 GPT\-2 labels \(1\.04%\), so on these checkpoints the roughly one percent label movement is associated with bf16 rather than with reduced precision as such \(Table\)\. Formats can also select different kernels, so the result does not establish general fp16 label invariance\. The two formats allocate the same 16 bits differently, since bf16 carries an 8\-bit exponent with a 7\-bit mantissa and fp16 a 5\-bit exponent with a 10\-bit mantissa, and the maximum logit movement of the bf16 to fp16 step \(0\.570\) matches that of fp32 to bf16 \(0\.564 at GPT\-2 large\), consistent with bf16 being the format that both fp32 and fp16 diverge from\. All moved labels fall in the lowest baseline\-margin bin, and BERT remains label\-stable on every step while still showing nonzero score drift\.

Table 10:Precision pairs on the matched six\-checkpoint set, 192 comparisons per family \(three model sizes, 64 targets each\)\. Each cell gives label changes over the denominator and the maximum absolute logit difference\.
### B\.19Separating the device from the quantization

Changing only the device moved logits by at most2\.86×10−62\.86\\times 10^\{\-6\}for BERT and1\.80×10−31\.80\\times 10^\{\-3\}for GPT\-2 with zero label changes in 128 comparisons per family, orders of magnitude below the quantization arithmetic \(Table\)\. Applying 8\-bit quantization with the device held fixed reproduced the movement on either side, with 1 of 192 changes per family when the device was held at the GPU and 1 of 128 GPT\-2 changes under dynamic int8 when held at the CPU, every change falling in the lowest baseline\-margin bin\. On these subsets, changing the device alone produced far smaller movement than applying quantization, which supports quantization as the main source of the whole\-migration effect in Section\. The controls use different checkpoint subsets from the main grid and do not decompose that full\-grid number\. The conclusion is bounded to the tested pair, one L40S GPU and one AMD EPYC host under one PyTorch build\.

Table 11:Decomposing the whole\-migration change \(GPU fp32 to CPU dynamic int8\) into a device factor and a quantization factor\. The GPU\-held row uses a six\-checkpoint set \(192 comparisons per family\); the CPU\-held rows use a separate medium\-size checkpoint per family on SST\-2 and AG News \(128 per family\), so the rows are not pooled\. The whole\-migration rate is measured on the main grid with different checkpoints\.
### B\.20Physical mobile devices

All phone runs used ONNX Runtime Mobile \(ONNX Runtime 1\.23\.2 for Android\) with fp32, dynamic int8, and static int8 exports of the same BERT SST\-2 classifier, evaluated on 64 balanced targets at batch size 8 and width 128 \(Table\)\. Sessions with the NNAPI accelerator enabled achieved partial delegation only on the Galaxy S25, and only for the fp32 and dynamic int8 graphs\. The Pixel 8a NNAPI sessions either failed to create or assigned every node to the CPU provider, so these are partial placements rather than accelerator\-only execution\. No supported comparison changed a label \(0 of 64 in every cell\)\. Under dynamic int8, changing the seven co\-batched texts moved all 64 of 64 probability vectors, with maximum total variation 0\.0048 on the phone CPUs and 0\.0057 under the partial Galaxy NNAPI placement, where it also produced one threshold\-action change and moved four of 64 targets by one rank\. This matches the server\-side dynamic int8 composition result, and the static int8 export was bitwise identical under the same composition change on both phone CPUs\. The target slot and the order of the other examples were bitwise identical for every format on every supported placement\.

Table 12:Serving\-context factors on physical phones\. Entries give the maximum total variation of the target probability vector, or “bitwise” when all 64 of 64 targets were bitwise identical, for CPU execution\. Cells marked n\.m\. were not measured because the int8 graphs were exported at a fixed batch size of 8\.
### B\.21Downstream decisions move more than labels

Tablereports, for three serving interventions, the fraction of evaluation\-example pairs whose predicted label, fitted per\-class threshold action, abstention decision at 90 percent target coverage, 90 percent prediction\-set membership, or top 1 percent ranking membership changed\. Every policy was fitted on reference\-context outputs only, then frozen before any evaluation metric was produced, so no threshold was refitted to a serving context\. In the tested configurations the threshold, abstention, prediction\-set, and ranking quantities change more often than the predicted label under padding and precision changes, and the batch\-size comparison changes zero labels among 799,620 pairs while still moving threshold actions, prediction\-set membership, and ranking membership at small nonzero rates\. These are rates over a balanced experimental grid, not production traffic prevalence\.

Table 13:Fraction of evaluation\-example pairs \(percent\) with a changed decision under each intervention\. Abstention and prediction sets use 90 percent target coverage\. Ranking churn is the fraction of the top 1 percent set whose membership differs,\|S​△​S′\|/2​k\|S\\triangle S^\{\\prime\}\|/2kfor selected sets of sizekkper class, not a fraction of all examples\.
### B\.22Recomputing low\-margin predictions in a canonical context

A deployable mitigation recomputes the least confident fast\-path predictions in a declared canonical context and releases the canonical result for those predictions\. The fast path here is bf16 execution at batch size 32, and the canonical context is fp32 execution at batch size one with static width, for the GPT\-2 medium SST\-2 checkpoint\. Each policy ranks predictions by a fast\-path risk score, the top\-two probability margin for the label policy and the distance of the positive\-class probability from the declared 0\.8 action threshold for the fixed\-threshold policy\. Cutoffs were fitted on a calibration split of 547 targets that was fixed before evaluation on the 1,274 held\-out targets; because the calibration split contained fewer disagreements than the predeclared minimum, the fitted operating points are underpowered\. The label\-margin policy recomputed 22\.45% of requests and detected all 44 observed label disagreements, while the fixed\-threshold policy recomputed 4\.00% and detected 18 of 21 threshold\-action disagreements \(Table\)\. The two policies cover different decisions, and neither covers complete probability vectors\.

Table 14:Held\-out replay results for GPT\-2 medium on SST\-2 at fast batch size 32 \(1,274 held\-out targets\)\.
### B\.23Output\-position faults that mimic padding instability

Some apparent padding instability is an implementation fault in the classification head rather than an effect of the position rule\. Under valid fixed\-shape repacking of batch neighbors, every rule that locates the correct semantic token or pools over exactly the packed set of real tokens changed 0 of 64 target labels for both tested checkpoints, and the BERT semantic\-token gathers were bitwise identical for 64 of 64 targets\. Rules that instead read a fixed physical column or include padding positions in the pooled set changed labels at fixed batch shape, from 3 of 64 for the BERT fixed\-last\-column rule up to 64 of 64 for the GPT\-2 fixed\-first\-column rule under end\-to\-end left versus right placement, with the GPT\-2 pad\-inclusive mean and sum each changing 28 of 64\. These faults are distinct from the structural mechanism in which real\-token positions depend on the padded width, and the two must be fixed separately, since correcting the positions does not correct an unsafe output position and correcting the output position does not remove width\-dependent positions\. The practical rule is to derive the output position from the attention mask and to exclude padding positions from pooling\.

### B\.24Flip\-rate distribution across the main grid

This table ranks the serving\-context comparisons by how often they change the predicted label, from the input\-side padding effect down to the within\-batch rearrangements that change no observed labels\.

Table 15:Serving\-context instability among the 158 from\-scratch checkpoints that exceed 1\.15 times chance validation accuracy\. Means average designed pairs rather than production traffic\. P90 is the 90th percentile of pair\-level flip rates\.
### B\.25Scope of the order and composition comparisons

The order comparison permutes the full test list before fixed\-width batching\. It preserves target IDs and tensor width but can change peers and target slot, so we do not interpret it as a pure within\-batch permutation effect\. The composition comparison holds the padded width fixed and changes which other examples share the batch\. It is the clean test of whether their content reaches the target\. A separate random\-peer width experiment allows peer lengths to change dynamic width and is therefore a joint peer/shape intervention\.

## Appendix CImplementation Details

##### Primary models and tokenization\.

Small, medium, and large models use hidden sizes 64, 384, and 768; depths 1, 6, and 12 layers; 1, 6, and 12 attention heads; and feed\-forward widths 256, 1,536, and 3,072, respectively, with maximum sequence length 512\. BERT families use thebert\-base\-uncasedWordPiece vocabulary and right padding\. GPT\-2 families use thegpt2byte\-pair vocabulary, reuse EOS as PAD, and are manually left padded\. The generative formulations add one generation marker and one synthetic token per class\.

##### Objectives and scores\.

The direct BERT model uses sequence\-classification cross\-entropy\. The masked\-label model applies standard masked\-language\-model cross\-entropy, randomly masks 15% of eligible tokens, and always masks the synthetic label positions\. The next\-label GPT\-2 model trains ontext \[label\_gen\] \[label\_y\]and classifies from the next\-token scores after the generation marker\. The class\-conditional GPT\-2 model trains on\[label\_gen\] \[label\_y\] textand scores each candidate class by summing non\-padding token log probabilities\. At inference, every formulation’s class scores are converted to fp32 and softmax\-normalized over only the candidate classes\. The reported generative probabilities are therefore class\-normalized verbalizer or sequence\-score probabilities, not full\-vocabulary probabilities\.

##### Primary optimization\.

We use AdamW with learning rate2×10−52\\times 10^\{\-5\}, zero warm\-up, linear decay, batch size 32, at most 60 epochs, and early\-stopping patience 10\. Training and checkpoint selection use validation weighted F1\. The best of the retained checkpoints is evaluated\. SST\-2, SST\-5, and Emotion use supplied validation splits\. AG News and IMDb use a stratified 20% training carve\-out with split seed 42\. Runs use fp32 and seeds 40 to 42 under the strict deterministic profile\.

##### Modern pretrained models\.

ModernBERT\-base and Qwen3\-0\.6B use LoRA rank 16, alpha 32, dropout 0\.05, and all linear modules\. We optimize trainable parameters with AdamW at2×10−42\\times 10^\{\-4\}for at most 10 epochs, with zero warm\-up, linear decay, and patience 3\. Adapters are merged before inference\. ModernBERT uses sequence\-classification cross\-entropy\. Qwen uses left\-padded raw text and class cross\-entropy over final\-position logits for single\-token digit labels\.

## Appendix DExtended Related Work

This appendix expands the related work summarized in the main text\. It groups prior work into six areas and, for each, states what that work establishes and how it differs from the question studied here, namely how the prediction of a single fixed trained classifier changes when only the inference\-time serving context varies while the model checkpoint, the input text, and the tokenizer are held fixed\.

##### Discriminative, generative, and pseudo\-generative text classifiers\.

A text classifier can be built in several ways that differ in what probability the model represents\. A discriminative classifier estimates the conditional probability of the label given the input text directly, for example logistic regression or a neural encoder with a classification head\. A generative classifier instead models the distribution of the input, either the joint distribution of text and label or the class\-conditional distribution of the text, and turns this into a label decision through Bayes’ rule\. The comparison between these two families has a long history, beginning with analyses of logistic regression, discriminant analysis, and naive Bayes, and of the conditions under which one family is more sample efficient or more accurate than the other\([Efron, 1975](https://arxiv.org/html/2610.09111#bib.bib35),[Ng and Jordan, 2001](https://arxiv.org/html/2610.09111#bib.bib36),[Xue and Titterington, 2008](https://arxiv.org/html/2610.09111#bib.bib37)\)\. Naive Bayes text classification, in which the class\-conditional distribution of words is modeled under an independence assumption, is an early and widely used generative approach for text\([Sahami et al\., 1998](https://arxiv.org/html/2610.09111#bib.bib53)\)\. Neural formulations extended the comparison\. This work includes recurrent generative and discriminative text classifiers\([Yogatama et al\., 2017](https://arxiv.org/html/2610.09111#bib.bib38)\), transformer\-based discriminative and generative models for sentence\-level classification\([Rezaee et al\., 2021](https://arxiv.org/html/2610.09111#bib.bib40),[Nogueira and Cho, 2019](https://arxiv.org/html/2610.09111#bib.bib54)\), and a theoretical account of when each family generalizes better\([Zheng et al\., 2023](https://arxiv.org/html/2610.09111#bib.bib41)\)\. A third formulation has become common with pretrained language models\. A masked\-language\-model classifier scores a label by how well a language model, trained to predict tokens that have been hidden, reconstructs a label\-conditioned version of the input\. Because the score is a product of conditional token probabilities rather than a properly normalized joint density over the whole sequence, it approximates a likelihood through pseudo\-likelihood\([Wang and Cho, 2019](https://arxiv.org/html/2610.09111#bib.bib39)\)\. We call this formulation pseudo\-generative to separate it from a classifier that models a normalized joint distribution\. Recent work uses autoregressive language models directly for classification\([Sun et al\., 2023](https://arxiv.org/html/2610.09111#bib.bib42)\)and compares encoder, autoregressive, masked\-language\-model, and discrete\-diffusion classifiers within one transformer\-era study\([Kasa et al\., 2025](https://arxiv.org/html/2610.09111#bib.bib12)\)\. Related work examines other design choices in text classification, such as the treatment of ordinal label structure\([Kasa et al\., 2024](https://arxiv.org/html/2610.09111#bib.bib50)\)\. Further studies characterize properties that separate the families\. Fully generative classifiers are not automatically resistant to shortcut features that correlate with the label in training but not in deployment\([Li et al\., 2025](https://arxiv.org/html/2610.09111#bib.bib43),[Kasa et al\., 2026](https://arxiv.org/html/2610.09111#bib.bib44)\), and modeling the input distribution can raise the risk that an attacker infers whether a specific example was in the training set\([Makroo et al\., 2026](https://arxiv.org/html/2610.09111#bib.bib45)\)\. These studies compare training objectives, model formulations, sample efficiency, robustness, calibration, and privacy across classifier families\. None of them holds one fixed trained checkpoint and one fixed input text and varies only the serving context, such as padding, batch composition, numerical precision, quantization, device, or inference backend, to measure whether the prediction of that fixed model changes\.

##### Non\-determinism and non\-invariance in language\-model inference\.

A separate line of work documents that the output of a language model can change when the way the computation is executed changes, even though the model parameters, the prompt, the decoding parameters, and any sampling seed are held fixed\. The root numerical cause is that floating\-point addition is not associative\. Adding the same set of numbers in a different order can yield a slightly different rounded result, because each intermediate sum is rounded to a finite number of bits\([Goldberg, 1991](https://arxiv.org/html/2610.09111#bib.bib8)\)\. This matters in practice because a serving system computes sums, such as the dot products inside a matrix multiplication or a reduction across the sequence, by splitting them across parallel hardware threads, and it selects which low\-level routine \(kernel\) to run and how to partition the reduction as a function of the shape of the input batch\. When the batch shape changes, for instance because more or fewer requests are processed together, the partitioning and therefore the accumulation order can change, and the rounded result moves\([Yuan et al\., 2025](https://arxiv.org/html/2610.09111#bib.bib34),[He, 2025](https://arxiv.org/html/2610.09111#bib.bib46),[Zhao et al\., 2026](https://arxiv.org/html/2610.09111#bib.bib31)\)\. The choice of inference engine and library version is a further source of divergence for the same model and inputs\([Pape et al\., 2026](https://arxiv.org/html/2610.09111#bib.bib32)\)\. At the level of hosted services, repeated requests under nominally deterministic settings change the generated output and the benchmark accuracy computed from it, without identifying which serving factor moved\([Atil et al\., 2024](https://arxiv.org/html/2610.09111#bib.bib55)\)\. That measurement is generation\-mediated, because the task answer is parsed out of generated text\. Three responses to this have appeared\. First, kernels can be written so that the reduction order does not depend on the batch shape, which restores identical results across batch sizes at a fixed execution configuration\([He, 2025](https://arxiv.org/html/2610.09111#bib.bib46)\)\. Second, requests can be padded so that each is processed at a fixed tensor shape, keeping a request’s arithmetic independent of which other requests share its batch\([Zhao et al\., 2026](https://arxiv.org/html/2610.09111#bib.bib31)\)\. Third, the default kernels can be kept and every decoded token verified against a deterministic recomputation, with disagreements repaired\([Gond et al\., 2026](https://arxiv.org/html/2610.09111#bib.bib56)\)\. The margin\-based recomputation we measure in Sectionis the classification counterpart of this response, applied once per input rather than once per decoded token\. The invariance that this padding\-for\-determinism work restores concerns a request that occupies a different row of the batch while the tensor shape is held fixed\. It is distinct from the change studied in this paper, in which a real token occupies a different absolute position within its own sequence because the padded width itself changed\. Related measurement work shows that reporting only aggregate, dataset\-level metrics hides the per\-example movement these effects cause\([Fang et al\., 2026](https://arxiv.org/html/2610.09111#bib.bib49)\), and that the token probabilities of a language model reveal non\-determinism that a discrete top\-one output conceals\([Fu et al\., 2026](https://arxiv.org/html/2610.09111#bib.bib33)\)\. All of this work studies language\-model generation, autoregressive or diffusion\-based, in which the output is a decoded sequence and a single perturbed step can propagate through the remaining decoding\. None of it isolates the single forward pass of a fixed text classifier, holding the checkpoint and the input text fixed, to determine whether and by how much these execution\-level mechanisms change a classification decision rather than a generated sequence\.

##### Prediction instability, churn, and underspecification\.

A different body of work studies how the predictions of a classifier vary across models rather than across executions of one model\. Fine\-tuning a pretrained model is itself unstable\. The final model depends on the random initialization of the added parameters and on the order in which training examples are presented, so the same recipe run twice can yield different models\([Dodge et al\., 2020](https://arxiv.org/html/2610.09111#bib.bib5)\)\. Prediction churn names the fraction of examples whose predicted label differs between two successively trained models, and procedures exist to reduce it during retraining\([Milani Fard et al\., 2016](https://arxiv.org/html/2610.09111#bib.bib7)\)\. Underspecification describes the situation in which a training pipeline admits many distinct solutions that reach nearly the same held\-out accuracy yet behave differently on axes that matter in deployment, so aggregate accuracy does not single out one model\([D’Amour et al\., 2022](https://arxiv.org/html/2610.09111#bib.bib6)\)\. Predictive multiplicity makes the same point at the level of individual inputs\. Several models with near\-equal accuracy can assign conflicting predictions to the same example, and this multiplicity can be quantified over the set of near\-optimal models\([Marx et al\., 2020](https://arxiv.org/html/2610.09111#bib.bib47),[Hsu and Calmon, 2022](https://arxiv.org/html/2610.09111#bib.bib48)\)\. In all of these settings the source of variation is a change of model, whether across independent training runs or across the set of models a pipeline could have produced\. This paper instead holds a single trained model fixed and varies only the inference\-time serving context, so the disagreement it measures is not attributable to a different choice of parameters\.

##### Imprecise probability, credal classification, and calibration\.

Some formal frameworks represent uncertainty with more than a single probability distribution\. Imprecise probability describes partial or unresolved knowledge using sets of distributions together with lower and upper probabilities, rather than one sharp distribution\([Walley, 1991](https://arxiv.org/html/2610.09111#bib.bib28)\)\. A credal classifier applies this idea to prediction and may return a set of admissible labels, rather than forcing a single label, when the available evidence does not determine one\. The naive credal classifier and its later refinement are the canonical examples for categorical data\([Zaffalon, 2002](https://arxiv.org/html/2610.09111#bib.bib29),[Corani and Zaffalon, 2008](https://arxiv.org/html/2610.09111#bib.bib2)\)\. The set of predictions studied in this paper is related in outward form to a set\-valued prediction but different in origin and in intended use\. It is the finite collection of outputs that one fixed model actually produces across the serving configurations that are tested, with no assumed convexity of a probability set, no generating prior, and no belief\-function semantics\. Taking the union of the predicted labels across configurations is one possible set\-valued reading of that collection, not a decision rule imported from the credal literature\. Calibration is a further, separate notion\. It measures whether the confidence a model reports matches the empirical frequency with which such predictions are correct, and modern neural networks are often miscalibrated toward overconfidence\([Guo et al\., 2017](https://arxiv.org/html/2610.09111#bib.bib9)\)\. The distinction between uncertainty that is irreducible given the data and uncertainty that reflects limited knowledge and could in principle be reduced has been formalized as aleatoric versus epistemic uncertainty\([Hüllermeier and Waegeman, 2021](https://arxiv.org/html/2610.09111#bib.bib10)\)\. This paper does not estimate calibration error and does not decompose uncertainty into these components\. It measures how much a fixed model’s output depends on hidden serving configuration\.

##### Numerical and positional mechanisms\.

Two implementation choices are central to how serving context can change a prediction\. The first is the numerical format used to store values and to carry out arithmetic\. Mixed\-precision execution keeps some values in a reduced\-precision floating\-point format, which uses fewer mantissa bits than the standard single\-precision format and therefore represents each number more coarsely, while accumulating selected sums in higher precision to limit error\([Micikevicius et al\., 2018](https://arxiv.org/html/2610.09111#bib.bib13)\)\. Quantization is a distinct technique that maps floating\-point values to low\-bit integers through a scale factor and then performs integer arithmetic, for example 8\-bit integer matrix multiplication for transformer models\([Dettmers et al\., 2022](https://arxiv.org/html/2610.09111#bib.bib3)\)\. The term quantization is reserved here for such integer methods and is not used for reduced\-precision floating point\. Both change how the same represented input is computed numerically, and this paper measures their effect on the individual predictions of a fixed model\. The second choice is how position information is supplied to the model, which matters because padding changes the absolute position index assigned to each real token\. Learned absolute position encoding, used by the original bidirectional encoder and by the autoregressive decoder considered here, assigns every absolute position index its own learned vector that is added to the token embedding, so the representation of a token depends on its absolute index\([Devlin et al\., 2019](https://arxiv.org/html/2610.09111#bib.bib4),[Radford et al\., 2019](https://arxiv.org/html/2610.09111#bib.bib15)\)\. Relative schemes instead encode the offset between a query position and a key position rather than the two absolute indices\([Shaw et al\., 2018](https://arxiv.org/html/2610.09111#bib.bib16)\)\. Rotary position embedding rotates the query and key vectors by an angle proportional to the absolute position, so that the attention score depends only on the relative offset and a shift common to query and key cancels\([Su et al\., 2024](https://arxiv.org/html/2610.09111#bib.bib26)\)\. Attention with linear biases adds a penalty to attention scores that grows with the distance between positions\([Press et al\., 2022](https://arxiv.org/html/2610.09111#bib.bib17)\)\. Models trained with no explicit positional encoding can still recover position information through the causal attention mask\([Haviv et al\., 2022](https://arxiv.org/html/2610.09111#bib.bib18)\)\. The choice of encoding scheme also affects how well a model generalizes to sequence lengths not seen during training\([Kazemnejad et al\., 2023](https://arxiv.org/html/2610.09111#bib.bib19),[Wang et al\., 2024](https://arxiv.org/html/2610.09111#bib.bib20)\)\. Current encoder and decoder models widely adopt rotary position embedding\([Warner et al\., 2025](https://arxiv.org/html/2610.09111#bib.bib27),[Qwen Team, 2025](https://arxiv.org/html/2610.09111#bib.bib14)\)\. A related training\-time intervention adds random padding during training so that real tokens are seen at many absolute positions, which improves the robustness of learned position embeddings\([Tao et al\., 2023](https://arxiv.org/html/2610.09111#bib.bib21)\)\. Together these mechanisms explain why a change in padded width can change the represented input for a model whose token representations depend on absolute position, while leaving it unchanged for a scheme that depends only on relative offsets\. Because the architecture name alone does not settle the question, the effect is measured directly rather than assumed\.

##### Behavioral and production\-readiness testing\.

A final line of work tests machine learning systems beyond a single aggregate accuracy number\. Behavioral testing constructs targeted suites that test specific capabilities and failure modes of a natural language processing model, exposing errors that overall accuracy hides\([Ribeiro et al\., 2020](https://arxiv.org/html/2610.09111#bib.bib22)\)\. Production\-readiness and reliability frameworks catalog the operational hazards of deployed systems, including hidden dependencies on data and configuration, and propose checklists that span data, model, and infrastructure\([Sculley et al\., 2015](https://arxiv.org/html/2610.09111#bib.bib24),[Breck et al\., 2017](https://arxiv.org/html/2610.09111#bib.bib1)\)\. The checks introduced in this paper are complementary to both\. For a fixed model and a fixed input text, they compare the predicted labels and the underlying scores across serving configurations\. They require every comparison to pass a task\-quality threshold, so that a configuration in which the model has collapsed to a constant prediction is excluded rather than counted as agreement\. They also include a deliberately vulnerable configuration whose sole purpose is to confirm that each measurement is able to detect the effect it is designed to test for\.

## Appendix ETheoretical Results and Proofs

Throughout this appendix,z⁡\(x,c\)∈ℝKz\(x,c\)\\in\\mathbb\{R\}^\{K\}denotes the logit vector produced for inputxxunder serving contextcc, where a serving context specifies the hidden execution conditions \(batch shape, padded width, numerical precision, quantizer state, and kernel configuration\) while the checkpoint, the text, and the tokenizer are held fixed\. Class probabilities are obtained by applying softmax to the displayed logits,p⁡\(y∣x,c\)=softmax⁡\(z⁡\(x,c\)\)yp\(y\\mid x,c\)=\\operatorname\{softmax\}\(z\(x,c\)\)\_\{y\}, and ties are broken by one fixed deterministic rule\. The total variation distance between the probability vectors of two contexts isTV⁡\(x,c,c′\)=12​∑y\|p⁡\(y∣x,c\)−p⁡\(y∣x,c′\)\|\\operatorname\{TV\}\(x,c,c^\{\\prime\}\)=\\tfrac\{1\}\{2\}\\sum\_\{y\}\|p\(y\\mid x,c\)\-p\(y\\mid x,c^\{\\prime\}\)\|\. The top\-two margin under a reference contextc0c\_\{0\}is the gap between the winning score and the strongest competitor, measured either on logits,g⁡\(x\)=zw​\(x,c0\)−maxj≠w⁡zj​\(x,c0\)g\(x\)=z\_\{w\}\(x,c\_\{0\}\)\-\\max\_\{j\\neq w\}z\_\{j\}\(x,c\_\{0\}\), or on probabilities,m⁡\(x\)=pw​\(x,c0\)−maxj≠w⁡pj​\(x,c0\)m\(x\)=p\_\{w\}\(x,c\_\{0\}\)\-\\max\_\{j\\neq w\}p\_\{j\}\(x,c\_\{0\}\), wherewwis the unique reference winner\. The results below formalize the mechanisms studied in the empirical sections: the structural position\-assignment mechanism, the margin conditions for label stability, the quantization scale\-policy contrast, the accumulation behavior of fully generative scoring and greedy decoding, the propagation of numerical discrepancies through depth, and the conditional feasibility of jointly attaining accuracy, speed, and serving\-context invariance\.

### E\.1Position assignment under changes of padded width

Proposition E\.1:Sufficient common\-shift condition for width invariance\.Let the real tokens of a fixed input occupy logical positionsiiandjj, and let addingssleading pad columns move their physical tensor columns toi\+si\+sandj\+sj\+s\. LetΦ⁡\(i,j\)\\Phi\(i,j\)denote every positional contribution to the real\-token computation\. Suppose all of the following hold:1\.Every positional contribution satisfiesΦ⁡\(i\+s,j\+s\)=Φ⁡\(i,j\)\\Phi\(i\+s,\\,j\+s\)=\\Phi\(i,j\)\.2\.Padding is excluded from attention and pooling, so the visibility relation among real tokens is unchanged\.3\.The class scores are read from the same semantic token or the same masked set of real tokens\.4\.Cache offsets, if present, use one consistent logical coordinate system\.5\.No other operation depends on the absolute physical tensor column\.Then a common shift of all real\-token columns leaves the ideal represented real\-token computation unchanged in exact arithmetic\.

###### Proof\.

The base real\-token contents are unchanged by the shift, since padding inserts columns without altering any real token embedding\. Condition 1 preserves every positional term entering the computation, so the first\-layer inputs restricted to real tokens are identical in the two executions\. Condition 2 preserves the dependency relation and the aggregation domain of every attention and pooling operation over real tokens, so each layer combines the same set of real\-token states with the same positional terms\. Conditions 3 and 4 preserve the semantic output position and any cached positional relations\. Condition 5 excludes any unmodeled path through which the absolute physical column could enter the computation\. Proceeding by induction over layers, the real\-token states after each layer are identical in exact arithmetic, and therefore so are the class scores read from them\. ∎

This proposition is a sufficient positional\-mechanism condition, not a complete serving\-invariance theorem\. A wider tensor can still select different kernels, tilings, or reduction trees and create numerical drift even when the represented input is preserved\. The condition explains the architecture control table\. Models whose positional terms depend only on token distance \(rotary position embeddings under left padding, and encoders whose real\-token positions do not move under right padding\) showed zero observed label changes out of 29,463 comparisons each\. A natively trained model with no explicit position term showed zero label changes out of 256 with only fp32\-scale numerical differences\.

Proposition E\.2:Learned absolute positions change the represented input\.Let the initial representation of real tokenxix\_\{i\}under a width\-derived left\-padding rule behi\(0\)=e⁡\(xi\)\+p⁡\(πi\)h\_\{i\}^\{\(0\)\}=e\(x\_\{i\}\)\+p\(\\pi\_\{i\}\), where positions are taken from the physical tensor column, so that addingssleading pad columns yields the shifted representationh~i\(0\)=e⁡\(xi\)\+p⁡\(πi\+s\)\\widetilde\{h\}\_\{i\}^\{\(0\)\}=e\(x\_\{i\}\)\+p\(\\pi\_\{i\}\+s\)\. Ifp⁡\(πi\+s\)≠p⁡\(πi\)p\(\\pi\_\{i\}\+s\)\\neq p\(\\pi\_\{i\}\)for any real token, then the two executions receive different represented inputs\. Consequently the same\-input premise of any numerical batch\-invariance guarantee is violated, and such a guarantee does not imply equal outputs\.

###### Proof\.

The difference betweenhi\(0\)h\_\{i\}^\{\(0\)\}andh~i\(0\)\\widetilde\{h\}\_\{i\}^\{\(0\)\}is present before the first transformer layer\. The same implemented function is then evaluated on two different represented inputs\. The outputs may coincidentally agree, but a numerical guarantee whose premise is an identical represented input does not apply, because that premise fails\. Correct masking does not remove this mechanism, since the changed vectors belong to real tokens rather than pads, and a semantic output position still consumes the changed hidden states\. ∎

This converse explains the other half of the architecture table\. The left\-padded family with learned absolute positions showed 35,591 label changes over 176,778 comparisons \(mean 20\.1%; 10\.3% when all comparisons from the 158 pairs that passed the accuracy threshold are pooled together\), while the right\-padded controls whose real\-token positions do not move showed zero observed label changes over the same comparison count\.

Proposition E\.3:Instantiation of the common\-shift condition across position\-encoding schemes\.Let all real\-token tensor columns shift by a common offsetss, and letΦ⁡\(i,j\)\\Phi\(i,j\)denote the positional contribution to the interaction between positionsiiandjj\. The common\-shift conditionΦ⁡\(i\+s,j\+s\)=Φ⁡\(i,j\)\\Phi\(i\+s,j\+s\)=\\Phi\(i,j\)holds for the following schemes\. \(i\) Positions assigned by order among real tokens: the assigned indices0,…,L−10,\\dots,L\-1do not change at all, so every positional term is identical\. \(ii\) Rotary embeddings: with rotation operatorRR, the query\-key positional term satisfies⟨R⁡\(i\)​q,R⁡\(j\)​k⟩=⟨q,R⁡\(j−i\)​k⟩\\langle R\(i\)q,R\(j\)k\\rangle=\\langle q,R\(j\-i\)k\\rangle, a function ofj−ij\-ionly\. \(iii\) ALiBi: the attention bias isBh​\(i,j\)=−mh​\(i−j\)B\_\{h\}\(i,j\)=\-m\_\{h\}\(i\-j\), a function of the distance only\. \(iv\) T5 bucketed relative bias:Bh​\(i,j\)=bh​\[bucket⁡\(j−i\)\]B\_\{h\}\(i,j\)=b\_\{h\}\[\\operatorname\{bucket\}\(j\-i\)\]\. \(v\) Shaw relative representations: relation vectors are selected byrclip⁡\(j−i,k\)r\_\{\\operatorname\{clip\}\(j\-i,\\,k\)\}\. \(vi\) Transformer\-XL relative attention: the relative\-position terms depend oni−ji\-j, provided cache indices use one coherent coordinate system\. \(vii\) A model with no positional term satisfies the condition vacuously\. The condition fails for learned absolute embeddings indexed by tensor column, sincep⁡\(πi\+s\)≠p⁡\(πi\)p\(\\pi\_\{i\}\+s\)\\neq p\(\\pi\_\{i\}\)in general, and it is not satisfied in general by sinusoidal absolute encodings\. AlthoughP​E​\(i\+s\)=As​P​E​\(i\)PE\(i\+s\)=A\_\{s\}\\,PE\(i\)for a block\-rotationAsA\_\{s\}within each frequency pair, the encoding is added to token content before arbitrary learned projections, and no general end\-to\-end cancellation identity exists\.

###### Proof\.

For \(i\), the position assignment is computed from the mask, so adding pad columns leaves every assigned index unchanged, and each positional term is evaluated at identical arguments\. For \(ii\) through \(vi\), substitutei\+si\+sandj\+sj\+sinto the stated positional term\. In each case the term depends on its arguments only through the difference\(j\+s\)−\(i\+s\)=j−i\(j\+s\)\-\(i\+s\)=j\-i\(after clipping or bucketing, which are functions of the difference\), so the term is unchanged\. For \(vii\) there is no positional term to change\. For learned absolute embeddings, the initial representation of a real token becomese⁡\(xi\)\+p⁡\(πi\+s\)e\(x\_\{i\}\)\+p\(\\pi\_\{i\}\+s\), which differs frome⁡\(xi\)\+p⁡\(πi\)e\(x\_\{i\}\)\+p\(\\pi\_\{i\}\)whenever the embedding table distinguishes the two rows, establishing the failure by Proposition\. For sinusoidal encodings, the block\-rotation identity shows that a common shift acts linearly on the encoding\. However, the encoding enters the model additively in content space and passes through learned projections that need not commute withAsA\_\{s\}; absent such structure, the represented input changes, so the condition is not satisfied in general\. ∎

Among these schemes, we directly measure learned absolute positions \(affected\), order positions, rotary embeddings \(Qwen3 and ModernBERT\), and a native model trained without any positional term \(all unaffected in the measured cells\)\. The ALiBi, T5, Shaw, Transformer\-XL, and sinusoidal statements are mechanism\-level consequences of the published formulas and are literature\-supported boundaries, not measurements from our experiments\. End\-to\-end invariance for any implementation additionally requires the masking, output\-position, cache, and no\-other\-path conditions of Proposition\.

### E\.2Margin certificates for label stability

Proposition E\.4:Sufficient logit\-margin condition\.Letwwbe the unique winning class under the reference contextc0c\_\{0\}and letg⁡\(x\)=zw​\(x,c0\)−maxj≠w⁡zj​\(x,c0\)g\(x\)=z\_\{w\}\(x,c\_\{0\}\)\-\\max\_\{j\\neq w\}z\_\{j\}\(x,c\_\{0\}\)be the reference top\-two logit margin\. Suppose that for every allowed contextcc,‖z⁡\(x,c\)−z⁡\(x,c0\)‖∞≤ϵ\.\\left\\\|z\(x,c\)\-z\(x,c\_\{0\}\)\\right\\\|\_\{\\infty\}\\leq\\epsilon\.Ifg⁡\(x\)\>2​ϵg\(x\)\>2\\epsilon, then the predicted class is invariant across all allowed contexts\.

###### Proof\.

Fix any allowed contextccand any competitor classj≠wj\\neq w\. The winning logit can decrease by at mostϵ\\epsilon, and the competitor logit can increase by at mostϵ\\epsilon\. Therefore

zw​\(x,c\)−zj​\(x,c\)≥\(zw​\(x,c0\)−ϵ\)−\(zj​\(x,c0\)\+ϵ\)≥g⁡\(x\)−2​ϵ\>0\.z\_\{w\}\(x,c\)\-z\_\{j\}\(x,c\)\\;\\geq\\;\\bigl\(z\_\{w\}\(x,c\_\{0\}\)\-\\epsilon\\bigr\)\-\\bigl\(z\_\{j\}\(x,c\_\{0\}\)\+\\epsilon\\bigr\)\\;\\geq\\;g\(x\)\-2\\epsilon\\;\>\\;0\.The reference winner remains strictly above every competitor in every allowed context, so the predicted label does not change\. ∎

The strict inequality is required for this norm bound alone to certify a strict winner\. Equality permits a tie but does not require one, so actual invariance can still hold below the threshold\. A sharper shift\-invariant condition replaces2​ϵ2\\epsilonby the oscillation of the perturbation,osc⁡\(Δ\)=maxk⁡Δk−mink⁡Δk≤2​‖Δ‖∞\\operatorname\{osc\}\(\\Delta\)=\\max\_\{k\}\\Delta\_\{k\}\-\\min\_\{k\}\\Delta\_\{k\}\\leq 2\\\|\\Delta\\\|\_\{\\infty\}, which discounts any common scalar added to all logits\. This certificate connects to the empirical margin stratification, in which precision\-induced and execution\-induced label changes concentrated at small pre\-intervention margins while wide\-margin examples remained stable\.

Proposition E\.5:Sufficient probability\-margin condition\.Letm⁡\(x\)=pw​\(x,c0\)−maxj≠w⁡pj​\(x,c0\)m\(x\)=p\_\{w\}\(x,c\_\{0\}\)\-\\max\_\{j\\neq w\}p\_\{j\}\(x,c\_\{0\}\)be the reference probability margin, and for one example and context defineTV⁡\(x,c,c0\)=12​∑y\|py​\(x,c\)−py​\(x,c0\)\|\\operatorname\{TV\}\(x,c,c\_\{0\}\)=\\tfrac\{1\}\{2\}\\sum\_\{y\}\|p\_\{y\}\(x,c\)\-p\_\{y\}\(x,c\_\{0\}\)\|\. Ifm⁡\(x\)\>2​TV⁡\(x,c,c0\),m\(x\)\>2\\operatorname\{TV\}\(x,c,c\_\{0\}\),then the predicted label cannot change in contextcc\. For a guarantee over all declared contexts, replaceTV\\operatorname\{TV\}by its supremum overcc\.

###### Proof\.

For an event consisting of a single class, the absolute difference of its probability between the two contexts is at most the total variation distance\. Hence the winner’s probability can decrease by at mostTV⁡\(x,c,c0\)\\operatorname\{TV\}\(x,c,c\_\{0\}\), and any competitor’s probability can increase by at mostTV⁡\(x,c,c0\)\\operatorname\{TV\}\(x,c,c\_\{0\}\)\. Every winner\-versus\-competitor probability gap therefore decreases by at most2​TV⁡\(x,c,c0\)2\\operatorname\{TV\}\(x,c,c\_\{0\}\), and since the reference gapm⁡\(x\)m\(x\)strictly exceeds this amount, the winner remains strictly ahead of every competitor\. ∎

Proposition E\.6:Corollary, necessary condition for a label change\.Under the assumptions of the two preceding propositions, if the predicted label changes between contextsc0c\_\{0\}andcc, then necessarilyg⁡\(x\)≤2​ϵandm⁡\(x\)≤2​TV⁡\(x,c,c0\)\.g\(x\)\\leq 2\\epsilon\\qquad\\text\{and\}\\qquad m\(x\)\\leq 2\\operatorname\{TV\}\(x,c,c\_\{0\}\)\.

###### Proof\.

Both statements are the contrapositives of the sufficient conditions just proved\. Ifg⁡\(x\)\>2​ϵg\(x\)\>2\\epsilonthe label cannot change, so a label change forcesg⁡\(x\)≤2​ϵg\(x\)\\leq 2\\epsilon; ifm⁡\(x\)\>2​TV⁡\(x,c,c0\)m\(x\)\>2\\operatorname\{TV\}\(x,c,c\_\{0\}\)the label cannot change, so a label change forcesm⁡\(x\)≤2​TV⁡\(x,c,c0\)m\(x\)\\leq 2\\operatorname\{TV\}\(x,c,c\_\{0\}\)\. ∎

This necessary condition held for all 375 of 375 observed reduced\-precision label changes in the movie\-review cell\. Every change satisfiedm≤2​TVm\\leq 2\\operatorname\{TV\}\. This is a consistency check implied by the proposition, not evidence that the bound is a strong prospective predictor\. A prospective verifier would additionally need an intervention\-specific total variation upper bound available before accepting the fast result, and the certificate covers only the top label, not calibration, ranking, thresholding, filtering, or abstention\.

### E\.3Quantization scale policy and peer coupling

Proposition E\.7:Dynamic batch\-global scale couples the target to its peers\.For a positive scalessand positive signed integer limitMM, define the dequantized scalar quantizerQs​\(a\)=s​clip⁡\(round⁡\(a/s\),−M,M\),Q\_\{s\}\(a\)=s\\,\\operatorname\{clip\}\\\!\\bigl\(\\operatorname\{round\}\(a/s\),\\,\-M,\\,M\\bigr\),and let a batch\-global dynamic quantizer chooses⁡\(B\)=maxa∈B⁡\|a\|/Ms\(B\)=\\max\_\{a\\in B\}\|a\|/Mor an analogous runtime range statistic, where the batchBBcontains the target row and its peers \(an all\-zero batch requires an explicit positive default scale\)\. LetBBandB′B^\{\\prime\}have the same shape and the same target activationaabut different peer values\. Ifs⁡\(B\)≠s⁡\(B′\)andQs⁡\(B\)​\(a\)≠Qs⁡\(B′\)​\(a\),s\(B\)\\neq s\(B^\{\\prime\}\)\\qquad\\text\{and\}\\qquad Q\_\{s\(B\)\}\(a\)\\neq Q\_\{s\(B^\{\\prime\}\)\}\(a\),then the target’s quantized activation depends on peer content\. If in addition the downstream target\-logit map is injective on these two activation points, or the relevant logit is strictly monotone along the segment connecting them, then the target logits also differ\.

###### Proof\.

The target valueaais fixed, but its quantized representation is evaluated on two different quantization grids determined by the two scales\. The stated inequalityQs⁡\(B\)​\(a\)≠Qs⁡\(B′\)​\(a\)Q\_\{s\(B\)\}\(a\)\\neq Q\_\{s\(B^\{\\prime\}\)\}\(a\)makes the dequantized target activation different in the two contexts\. Injectivity of the downstream map on these two points, or strict monotonicity of a relevant logit along their connecting segment, then maps the two distinct activations to two distinct logit values\. ∎

This proposition establishes a possible scale\-mediated peer dependence, not inevitable movement\. Some target values lie on both grids, downstream computation can map distinct activations to the same logits, and the formula models a batch\-global quantizer rather than dynamic quantizers with request\-local, row\-local, or token\-local scales\. It corresponds to the dynamic int8 experiment, in which changing only peer content at fixed shape moved target logits by up to 0\.774 for the fully generative model and about 0\.09 for the discriminative model, with up to 2 of 64 label changes, for the one tested dynamic\-quantization implementation and module set\.

Proposition E\.8:Fixed peer\-independent scale does not depend on peer content at fixed shape\.Assume all of the following:1\.The target tokens, masks, positions, initial activations, and quantization state are fixed\.2\.The activation scale is positive, fixed before serving, and independent of peer values\.3\.Every target\-row intermediate depends only on the preceding target\-row intermediate and fixed constants\.4\.Kernel selection, operation order, rounding path, and runtime state used for the target are independent of peer values at the fixed shape\.5\.Execution is deterministic under these conditions\.Then changing peer content at fixed shape cannot change the target output\.

###### Proof\.

By assumption 2 the quantizer scale does not respond to the peers, so the fixed quantizer maps the fixed target activation to the same quantized value in both contexts\. By assumptions 3 and 4 the target\-row computation applies the same operations, in the same order, with the same rounding path and runtime state, to the same target intermediates, regardless of peer values\. By assumption 5 there is no residual source of variation\. Induction over the target\-row computation therefore yields identical target intermediates at every stage and an identical target output\. ∎

The proposition does not guarantee invariance across batch shapes, kernels, hardware, precision, or scale\-application order\. The server static\-int8 configuration and the dynamic\-scale control in the same evaluation setup both use fake\-quantized linear modules with fp32 accumulation and differ only in the activation scale source\. This comparison isolates that variable but does not use a production integer kernel\. The dynamic\-int8 experiment in Tableuses the integer CPU kernel, and the phone rows use exported integer graphs\. The proposition predicts the observed contrast between the dynamic and fixed\-scale int8 experiments\. The same peer\-content intervention that moved logits under a batch\-dependent runtime scale became bitwise identical on the target once the activation scale was fixed offline and made independent of the peers\.

### E\.4Fully generative scoring and greedy decoding

Proposition E\.9:Length\-proportional drift bound for summed sequence scores\.For classyy, let the token\-level log\-probability contribution under contextccbeℓy,t​\(c\)=log⁡pθ,c​\(xt∣x<t,y\)\\ell\_\{y,t\}\(c\)=\\log p\_\{\\theta,c\}\(x\_\{t\}\\mid x\_\{<t\},y\)and the class score beSy​\(c\)=∑t=1Lℓy,t​\(c\)S\_\{y\}\(c\)=\\sum\_\{t=1\}^\{L\}\\ell\_\{y,t\}\(c\), with padding positions excluded\. Assume each token contribution changes by at mostϵ\\epsilonacross two contexts:\|ℓy,t​\(c\)−ℓy,t​\(c′\)\|≤ϵfor all​t\.\|\\ell\_\{y,t\}\(c\)\-\\ell\_\{y,t\}\(c^\{\\prime\}\)\|\\leq\\epsilon\\quad\\text\{for all \}t\.Then for each class,\|Sy​\(c\)−Sy​\(c′\)\|≤L​ϵ,\|S\_\{y\}\(c\)\-S\_\{y\}\(c^\{\\prime\}\)\|\\leq L\\epsilon,and for any two classesyyandjj, the score\-gap drift is bounded by\|\[Sy​\(c\)−Sj​\(c\)\]−\[Sy​\(c′\)−Sj​\(c′\)\]\|≤2​L​ϵ\.\\bigl\|\[S\_\{y\}\(c\)\-S\_\{j\}\(c\)\]\-\[S\_\{y\}\(c^\{\\prime\}\)\-S\_\{j\}\(c^\{\\prime\}\)\]\\bigr\|\\leq 2L\\epsilon\.

###### Proof\.

For the first bound, apply the triangle inequality to the sum ofLLtoken\-level differences:

\|Sy​\(c\)−Sy​\(c′\)\|=\|∑t=1L\(ℓy,t​\(c\)−ℓy,t​\(c′\)\)\|≤∑t=1L\|ℓy,t​\(c\)−ℓy,t​\(c′\)\|≤L​ϵ\.\|S\_\{y\}\(c\)\-S\_\{y\}\(c^\{\\prime\}\)\|=\\Bigl\|\\sum\_\{t=1\}^\{L\}\\bigl\(\\ell\_\{y,t\}\(c\)\-\\ell\_\{y,t\}\(c^\{\\prime\}\)\\bigr\)\\Bigr\|\\leq\\sum\_\{t=1\}^\{L\}\|\\ell\_\{y,t\}\(c\)\-\\ell\_\{y,t\}\(c^\{\\prime\}\)\|\\leq L\\epsilon\.For the second bound, apply the triangle inequality once more to the difference of the two class\-score drifts:

\|\[Sy​\(c\)−Sj​\(c\)\]−\[Sy​\(c′\)−Sj​\(c′\)\]\|≤\|Sy​\(c\)−Sy​\(c′\)\|\+\|Sj​\(c\)−Sj​\(c′\)\|≤2​L​ϵ\.\\bigl\|\[S\_\{y\}\(c\)\-S\_\{j\}\(c\)\]\-\[S\_\{y\}\(c^\{\\prime\}\)\-S\_\{j\}\(c^\{\\prime\}\)\]\\bigr\|\\leq\|S\_\{y\}\(c\)\-S\_\{y\}\(c^\{\\prime\}\)\|\+\|S\_\{j\}\(c\)\-S\_\{j\}\(c^\{\\prime\}\)\|\\leq 2L\\epsilon\.∎

Proposition E\.10:Corollary, margin certificate for fully generative prediction\.Under the assumptions of the preceding proposition, if the reference class\-score margin exceeds2​L​ϵ2L\\epsilon, then the fully generative prediction is invariant under the assumed token\-level perturbation bound\. If class\-conditioned sequences have different lengthsLyL\_\{y\}andLjL\_\{j\}, the pairwise bound is\(Ly\+Lj\)​ϵ\(L\_\{y\}\+L\_\{j\}\)\\epsilonrather than2​L​ϵ2L\\epsilon\.

###### Proof\.

By the preceding proposition, every winner\-versus\-competitor score gap changes by at most2​L​ϵ2L\\epsilonbetween the two contexts \(or by at most\(Ly\+Lj\)​ϵ\(L\_\{y\}\+L\_\{j\}\)\\epsilonunder class\-dependent lengths\)\. If the reference margin strictly exceeds this amount, the reference winner remains strictly ahead of every competitor, by the same argument as in the logit\-margin certificate\. ∎

These bounds establish a larger admissible drift for summed sequence scores than for a classifier that reads a single output position, whose corresponding factor is2​ϵ2\\epsilonrather than2​L​ϵ2L\\epsilon\. They do not prove larger realized drift, and length normalization, shared computation, higher margins, contraction, or exact arithmetic can reverse the ordering\. The comparison isolates the aggregation mechanism behind two empirical observations: the difference between formulations, in which the fully generative model showed maximum batch\-shape logit movement of approximately 0\.447 under reduced precision against 0\.03125 for the discriminative model in the tested cells, and the within\-example trend of drift increasing with real\-token length at fixed padded width\.

Proposition E\.11:First\-divergence certificate for greedy decoding\.Consider two executions of greedy decoding under contextsccandc′c^\{\\prime\}from the same prompt with a fixed deterministic tie rule\. Before the two trajectories diverge, their prefixes are identical\. Letgtg\_\{t\}be the top\-two logit margin at steptton the common prefix under the reference contextc0c\_\{0\}, and suppose the two\-context logit difference at that same prefix is bounded in the infinity norm byϵt\\epsilon\_\{t\}\. Ifgt\>2​ϵtat every step​t=1,…,T,g\_\{t\}\>2\\epsilon\_\{t\}\\quad\\text\{at every step \}t=1,\\dots,T,then the selected token is identical in both executions at every step throughTT\.

###### Proof\.

Proceed by induction on the decoding step\. At step 1 the two executions share the prompt as their common prefix\. Assume the prefixes are identical through stept−1t\-1\. Then at stepttboth executions evaluate the model on the same prefix, and their logit vectors differ by at mostϵt\\epsilon\_\{t\}in the infinity norm\. By the logit\-margin certificate, the conditiongt\>2​ϵtg\_\{t\}\>2\\epsilon\_\{t\}guarantees that the same token is selected in both executions, so the prefixes remain identical through steptt\. By induction the selected tokens agree at every step throughTT\. ∎

Once one token differs, the prefixes are no longer equal and the same\-input perturbation model no longer applies\. Subsequent divergence can then reflect ordinary sensitivity to different inputs rather than continued numerical error accumulation alone\. This certificate frames the autoregressive accumulation experiment, in which reduced\-precision batch\-shape perturbations changed all three tested generations within the first several steps while full\-precision generation did not change within the measured horizon, and every same\-context repeat was bitwise identical\.

### E\.5Propagation of discrepancies through depth

Proposition E\.12:Layerwise error propagation\.Let the layer state under contextccsatisfyhℓc=Fℓc​\(hℓ−1c\)h\_\{\\ell\}^\{c\}=F\_\{\\ell\}^\{c\}\(h\_\{\\ell\-1\}^\{c\}\)for layersℓ=1,…,L\\ell=1,\\dots,L\. Assume that over all reachable states each context\-specific layer map is Lipschitz,‖Fℓc​\(u\)−Fℓc​\(v\)‖≤Kℓ​‖u−v‖,\\\|F\_\{\\ell\}^\{c\}\(u\)\-F\_\{\\ell\}^\{c\}\(v\)\\\|\\leq K\_\{\\ell\}\\\|u\-v\\\|,and that the same\-state discrepancy between the two context\-specific maps is uniformly bounded,‖Fℓc​\(v\)−Fℓc′​\(v\)‖≤ϵℓ\.\\\|F\_\{\\ell\}^\{c\}\(v\)\-F\_\{\\ell\}^\{c^\{\\prime\}\}\(v\)\\\|\\leq\\epsilon\_\{\\ell\}\.Then the final discrepancy satisfies‖hLc−hLc′‖≤‖h0c−h0c′‖​∏j=1LKj\+∑ℓ=1Lϵℓ​∏j=ℓ\+1LKj\.\\\|h\_\{L\}^\{c\}\-h\_\{L\}^\{c^\{\\prime\}\}\\\|\\leq\\\|h\_\{0\}^\{c\}\-h\_\{0\}^\{c^\{\\prime\}\}\\\|\\prod\_\{j=1\}^\{L\}K\_\{j\}\+\\sum\_\{\\ell=1\}^\{L\}\\epsilon\_\{\\ell\}\\prod\_\{j=\\ell\+1\}^\{L\}K\_\{j\}\.

###### Proof\.

For each layerℓ\\ell, insert and subtract the mixed termFℓc​\(hℓ−1c′\)F\_\{\\ell\}^\{c\}\(h\_\{\\ell\-1\}^\{c^\{\\prime\}\}\):

‖hℓc−hℓc′‖≤‖Fℓc​\(hℓ−1c\)−Fℓc​\(hℓ−1c′\)‖\+‖Fℓc​\(hℓ−1c′\)−Fℓc′​\(hℓ−1c′\)‖\.\\\|h\_\{\\ell\}^\{c\}\-h\_\{\\ell\}^\{c^\{\\prime\}\}\\\|\\leq\\\|F\_\{\\ell\}^\{c\}\(h\_\{\\ell\-1\}^\{c\}\)\-F\_\{\\ell\}^\{c\}\(h\_\{\\ell\-1\}^\{c^\{\\prime\}\}\)\\\|\+\\\|F\_\{\\ell\}^\{c\}\(h\_\{\\ell\-1\}^\{c^\{\\prime\}\}\)\-F\_\{\\ell\}^\{c^\{\\prime\}\}\(h\_\{\\ell\-1\}^\{c^\{\\prime\}\}\)\\\|\.The first term is at mostKℓ​‖hℓ−1c−hℓ−1c′‖K\_\{\\ell\}\\\|h\_\{\\ell\-1\}^\{c\}\-h\_\{\\ell\-1\}^\{c^\{\\prime\}\}\\\|by the Lipschitz assumption, and the second is at mostϵℓ\\epsilon\_\{\\ell\}by the same\-state discrepancy assumption\. This yields the recurrence

‖hℓc−hℓc′‖≤Kℓ​‖hℓ−1c−hℓ−1c′‖\+ϵℓ\.\\\|h\_\{\\ell\}^\{c\}\-h\_\{\\ell\}^\{c^\{\\prime\}\}\\\|\\leq K\_\{\\ell\}\\\|h\_\{\\ell\-1\}^\{c\}\-h\_\{\\ell\-1\}^\{c^\{\\prime\}\}\\\|\+\\epsilon\_\{\\ell\}\.Iterating the recurrence fromℓ=1\\ell=1toLL, the initial discrepancy is multiplied by every later Lipschitz factor, and each local map discrepancyϵℓ\\epsilon\_\{\\ell\}is multiplied only by the Lipschitz factors of the layers after it, which gives the stated bound\. ∎

The bound formally separates the two regimes measured in the paper\. For a numerical execution comparison with an identical represented input, the initial term is zero and only accumulated local discrepancies remain, whereas for structural padding that changes learned absolute position embeddings the initial term is generally nonzero\. No monotonic depth conclusion follows without additional assumptions on theKℓK\_\{\\ell\}andϵℓ\\epsilon\_\{\\ell\}values, since contracting later layers can shrink earlier discrepancies and perturbations can cancel\. The observed rise of hidden score drift with nominal model size is therefore a descriptive association consistent with one possible parameter regime, not a consequence of depth alone\.

### E\.6Conditional infeasibility of accuracy, speed, and invariance

Theorem E\.1:Conditional triple infeasibility\.Fix a finite label set, a task distribution over\(X,Y\)\(X,Y\), a bounded set of allowed serving contexts𝒞\\mathcal\{C\}, a bounded workload, one hardware and software stack, an implementation familyℋ\\mathcal\{H\}, and predeclared thresholdsα\\alpha\(accuracy\),τ\\tau\(latency\), andδ\\delta\(probability drift in total variation\)\. Define worst\-context accuracyA\(f\)=infc∈𝒞Pr\[argmaxypf\(y∣X,c\)=Y\]A\(f\)=\\inf\_\{c\\in\\mathcal\{C\}\}\\Pr\[\\arg\\max\_\{y\}p\_\{f\}\(y\\mid X,c\)=Y\], worst\-case latencyT⁡\(f\)T\(f\)over the bounded workload, and driftV⁡\(f\)=supx,c,c′TV⁡\(x,c,c′\)V\(f\)=\\sup\_\{x,c,c^\{\\prime\}\}\\operatorname\{TV\}\(x,c,c^\{\\prime\}\)\. Define the fast and invariant subsetℋF​I=\{f∈ℋ:T\(f\)≤τ,V\(f\)≤δ\}\.\\mathcal\{H\}\_\{FI\}=\\\{f\\in\\mathcal\{H\}:T\(f\)\\leq\\tau,\\;V\(f\)\\leq\\delta\\\}\.Assume there exists a measurable statisticQ=q⁡\(X\)Q=q\(X\)such that, for everyf∈ℋF​If\\in\\mathcal\{H\}\_\{FI\}and every allowed contextcc, the final prediction is a deterministic measurable function ofQQ:Y^f​\(c\)=gf,c​\(Q\)\.\\widehat\{Y\}\_\{f\}\(c\)=g\_\{f,c\}\(Q\)\.Define the Bayes accuracy available fromQQasAQ⋆=𝔼Q​\[maxy⁡Pr⁡\(Y=y∣Q\)\]\.A\_\{Q\}^\{\\star\}=\\mathbb\{E\}\_\{Q\}\\Bigl\[\\max\_\{y\}\\Pr\(Y=y\\mid Q\)\\Bigr\]\.IfAQ⋆<αA\_\{Q\}^\{\\star\}<\\alpha, then no implementation inℋ\\mathcal\{H\}is simultaneouslyτ\\tau\-fast,α\\alpha\-accurate, andδ\\delta\-invariant\.

###### Proof\.

Fix anyf∈ℋF​If\\in\\mathcal\{H\}\_\{FI\}and any allowed contextcc\. Because the decision offfunderccfactors throughQQ,

Pr\[Y^f\(c\)=Y\]=𝔼Q\[Pr\(Y=gf,c\(Q\)∣Q\)\]≤𝔼Q\[maxyPr\(Y=y∣Q\)\]=AQ⋆<α\.\\Pr\\bigl\[\\widehat\{Y\}\_\{f\}\(c\)=Y\\bigr\]=\\mathbb\{E\}\_\{Q\}\\bigl\[\\Pr\(Y=g\_\{f,c\}\(Q\)\\mid Q\)\\bigr\]\\leq\\mathbb\{E\}\_\{Q\}\\Bigl\[\\max\_\{y\}\\Pr\(Y=y\\mid Q\)\\Bigr\]=A\_\{Q\}^\{\\star\}<\\alpha\.The bound holds for every allowed context, so it also bounds the infimum over contexts that definesA⁡\(f\)A\(f\); henceA⁡\(f\)<αA\(f\)<\\alpha\. Every fast and invariant implementation therefore fails the accuracy threshold, and the intersection of the three feasible sets is empty\. ∎

The information\-restriction premise, namely that every fast and invariant implementation factors through the statisticQQ, is an assumption of the theorem and has not been established for the classifier family studied in this paper\. The result is a generic conditional feasibility theorem, not an architecture\-specific impossibility result\. The theorem can also be vacuous if the fast and invariant subset is empty or ifα\\alphaexceeds the full\-information Bayes accuracy, and a strict three\-way incompatibility would additionally require three separately established pairwise witnesses under common thresholds\. The theorem’s role is interpretive\. The measured mitigation costs \(batch\-invariant execution at an end\-to\-end latency ratio near 1\.3 to 1\.5 at large batch and roughly neutral at batch size 1, despite isolated invariant matrix multiplications running several times slower per call, and position assignment that does not depend on the padded width preserving family\-level quality\) show that the fast and invariant subset need not be empty in practice, which constrains how the theorem can be instantiated\.

Proposition E\.13:Approximate bottleneck bound\.Suppose that for every fast and invariant implementationf∈ℋF​If\\in\\mathcal\{H\}\_\{FI\}there exists a family of deterministicQQ\-based rules indexed by context,\{hf,c:c∈𝒞\}\\\{h\_\{f,c\}:c\\in\\mathcal\{C\}\\\}, such thatsupc∈𝒞Pr\[Y^f\(c\)≠hf,c\(Q\)\]≤ρ\.\\sup\_\{c\\in\\mathcal\{C\}\}\\Pr\\bigl\[\\widehat\{Y\}\_\{f\}\(c\)\\neq h\_\{f,c\}\(Q\)\\bigr\]\\leq\\rho\.Then every such implementation satisfiesA⁡\(f\)≤AQ⋆\+ρ,A\(f\)\\leq A\_\{Q\}^\{\\star\}\+\\rho,and ifAQ⋆\+ρ<αA\_\{Q\}^\{\\star\}\+\\rho<\\alpha, the triple of thresholds remains infeasible\.

###### Proof\.

Fix any contextcc\. The event thatffpredicts correctly underccis contained in the union of the event thathf,c​\(Q\)h\_\{f,c\}\(Q\)predicts correctly and the event that the two rules disagree\. Therefore

Pr⁡\(Y^f​\(c\)=Y\)≤Pr⁡\(hf,c​\(Q\)=Y\)\+Pr⁡\(Y^f​\(c\)≠hf,c​\(Q\)\)\.\\Pr\\bigl\(\\widehat\{Y\}\_\{f\}\(c\)=Y\\bigr\)\\leq\\Pr\\bigl\(h\_\{f,c\}\(Q\)=Y\\bigr\)\+\\Pr\\bigl\(\\widehat\{Y\}\_\{f\}\(c\)\\neq h\_\{f,c\}\(Q\)\\bigr\)\.The first term is at mostAQ⋆A\_\{Q\}^\{\\star\}, sincehf,ch\_\{f,c\}is a deterministic function ofQQand no such rule can exceed the Bayes accuracy available fromQQ; the second term is at mostρ\\rho, uniformly over contexts\. Taking the infimum over contexts givesA⁡\(f\)≤AQ⋆\+ρA\(f\)\\leq A\_\{Q\}^\{\\star\}\+\\rho\. IfAQ⋆\+ρ<αA\_\{Q\}^\{\\star\}\+\\rho<\\alpha, thenA⁡\(f\)<αA\(f\)<\\alphafor every fast and invariant implementation, so no implementation satisfies all three thresholds\. ∎

This relaxation weakens exact factorization to approximate agreement with aQQ\-based rule\. Both results take the restriction toQQas a premise\. The remainder of this subsection derives that premise from an execution model, exhibits a finite instance in which the three thresholds are pairwise feasible but jointly infeasible, and extends the derivation to the approximate setting of Proposition\.

Proposition E\.14:Collision lemma, exact invariance factors through the fast representation\.Let the input support𝒳\\mathcal\{X\}be finite\. Suppose that for every contextc∈𝒞c\\in\\mathcal\{C\}the fast execution path exposes a representationRc​\(x\)R\_\{c\}\(x\)of the input, and that every implementation withT⁡\(f\)≤τT\(f\)\\leq\\taupredicts from it,pf\(⋅∣x,c\)=hf,c\(Rc\(x\)\)p\_\{f\}\(\\cdot\\mid x,c\)=h\_\{f,c\}\(R\_\{c\}\(x\)\), where the postprocessorhf,ch\_\{f,c\}may depend on the context but has no other access toxx\. Form the undirected graph on𝒳\\mathcal\{X\}that joinsxxandx′x^\{\\prime\}wheneverRc​\(x\)=Rc​\(x′\)R\_\{c\}\(x\)=R\_\{c\}\(x^\{\\prime\}\)for somec∈𝒞c\\in\\mathcal\{C\}, and letQ⁡\(x\)Q\(x\)denote the connected component ofxx\. Ifffisτ\\tau\-fast and label\-invariant across𝒞\\mathcal\{C\}, then its predicted label is a function ofQQ; ifffis exactly score\-invariant \(V⁡\(f\)=0V\(f\)=0\), then its whole probability vector is a function ofQQ\.

###### Proof\.

Letxxandx′x^\{\\prime\}be joined by an edge witnessed bycc, soRc​\(x\)=Rc​\(x′\)R\_\{c\}\(x\)=R\_\{c\}\(x^\{\\prime\}\)\. Thenpf\(⋅∣x,c\)=hf,c\(Rc\(x\)\)=hf,c\(Rc\(x′\)\)=pf\(⋅∣x′,c\)p\_\{f\}\(\\cdot\\mid x,c\)=h\_\{f,c\}\(R\_\{c\}\(x\)\)=h\_\{f,c\}\(R\_\{c\}\(x^\{\\prime\}\)\)=p\_\{f\}\(\\cdot\\mid x^\{\\prime\},c\), so in contextccthe fast path cannot distinguish the two inputs\. Label invariance givesY^f​\(x,c′\)=Y^f​\(x,c\)=Y^f​\(x′,c\)=Y^f​\(x′,c′\)\\widehat\{Y\}\_\{f\}\(x,c^\{\\prime\}\)=\\widehat\{Y\}\_\{f\}\(x,c\)=\\widehat\{Y\}\_\{f\}\(x^\{\\prime\},c\)=\\widehat\{Y\}\_\{f\}\(x^\{\\prime\},c^\{\\prime\}\)for everyc′c^\{\\prime\}, so the predicted label agrees on the two endpoints in every context\. Agreement is transitive along paths, hence the label is constant on each connected component and is a function ofQQ\. UnderV⁡\(f\)=0V\(f\)=0the same chain of equalities holds for the full probability vector\. ∎

The lemma supplies the premise of TheoremwithQQequal to the component map, so thatA⁡\(f\)≤AQ⋆A\(f\)\\leq A\_\{Q\}^\{\\star\}for every fast and exactly invariant implementation\. The collision must occur within a single context\.Rc​\(x\)=Rc′​\(x′\)R\_\{c\}\(x\)=R\_\{c^\{\\prime\}\}\(x^\{\\prime\}\)forc≠c′c\\neq c^\{\\prime\}carries no constraint, because a context\-aware postprocessor may treat the two contexts differently\. The lemma also locates what an instantiation must establish\. It must identify a lossyRcR\_\{c\}for the actual fast path and show that every implementation inside the latency budget is confined to it\. Measuring the latency of particular kernels does not by itself establish that lower bound over implementations\. If distinct fast policies expose distinct maps, each policy has its own graph and its own bound\.

Proposition E\.15:A finite instance in which each pair of thresholds is feasible and the triple is not\.LetXXbe uniform on\{a,b,c,d\}\\\{a,b,c,d\\\}with deterministic labels0,1,0,10,1,0,1, and let𝒞=\{c1,c2,c3\}\\mathcal\{C\}=\\\{c\_\{1\},c\_\{2\},c\_\{3\}\\\}with fast representations given by the partitionsc1:\{a,b\},\{c\},\{d\}c\_\{1\}:\\\{a,b\\\},\\\{c\\\},\\\{d\\\};c2:\{a\},\{b,c\},\{d\}c\_\{2\}:\\\{a\\\},\\\{b,c\\\},\\\{d\\\};c3:\{a\},\{b\},\{c,d\}c\_\{3\}:\\\{a\\\},\\\{b\\\},\\\{c,d\\\}, where inputs in one cell share the sameRcR\_\{c\}\. Let the fast path cost one unit and a slow path that exposes the input identity cost two units, and setτ=1\\tau=1,δ=0\\delta=0, and anyα∈\(1/2,3/4\]\\alpha\\in\(1/2,3/4\]\. Then each pair of thresholds is attainable while the triple is not\.

###### Proof\.

In the fast and accurate case, predict in each context the true label on the two singleton cells and one fixed label on the merged pair; exactly one of the four inputs is wrong, so the accuracy is3/4≥α3/4\\geq\\alphain every context, but the predictions differ across contexts\. In the accurate and invariant case, the slow path returns the true label everywhere, with accuracy11, at cost2\>τ2\>\\tau\. In the fast and invariant case, a constant label has accuracy1/21/2\. For the triple, the collision graph has edgesaa–bb\(witnessed byc1c\_\{1\}\),bb–cc\(c2c\_\{2\}\), andcc–dd\(c3c\_\{3\}\), so it is connected andQQis a single component\. By Propositionevery fast and invariant rule is constant, andAQ⋆=1/2<αA\_\{Q\}^\{\\star\}=1/2<\\alpha\. ∎

No single context forces the accuracy down to1/21/2\. The loss arises from requiring agreement across contexts whose collisions overlap\. The costs define the abstract execution model of the example and are not measured latencies\.

Proposition E\.16:Approximate collision bound\.In the setting of Proposition, suppose every connected component has graph diameter at mostDD, and letffbeτ\\tau\-fast withV⁡\(f\)≤δV\(f\)\\leq\\delta\. Fix a reference contextc0c\_\{0\}and one representativer⁡\(q\)r\(q\)in each componentqq\. Then for everyxx,TV\(pf\(⋅∣x,c0\),pf\(⋅∣r\(Q\(x\)\),c0\)\)≤2Dδ,\\operatorname\{TV\}\\bigl\(p\_\{f\}\(\\cdot\\mid x,c\_\{0\}\),\\;p\_\{f\}\(\\cdot\\mid r\(Q\(x\)\),c\_\{0\}\)\\bigr\)\\leq 2D\\delta,and, writingmf​\(x,c0\)m\_\{f\}\(x,c\_\{0\}\)for the top\-two probability margin ofpf\(⋅∣x,c0\)p\_\{f\}\(\\cdot\\mid x,c\_\{0\}\)andρf=Pr\{mf\(X,c0\)≤4Dδ\}\\rho\_\{f\}=\\Pr\\\{m\_\{f\}\(X,c\_\{0\}\)\\leq 4D\\delta\\\},A⁡\(f\)≤AQ⋆\+ρf\.A\(f\)\\leq A\_\{Q\}^\{\\star\}\+\\rho\_\{f\}\.

###### Proof\.

For an edgexx–x′x^\{\\prime\}witnessed bycc, the fast path givespf\(⋅∣x,c\)=pf\(⋅∣x′,c\)p\_\{f\}\(\\cdot\\mid x,c\)=p\_\{f\}\(\\cdot\\mid x^\{\\prime\},c\), andV⁡\(f\)≤δV\(f\)\\leq\\deltabounds the movement of each endpoint betweenccandc0c\_\{0\}byδ\\delta; the triangle inequality givesTV\(pf\(⋅∣x,c0\),pf\(⋅∣x′,c0\)\)≤2δ\\operatorname\{TV\}\(p\_\{f\}\(\\cdot\\mid x,c\_\{0\}\),p\_\{f\}\(\\cdot\\mid x^\{\\prime\},c\_\{0\}\)\)\\leq 2\\delta\. Composing along a path of length at mostDDtor⁡\(Q⁡\(x\)\)r\(Q\(x\)\)gives the first bound\. Proposition, applied to the two vectorspf\(⋅∣x,c0\)p\_\{f\}\(\\cdot\\mid x,c\_\{0\}\)andpf\(⋅∣r\(Q\(x\)\),c0\)p\_\{f\}\(\\cdot\\mid r\(Q\(x\)\),c\_\{0\}\), states that a total\-variation distance of at most2​D​δ2D\\deltacannot change the predicted label when the margin exceeds4​D​δ4D\\delta; on the complement of the event\{mf\(X,c0\)≤4Dδ\}\\\{m\_\{f\}\(X,c\_\{0\}\)\\leq 4D\\delta\\\},fftherefore predicts atc0c\_\{0\}the label of theQQ\-based rulex↦Y^f​\(r⁡\(Q⁡\(x\)\),c0\)x\\mapsto\\widehat\{Y\}\_\{f\}\(r\(Q\(x\)\),c\_\{0\}\)\. Propositionwith this rule andρ=ρf\\rho=\\rho\_\{f\}givesPr\[Y^f\(c0\)=Y\]≤AQ⋆\+ρf\\Pr\[\\widehat\{Y\}\_\{f\}\(c\_\{0\}\)=Y\]\\leq A\_\{Q\}^\{\\star\}\+\\rho\_\{f\}, andA⁡\(f\)A\(f\)is at most the accuracy at any single context\. ∎

The approximate bound derives the disagreement rateρ\\rhoof Propositionfrom four quantities: the collision structure, the component diameter, the score tolerance, and the margin tail at the reference context\. A family\-wide statement needs a uniform boundρf≤ρ\\rho\_\{f\}\\leq\\rhoover the fast andδ\\delta\-invariant implementations andα\>AQ⋆\+ρ\\alpha\>A\_\{Q\}^\{\\star\}\+\\rho\. The bound can be loose\. IfDDis large, orAQ⋆\+ρA\_\{Q\}^\{\\star\}\+\\rhoreaches one, it excludes nothing\. These results establish the theorem’s premise for any execution model of the stated form\. They do not identify such a model for the GPU and CPU paths measured in this paper, and the measured mitigation costs indicate that speed, quality, and invariance were not observed to be pairwise incompatible in the tested configurations\. No controlled joint frontier over all three objectives has been measured\.

## Appendix FExtended Limitations

The study isolates each serving factor on a fixed checkpoint and fixed text, and the boundaries below state where its measured magnitudes apply\. The mechanism results are stated in terms of the position rule and the top\-two margin, which do not depend on these boundaries\.

##### Model and task scope\.

We study transformer text classifiers, trained from initialization in the primary grid following the protocol of[Kasa et al\. \(2025\)](https://arxiv.org/html/2610.09111#bib.bib12)and fine\-tuned in the pretrained controls, in four formulations: a discriminative encoder head, a masked\-language\-model label predictor, a GPT\-2 classifier that predicts the next label token, and a GPT\-2 classifier that scores the input under each class\. We focus on conventional fine\-tuning and include LoRA as the parameter\-efficient control on the two modern backbones; in\-context classification with instruction\-tuned language models and other adaptation methods are left for future work, where the same position and margin analysis applies\. The inputs are unimodal text, so classifiers that combine text with tabular, image, or audio features remain to be measured\.

##### Hardware and backend scope\.

Measurements run on one representative GPU \(an NVIDIA L40S\), one representative CPU \(an AMD EPYC\), and two Android phones, with PyTorch eager execution compared against torch\.compile and an ONNX Runtime export\. The reported magnitudes are specific to these configurations\. Production engines such as vLLM and SGLang, tensor\-parallel execution, and other accelerators change the arithmetic through their own kernel choices and require their own measurement, for which the repeatability floor and target\-anchored design transfer unchanged\. The GPU\-fp32 to CPU\-int8 comparison is reported as a whole migration; Appendixseparates its device and quantization components on matched subsets\.

##### Evaluation scope\.

We evaluate on standard independent and identically distributed test splits\. Behavior under distribution shift, where both the baseline predictions and the perturbation sensitivity may differ, is a natural extension\. The balanced grids report how often an effect occurs under a designed set of conditions rather than its incidence in production traffic, and each serving context is executed once, with same\-context run\-to\-run variation measured separately by the repeatability floor\.

##### Mechanism and architecture scope\.

The architecture comparison holds the reference\-code conventions of each family fixed rather than randomizing padding side, position representation, pretraining, and formulation independently\. The Qwen3 rotary control and the direct position\-index intervention measure the two cases that the position theorem separates, and the statements about ALiBi, T5, Shaw, Transformer\-XL, and sinusoidal encodings follow from their published formulas\. The from\-scratch size series scales depth, width, and head count jointly and conditions on all three sizes passing the accuracy threshold, so its trends describe how the effect moves with capacity at matched data and seeds\. The no\-change score summaries are computed per size, and probability total variation provides the scale\-free comparison across independently trained checkpoints alongside raw logit movement\.

##### Pretrained\-model scope\.

The pretrained GPT\-2 experiment establishes padding vulnerability on public checkpoints\. Because changing the position rule only at inference gives a model position inputs it was never trained on, the accuracy threshold excludes that configuration \(it collapses five of fifteen models to a constant prediction: SST\-2 seeds 41 and 42 and SST\-5 seeds 40, 41, and 42\), and the mitigation evidence comes from the three\-seed experiment that trains and serves the same rule, which establishes agreement on the predicted label\. The pretrained replication covers one 124M\-parameter GPT\-2 family and formulation, target\-first batch construction, and five dataset\-level bootstrap clusters; Appendixand the modern\-backbone controls extend the family coverage, and larger pretrained families are the next measurement\.