Reusing Calibration Gains across Model Representations

# Reusing Calibration Gains across Model Representations
Source: [https://arxiv.org/html/2610.22771](https://arxiv.org/html/2610.22771)
###### Abstract

Scale calibration learns an update for one model representation\. We study how much of that update can be reused by another representation of the same pretrained model\. Our transfer procedure extracts positive group\-wise gains and installs them in the recipient’s weights or native quantization scales without recipient\-side optimization\. On a quantized recipient, it preserves integer codes, zero points and storage layout\. For a Qwen3\-1\.7B floating\-point/GPTQ INT3 pair, bidirectional transfer retains 76\.9–90\.5% of the text\-loss improvement from recipient\-specific calibration on fresh WikiText and C4 cohorts excluded from project development\. Shuffling the same gain values within each projection worsens both recipients, showing that their learned group assignment matters after transfer\. A Llama\-3\.2\-3B control keeps the initial native tensors at all four calibrated projections identical and changes nine projections elsewhere\. Cross\-installation retains 98\.6–99\.3% of the text\-calibration benefit, alongside a small, resolved preference for each recipient’s own gain\. These results quantify both the adaptation carried by a reusable calibration update and the remaining benefit of recipient\-specific training\.

## 1Introduction

Calibrating a quantized language model can improve its predictions while leaving its integer codes fixed\. Group\-wise scales provide a compact trainable interface for this purpose, supporting downstream fine\-tuning\[[2](https://arxiv.org/html/2610.22771#bib.bib2)\]and end\-to\-end quantization refinement\[[3](https://arxiv.org/html/2610.22771#bib.bib3)\]\. Once calibration is complete, the learned change can be retained separately from the final checkpoint\. Reusing that change would give another precision or quantization configuration of the same pretrained model an adapted starting point without repeating calibration\.

Aligned weight groups can respond differently to the same gain\. Scaling a quantized group changes different weights from scaling its floating\-point counterpart\. Even identical local weights can operate inside different surrounding networks\. A learned update may therefore carry adaptation useful across representations while fitting its training model more closely\. We distinguish these outcomes by installing the same update in complete recipients and comparing it with the gain learned on each recipient\.

We use dimensionless, group\-wise gains as that shared description\. On a floating\-point model, a gain multiplies a weight group\. On an affine\-quantized model, the same multiplication can be absorbed into the group’s native scale\. A quantized donor supplies its gain through the ratio of final to initial scales, while a floating\-point donor supplies the multipliers saved during calibration\. This construction moves the learned change without replacing the recipient’s original representation\. It produces a calibrated recipient with no further optimizer steps and no additional inference\-time operation\.

We evaluate transfer in two crossed experiments, with a common training recipe and data stream for the donors within each experiment\. On a Qwen3\-1\.7B floating\-point/INT3 pair, installed gains retain 76\.9–90\.5% of the negative log\-likelihood \(NLL\) reduction from self\-calibration on fresh cohorts excluded from project development\. Earlier development evaluations give a similar range of 78\.0–90\.2%\. Both recipients obtain an additional text benefit from their own gains\. Shuffling gains within each projection raises text loss, linking the transfer benefit to the learned assignment of values to groups\.

The Llama\-3\.2\-3B experiment separates local representation differences from surrounding\-model differences\. Its two configurations have identical initial tensors at all four calibrated projections but differ at nine projections elsewhere\. Cross\-installation preserves 98\.6–99\.3% of their text\-calibration benefit, while each recipient’s own gain improves perplexity by a further 0\.08–0\.13%\. The control exposes a small context\-dependent residual after nearly complete reuse\. PIQA supplies a separate task readout whose ordering does not uniformly follow text loss\.

## 2Transferring a calibration update

### 2\.1A common multiplicative interface

Consider representations of the same pretrained model with aligned projection names and weight\-group coordinates\. We first describe transfer between a floating\-point model F and a quantized model Q; the same scale operation applies between two quantized configurations\. LetW0,gFW^\{F\}\_\{0,g\}denote an original floating\-point weight group\. A positive gainaga\_\{g\}produces the calibrated group

WgF=ag​W0,gF\.W^\{F\}\_\{g\}=a\_\{g\}W^\{F\}\_\{0,g\}\.
For the corresponding quantized group, integer codescgc\_\{g\}, zero pointzgz\_\{g\}and positive scalesgs\_\{g\}decode to

WgQ=sg​\(cg−zg\)\.W^\{Q\}\_\{g\}=s\_\{g\}\(c\_\{g\}\-z\_\{g\}\)\.
Multiplying the scale byaga\_\{g\}gives

\(ag​sg\)​\(cg−zg\)=ag​WgQ\.\(a\_\{g\}s\_\{g\}\)\(c\_\{g\}\-z\_\{g\}\)=a\_\{g\}W^\{Q\}\_\{g\}\.
The same gain therefore acts on both representations through their existing parameters\. This is the scale parameterization used in quantization\-based adaptation\[[2](https://arxiv.org/html/2610.22771#bib.bib2),[3](https://arxiv.org/html/2610.22771#bib.bib3)\]\. Its role here is to provide a transferable update: the recipient keeps its own base weights, and the donor specifies how strongly to scale each aligned group\.

Alignment refers to the original weight coordinates, not merely to matching tensor dimensions\. In our GPTQ representation, each group spans 128 consecutive original input columns in one output row\. The group mapping retained by activation\-ordered quantization is respected during installation\. All parameters outside the selected groups remain those of the recipient\. Exact absorption into one recipient scale requires a common gain across the nonzero decoded weights controlled by that scale\. A coarser or differently arranged recipient grouping therefore requires a compatibility check beyond matching tensor shapes\.

### 2\.2Extracting and installing the gain

The floating\-point calibration path parameterizes gains asexp⁡\(θ\)\\exp\(\\theta\)and stores their final values\. We denote the saved gain vector byaFa\_\{F\}\. For a quantized donor, we extract the gain from its original and calibrated native scales:

aQ=sQ,final/sQ,0\.a\_\{Q\}=s\_\{Q,\\mathrm\{final\}\}/s\_\{Q,0\}\.
The ratio removes the donor’s initial scale and retains its learned relative change\. It is defined even when some decoded weights are zero, unlike a ratio of individual final and initial weights\. For two quantized configurations with identical initial scales at the selected groups, transferring the relative gain reduces to copying the donor’s final scales into the recipient\. The recipient still supplies the rest of the model\.

Transfer then consists of two possible installations:

F←Q:WgF=aQ,g​W0,gF,Q←F:sgQ=aF,g​s0,gQ\.F\\leftarrow Q:\\quad W^\{F\}\_\{g\}=a\_\{Q,g\}W^\{F\}\_\{0,g\},\\qquad Q\\leftarrow F:\\quad s^\{Q\}\_\{g\}=a\_\{F,g\}s^\{Q\}\_\{0,g\}\.
We promote the stored FP32 operands to FP64 for construction and round each installed result to FP32 once\. Applying each extracted gain to its own original base reconstructs the saved calibrated endpoint exactly in our implementation\. This self\-installation check ensures that the transferred object is the update actually learned by the donor\. Numerical details are given in Appendix A\.

The resulting procedure takes an original recipient, a calibrated donor and the donor’s original scales or saved gains\. It aligns the groups, extracts the relative update and multiplies it into the recipient’s weights or scales\. The output is a materialized recipient model\. For Q, the code tensors, zero points, group layout and scale slots are retained\. The installed model requires no separate gain application during inference\.

Installation usesO⁡\(G\)O\(G\)scale arithmetic for G quantized groups, orO⁡\(N\)O\(N\)multiplications to materialize N selected floating\-point weights\. The Qwen gain vector has 393,216 entries, occupying 1\.5 MiB in FP32 before serialization metadata; the Llama vector has 786,432 entries\. Reuse presupposes an existing donor and also incurs model\-loading and file\-I/O costs\. The quantized output occupies the recipient’s existing scale slots\. Appendix F separates CPU arithmetic, identity checks and serialization for construction of the four\-projection update payloads\.

## 3Evaluation design

### 3\.1Cross\-precision donors

We use the public Qwen3\-1\.7B checkpoint\[[5](https://arxiv.org/html/2610.22771#bib.bib5),[15](https://arxiv.org/html/2610.22771#bib.bib15)\], which includes pretraining and post\-training, as F and its native activation\-ordered, static\-group GPTQ INT3 quantization\[[6](https://arxiv.org/html/2610.22771#bib.bib6)\]as Q\. Both calibration paths update the down projections in layers 6, 13, 20 and 27, the endpoints of four equal\-depth partitions\. Group size 128 gives 393,216 trainable log\-gains\. The other 192 projections and all embedding, normalization, bias and output\-head parameters retain their recipient\-specific original values\. Q’s integer codes and zero points are fixed throughout training and transfer\.

The donors use the same 600\-update recipe\. Each update combines eight ARC items\[[10](https://arxiv.org/html/2610.22771#bib.bib10)\], one WikiText block\[[7](https://arxiv.org/html/2610.22771#bib.bib7)\]and one C4 block\[[8](https://arxiv.org/html/2610.22771#bib.bib8)\]\. The loss is

ℒ=λA​ℒARC\+λW​ℒWiki\+λC​ℒC4,\\mathcal\{L\}=\\lambda\_\{A\}\\mathcal\{L\}\_\{\\mathrm\{ARC\}\}\+\\lambda\_\{W\}\\mathcal\{L\}\_\{\\mathrm\{Wiki\}\}\+\\lambda\_\{C\}\\mathcal\{L\}\_\{\\mathrm\{C4\}\},
where the ARC term is the Kullback\-Leibler divergenceDKL\(pF​0∥pθ\)D\_\{\\mathrm\{KL\}\}\(p\_\{F0\}\\\|p\_\{\\theta\}\), from the original floating\-point teacher’s option distribution to the calibrated model’s distribution, and the text terms are next\-token losses\. ARC uses a plain\-text question\-and\-options prompt followed by a bare option label\. The saved labels are single tokens without a leading space; their conditional probabilities are renormalized over the valid options, with temperature 1 and no length normalization\. Appendix A\.4 gives the exact scoring rule and a prompt example for both model families\.

The loss weights are 0\.080624, 0\.406788 and 0\.512588, respectively\. They were determined on the quantized path and reused on F\. Both paths use Adam\[[11](https://arxiv.org/html/2610.22771#bib.bib11)\], initially zero moments, learning rate 0\.01 for 200 updates and a linear decrease to 0\.001 over the next 400\. Optimizer moments and data cursors continue across this boundary\. The common recipe holds the training intervention fixed while allowing each base to learn its own gain\. Appendix A gives the optimizer and data details\.

### 3\.2Separating transfer benefit from source preference

Each complete recipient is evaluated without calibration, with its own gain and with the other donor’s gain\. The two transferred states are fixed before evaluation\. The crossed study adds no training or checkpoint selection\.

LetL⁡\(B,a\)L\(B,a\)be the loss of recipient B with installed gain a, and leta=1a=1denote the uncalibrated model\. Transfer benefit is measured byL⁡\(Q,aF\)−L⁡\(Q,1\)L\(Q,a\_\{F\}\)\-L\(Q,1\)andL⁡\(F,aQ\)−L⁡\(F,1\)L\(F,a\_\{Q\}\)\-L\(F,1\)\. To compare gain sources while holding the recipient fixed, we use

dQ=L⁡\(Q,aQ\)−L⁡\(Q,aF\),dF=L⁡\(F,aQ\)−L⁡\(F,aF\)\.d\_\{Q\}=L\(Q,a\_\{Q\}\)\-L\(Q,a\_\{F\}\),\\qquad d\_\{F\}=L\(F,a\_\{Q\}\)\-L\(F,a\_\{F\}\)\.
Both differences use the convention Q\-source minus F\-source\. ThusdQ<0d\_\{Q\}<0anddF\>0d\_\{F\}\>0indicate that each recipient favors its own gain\. We also report the interactionℐ=dQ−dF\\mathcal\{I\}=d\_\{Q\}\-d\_\{F\}, which measures how the source effect changes between recipients\. The two direct differences determine which source each recipient favors\.

To express the practical size of transfer, we divide its NLL reduction by the reduction from self\-calibration on the same recipient\. For Q, this retained\-benefit ratio is

rQ=L⁡\(Q,1\)−L⁡\(Q,aF\)L⁡\(Q,1\)−L⁡\(Q,aQ\),r\_\{Q\}=\\frac\{L\(Q,1\)\-L\(Q,a\_\{F\}\)\}\{L\(Q,1\)\-L\(Q,a\_\{Q\}\)\},
with the analogous definition for F\. This is a descriptive ratio of observed mean losses\. Paired loss differences, rather than the ratios, provide the statistical comparisons\.

### 3\.3Controlling the local starting representation

The Qwen comparison changes both local weights and the surrounding network\. To isolate the latter, we use two existing Llama\-3\.2\-3B\[[12](https://arxiv.org/html/2610.22771#bib.bib12)\]INT3/g128 configurations, denoted S and J\. Each of their 196 projections selects a complete native encoding from GPTQ with 1% damping, round\-to\-nearest quantization, or GPTQ with 0\.1% damping\. The configurations differ at nine projections, all outside the four calibrated down projections in layers 6, 13, 20 and 27\. Those four projections have identical initial codes, scales and zero points: GPTQ at layers 6, 20 and 27, and round\-to\-nearest at layer 13\.

Both donors calibrate the same 786,432 scales for 200 Adam updates, using a common data stream, objective and learning rate\. The saved gains are installed in both complete recipients\. Because their local initial scales agree, cross\-installation copies the donor’s saved FP32 scales exactly\. The other 192 projections remain those of the recipient\. The four states therefore vary the surrounding configuration and gain source while holding the local starting representation fixed\.

For each recipient we report cross\-gain loss minus own\-gain loss\. Positive differences on both rows establish an own\-gain preference separately for S and J\. Their sum measures the matching interaction\. This experiment examines context dependence between quantized configurations; the Qwen experiment measures bidirectional transfer across precision\.

### 3\.4Data and readouts

The primary Qwen evidence is a fresh\-text confirmation with 256 WikiText blocks from 139 article clusters and 256 C4 blocks, selected after the development studies while keeping the six model states fixed\. Section 3\.6 describes its construction\. Two earlier development evaluations, archived as REPLAY and VALIDATION, contain 255/255 WikiText blocks from 121/113 article clusters and 261/260 C4 blocks, respectively\. Each text block has 2,048 input tokens and 2,047 prediction positions\. The original training pool contains 264 WikiText blocks and 260 C4 blocks, with source documents disjoint from the evaluation sets\. Appendix A\.2 separates training from the smaller FIT pool used for objective weighting and endpoint diagnostics\.

The Llama control uses 256 WikiText blocks from 114 article clusters, 256 C4 blocks and 569 ARC items\. ARC measures KL to the fixed floating\-point teacher’s option distribution\. All four own\-gain and cross\-gain models are reevaluated in one run on the same GPU, using their saved native tensors and unchanged inputs\. Table 3 uses these complete contemporaneous loss vectors\. Appendix E compares them with the historical measurements and gives the complete states\. These Llama data and the two original Qwen text sets were used during development; the fresh Qwen confirmation is described in Section 3\.6\. Every crossed comparison fixes the donor checkpoints before scoring\.

PIQA\[[9](https://arxiv.org/html/2610.22771#bib.bib9)\]supplies a separate task\-accuracy measure on 1,838 questions\. We score full\-answer continuations and report character\-length\-normalized accuracy as the primary metric, alongside unnormalized accuracy\. Llama’s own\-gain and cross\-gain states are evaluated together for this task\. The task readout tests whether the text\-loss preference also appears in discrete answer decisions\.

Evaluation uses FP32 computation, scaled dot\-product attention \(SDPA\) and disabled TensorFloat\-32 \(TF32\)\. All differences are paired on the same questions or text units\. We use 10,000 paired bootstrap draws for pointwise 95% intervals: WikiText resamples articles with retained\-block weighting, C4 resamples blocks and PIQA resamples questions\. The intervals quantify variation across these evaluation units for the fixed models and training trajectories\. Appendix H checks sensitivity to adjacent\-block dependence in the fresh Qwen and Llama C4 comparisons\.

### 3\.5Testing group\-level correspondence

We test whether transferred gains depend on their correspondence with the recipient’s weight groups\. For each of the four Qwen projections, we flatten the 98,304 row\-group gains and apply one permutation drawn with PCG64 and seed 20261014\. Both donors use the same permutation at corresponding projections\. This intervention preserves each projection’s exact gain multiset, parameter count and log\-gain norm while changing which group receives each value\. The resulting weight\-displacement norm can change because the recipient groups have different weights\.

In each transfer direction, the aligned and shuffled 600\-update gains are installed and evaluated together on the same GPU, using the REPLAY development text set\. The contrast is shuffled minus aligned NLL, paired over the same units as Section 3\.4\. The permutation is fixed before scoring, with no training or checkpoint selection\. Its paired intervals describe evaluation\-unit variation for this fixed intervention, rather than variation across permutation seeds\.

### 3\.6Fresh\-text confirmation with fixed donors

After the development comparisons, we evaluate the same six Qwen states on newly selected text without changing the donors, gains or installation\. The selection rule and token arrays are fixed before model evaluation\. WikiText contributes 256 blocks from 139 articles, excluding recorded project\-consumed articles and the official validation and test articles by title and normalized content\. Eligible articles are ordered by a salted hash, and complete blocks are retained within articles until the fixed total is reached\. C4 contributes 256 blocks from validation shard 7 after excluding normalized documents and nonempty URLs from the previously used shards 0–6\. We preserve source order and the original two\-newline document separator when packing C4\. Table 1 gives the six means and Appendix G gives the source details\.

This confirmation tests transfer and source preference on text not used to develop the calibration recipe or inspect the earlier transfer results\. It adds no optimization, donor selection or gain\-strength adjustment\. We retain the same paired estimands, with 139 article clusters for WikiText and 256 blocks for C4, and report these cohorts separately from the development sets\.

## 4Results

### 4\.1Existing gains improve both recipients without retraining

Transfer improves text prediction in both directions on the fresh cohorts \(Table 1\)\. The floating\-point gain reduces the quantized recipient’s WikiText perplexity by 42\.408% and its C4 perplexity by 21\.608%\. The reverse transfer reduces the floating\-point recipient’s perplexities by 36\.658% and 19\.059%\. All four paired NLL differences have intervals below zero \(Table 2\)\.

Table 1:Six fixed Qwen states on fresh text: 256 WikiText blocks from 139 articles and 256 C4 blocks\. Perplexity is exp\(mean NLL\)\. Each recipient retains its original parameters outside the four calibrated projections\. Gain\-bearing states use the original 600\-update donors; None denotes the original uncalibrated base\.Transfer recovers more of the self\-calibration benefit on WikiText than on C4\. For Q, the floating\-point gain retains 90\.51% and 76\.89% of the self\-calibration NLL reduction on WikiText and C4\. For F, the quantized gain retains 90\.07% and 78\.99%\.

The quantized donor’s gain improves the unquantized model, and direct calibration of that floating\-point model also yields a large text improvement\. The transferred gain thus carries adaptation useful beyond correcting the donor’s quantization error\.

The earlier development evaluations show the same transfer pattern\. REPLAY retains 78\.0–90\.2% of self\-calibration benefit, close to the fresh\-text range\. Appendix B gives its complete means and Appendix G reports its paired comparisons alongside the second development set\.

### 4\.2Recipient\-specific calibration captures an additional benefit

Figure 1 shows that both recipients attain their lowest fresh\-text NLL with their own gains\. Table 2 holds each recipient fixed while changing the gain source\. Q favors its own gain, giving negative Q\-source\-minus\-F\-source differences; F favors its own gain, giving positive differences\. All four source\-preference intervals exclude zero\.

Figure 1:Gain reuse and source preference on fresh text\. The upper panels show NLL for the six fixed states in Table 1\. Lines connect gain choices, not training steps\. The lower panels compare Q\-source and F\-source gains on each recipient, with pointwise 95% paired intervals\. Negative values favor Q\-source gains and positive values favor F\-source gains\.Table 2:Paired effects on the fresh cohorts in Table 1\. The first two rows compare transferred gains with no calibration\. The next two hold the recipient fixed and change the gain source\. Their difference isℐ\\mathcal\{I\}\. All intervals are pointwise 95%\.The interaction is−\-0\.108193 NLL on WikiText and−\-0\.129393 on C4\. The two development sets give the same source preferences \(Appendix G\)\. Appendix H’s consecutive\-block sensitivity analysis preserves the C4 preferences while allowing neighboring blocks to be resampled together\.

### 4\.3Source preference survives identical local starting tensors

The Llama control shows that the surrounding configuration affects gain compatibility\. Table 3 holds each complete recipient fixed and compares its own gain with the other donor’s gain\. Both recipients favor their own gains on all three losses, with each paired interval above zero\.

Table 3:Llama source preference with identical initial tensors at all four calibrated projections\. All four states are measured in the same GPU run\. Positive values favor the recipient’s own gain\. Each interval is a pointwise 95% paired interval; ARC KL is a teacher\-distribution loss, not answer accuracy\.Cross\-installation nevertheless retains nearly all of the observed text\-calibration benefit: 98\.77% on WikiText and 98\.62% on C4 for S, and 99\.25% and 98\.62% for J\. These descriptive ratios combine the archived uncalibrated baselines with the four contemporaneously evaluated calibrated states in Appendix E\. The residual own\-gain advantage is approximately 0\.08–0\.13% in perplexity\.

The matching interaction is \+0\.010631 \[0\.008917, 0\.012355\] for ARC KL, \+0\.002088 \[0\.001976, 0\.002208\] for WikiText NLL and \+0\.002235 \[0\.001932, 0\.002648\] for C4 NLL\. Because the calibrated projections start with identical native tensors, this preference reflects differences elsewhere in the model\. In this control, detectable recipient dependence coexists with almost complete reuse of the text benefit\.

### 4\.4Task accuracy gives a distinct readout of transfer

On PIQA, the quantized recipient’s own gain raises primary accuracy from 68\.226% to 70\.131%, an improvement of 1\.904 percentage points \[0\.598, 3\.210\]\. Transferring the floating\-point gain reaches 69\.260%, an improvement of 1\.034 points \[−\-0\.163, 2\.285\]\. The direct advantage of its own gain is 0\.871 points \[−\-0\.109, 1\.850\]\. Table 4 reports both readouts for all six states\.

Table 4:Task accuracy of the fixed Qwen states on the original 1,838 PIQA questions\. Character\-length\-normalized accuracy is primary; raw accuracy uses summed continuation log probability\. This task readout is separate from the fresh\-text confirmation\. Appendix B gives the correct counts\.On the floating\-point recipient, the transferred quantized gain has the highest primary accuracy in Table 4, despite its higher text losses than self\-calibration\. Its advantage over the floating\-point gain is 0\.163 points \[−\-0\.707, 1\.034\]\. The accuracy interaction is 0\.707 points \[−\-0\.598, 1\.959\], so the text source preference has no resolved counterpart in the primary PIQA interaction\.

Answer normalization changes the observed direction of transfer on Q\. The floating\-point gain raises its character\-normalized accuracy but changes raw accuracy from 67\.791% to 67\.628%\. On F, the quantized gain raises raw accuracy from 72\.416% to 73\.667%\. These readouts give a task assessment distinct from the consistent text\-loss preference for each recipient’s own gain\.

For Llama, cross\-gain minus own\-gain normalized PIQA accuracy is−\-0\.054 points \[−\-0\.435, 0\.326\] on S and \+0\.163 points \[−\-0\.272, 0\.598\] on J\. Neither comparison resolves an own\-gain advantage\. Under raw scoring, cross\-installation on J loses 14 correct answers, or−\-0\.762 points \[−\-1\.251,−\-0\.326\]\. Appendix E reports the complete counts under both readouts\.

### 4\.5Transfer depends on group\-level correspondence

The permutation control separates learned placement from the distribution of gain values\. On the REPLAY development cohort, shuffling gains raises loss in both transfer directions and both text domains \(Table 5\)\. For floating\-point gains installed on Q, perplexity rises by 81\.46% on WikiText and 32\.60% on C4 relative to aligned transfer\. For quantized gains installed on F, the increases are 62\.49% and 24\.99%\. All four paired NLL intervals lie above zero\.

Table 5:Group\-level correspondence in the transferred 600\-update gains on development text\. Aligned and shuffled states are evaluated on the same GPU\. Each projection retains exactly the same gain values under the fixed permutation; positive differences favor their learned assignment to weight groups\.The aligned reruns reproduce the historical crossed states to within9\.55×10−79\.55\\times 10^\{\-7\}NLL per block, far below the observed permutation effects\. With the per\-layer gain distribution and log\-gain norm held fixed, reassignment worsens text loss in both directions\. The learned group assignment therefore remains useful after the recipient’s precision changes\.

### 4\.6Transfer benefit and representation gap are different quantities

The six states also allow the two representations to be compared under a common gain\. DefineG⁡\(a\)=L⁡\(Q,a\)−L⁡\(F,a\)G\(a\)=L\(Q,a\)\-L\(F,a\)\. The interaction in Table 2 is exactly

ℐ=G⁡\(aQ\)−G⁡\(aF\)\.\\mathcal\{I\}=G\(a\_\{Q\}\)\-G\(a\_\{F\}\)\.
Thus the interaction and the change in representation gap are two readings of the same crossed comparison\. Adding the uncalibrated gapG⁡\(1\)G\(1\)shows whether a common gain brings the two recipients closer together \(Table 6\)\.

Table 6:Descriptive representation gaps G\(a\) on fresh text, computed from Table 1’s unrounded means\. Both recipients receive the same gain in each row\.On C4, the common floating\-point gain increases the gap from 0\.277428 to 0\.301667 while improving both recipients: F benefits more in absolute NLL\. The common quantized gain reduces the gap to 0\.172273\. Transfer benefit and the fraction of self\-calibration benefit retained therefore measure something different from closing the loss gap between precision versions\.

## 5Related work

Retaining an adaptation separately from its base model is useful beyond quantization\. Task Arithmetic represents fine\-tuning by an additive weight difference that can be combined or moved in weight space\[[13](https://arxiv.org/html/2610.22771#bib.bib13)\]\. IA3 learns vectors that scale activations for parameter\-efficient adaptation\[[14](https://arxiv.org/html/2610.22771#bib.bib14)\]\. Our update is a positive, dimensionless gain on aligned weight groups, extracted from saved calibration and absorbed into each recipient’s own weights or native scales\. Its coordinate system makes the same relative operation available on floating\-point and integer representations; the crossed experiments measure how much of its benefit survives that change of recipient\.

Scale\-based adaptation makes quantization parameters trainable without replacing the integer representation\. PEQA adapts compressed models by learning their scales\[[2](https://arxiv.org/html/2610.22771#bib.bib2)\]\. EfficientQAT combines block\-wise training with end\-to\-end optimization of quantization step sizes\[[3](https://arxiv.org/html/2610.22771#bib.bib3)\]\. These methods motivate the calibration interface used here\. We study a different operation on its output: extracting a learned relative gain and installing it on another aligned precision version\. GPTQ provides our quantized base\[[6](https://arxiv.org/html/2610.22771#bib.bib6)\], whose codes, zero points and group layout are held fixed during both calibration and transfer\.

Reuse across quantization configurations is also the goal of configuration\-aware adaptation\. CoA\-LoRA predicts low\-rank adjustments for different per\-layer configurations through a trained configuration\-aware model\[[4](https://arxiv.org/html/2610.22771#bib.bib4)\]\. Direct gain transfer uses a fixed update and the recipient’s existing weights or scales, without training a configuration\-to\-update mapping\. Our crossed comparisons measure the benefit and recipient dependence of that direct operation\. The Llama control additionally keeps the adapted projections’ initial tensors identical, separating surrounding\-model dependence from a change in those local starting tensors\.

## 6Discussion

When a calibrated donor is available, direct transfer provides an adapted recipient without repeating optimization\. The quantized recipient keeps its existing encoding format and inference operations\. In the Qwen comparison, transfer recovers most of the self\-calibration text benefit, while recipient\-specific calibration yields a further improvement\. Reuse still requires model loading and installation; Appendix F separates those construction costs from training and research evaluation\.

The controls distinguish two requirements for reuse\. Group alignment makes installation well defined, and the permutation experiment shows that the donor’s assignment of gain values to groups matters for the transferred benefit\. The surrounding model affects which gain fits best, as the Llama control demonstrates with identical local starting tensors\. Compatibility can therefore be checked at the coordinate level for installation, but the resulting quality must be measured on the complete recipient\.

Update placement and the evaluation interface also affect calibration outcomes\. At Qwen’s 200\-update checkpoints, removing one layer’s gain gives higher text loss than shrinking all four gains to the same total log\-gain norm \(Appendix C\)\. The 200\-to\-600 checkpoint comparison changes direction across ARC label and chat readouts \(Appendix D\)\. These observations, together with the PIQA results, motivate evaluating a reused gain on the recipient’s intended task\.

The evidence covers one Qwen floating\-point/GPTQ pair and one Llama pair of quantized configurations, each with four calibrated projections and fixed training trajectories\. The matching gaps compare saved gains learned under a common recipe\. They measure the extra benefit of those recipient\-specific gains, without estimating the best calibration achievable on each recipient\.

## 7Conclusion

Group\-wise gains let a learned calibration be installed in another aligned representation without recipient\-side training\. On fresh Qwen3\-1\.7B text cohorts, bidirectional transfer retains 76\.9–90\.5% of the self\-calibration benefit\. The permutation control shows that the benefit depends on the learned group assignment, and the Llama control shows that gain preference depends on the surrounding model even when local starting tensors agree\. Saving the gains separately provides a reusable update that fits the recipient’s existing parameterization\.

## References

- \[2\]Jeonghoon Kim, Jung Hyun Lee, Sungdong Kim, Joonsuk Park, Kang Min Yoo, Se Jung Kwon, and Dongsoo Lee\.*Memory\-Efficient Fine\-Tuning of Compressed Large Language Models via sub\-4\-bit Integer Quantization*\. NeurIPS 2023\.[Paper](https://arxiv.org/abs/2305.14152)\.
- \[3\]Mengzhao Chen, Wenqi Shao, Peng Xu, Jiahao Wang, Peng Gao, Kaipeng Zhang, and Ping Luo\.*EfficientQAT: Efficient Quantization\-Aware Training for Large Language Models*\. ACL 2025, pp\. 10081–10100\.[Paper](https://aclanthology.org/2025.acl-long.498/)\.
- \[4\]Rongguang Ye, Ming Tang, and Edith C\. H\. Ngai\.*On\-the\-Fly Adaptation to Quantization: Configuration\-Aware LoRA for Efficient Fine\-Tuning of Quantized LLMs*\. arXiv:2509\.25214, version 4, 2026\.[Paper](https://arxiv.org/abs/2509.25214v4)\.
- \[5\]Qwen Team\.*Qwen3 Technical Report*\. arXiv:2505\.09388, 2025\.[Paper](https://arxiv.org/html/2505.09388v1)\.
- \[6\]Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh\.*GPTQ: Accurate Post\-Training Quantization for Generative Pre\-trained Transformers*\. ICLR 2023\.[Paper](https://arxiv.org/abs/2210.17323)\.
- \[7\]Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher\.*Pointer Sentinel Mixture Models*\. arXiv:1609\.07843, 2016\.[Paper](https://arxiv.org/abs/1609.07843)\.
- \[8\]Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J\. Liu\.*Exploring the Limits of Transfer Learning with a Unified Text\-to\-Text Transformer*\. Journal of Machine Learning Research, 21\(140\):1–67, 2020\.[Paper](https://www.jmlr.org/papers/v21/20-074.html)\.
- \[9\]Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi\.*PIQA: Reasoning about Physical Commonsense in Natural Language*\. arXiv:1911\.11641, 2019\.[Paper](https://arxiv.org/abs/1911.11641)\.
- \[10\]Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord\.*Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge*\. arXiv:1803\.05457, 2018\.[Paper](https://arxiv.org/abs/1803.05457)\.
- \[11\]Diederik P\. Kingma and Jimmy Ba\.*Adam: A Method for Stochastic Optimization*\. ICLR 2015\.[Paper](https://arxiv.org/abs/1412.6980)\.
- \[12\]Meta\.*Llama 3\.2 Model Card*\. 2024\.[Model card](https://huggingface.co/meta-llama/Llama-3.2-3B)\.
- \[13\]Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi\.*Editing Models with Task Arithmetic*\. ICLR 2023\.[Paper](https://arxiv.org/abs/2212.04089)\.
- \[14\]Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin Raffel\.*Few\-Shot Parameter\-Efficient Fine\-Tuning is Better and Cheaper than In\-Context Learning*\. NeurIPS 2022\.[Paper](https://arxiv.org/abs/2205.05638)\.
- \[15\]Qwen Team\.*Qwen3\-1\.7B Model Card*\. 2025\.[Model card](https://huggingface.co/Qwen/Qwen3-1.7B)\.

## Appendix AImplementation and calibration details

### A\.1Native installation

The installer reads the actual saved FP32 gains and native scales\. For the quantized donor, both scale tensors are promoted to FP64 before forming the ratio\. For the floating\-point donor, the saved FP32 gain is promoted directly\. Multiplication uses FP64, followed by one FP32 cast of each installed weight or scale\. The implementation checks scale positivity and finiteness, preserves codes and zero points, and matches original input\-column groups\. Self\-installation reproduces the saved source endpoints, including their unchanged model context\.

The decoding identity in Section 2 holds over real numbers\. The stated construction specifies the numerical realization used in this study\. The recipient’s complete context is retained: installing a gain into F leaves all unmodified projections floating\-point, while installing into Q leaves them quantized\. Group shape alone is insufficient to establish compatibility when a quantizer uses a different coordinate mapping\.

A common gain also need not produce the same weight displacement on the two bases\. For one aligned group and scalar gain a,

Δ​WF=\(a−1\)​WF,0,Δ​WQ=\(a−1\)​WQ,0,\\Delta W\_\{F\}=\(a\-1\)W\_\{F,0\},\\qquad\\Delta W\_\{Q\}=\(a\-1\)W\_\{Q,0\},
and hence

Δ​WQ−Δ​WF=\(a−1\)​\(WQ,0−WF,0\)\.\\Delta W\_\{Q\}\-\\Delta W\_\{F\}=\(a\-1\)\(W\_\{Q,0\}\-W\_\{F,0\}\)\.
This identity describes the difference in the local weight updates\. The complete models also differ in their surrounding layers\. Source preference in Section 4 is measured on those complete recipients\.

The 393,216\-entry gain vector contains 1,572,864 bytes in FP32\. Storing the extracted ratios in FP64 uses 3 MiB, excluding serialization metadata\. Once installed on Q, the values occupy the original scale slots\. The crossed study’s formal runtime was 759\.211 seconds, including new evaluations and surrounding work\. That figure is research execution cost, not an installation\-latency benchmark\.

### A\.2Training recipe

Adam uses betas \(0\.9, 0\.999\), epsilon 1e−\-8 and zero weight decay\. The exact objective weights are 0\.08062415495688265 for ARC, 0\.40678819283872103 for WikiText and 0\.5125876522043963 for C4\. The original quantized\-path weights are reused on F, including the ARC weight\. F initially matches its fixed FP0 teacher, so recomputing inverse\-gradient\-norm weights at that point would define a different recipe\.

The saved 600\-update donors were originally trained on 592 ARC items, 264 WikiText blocks and 260 C4 blocks\. Each update combines eight ARC items and one block from each training text pool\. The common FIT pool contains the same ARC and C4 inputs but a 29\-block WikiText subset\. It supplies the original quantized\-path objective weighting and common endpoint diagnostics\. The saved WikiText sampler cursors record two traversals of the 264\-block training pool plus 72 blocks\. Both paths use sampler seed 20261012 and preserve their optimizer moments and cursors across the 200\-update boundary\.

Table A1:Qwen data roles\. All text blocks contain 2,048 tokens\. Common FIT’s WikiText inputs are included in training\. The two development evaluation sources had been inspected before the transfer study; fresh confirmation excludes project\-consumed sources and uses 139 WikiText articles\. PIQA’s 1,838 questions are a separate development task readout\.
The expanded training pool combines an original WikiText prefix covering 81 articles with a continuation covering 103 other articles\. Their article identifiers and normalized content identities are disjoint from the 114 source articles used for VALIDATION and the 121 used for REPLAY; the two evaluation sources are also disjoint\. Retokenization with Qwen leaves complete scoring blocks in 113 of the 114 VALIDATION articles\. C4 training/FIT, VALIDATION and REPLAY draw on 1,244, 1,128 and 1,145 source documents, respectively, with no shared recorded content identities\. Exact token\-block intersections between training and evaluation are empty\. These checks establish source isolation within the project, rather than absence from the base model’s pretraining corpus\.

The models and recipe originated in earlier development experiments\. The four projection positions follow⌈28​k/4⌉−1\\lceil 28k/4\\rceil\-1fork=1,…,4k=1,\\ldots,4\. The quantized study compared fixed 600\-update learning\-rate schedules, and the present donor retains its predesignated linear\-decay endpoint even though another schedule achieved lower text loss in that earlier study\. The floating\-point donor then used this same recipe\. Both text evaluation sets had already been inspected during development before the six\-state transfer study\. The transfer and permutation experiments fix the donor endpoints before new scoring and make no additional checkpoint or gain\-source selection\.

The floating\-point weights and tokenizer come from Qwen/Qwen3\-1\.7B revision[70d244cc](https://huggingface.co/Qwen/Qwen3-1.7B/tree/70d244cc86ccca08cf5af4e1e306ecf908b1ad5e)\. Text evaluation uses ordinary token sequences without a chat or thinking template\. The native Q base uses asymmetric INT3, group size 128, GPTQ block size 128, damping 0\.01, activation ordering and static original\-column groups, without MSE clipping\. The configuration and saved payload identities are provided with the reproducibility material\.

### A\.3Paired estimates

Bootstrap resampling uses 10,000 draws with seed 20261014\. A WikiText article contributes all of its retained blocks when selected, and the mean is weighted by the resulting block count\. C4 uses block resampling\. PIQA uses shared question indices across states\. Interactions are formed within each evaluation unit before averaging and resampling\. All reported intervals are pointwise 95% intervals\.

The retained\-benefit ratios divide transfer NLL reduction by self\-calibration NLL reduction on the same recipient and domain\. They are functions of the observed means, not estimates of a causal repair fraction\. The two development text sets had been consumed during development; the fresh cohorts in Section 3\.6 were excluded from that use\. All three six\-state comparisons use the same fixed checkpoints\.

The Llama cross\-installation analysis uses 10,000 paired bootstrap draws with seed 20261004\. WikiText resamples its 114 article clusters with retained\-block weighting; C4, ARC and PIQA use paired blocks or questions\. Its matching interaction is formed per evaluation unit before resampling\. The models and data differ from the Qwen study, so their numerical effects are reported separately\.

The Qwen permutation control uses 10,000 paired draws with seed 20261014, resampling REPLAY’s 121 WikiText article clusters or 261 C4 blocks\. All four fixed states are rescored on one RTX 4080 SUPER with PyTorch 2\.8\.0\+cu128, Transformers 5\.9\.0, FP32/SDPA and TF32 disabled\. The two aligned states preserve the original crossed model tensors and token sequences\. Their largest absolute mean\-NLL difference from the archived evaluation is3\.04×10−83\.04\\times 10^\{\-8\}\. This comparison checks the rerun; Table 5’s intervention effects use the contemporaneous aligned states\.

### A\.4ARC option distributions

Both calibration studies use the same plain\-text question\-and\-options format\. One saved example is:

> ``` Question: Which technology was developed most recently? A. cellular telephone B. television C. refrigerator D. airplane Answer with only one label from A, B, C, D. Answer: ```

The candidates for this item are the separate continuations A, B, C and D, without spaces or newlines\. Other items retain their original A–E or 1–4 labels\. Prompts are encoded with special tokens enabled and continuations independently with special tokens disabled, then concatenated as token sequences\. The Llama prompt begins with BOS token 128000; the Qwen tokenizer adds no BOS under this call\. Both tokenizers encode every stored candidate label as one token\.

For prompt x and candidate token sequenceycy\_\{c\}, the implementation computes

zc=∑t=1\|yc\|log⁡pθ​\(yc,t∣x,yc,<t\),pθ​\(c∣x\)=exp⁡zc∑j∈C⁡\(x\)exp⁡zj\.z\_\{c\}=\\sum\_\{t=1\}^\{\|y\_\{c\}\|\}\\log p\_\{\\theta\}\(y\_\{c,t\}\\mid x,y\_\{c,<t\}\),\\qquad p\_\{\\theta\}\(c\\mid x\)=\\frac\{\\exp z\_\{c\}\}\{\\sum\_\{j\\in C\(x\)\}\\exp z\_\{j\}\}\.
Token probabilities use FP32 log\-softmax over the full vocabulary; the outer softmax includes only the item’s valid candidatesC⁡\(x\)C\(x\)\. Temperature is 1 and no length normalization is applied\. With the saved one\-token labels, this reduces to renormalizing next\-token probabilities over those labels\. Each family uses its own original floating\-point model as teacher and averagesDKL\(pF​0∥pθ\)D\_\{\\mathrm\{KL\}\}\(p\_\{F0\}\\\|p\_\{\\theta\}\)over items\. This rule defines the training ARC term, the Llama ARC losses and the bare\-label readout in Appendix D\. PIQA instead scores complete answers as described in Section 3\.4\.

## Appendix BText means and PIQA counts

Table A2:Original REPLAY development text and PIQA correct counts\. The PIQA denominator is 1,838 in every cell\. These observations precede the fresh\-text confirmation\.

## Appendix CGain placement at matched parameter norm

This intervention uses the fixed 200\-update checkpoints to compare where a gain acts\. For each calibrated layer, DROP restores that layer to its original base while retaining the other three updates\. UNIFORM instead scales all four log\-gains by a common factor chosen to match DROP’s unweighted L2 norm\. For a log\-gain vectorθ\\thetapartitioned by layer, the matching factor for withdrawing layer l is

αl=1−‖θl‖22/‖θ‖22\.\\alpha\_\{l\}=\\sqrt\{1\-\\\|\\theta\_\{l\}\\\|\_\{2\}^\{2\}/\\\|\\theta\\\|\_\{2\}^\{2\}\}\.
DROP and UNIFORM therefore retain the same total log\-gain norm while distributing it differently\. This controls the size of the update in log\-gain coordinates; actual weight displacement also depends on the base and layer\.

Figure A1:Removing a layer’s gain produces higher text loss than distributing the same log\-gain norm across all four layers\. Points and whiskers show DROP−\-UNIFORM NLL and pointwise 95% paired intervals for the 200\-update states\.Across both bases, four layers and two text domains, all sixteen DROP−\-UNIFORM NLL differences are positive, with individual intervals above zero\. WikiText differences range from 0\.016099 to 0\.080062 on Q and from 0\.011932 to 0\.072699 on F\. Distributed shrinkage preserves more text benefit than each tested layer withdrawal at this matched norm\.

The placement change also affects generation behavior\. On F, withdrawing the layer\-27 gain yields 479 EOS endings in 569 greedy generations, compared with 420 for its matched UNIFORM control, while increasing both text losses\. These counts measure observed termination within a fixed 64\-token window\.

## Appendix DCheckpoint comparison under different readouts

We compare the fixed 200\- and 600\-update states on the same 569 ARC items under four interfaces: bare labels, space\-prefixed labels, labels in a native non\-thinking chat wrapper, and complete answer text\. Each interface defines an option distribution and compares it with FP0 evaluated under that same interface\. The metric is option\-distribution KL, not answer accuracy\.

Figure A2:The 200\-to\-600 update changes Q’s option KL in opposite directions under bare\-label and chat\-label readouts\. Both bases and all four interfaces are shown with pointwise 95% paired intervals\.For Q, later calibration increases KL by 0\.050054 \[0\.017590, 0\.083242\] with bare labels but decreases it by 0\.287473 \[0\.226663, 0\.349671\] with the chat wrapper\. Adding a space to the labels yields an increase of 0\.048508, and scoring complete answer text yields an increase of 0\.081679\. The native chat wrapper changes the model’s input context beyond whitespace, so these interfaces measure distinct behavioral responses to the same checkpoint change\.

F exhibits a different pattern: its bare\-label change is unresolved, while chat\-label KL increases\. The comparison shows why a calibration checkpoint should be associated with a specified evaluation interface\. The 200\-to\-600 readout intervention and the 200\-update placement intervention characterize fixed calibration paths; the main experiment separately measures transfer of the 600\-update gains\.

## Appendix ELlama context control

The two recipients come from a previously fixed development slate: a static replacement construction produced S with 31 changes from the original GPTQ base, and a joint ARC\-oriented construction produced J with 39\. They were retained for this control because their four calibrated projections have identical initial native entries while nine projections elsewhere differ\. They are not nested configurations and were not selected by the gain\-transfer results\. Each donor trains 786,432 log\-scales with Adam, learning rate 0\.01, betas \(0\.9, 0\.999\), epsilon 1e−\-8 and zero weight decay\. The common sampler seed is 20261002\. Each of the 200 updates combines eight ARC items with one 2,048\-token WikiText block and one C4 block\.

Here FIT is also the actual training pool: 592 ARC items, 28 WikiText blocks and 256 C4 blocks\. The 200\-update WikiText cursor records seven complete traversals plus four blocks\. The evaluation uses separate article and document sources: the construction excludes the training WikiText articles, the selected C4 document\-content identities do not overlap, and the ARC item\-ID intersection is empty\. The fixed objective weights for ARC KL, WikiText NLL and C4 NLL are 0\.02658740554232893, 0\.34240306533240217 and 0\.631009529125269\. The two donors share these weights, input order and zero\-initialized optimizer moments\. ARC follows Appendix A\.4’s bare\-label interface\. The cross\-installation experiment adds no training\.

Table A3 includes the uncalibrated S/J losses and the original GPTQ configuration after the same 200\-update recipe as references\. GPTQ’s layer\-13 initial encoding differs from S and J, so it is not a recipient in their controlled gain swap\. The five calibrated states receive the same PIQA full\-answer scoring protocol\. The four crossed states’ text and ARC losses are fully reevaluated on one RTX 5090, with PyTorch 2\.8\.0\+cu128, Transformers 5\.9\.0, FP32/SDPA and TF32 disabled\. Historical baseline and PIQA records remain unchanged\.

Compared with the archived loss vectors, the largest absolute changes in mean WikiText NLL, C4 NLL and ARC KL across the four states are9\.22×10−79\.22\\times 10^\{\-7\},5\.89×10−85\.89\\times 10^\{\-8\}and2\.30×10−72\.30\\times 10^\{\-7\}, respectively\. The corresponding largest per\-unit changes are1\.77×10−41\.77\\times 10^\{\-4\},2\.62×10−62\.62\\times 10^\{\-6\}and3\.96×10−53\.96\\times 10^\{\-5\}\. The complete same\-machine comparison preserves all six own\-gain preferences\. It evaluates 2,048 model\-text blocks and 2,276 model\-ARC items in 995\.054 s, including setup and surrounding work, with no additional training or PIQA calls\.

Table A3:Llama crossed states and uncalibrated loss baselines\. Reported PIQA counts use 1,838 questions; n/a denotes a state outside the original five\-state PIQA evaluation\. S and J share the four calibrated projections’ initial native tensors; the GPTQ reference does not share all four and is shown separately from the crossed comparison\.
Table A4:All nine differences between S and J, with zero\-based layer numbers\. Percentages specify GPTQ damping; RTN denotes round\-to\-nearest\. Every encoding uses INT3 and group size 128\. The supplement gives the complete 196\-projection manifest and the native equality records for the four calibrated sites\.

## Appendix FCPU update\-payload construction

We measure construction of the Qwen cross\-transfer update payloads from the saved donor checkpoints\. The floating\-point output contains four materialized weight matrices; the quantized output contains four native entries with updated scales and unchanged codes and zero points\. Both aligned outputs reproduce the original cross\-transfer tensors and complete model identities\. The operation uses FP64 multiplication followed by one FP32 cast, as in the quality experiment\.

Table A5 reports one CPU invocation on an AMD Ryzen 5 5600GT with six PyTorch threads, Windows 11, PyTorch 2\.13\.0\+cpu and NumPy 2\.5\.2\. Source\-file identity checks precede loading, so these measurements use an uncontrolled, potentially warm OS file cache\. Loading the shared floating source, two donor payloads and complete native base takes 0\.347 s; extracting both relative gains takes 0\.011 s\. These shared times are counted once\. Arithmetic includes allocation and output materialization; identity checking and output serialization with flush/fsync are timed separately\.

Table A5:Single\-invocation CPU costs for the four\-projection update payloads\. Identity checking includes the unchanged native projection context, which explains its larger cost for Q\. The table measures payload construction, not loading and publishing a complete floating\-point checkpoint or GPU installation\.
The native scale update requires much less arithmetic than materializing the floating\-point matrices\. In both cases, validation and I/O contribute appreciably to payload preparation\. These timings describe the installation operation separately from the earlier donor training and scientific evaluation; they are not a same\-machine speed comparison against training\.

## Appendix GText sources and development comparisons

The WikiText confirmation uses the raw WikiText\-103 training split at revision[b08601e0](https://huggingface.co/datasets/Salesforce/wikitext/tree/b08601e04326c79dfdd32d625aee71d232d685c3)\. The project history contains 932 consumed article identifiers\. Together with the official validation and test articles, the exclusion covers 1,054 normalized titles and 1,054 content identities\. The salt calibration\-transfer\-confirmation\-20261014 orders the remaining articles before complete 2,048\-token blocks are retained, yielding 139 articles and 256 blocks\. The fresh blocks have no exact intersection with the archived training, FIT, VALIDATION or REPLAY blocks\.

C4 uses validation shard 7 at revision[1588ec45](https://huggingface.co/datasets/allenai/c4/tree/1588ec454efa1a09f29cd18ddd04fe05fc8653a2)\. Document\-content and nonempty\-URL exclusions cover all 319,032 documents in the old shards 0–6\. The retained prefix contains 1,103 new documents, packed into 256 complete blocks\. These exclusions concern project data use; the underlying corpora may also occur in foundation\-model pretraining\.

All six fresh\-text states are evaluated on one RTX 5090 with PyTorch 2\.8\.0\+cu128, Transformers 5\.9\.0, FP32/SDPA and TF32 disabled\. Their complete means are in Table 1\. The completed evaluation takes 704\.872 s, separately from gain construction and donor training\.

The original REPLAY development comparisons are retained in Table A6\. Cross\-gain benefit retention is 90\.10%/78\.03% for Q and 90\.21%/80\.07% for F on WikiText/C4\. On the second development set, VALIDATION, floating\-point gains reduce Q’s WikiText/C4 perplexities by 43\.103%/21\.898%, while quantized gains reduce F’s by 36\.899%/19\.700%; the corresponding source interactions are−\-0\.112485/−\-0\.126137 NLL\. These evaluations use the same fixed donors as fresh confirmation, but had been inspected during development\.

Table A6:Paired comparisons on the original REPLAY development text\. Definitions match the fresh comparisons in Table 2\.
The development representation gapsG⁡\(1\)G\(1\),G⁡\(aF\)G\(a\_\{F\}\)andG⁡\(aQ\)G\(a\_\{Q\}\)are 0\.294806, 0\.250434 and 0\.137876 on WikiText, and 0\.254624, 0\.275165 and 0\.153047 on C4\. As on fresh text, a common floating\-point gain improves both recipients while increasing their C4 gap\.

## Appendix HSensitivity to consecutive C4 blocks

C4 packs source\-ordered documents into 2,048\-token blocks, so a document can span neighboring blocks\. Reconstructing the original token streams from the selected documents exactly reproduces both frozen arrays\. In Llama’s 1,128\-document evaluation prefix, 221 documents cross a block boundary and the longest spans nine blocks; 252 of the 255 adjacent\-block boundaries lie inside a document\. In the fresh Qwen prefix, the corresponding counts are 220 of 1,103 documents, seven blocks, and 255 of 255 boundaries\.

We examine this dependence in a post\-hoc sensitivity analysis using the saved paired losses\. A circular moving\-block bootstrap resamples consecutive segments of length 2, 4, 8 or 16, retaining 256 blocks per draw; length 1 supplies a single\-block reference\. Segment starts are uniform over the 256 positions, with wraparound at the end\. Each length uses 10,000 draws with seed 20261014 and shared indices across the model states\. These diagnostics leave the original pointwise intervals unchanged\.

Table A7:Consecutive\-block sensitivity of the two small Llama C4 source preferences\. Positive values favor the recipient’s own gain\. Length 1 uses this diagnostic’s seed, while Table 3 retains the original seed 20261004\.
Both Llama preferences remain positive for every segment length\. On fresh Qwen text, both transfer\-benefit intervals remain below zero, both cross\-minus\-own intervals remain above zero, and the source interaction remains negative at all five lengths\. Some intervals widen: for F’s cross\-minus\-own effect, the length\-1 interval \[0\.054000, 0\.058631\] becomes \[0\.052538, 0\.060974\] at length 16\. The supplement reports every length and contrast\. The sensitivity covers these segment scales for the fixed models, rather than variability across training runs\.