Error-Subspace Calibration for Multi-Task Model Merging

# Error-Subspace Calibration for Multi-Task Model Merging
Source: [https://arxiv.org/html/2610.33483](https://arxiv.org/html/2610.33483)
\\reportnumber

Hanseul ChoChulhee YunEmail:[cyseul@kaist\.ac\.kr](mailto:[email protected])Email:[chulhee\.yun@kaist\.ac\.kr](mailto:[email protected])Corresponding author:Correspondence to: and \.Affiliation:Kim Jaechul Graduate School of AI, KAIST

###### Abstract

Model merging combines task\-specific expert models into a single model without joint training on their original datasets, but often incurs a performance gap relative to the individual experts\. Post\-merging calibration can reduce this gap through task\-specific representation alignment, but the alignment objective alone does not specify which feature directions to correct\. We proposeError\-Subspace Calibration\(ESCal\), a post\-merging calibration framework that identifies low\-dimensional error subspaces from weight differences between expert and merged models\. At each intermediate block boundary, aBridgemodule projects the merged representation onto the next block’s input\-projection error subspace and maps these coordinates to an additive correction in the current block’s output\-projection error subspace\. The subspace bases remain fixed, so calibration learns only these linear maps, yielding a parameter\-efficient mechanism for feature correction\. We find thatESCalbrings the merged model closer to the experts in both representations and predictions, indicating more faithful recovery of task\-specific behavior\. Experiments on vision and natural language understanding benchmarks show thatESCalimproves merged\-model performance across diverse architectures and merging methods using a small calibration set for each task\. Our code is available at[ESCal](https://github.com/cyseul/ESCal)\.

## 1Introduction

SUN397CarsRESISC45EuroSATSVHNGTSRBMNISTDTDAverage005510101515202025253030Accuracy gain \(pp\) overuncalibrated base merging11223344556677889910101111121210−310^\{\-3\}10−210^\{\-2\}10−110^\{\-1\}Transformer blockAccumulated error\(MSE, log scale\)Task Arithmeticw/ SurgeryV2w/ IntervMergew/ FeatCalw/ ESCalFigure 1:Post\-merging calibration of an eight\-taskTask Arithmeticmerge of ViT\-B/32 experts\.\(Left\)Test\-accuracy gains over the uncalibrated merge \(percentage points\); “Average” denotes the eight\-task mean\.\(Right\)Mean squared error between merged and expert hidden features at depthℓ\\ell, averaged over eight tasks; shading indicates standard errors across tasks\. IntervMerge uses the selected ViT\-B/32–Task Arithmetic setting \(r=16r=16,η=3×10−4\\eta=3\\times 10^\{\-4\}, 1,000 outer iterations\)\. Its results are averaged over five seeds in both panels; for feature error, seed numerators and element counts are pooled within each task before averaging across tasks\.ESCalachieves the largest accuracy gains on six tasks and on average, and the lowest feature error at most block boundaries\.Model merging aims to integrate the capabilities of multiple task\-specificexpertmodels into a singlemergedmodel without jointly training it on their original datasets\([Ilharco et al\., 2023](https://arxiv.org/html/2610.33483#bib.bib7);[Yadav et al\., 2023](https://arxiv.org/html/2610.33483#bib.bib25);[Cheng et al\., 2025](https://arxiv.org/html/2610.33483#bib.bib2);[Xu et al\., 2025](https://arxiv.org/html/2610.33483#bib.bib24)\)\. By reusing fine\-tuned expert models, it offers a practical alternative to multi\-task training or deploying a separate model for each task\. However, the merged model often performs worse than the expert models, leaving a performance gap that limits the effectiveness of multi\-task model merging\. Reducing this gap while retaining the efficiency of model merging remains an important research question\([Yang et al\., 2026](https://arxiv.org/html/2610.33483#bib.bib29)\)\.

To narrow this gap, post\-merging calibration methods use a small calibration set to align the representations of merged models with those of the corresponding experts\([Yang et al\., 2024a](https://arxiv.org/html/2610.33483#bib.bib26);[Yang et al\., 2024b](https://arxiv.org/html/2610.33483#bib.bib27);[Gu et al\., 2026](https://arxiv.org/html/2610.33483#bib.bib5)\)\. The expert representations serve as alignment targets, enabling calibration without labels\. Calibration can be applied at the final representation or at intermediate layers to address discrepancies that arise throughout the network\.

Although representation alignment is effective, the alignment objective alone does not explicitly specify the feature directions that calibration modules can modify\. For example,SurgeryandSurgeryV2 learn the projection matrices of their low\-rank calibration modules from calibration samples, with the correction directions determined by the alignment objective\. This motivates us to ask whether these directions can be identified from merging errors before calibration, leaving fewer parameters to learn from the calibration data\.Can we improve both the effectiveness and efficiency of representation alignment by restricting calibration to the feature subspaces associated with merging errors?

To investigate this question, we examinemerging error, defined as the difference between expert weights and the merged weights\. Its dominant singular subspaces capture the largest components of these weight differences\. Based on this insight, we proposeError\-Subspace Calibration\(ESCal\), which constructs task\-specific calibration subspaces from merging errors\. At each intermediate block boundary, aBridgemodule reads coordinates from the next block’s input\-projection error subspace and maps them to an additive correction in the current block’s output\-projection error subspace\. With both subspaces fixed, calibration learns only a mapping between them \(Section[3](https://arxiv.org/html/2610.33483#S3)\)\.

We evaluateESCalon vision and natural language understanding benchmarks across various architectures and base merging methods, with additional experiments on natural language generation\.ESCalimproves merged\-model performance using a small calibration set per task \(Section[4](https://arxiv.org/html/2610.33483#S4)\)\. Our ablation studies examine how subspace construction and constraints affect calibration performance \(Section[5\.1](https://arxiv.org/html/2610.33483#S5.SS1)\)\. Our alignment analyses further show thatESCalbrings the merged model closer to the expert models in both representations and predictions \(Section[5\.2](https://arxiv.org/html/2610.33483#S5.SS2)\)\.

Keeping the subspace bases fixed also makesESCalparameter\-efficient, as eachBridgemodule learns only anr×rr\\times rmapping for each task and block boundary\. Across all evaluated configurations,ESCaluses fewer trainable parameters thanSurgeryV2, while achieving higher average performance in most settings \(Section[5\.3](https://arxiv.org/html/2610.33483#S5.SS3)\)\.

## 2Related Work

### 2\.1Multi\-task Model Merging

A foundational approach isTask Arithmetic\([Ilharco et al\., 2023](https://arxiv.org/html/2610.33483#bib.bib7)\), which represents the knowledge acquired during fine\-tuning as a*task vector*, defined as the difference between the fine\-tuned and pretrained model parameters\. It constructs a multi\-task model by adding the sum of task vectors, scaled by a single merging coefficient, to the pretrained model parameters\. Building on this approach,AdaMerging\([Yang et al\., 2024c](https://arxiv.org/html/2610.33483#bib.bib28)\)adaptively learns task\-wise or layer\-wise merging coefficients using unlabeled test data, rather than relying on a single fixed coefficient\.

A major line of work focuses on mitigating interference among task updates\.TIES\-Merging\([Yadav et al\., 2023](https://arxiv.org/html/2610.33483#bib.bib25)\)reduces interference by removing redundant parameter updates and resolving sign conflicts before merging\.WUDI\-Merging\([Cheng et al\., 2025](https://arxiv.org/html/2610.33483#bib.bib2)\)leverages task\-vector\-induced subspaces to reduce parameter interference without additional data\.ProDistill\([Xu et al\., 2025](https://arxiv.org/html/2610.33483#bib.bib24)\)instead uses few\-shot data to progressively calibrate intermediate features through layer\-wise distillation with element\-wise merging coefficients, allowing more fine\-grained adaptation of the merged model toward task\-specific experts\. These approaches modify how the merged model is constructed, in contrast to post\-merging calibration methods that operate on an already merged checkpoint\. Additional related work on model merging is discussed in Appendix[A](https://arxiv.org/html/2610.33483#A1)\.

### 2\.2Post\-merging Calibration

Post\-merging calibration aims to reduce the performance gap between the merged model and its task\-specific experts by correcting representation discrepancies after merging\.

Surgery\([Yang et al\., 2024a](https://arxiv.org/html/2610.33483#bib.bib26)\)characterizes these discrepancies as*representation bias*and learns a lightweight task\-specific low\-rank module to align the merged model’s final\-layer representation with that of the corresponding expert\. Its projection matrices are learned from unlabeled test samples, allowing the directions of the representation correction to be determined by the alignment objective\.ProbSurgery\([Wei et al\., 2025](https://arxiv.org/html/2610.33483#bib.bib21)\)extends this framework by probabilistically modeling representation bias and its uncertainty\.

While these approaches calibrate only the final representation,SurgeryV2\([Yang et al\., 2024b](https://arxiv.org/html/2610.33483#bib.bib27)\)extends calibration to intermediate layers, motivated by the observation that representation bias can arise throughout the network and accumulate with depth\. It places task\-specific calibration modules at multiple locations and aligns the merged representations with those of the corresponding expert\.IntervMerge\([Osial et al\., 2024](https://arxiv.org/html/2610.33483#bib.bib14)\)uses a similar layer\-wise correction strategy, formulating task\-specific feature adjustment as low\-rank interventions within Transformer blocks\. Its parameter\-efficient mini\-interventions restrict the correction to selected dimensions of the intermediate representation\.

More recently,FeatCal\([Gu et al\., 2026](https://arxiv.org/html/2610.33483#bib.bib5)\)takes a different approach by interpreting the discrepancy between merged and expert representations as*feature drift*\. It explicitly accounts for both locally introduced errors and their propagation through subsequent layers and, rather than introducing additional calibration modules, directly recalibrates the merged weights layer by layer using closed\-form updates\. In contrast,ESCalkeeps the merged weights frozen and learns only small mappings between fixed error subspaces derived from weight differences between the expert and merged models\.

𝒟t\\mathcal\{D\}\_\{t\}𝒙\{\\bm\{x\}\}=𝒉0=\{\\bm\{h\}\}\_\{0\}11ℓ\\ellℓ\+1\\ell\+1LL…\\ldots…\\ldots…\\ldots…\\ldotsESCalMerged𝒉¯L′,t\\overline\{\{\\bm\{h\}\}\}\_\{L\}^\{\\prime,t\}𝒉Lt\{\\bm\{h\}\}\_\{L\}^\{t\}Expert\(t​a​s​k=t\)\(task=t\)Task\-specificclassificationheadoutoutMerged blockℓ\\ellAttentionMLP𝒉¯ℓt\\overline\{\{\\bm\{h\}\}\}\_\{\\ell\}^\{t\}BridgeNormalization omitted𝑾¯ℓ\\overline\{\{\\bm\{W\}\}\}\_\{\\ell\}𝑾¯ℓ\+1\\overline\{\{\\bm\{W\}\}\}\_\{\\ell\+1\}𝑾ℓt\{\\bm\{W\}\}\_\{\\ell\}^\{t\}𝑾ℓ\+1t\{\\bm\{W\}\}\_\{\\ell\+1\}^\{t\}𝒉¯ℓt\\overline\{\{\\bm\{h\}\}\}\_\{\\ell\}^\{t\}𝑼in,ℓ\+1t\{\\bm\{U\}\}\_\{\\mathrm\{in\},\\ell\+1\}^\{t\} ![Refer to caption](https://arxiv.org/html/2610.33483v1/Figures/escal-snow.png)Readddrr𝑪ℓt\{\\bm\{C\}\}\_\{\\ell\}^\{t\}![Refer to caption](https://arxiv.org/html/2610.33483v1/Figures/escal-fire.png)Map\(𝑽out,ℓt\)⊤\\bigl\(\{\\bm\{V\}\}\_\{\\mathrm\{out\},\\ell\}^\{t\}\\bigr\)^\{\\top\} ![Refer to caption](https://arxiv.org/html/2610.33483v1/Figures/escal-snow.png)Writerrdd𝒉¯ℓ′,t\\overline\{\{\\bm\{h\}\}\}\_\{\\ell\}^\{\\prime,t\}𝑬out,ℓt\{\\bm\{E\}\}\_\{\\mathrm\{out\},\\ell\}^\{t\}≈𝑼out,ℓt​𝚺out,ℓt​\(𝑽out,ℓt\)⊤\\approx\{\\bm\{U\}\}\_\{\\mathrm\{out\},\\ell\}^\{t\}\{\\bm\{\\Sigma\}\}\_\{\\mathrm\{out\},\\ell\}^\{t\}\\bigl\(\{\\bm\{V\}\}\_\{\\mathrm\{out\},\\ell\}^\{t\}\\bigr\)^\{\\top\} 𝑬in,ℓ\+1t\{\\bm\{E\}\}\_\{\\mathrm\{in\},\\ell\+1\}^\{t\}𝑼in,ℓ\+1t​𝚺in,ℓ\+1t​\(𝑽in,ℓ\+1t\)⊤≈\{\\bm\{U\}\}\_\{\\mathrm\{in\},\\ell\+1\}^\{t\}\{\\bm\{\\Sigma\}\}\_\{\\mathrm\{in\},\\ell\+1\}^\{t\}\\bigl\(\{\\bm\{V\}\}\_\{\\mathrm\{in\},\\ell\+1\}^\{t\}\\bigr\)^\{\\top\}\\approx 𝒉ℓt\{\\bm\{h\}\}\_\{\\ell\}^\{t\}MSE![Refer to caption](https://arxiv.org/html/2610.33483v1/Figures/escal-fire.png)Learnable core![Refer to caption](https://arxiv.org/html/2610.33483v1/Figures/escal-snow.png)Frozen read/write basesFrozen model weights

Figure 2:Overview of Error\-Subspace Calibration \(ESCal\)\.Task\-specificBridgemodules align the frozen merged model with its expert at complete block outputs, after both residual additions\. At intermediate boundaries, read and write bases come from merging errors in the next block’s joint QKV input projection and the current block’s MLP output projection, respectively\.

## 3Method: Error\-Subspace Calibration

We characterize the discrepancy introduced by model merging through low\-dimensional error subspaces derived from weight differences between expert and merged models, and use them to constrain post\-merging calibration\. The subspaces determine where a calibration module reads from and where its correction can act, leaving only the mapping between them to be learned\.

#### Notation\.

For a positive integerNN, we write\[N\]=\{1,…,N\}\[N\]=\\left\\\{1,\\ldots,N\\right\\\}\. We considerTTtasks and a Transformer backbone withLLblocks, usingt∈\[T\]t\\in\[T\]to identify a task andℓ∈\[L\]\\ell\\in\[L\]to identify a block\. For a given linear transformation in blockℓ\\ell,𝑾ℓt\{\\bm\{W\}\}^\{t\}\_\{\\ell\}denotes the weight matrix of the expert for tasktt, while𝑾¯ℓ\\overline\{\{\\bm\{W\}\}\}\_\{\\ell\}denotes the corresponding weight matrix of the merged model\. The merged weights are shared across tasks so we omit the task index\. Throughout, we represent hidden features as row vectors: an input𝒙\{\\bm\{x\}\}is multiplied by the weight matrix on the right, giving𝒙​𝑾\{\\bm\{x\}\}\{\\bm\{W\}\}for a linear transformation\.

### 3\.1Merging Error Subspaces

For a linear transformation, the discrepancy between expert and merged outputs under a common input is determined by their weight difference\. We define the merging error as

𝑬ℓt=𝑾ℓt−𝑾¯ℓ\.\{\\bm\{E\}\}\_\{\\ell\}^\{t\}=\{\\bm\{W\}\}\_\{\\ell\}^\{t\}\-\\overline\{\{\\bm\{W\}\}\}\_\{\\ell\}\.\(1\)
Write its compact singular value decomposition \(SVD\) as

𝑬ℓt=𝑼ℓt​𝚺ℓt​\(𝑽ℓt\)⊤,\{\\bm\{E\}\}\_\{\\ell\}^\{t\}=\{\\bm\{U\}\}\_\{\\ell\}^\{t\}\{\\bm\{\\Sigma\}\}\_\{\\ell\}^\{t\}\\left\(\{\\bm\{V\}\}\_\{\\ell\}^\{t\}\\right\)^\{\\top\},\(2\)where𝚺ℓt\{\\bm\{\\Sigma\}\}\_\{\\ell\}^\{t\}contains all nonzero singular values in descending order\.

Under our row\-vector convention, the columns of𝑼ℓt\{\\bm\{U\}\}\_\{\\ell\}^\{t\}and𝑽ℓt\{\\bm\{V\}\}\_\{\\ell\}^\{t\}span the input\-side and output\-side error subspaces, respectively\. For a common input𝒙\{\\bm\{x\}\}, the output discrepancy is𝒙​𝑬ℓt=\(𝒙​𝑼ℓt\)​𝚺ℓt​\(𝑽ℓt\)⊤\{\\bm\{x\}\}\{\\bm\{E\}\}\_\{\\ell\}^\{t\}=\(\{\\bm\{x\}\}\{\\bm\{U\}\}\_\{\\ell\}^\{t\}\)\{\\bm\{\\Sigma\}\}\_\{\\ell\}^\{t\}\(\{\\bm\{V\}\}\_\{\\ell\}^\{t\}\)^\{\\top\}\. Thus, the discrepancy depends only on the input coordinates𝒙​𝑼ℓt\{\\bm\{x\}\}\{\\bm\{U\}\}\_\{\\ell\}^\{t\}, and its transpose lies inspan⁡\(𝑽ℓt\)\\operatorname\{span\}\(\{\\bm\{V\}\}\_\{\\ell\}^\{t\}\)\.

We retain the leadingrrleft and right singular vectors to construct low\-dimensional approximations of these subspaces\. For a fixed rank budget, truncated SVD minimizes the approximation error of the weight difference in Frobenius norm, supporting the choice of leading singular directions\. This argument concerns the local output discrepancy of a linear transformation under a common input\. Appendix[B](https://arxiv.org/html/2610.33483#A2)provides the linear\-map analysis underlying this argument\.

### 3\.2Cross\-Layer Error\-Subspace Calibration

Feature drift in a merged model reflects both locally introduced mismatch and drift propagated from preceding layers\([Gu et al\., 2026](https://arxiv.org/html/2610.33483#bib.bib5)\)\. Thus, a discrepancy at one block also affects the representations processed by subsequent blocks\. This propagation view suggests calibrating at block boundaries, where a representation serves as both the output of the current block and the input to the next\.

For a selected linear transformation under a common input, its weight\-induced output discrepancy depends only on the input’s coordinates in its full input\-side error subspace\. This motivates using the next block’s input\-projection error subspace to select the coordinates the calibration module reads, while using the current block’s output\-projection error subspace to specify its correction directions\.

#### Read and write bases\.

For each Transformer blockℓ\\ell, we construct the read basis from the leading input\-side error directions of the joint query, key, and value \(QKV\) input projection, and the write basis from the leading output\-side error directions of the MLP output projection\. We refer to the resulting subspaces as the input\-projection and output\-projection error subspaces, respectively\. We denote the expert weights of these two projections by𝑾in,ℓt\{\\bm\{W\}\}\_\{\\mathrm\{in\},\\ell\}^\{t\}and𝑾out,ℓt\{\\bm\{W\}\}\_\{\\mathrm\{out\},\\ell\}^\{t\}\. We write both projection weights in input\-by\-output form\. The input projection hasddrows, and the output projection hasddcolumns, so their left and right singular vectors, respectively, lie in the samedd\-dimensional hidden representation space\.

Applying the merging error definition to these projections gives

𝑬in,ℓt=𝑾in,ℓt−𝑾¯in,ℓ,𝑬out,ℓt=𝑾out,ℓt−𝑾¯out,ℓ\.\{\\bm\{E\}\}\_\{\\mathrm\{in\},\\ell\}^\{t\}=\{\\bm\{W\}\}\_\{\\mathrm\{in\},\\ell\}^\{t\}\-\\overline\{\{\\bm\{W\}\}\}\_\{\\mathrm\{in\},\\ell\},\\qquad\{\\bm\{E\}\}\_\{\\mathrm\{out\},\\ell\}^\{t\}=\{\\bm\{W\}\}\_\{\\mathrm\{out\},\\ell\}^\{t\}\-\\overline\{\{\\bm\{W\}\}\}\_\{\\mathrm\{out\},\\ell\}\.\(3\)We use𝑼in,ℓt\{\\bm\{U\}\}\_\{\\mathrm\{in\},\\ell\}^\{t\}and𝑽out,ℓt\{\\bm\{V\}\}\_\{\\mathrm\{out\},\\ell\}^\{t\}to denote their top\-rrleft and right singular\-vector bases, respectively\.

At the boundary between blocksℓ\\ellandℓ\+1\\ell\+1, we pair the read basis𝑼in,ℓ\+1t\{\\bm\{U\}\}\_\{\\mathrm\{in\},\\ell\+1\}^\{t\}with the write basis𝑽out,ℓt\{\\bm\{V\}\}\_\{\\mathrm\{out\},\\ell\}^\{t\}\. This pairing reads coordinates along the dominant input\-side error directions of blockℓ\+1\\ell\+1’s QKV input projection and writes corrections along the dominant output\-side error directions of blockℓ\\ell’s MLP output projection\.

#### Bridgemodule\.

Using these fixed rank\-rrbases, we define the task\-specificBridgemodule at each intermediate block boundaryℓ=1,…,L−1\\ell=1,\\ldots,L\-1as

𝑩ℓt=𝑼in,ℓ\+1t​𝑪ℓt​\(𝑽out,ℓt\)⊤,𝑪ℓt∈ℝr×r,\{\\bm\{B\}\}\_\{\\ell\}^\{t\}=\{\\bm\{U\}\}\_\{\\mathrm\{in\},\\ell\+1\}^\{t\}\{\\bm\{C\}\}\_\{\\ell\}^\{t\}\\left\(\{\\bm\{V\}\}\_\{\\mathrm\{out\},\\ell\}^\{t\}\\right\)^\{\\top\},\\qquad\{\\bm\{C\}\}\_\{\\ell\}^\{t\}\\in\\mathbb\{R\}^\{r\\times r\},\\addcontentsline\{lla\}\{section\}\{\\hbox to20\.74pt\{\\crtrefnumber\{eq:bridge\}\\hfil\}eq:bridge\}\(4\)where the core𝑪ℓt\{\\bm\{C\}\}\_\{\\ell\}^\{t\}maps coordinates in the read basis to correction coefficients in the write basis\. Only𝑪ℓt\{\\bm\{C\}\}\_\{\\ell\}^\{t\}is trainable, requiringr2r^\{2\}parameters per task and location\.

At the terminal location, where there is no following Transformer block, we use the task\-specific output head’s top\-rrinput singular vectors as the read basis𝑼headt\{\\bm\{U\}\}\_\{\\mathrm\{head\}\}^\{t\}, giving

𝑩Lt=𝑼headt​𝑪Lt​\(𝑽out,Lt\)⊤\.\{\\bm\{B\}\}\_\{L\}^\{t\}=\{\\bm\{U\}\}\_\{\\mathrm\{head\}\}^\{t\}\{\\bm\{C\}\}\_\{L\}^\{t\}\\left\(\{\\bm\{V\}\}\_\{\\mathrm\{out\},L\}^\{t\}\\right\)^\{\\top\}\.\(5\)Unlike the intermediate read bases,𝑼headt\{\\bm\{U\}\}\_\{\\mathrm\{head\}\}^\{t\}is derived from the head weights, capturing directions to which the head is most sensitive\.

### 3\.3Module Training and Feature Calibration

Calibration is performed independently for each taskttusing a task\-specific calibration set𝒟t\\mathcal\{D\}\_\{t\}\. The read and write bases are computed once and kept fixed, while both the merged model and the corresponding expert remain frozen\. We optimize only theBridgecores𝑪ℓt\{\\bm\{C\}\}\_\{\\ell\}^\{t\}to align the merged model’s representations with those of the expert\.

For an input𝒙\{\\bm\{x\}\}, let𝒉¯ℓt​\(𝒙\)\\overline\{\{\\bm\{h\}\}\}\_\{\\ell\}^\{t\}\(\{\\bm\{x\}\}\)denote the output of merged Transformer blockℓ\\ellafter both residual additions in its self\-attention and MLP sublayers, includingBridgecorrections propagated from preceding block boundaries\. At each intermediate boundaryℓ=1,…,L−1\\ell=1,\\ldots,L\-1,Bridgeapplies the residual correction

𝒉¯ℓ′,t\(𝒙\)\\displaystyle\\overline\{\{\\bm\{h\}\}\}\_\{\\ell\}^\{\\prime,t\}\(\{\\bm\{x\}\}\)=𝒉¯ℓt​\(𝒙\)\+𝒉¯ℓt​\(𝒙\)​𝑩ℓt\\displaystyle=\\overline\{\{\\bm\{h\}\}\}\_\{\\ell\}^\{t\}\(\{\\bm\{x\}\}\)\+\\overline\{\{\\bm\{h\}\}\}\_\{\\ell\}^\{t\}\(\{\\bm\{x\}\}\)\{\\bm\{B\}\}\_\{\\ell\}^\{t\}\(6\)=𝒉¯ℓt​\(𝒙\)\+\(𝒉¯ℓt​\(𝒙\)​𝑼in,ℓ\+1t\)​𝑪ℓt​\(𝑽out,ℓt\)⊤\.\\displaystyle=\\overline\{\{\\bm\{h\}\}\}\_\{\\ell\}^\{t\}\(\{\\bm\{x\}\}\)\+\\left\(\\overline\{\{\\bm\{h\}\}\}\_\{\\ell\}^\{t\}\(\{\\bm\{x\}\}\)\{\\bm\{U\}\}\_\{\\mathrm\{in\},\\ell\+1\}^\{t\}\\right\)\{\\bm\{C\}\}\_\{\\ell\}^\{t\}\\left\(\{\\bm\{V\}\}\_\{\\mathrm\{out\},\\ell\}^\{t\}\\right\)^\{\\top\}\.The corrected representation𝒉¯ℓ′,t\(𝒙\)\\overline\{\{\\bm\{h\}\}\}\_\{\\ell\}^\{\\prime,t\}\(\{\\bm\{x\}\}\)is then passed to merged blockℓ\+1\\ell\+1, so corrections are propagated sequentially through the backbone\. At the terminal location, the analogous update using𝑩Lt\{\\bm\{B\}\}\_\{L\}^\{t\}produces the calibrated final representation𝒉¯L′,t\(𝒙\)\\overline\{\{\\bm\{h\}\}\}\_\{L\}^\{\\prime,t\}\(\{\\bm\{x\}\}\)\.

Let𝒉ℓt​\(𝒙\)\{\\bm\{h\}\}\_\{\\ell\}^\{t\}\(\{\\bm\{x\}\}\)denote the corresponding expert representation\. We jointly optimize allBridgecores for taskttby minimizing the feature discrepancy across calibration locations:

min\{𝑪ℓt\}ℓ=1L∑ℓ=1L𝔼𝒙∼𝒟t\[‖𝒉¯ℓ′,t\(𝒙\)−𝒉ℓt\(𝒙\)‖F2\]\.\\min\_\{\\\{\{\\bm\{C\}\}\_\{\\ell\}^\{t\}\\\}\_\{\\ell=1\}^\{L\}\}\\sum\_\{\\ell=1\}^\{L\}\\mathbb\{E\}\_\{\{\\bm\{x\}\}\\sim\\mathcal\{D\}\_\{t\}\}\\left\[\\left\\\|\\overline\{\{\\bm\{h\}\}\}\_\{\\ell\}^\{\\prime,t\}\(\{\\bm\{x\}\}\)\-\{\\bm\{h\}\}\_\{\\ell\}^\{t\}\(\{\\bm\{x\}\}\)\\right\\\|\_\{F\}^\{2\}\\right\]\.\\addcontentsline\{lla\}\{section\}\{\\hbox to20\.74pt\{\\crtrefnumber\{eq:calibration\_loss\}\\hfil\}eq:calibration\_\{l\}oss\}\(7\)Since the frozen expert provides the target representations, calibration requires no task labels\.

## 4Experimental Results

We compareESCalwith existing post\-merging calibration methods under different base merging methods\. Our evaluation examines both the improvement over uncalibrated merged models and the remaining performance gap to the expert models\. We conduct our main evaluation on vision and natural language understanding \(NLU\), and additionally evaluate natural language generation \(NLG\) to examine the applicability ofESCalbeyond these settings\. The NLG results and discussion are provided in Appendix[C\.5](https://arxiv.org/html/2610.33483#A3.SS5)\.

#### Tasks and Models\.

For vision, we use CLIP ViT\-B/32, ViT\-B/16, and ViT\-L/14 models\([Radford et al\., 2021](https://arxiv.org/html/2610.33483#bib.bib15)\)fine\-tuned on eight image classification tasks\. For NLU, we use RoBERTa\-base\([Liu et al\., 2019](https://arxiv.org/html/2610.33483#bib.bib10)\)models fine\-tuned on eight GLUE tasks\([Wang et al\., 2018](https://arxiv.org/html/2610.33483#bib.bib20)\)\. In both settings, we merge the eight task\-specific models and evaluate the resulting model across all tasks\. For NLG, we use three LLaMA\-2\-13B\([Touvron et al\., 2023](https://arxiv.org/html/2610.33483#bib.bib19)\)experts specialized for instruction following, mathematical reasoning, and code generation, and consider all pairwise merges\.

#### Merging and Calibration Methods\.

We considerTask Arithmetic\([Ilharco et al\., 2023](https://arxiv.org/html/2610.33483#bib.bib7)\),TIES\-Merging\([Yadav et al\., 2023](https://arxiv.org/html/2610.33483#bib.bib25)\),AdaMerging\([Yang et al\., 2024c](https://arxiv.org/html/2610.33483#bib.bib28)\),WUDI\-Merging\([Cheng et al\., 2025](https://arxiv.org/html/2610.33483#bib.bib2)\), andProDistill\([Xu et al\., 2025](https://arxiv.org/html/2610.33483#bib.bib24)\)as base merging methods\. For post\-merging calibration, we compareSurgery\([Yang et al\., 2024a](https://arxiv.org/html/2610.33483#bib.bib26)\),SurgeryV2\([Yang et al\., 2024b](https://arxiv.org/html/2610.33483#bib.bib27)\),IntervMerge\([Osial et al\., 2024](https://arxiv.org/html/2610.33483#bib.bib14)\),FeatCal\([Gu et al\., 2026](https://arxiv.org/html/2610.33483#bib.bib5)\), andESCal\(ours\)\. We report results for each base merged model before and after calibration\. We also include the pretrained model and task\-specific experts as references\.

#### Calibration Protocol\.

For each base merging method, all calibration methods start from the same merged checkpoint and use the same task\-specific calibration sets, each containing 64 input samples\.Surgery,SurgeryV2,IntervMerge, andESCaloptimize only their calibration modules with the merged weights frozen, whereasFeatCalapplies closed\-form updates to the merged weights\. Calibration and evaluation sets are disjoint, and results are averaged over five random seeds unless otherwise specified\. ForIntervMerge, we select settings separately for each architecture and base using the existing seed\-0 search scores \(vision test accuracy and NLU GLUE validation score\), then fix these settings across seeds 0–4\. Detailed hyperparameter settings are provided in Appendix[C\.2](https://arxiv.org/html/2610.33483#A3.SS2)\.

### 4\.1Results on Vision Tasks

Table[1](https://arxiv.org/html/2610.33483#S4.T1)presents representative ViT\-B/32 results withTask Arithmetic,TIES\-Merging, andWUDI\-Merging\. Complete results across architectures and base merging methods are provided in Appendix[C\.3](https://arxiv.org/html/2610.33483#A3.SS3)\.

Table 1:Vision model merging results on ViT\-B/32\.Task\-wise test accuracy \(%\) and average performance across eight image classification tasks\.MethodSUN397CarsRESISC45EuroSATSVHNGTSRBMNISTDTDAvg\.Pretrained63\.1859\.7360\.3345\.3731\.6132\.5648\.2543\.9948\.13Expert75\.3477\.7395\.9899\.7497\.4698\.7399\.6979\.3690\.50Task Arithmetic55\.3254\.9866\.8277\.7880\.2169\.6897\.3450\.3769\.06w/Surgery60\.3952\.6271\.3888\.6581\.6774\.2797\.8252\.7772\.45w/SurgeryV270\.7066\.4190\.6599\.1296\.6195\.8899\.4770\.9086\.22w/IntervMerge66\.3558\.8785\.8897\.6193\.4487\.4998\.6264\.3181\.57w/FeatCal69\.2270\.3189\.0698\.0095\.0192\.4099\.1371\.1185\.53w/ESCal70\.3369\.0293\.6099\.4897\.1097\.7099\.5573\.5287\.54TIES\-Merging65\.0564\.4374\.9276\.4881\.2869\.3796\.5354\.5272\.82w/Surgery67\.6459\.5778\.5089\.2481\.3076\.0797\.1460\.0376\.19w/SurgeryV270\.8766\.2690\.1498\.9496\.5495\.5299\.4570\.8986\.08w/IntervMerge68\.6560\.5986\.2197\.3992\.9987\.6098\.2465\.1482\.10w/FeatCal70\.1270\.7489\.2897\.9394\.7192\.5599\.1071\.1385\.69w/ESCal70\.9369\.2393\.3199\.3697\.0997\.7099\.5373\.3587\.56WUDI\-Merging69\.1171\.3885\.6295\.6394\.2794\.5199\.4769\.4784\.93w/Surgery69\.2469\.4287\.7097\.5394\.8195\.4399\.4871\.3785\.62w/SurgeryV273\.2473\.5794\.1599\.5997\.1798\.3899\.6477\.6689\.18w/IntervMerge71\.7272\.5592\.6599\.2196\.3397\.9799\.5975\.8988\.24w/FeatCal71\.9875\.8093\.7199\.3196\.5398\.4299\.5077\.5289\.10w/ESCal73\.5574\.6894\.8099\.5897\.3198\.6899\.6477\.5289\.47

ESCalimproves the average accuracy of all three base merged models shown in the table and achieves the highest average among the compared calibration methods\. The largest improvement over the uncalibrated model occurs withTask Arithmetic, whose average accuracy rises from69\.06%69\.06\\%to87\.54%87\.54\\%, a gain of18\.4818\.48percentage points\.ESCalalso improvesWUDI\-Merging, which has the highest uncalibrated average accuracy among the three base merging methods, from84\.93%84\.93\\%to89\.47%89\.47\\%, a gain of4\.544\.54percentage points\. These results show that calibration remains beneficial even when the base merged model already achieves high accuracy\.

These gains extend to both task\-wise and layer\-wiseAdaMerging\(Appendix[C\.3](https://arxiv.org/html/2610.33483#A3.SS3)\)\. On ViT\-B/32,ESCalincreases their average accuracies from70\.82%70\.82\\%to86\.87%86\.87\\%and from79\.31%79\.31\\%to87\.91%87\.91\\%, respectively, showing that calibration can provide further gains after merging coefficients have been adapted\. The improvements also hold forProDistillacross all three architectures\. Thus,ESCalcan complement merging methods that already use intermediate\-feature distillation\.

### 4\.2Results on NLU Tasks

Table[2](https://arxiv.org/html/2610.33483#S4.T2)reports representative results withTask Arithmetic,TIES\-Merging, andWUDI\-Merging\. Complete results across base merging methods are provided in Appendix[C\.4](https://arxiv.org/html/2610.33483#A3.SS4)\.

Table 2:NLU model merging results on RoBERTa\-base\.Task\-wise performance and average score across eight GLUE tasks\.MethodCoLASST\-2MRPCSTS\-BQQPMNLIQNLIRTEAvg\.Pretrained0\.00000\.59750\.1581\-0\.07580\.46330\.31830\.50780\.52710\.3120Expert0\.58950\.94380\.90910\.89970\.89940\.87180\.91980\.77260\.8507Task Arithmetic0\.12320\.61350\.55000\.53560\.58810\.62700\.53650\.53430\.5135w/Surgery0\.25530\.85320\.76570\.78770\.77930\.72150\.78390\.71840\.7081w/SurgeryV20\.53580\.93260\.89080\.89780\.80020\.84040\.90840\.74080\.8184w/IntervMerge0\.46470\.91350\.84770\.87060\.77790\.76250\.85240\.71990\.7762w/FeatCal0\.38790\.91400\.81620\.49000\.76800\.75690\.84990\.70110\.7105w/ESCal0\.55880\.93190\.89370\.89980\.80610\.84690\.91170\.74150\.8238TIES\-Merging0\.19720\.76260\.50750\.43900\.60390\.66700\.53800\.49460\.5262w/Surgery0\.26800\.86350\.75700\.77700\.77700\.73470\.79880\.69890\.7094w/SurgeryV20\.54200\.93190\.88480\.89650\.82050\.84740\.90790\.73940\.8213w/IntervMerge0\.43820\.91740\.83680\.87020\.80230\.78050\.86600\.72490\.7796w/FeatCal0\.40550\.91060\.81110\.47650\.77020\.75690\.84330\.70610\.7100w/ESCal0\.55520\.93390\.87690\.89350\.81440\.84430\.90700\.74300\.8210WUDI\-Merging0\.06250\.80730\.25650\.09270\.76800\.44310\.46220\.48740\.4225w/Surgery0\.22260\.90250\.78600\.77580\.82700\.77550\.81180\.72060\.7277w/SurgeryV20\.55290\.93920\.89080\.88680\.87980\.86200\.91360\.76030\.8357w/IntervMerge0\.47660\.92820\.83030\.86680\.85620\.83470\.88840\.75160\.8041w/FeatCal0\.43870\.91900\.82150\.49530\.82340\.77480\.83580\.68880\.7247w/ESCal0\.56630\.93970\.89500\.89530\.87440\.86160\.91440\.75450\.8376

ESCalimproves the average GLUE score for all three base merging methods\.WUDI\-Mergingbenefits most, with its average score rising from0\.42250\.4225to0\.83760\.8376\. Despite having the lowest uncalibrated score among the three methods,WUDI\-Mergingachieves the highest average afterESCalcalibration\. Across these settings,ESCalconsistently outperformsSurgeryandFeatCal\. Relative toSurgeryV2, it achieves higher averages withTask ArithmeticandWUDI\-Merging, and a comparable average withTIES\-Merging\.

The improvements also extend to the additional base merging methods in Appendix[C\.4](https://arxiv.org/html/2610.33483#A3.SS4)\. For task\-wise and layer\-wiseAdaMerging,ESCalraises the average scores from0\.49890\.4989to0\.82020\.8202and from0\.48230\.4823to0\.82760\.8276, respectively; forProDistill, it improves the average from0\.72980\.7298to0\.79990\.7999\. These results support the effectiveness of error\-subspace calibration across merged checkpoints produced by different merging strategies\.

## 5Further Analyses

We further analyzeESCalto examine how its error\-subspace design contributes to calibration performance and how closely the calibrated model matches expert predictions\. We also assess its calibration data and parameter requirements, along with the additional storage and inference overhead of its calibration modules\.

### 5\.1Effectiveness of Error\-Subspace Construction

Table 3:We compare different basis sources on ViT\-B/32 merged withTask Arithmeticusing rankr=128r=128\. We report mean accuracy here, with standard deviations provided in Table[18](https://arxiv.org/html/2610.33483#A4.T18)in Appendix[D\.1](https://arxiv.org/html/2610.33483#A4.SS1)\.BasisSUN397CarsRESISC45EuroSATSVHNGTSRBMNISTDTDAvg\.Random orthonormal65\.4263\.8888\.5498\.6196\.0695\.8199\.3264\.9884\.08Fine\-tuned weights, top\-rr66\.9465\.0289\.8599\.0396\.1796\.3299\.3665\.7084\.80Task vector, top\-rr69\.2967\.7293\.0199\.4497\.0897\.4799\.5372\.5387\.01Merging error, top\-rr70\.3369\.0293\.6099\.4897\.1097\.7099\.5573\.5287\.54

#### Source of calibration subspaces\.

We first compare top\-rrsingular subspaces derived from fine\-tuned weights, task vectors, and merging errors, together with a random orthonormal baseline\. We construct the random baseline by sampling Gaussian random matrices and applying reduced QR decomposition\.

As shown in Table[3](https://arxiv.org/html/2610.33483#S5.T3), error\-derived bases achieve the highest average accuracy of87\.54%87\.54\\%, followed by task\-vector bases with87\.01%87\.01\\%and fine\-tuned\-weight bases with84\.80%84\.80\\%\. Fine\-tuned weights represent the full expert transformation and do not explicitly isolate the parameter changes introduced by fine\-tuning\. Task vectors capture these changes relative to the pretrained model, whereas merging errors isolate the discrepancy between the expert and the merged model being calibrated\. The observed ordering, therefore, supports constructing calibration subspaces from the discrepancy that the calibration is intended to correct\.

Table 4:We independently vary the read and write directions forESCalwithTask Arithmeticon ViT\-B/32 using rankr=128r=128\. Random\-rrdenotesrrdirections selected from the singular\-vector basis of the merging error\.Read basisWrite basisTop\-rrRandom\-rrBottom\-rrTop\-rr87\.54±0\.07\\mathbf\{87\.54\\pm 0\.07\}84\.24±0\.1884\.24\\pm 0\.1881\.69±0\.1281\.69\\pm 0\.12Random\-rr85\.96±0\.1585\.96\\pm 0\.1582\.93±0\.2282\.93\\pm 0\.2280\.89±0\.1980\.89\\pm 0\.19Bottom\-rr85\.35±0\.0785\.35\\pm 0\.0782\.63±0\.2082\.63\\pm 0\.2080\.86±0\.1380\.86\\pm 0\.13

#### Selection of read and write directions\.

We next examine which directions within the merging\-error subspaces are most effective for reading and writing corrections\. Table[4](https://arxiv.org/html/2610.33483#S5.T4)independently varies the read and write bases among the top\-rr, random\-rr, and bottom\-rrsingular directions of the merging error\. Unlike the random orthonormal baseline considered above, Random\-rrselectsrrdirections from the singular vectors of the merging error\. Using the top\-rrsingular directions for both the read and write bases achieves the highest average accuracy of87\.54%87\.54\\%\.

We find that the choice of write basis has a larger effect on performance\. For a fixed read basis, replacing the top\-rrwrite directions with random\-rror bottom\-rrdirections consistently causes substantial degradation\. This suggests that corrections should be written along the dominant output\-error directions rather than random or bottom singular directions\.

We also find that, for a fixed write basis, the top\-rrread directions consistently perform best, while the bottom\-rrdirections perform worst\. This indicates that, beyond correcting directions in which the current representation deviates from the expert, reading from input directions that most strongly affect the following transformation’s local output discrepancy is beneficial\. Together, these results support theESCaldesign of writing along the dominant current output\-error directions while reading from the dominant input\-error directions of the next transformation\.

Table 5:Constrained uses a fixed top\-rrerror basis\. Unconstrained learns the corresponding projection from a random initialization without restricting its directions to the error subspace\.ReadWriter=64r=64r=128r=128r=256r=256ConstrainedConstrained86\.91±0\.0686\.91\\pm 0\.0687\.54±0\.0787\.54\\pm 0\.0787\.62±0\.1287\.62\\pm 0\.12UnconstrainedConstrained87\.23±0\.0587\.23\\pm 0\.0587\.50±0\.0487\.50\\pm 0\.0487\.62±0\.1487\.62\\pm 0\.14ConstrainedUnconstrained87\.53±0\.1087\.53\\pm 0\.1087\.76±0\.0487\.76\\pm 0\.0487\.75±0\.0787\.75\\pm 0\.07
#### Role of the read\- and write\-subspace constraints\.

The preceding ablations examine which basis sources and singular directions are most effective when the read and write bases are fixed\. We next examine whether relaxing either subspace constraint provides additional gains by replacing the corresponding fixed basis with a learnable projection\. Specifically, we make either the read or write projection learnable from a random initialization while keeping the other constrained to its fixed top\-rrerror basis\.

As shown in Table[5](https://arxiv.org/html/2610.33483#S5.T5), relaxing the write constraint improves accuracy across all three ranks, whereas relaxing the read constraint provides a gain only atr=64r=64\. The more consistent gains from adapting the write projection further highlight the importance of the directions along which corrections are applied\. However, relaxing either constraint requires learning ad×rd\\times rprojection in place of the corresponding fixed basis, increasing the number of trainable parameters for that projection by a factor ofd/rd/rcompared with ther×rr\\times rcore\. Given the modest performance gains relative to this additional parameter cost, we keep both bases fixed inESCal\.

### 5\.2Prediction Alignment with Expert Models

To complement task\-performance evaluation, we measure how closely the calibrated merged models match expert predictions using KL divergence and prediction agreement\. Following the standard evaluation protocol for multi\-task model merging, we pair each calibrated merged backbone with the corresponding fixed expert head and compare its predictions with those of the expert\. For each task, we compute KL divergence from the expert predictive distribution to that of the calibrated merged model and prediction agreement, defined as the fraction of matching predicted labels, and report the arithmetic mean across tasks for KL divergence and the sample\-count\-weighted mean across tasks for prediction agreement\. For RoBERTa, we compute these metrics over the seven classification tasks, excluding STS\-B, which is a regression task\.

Table 6:Prediction alignment with expert models\.KL divergence and prediction agreement are reported across vision and NLU architectures\.MethodViT\-B/32ViT\-B/16ViT\-L/14RoBERTaKL↓\\downarrowAgreement↑\\uparrowKL↓\\downarrowAgreement↑\\uparrowKL↓\\downarrowAgreement↑\\uparrowKL↓\\downarrowAgreement↑\\uparrowTask Arithmetic0\.938374\.51%0\.791879\.56%0\.444786\.83%0\.692661\.38%w/SurgeryV20\.267190\.70%0\.205492\.94%0\.125995\.27%0\.287287\.76%w/FeatCal0\.320489\.48%0\.298790\.96%0\.161094\.40%0\.469783\.35%w/ESCal0\.182192\.09%0\.134194\.16%0\.091096\.09%0\.243988\.74%

WithTask Arithmeticas the base merging method,ESCalachieves the lowest KL divergence and the highest prediction agreement among the compared methods across all four evaluated architectures\. On ViT\-B/32, for example, relative toSurgeryV2,ESCalreduces KL divergence from0\.26710\.2671to0\.18210\.1821and increases prediction agreement from90\.70%90\.70\\%to92\.09%92\.09\\%\. These results show thatESCalmore closely matches expert predictions, with stronger alignment in both predictive distributions and predicted labels\.

### 5\.3Calibration Efficiency and Overhead

We examine the parameter efficiency and calibration data requirements ofESCalon ViT\-B/32 withTask Arithmetic\. We also assess the additional storage and inference overhead associated with its calibration modules\.

Table 7:Calibration cost and accuracy\.MethodTrainableparams\.Runtime\(min\)Avg\. acc\.\(%\)Surgery131,0728\.4872\.45SurgeryV22,490,36813\.7686\.22FeatCal00\.6385\.53ESCal1,572,86413\.2287\.54
Table 8:Calibration budget\.Examplesper taskAvg\. acc\.\(%\)3286\.48±0\.2186\.48\\pm 0\.216487\.54±0\.0787\.54\\pm 0\.0712888\.07±0\.0688\.07\\pm 0\.0625688\.29±0\.0788\.29\\pm 0\.07

#### Parameter efficiency and inference overhead\.

ESCalachieves higher average accuracy with36\.8%36\.8\\%fewer trainable parameters thanSurgeryV2 on ViT\-B/32 \(Table[8](https://arxiv.org/html/2610.33483#S5.T8)\), while requiring comparable calibration runtime under the selected configurations\. ForFeatCal, we report zero trainable parameters because its calibration updates are computed in closed form without gradient\-based optimization\. While only the core mappings are learned, the fixed read/write bases must also be retained at inference time, requiring additional storage beyond the trainable parameters\. ForTask Arithmeticon ViT\-B/32 withr=128r=128, the bases and cores together add2\.91%2\.91\\%per task to the shared image encoder’s parameter storage at the same precision\. We additionally measure the inference overhead relative to the uncalibrated model in terms of latency and throughput, and report these results together with detailed storage accounting in Appendix[D\.2](https://arxiv.org/html/2610.33483#A4.SS2)\.

#### Calibration data budget\.

We vary the number of calibration examples per task from 4 to 256 forESCalwithTask Arithmeticon ViT\-B/32 and evaluate on the full test sets\. Table[8](https://arxiv.org/html/2610.33483#S5.T8)reports the results from 32 to 256 examples, with the full sweep provided in Appendix[D\.3](https://arxiv.org/html/2610.33483#A4.SS3)\. Our default budget of 64 examples per task provides a favorable trade\-off between accuracy and calibration data requirements\.

## 6Conclusion and Limitations

We introducedESCal, a post\-merging calibration framework that restricts feature correction to low\-dimensional error subspaces derived from weight differences between expert and merged models\. ItsBridgemodules learn only small linear mappings between fixed subspaces, enabling parameter\-efficient calibration\. Across vision and natural language understanding, our results show that these error subspaces provide an effective structure for recovering task\-specific capabilities after merging\. This highlights the importance of identifying*which directions*to correct, in addition to optimizing representation alignment\.

#### Limitations & Future Directions\.

First, eachBridgelearns a low\-rank linear correction to the blockwise hidden\-representation gap between the merged model and an expert, with both subspace bases fixed\. This design draws on prior work on lightweight post\-merging calibration\([Yang et al\., 2024a](https://arxiv.org/html/2610.33483#bib.bib26);[Yang et al\., 2024b](https://arxiv.org/html/2610.33483#bib.bib27)\)\. Our experiments demonstrate its effectiveness in the evaluated settings, but its general applicability remains unverified\. Thus,ESCalmay be less effective when the required corrections fall outside the fixed error subspaces or require nonlinear transformations\.

Second, the error\-subspace dimensionrrmust be chosen beforeBridgetraining and may need to vary across architectures and domains\. We treatrras a hyperparameter, but do not systematically investigate its relationship with the correction capacity ofBridgemodules\. We leave this analysis and adaptive rank allocation to future work\.

Third,ESCalretains its fixed read/write bases at inference time, requiring additional storage beyond the learnedBridgecore modules\. However, this overhead is relatively small\. For example, withTask Arithmeticon ViT\-B/32 andr=128r=128, the bases and cores together add2\.91%2\.91\\%per task to the shared image encoder’s parameter storage at the same precision \(Table[19](https://arxiv.org/html/2610.33483#A4.T19)\)\.

Finally, we studyBridgemodules only at block boundaries\. Exploring alternative placements, including within blocks, could help clarify how correction placement affects representation alignment\. More expressive correction modules also remain an architectural direction for future work\.

## References

- Cheng et al\. \(2017\)Gong Cheng, Junwei Han, and Xiaoqiang Lu\.Remote sensing image scene classification: Benchmark and state of the art\.*Proceedings of the IEEE*, 105\(10\):1865–1883, 2017\.
- Cheng et al\. \(2025\)Runxi Cheng, Feng Xiong, Yongxian Wei, Wanyun Zhu, and Chun Yuan\.Whoever started the interference should end it: Guiding data\-free model merging via task vectors\.In*Forty\-second International Conference on Machine Learning*, 2025\.URL[https://openreview\.net/forum?id=xR9msNaREW](https://openreview.net/forum?id=xR9msNaREW)\.
- Cimpoi et al\. \(2014\)Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi\.Describing textures in the wild\.In*Proceedings of the IEEE conference on computer vision and pattern recognition*, pages 3606–3613, 2014\.
- Gargiulo et al\. \(2025\)Antonio Andrea Gargiulo, Donato Crisostomi, Maria Sofia Bucarelli, Simone Scardapane, Fabrizio Silvestri, and Emanuele Rodola\.Task singular vectors: Reducing task interference in model merging\.In*2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, pages 18695–18705\. IEEE, 2025\.
- Gu et al\. \(2026\)Yanggan Gu, Shuo Cai, Zihao Wang, Wenjun Wang, Yuanyi Wang, Pengkai Wang, Sirui Huang, Su Lu, Jianmin Wu, and Hongxia Yang\.FeatCal: Feature calibration for post\-merging models\.*arXiv preprint arXiv:2605\.13030*, 2026\.
- Helber et al\. \(2019\)Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth\.EuroSAT: A novel dataset and deep learning benchmark for land use and land cover classification\.*IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing*, 12\(7\):2217–2226, 2019\.
- Ilharco et al\. \(2023\)Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi\.Editing models with task arithmetic\.In*The Eleventh International Conference on Learning Representations*, 2023\.URL[https://openreview\.net/forum?id=6t0Kwf8\-jrj](https://openreview.net/forum?id=6t0Kwf8-jrj)\.
- Krause et al\. \(2013\)Jonathan Krause, Michael Stark, Jia Deng, and Li Fei\-Fei\.3D object representations for fine\-grained categorization\.In*Proceedings of the IEEE international conference on computer vision workshops*, pages 554–561, 2013\.
- LeCun et al\. \(1998\)Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner\.Gradient\-based learning applied to document recognition\.*Proceedings of the IEEE*, 86\(11\):2278–2324, 1998\.
- Liu et al\. \(2019\)Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov\.RoBERTa: A robustly optimized BERT pretraining approach\.*arXiv preprint arXiv:1907\.11692*, 2019\.
- Luo et al\. \(2025\)Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jian\-Guang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, Yansong Tang, et al\.WizardMath: Empowering mathematical reasoning for large language models via reinforced evol\-instruct\.In*International Conference on Learning Representations*, pages 49573–49609, 2025\.
- Marczak et al\. \(2025\)Daniel Marczak, Simone Magistri, Sebastian Cygert, Bartłomiej Twardowski, Andrew D\. Bagdanov, and Joost van de Weijer\.No task left behind: Isotropic model merging with common and task\-specific subspaces\.In*Forty\-second International Conference on Machine Learning*, 2025\.URL[https://openreview\.net/forum?id=RBZpAa27ls](https://openreview.net/forum?id=RBZpAa27ls)\.
- Netzer et al\. \(2011\)Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, and Andrew Y Ng\.Reading digits in natural images with unsupervised feature learning\.In*NIPS workshop on deep learning and unsupervised feature learning*, number 2, page 4\. Granada, 2011\.
- Osial et al\. \(2024\)Marcin Osial, Daniel Marczak, et al\.Parameter\-efficient interventions for enhanced model merging\.*arXiv preprint arXiv:2412\.17023*, 2024\.
- Radford et al\. \(2021\)Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al\.Learning transferable visual models from natural language supervision\.In*International conference on machine learning*, pages 8748–8763\. PMLR, 2021\.
- Skorobogat et al\. \(2025\)Ronald Skorobogat, Karsten Roth, and Mariana\-Iuliana Georgescu\.Subspace\-boosted model merging\.*arXiv preprint arXiv:2506\.16506*, 2025\.
- Stallkamp et al\. \(2011\)Johannes Stallkamp, Marc Schlipsing, Jan Salmen, and Christian Igel\.The German traffic sign recognition benchmark: a multi\-class classification competition\.In*The 2011 international joint conference on neural networks*, pages 1453–1460\. IEEE, 2011\.
- Stoica et al\. \(2025\)George Stoica, Pratik Ramesh, Boglarka Ecsedi, Leshem Choshen, and Judy Hoffman\.Model merging with SVD to tie the knots\.In*International Conference on Learning Representations*, volume 2025, pages 4501–4519, 2025\.
- Touvron et al\. \(2023\)Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al\.Llama 2: Open foundation and fine\-tuned chat models\.*arXiv preprint arXiv:2307\.09288*, 2023\.
- Wang et al\. \(2018\)Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman\.GLUE: A multi\-task benchmark and analysis platform for natural language understanding\.In*Proceedings of the 2018 EMNLP workshop BlackboxNLP: Analyzing and interpreting neural networks for NLP*, pages 353–355, 2018\.
- Wei et al\. \(2025\)Qi Wei, Shuo He, Enneng Yang, Tingcong Liu, Haobo Wang, Lei Feng, and Bo An\.Representation surgery in model merging with probabilistic modeling\.In*Forty\-second International Conference on Machine Learning*, 2025\.
- Xiao et al\. \(2010\)Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba\.SUN database: Large\-scale scene recognition from abbey to zoo\.In*2010 IEEE computer society conference on computer vision and pattern recognition*, pages 3485–3492\. IEEE, 2010\.
- Xu et al\. \(2024\)Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang\.WizardLM: Empowering large pre\-trained language models to follow complex instructions\.In*International Conference on Learning Representations*, pages 30745–30766, 2024\.
- Xu et al\. \(2025\)Jing Xu, Jiazheng Li, and Jingzhao Zhang\.Scalable model merging with progressive layer\-wise distillation\.In*Forty\-second International Conference on Machine Learning*, 2025\.URL[https://openreview\.net/forum?id=xX8NJShgny](https://openreview.net/forum?id=xX8NJShgny)\.
- Yadav et al\. \(2023\)Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal\.TIES\-Merging: Resolving interference when merging models\.*Advances in neural information processing systems*, 36:7093–7115, 2023\.
- Yang et al\. \(2024a\)Enneng Yang, Li Shen, Zhenyi Wang, Guibing Guo, Xiaojun Chen, Xingwei Wang, and Dacheng Tao\.Representation surgery for multi\-task model merging\.In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors,*Proceedings of the 41st International Conference on Machine Learning*, volume 235 of*Proceedings of Machine Learning Research*, pages 56332–56356\. PMLR, 21–27 Jul 2024a\.URL[https://proceedings\.mlr\.press/v235/yang24t\.html](https://proceedings.mlr.press/v235/yang24t.html)\.
- Yang et al\. \(2024b\)Enneng Yang, Li Shen, Zhenyi Wang, Guibing Guo, Xingwei Wang, Xiaocun Cao, Jie Zhang, and Dacheng Tao\.SurgeryV2: Bridging the gap between model merging and multi\-task learning with deep representation surgery\.*arXiv preprint arXiv:2410\.14389*, 2024b\.
- Yang et al\. \(2024c\)Enneng Yang, Zhenyi Wang, Li Shen, Shiwei Liu, Guibing Guo, Xingwei Wang, and Dacheng Tao\.AdaMerging: Adaptive model merging for multi\-task learning\.In*International Conference on Learning Representations*, pages 22743–22763, 2024c\.
- Yang et al\. \(2026\)Enneng Yang, Li Shen, Guibing Guo, Xingwei Wang, Xiaochun Cao, Jie Zhang, and Dacheng Tao\.Model merging in LLMs, MLLMs, and beyond: Methods, theories, applications, and opportunities\.*ACM Computing Surveys*, 58\(8\), February 2026\.ISSN 0360\-0300\.[10\.1145/3787849](https://doi.org/10.1145/3787849)\.URL[https://doi\.org/10\.1145/3787849](https://doi.org/10.1145/3787849)\.

Contents

## Appendix AAdditional Related Work on Model Merging

Several model\-merging approaches study the spectral structure of task updates to reduce interference and preserve task\-specific information\.TSV\-Merge\([Gargiulo et al\., 2025](https://arxiv.org/html/2610.33483#bib.bib4)\)decomposes layer\-wise task matrices into singular components, retains their dominant directions, and orthogonalizes these singular vectors across tasks before reconstructing the merged update\. This combines low\-rank compression with interference reduction\.Iso\-Merging\([Marczak et al\., 2025](https://arxiv.org/html/2610.33483#bib.bib12)\)studies the alignment between task\-specific and merged updates and proposes flattening the singular value spectrum to balance the contributions of different directions\. Its framework also incorporates common and task\-specific subspaces to preserve information that may be underrepresented in the aggregate update\.Subspace Boosting\([Skorobogat et al\., 2025](https://arxiv.org/html/2610.33483#bib.bib16)\)examines rank collapse as more experts are merged and boosts smaller singular values of the merged task update to retain a broader set of task\-relevant directions\. Alignment between task updates also matters when merging models fine\-tuned with low\-rank adaptation\.KnOTS\([Stoica et al\., 2025](https://arxiv.org/html/2610.33483#bib.bib18)\)observes that LoRA task updates exhibit less alignment than their fully fine\-tuned counterparts, which can limit the effectiveness of existing merging methods\. It uses SVD to jointly transform the task updates into a common aligned space, applies an existing merging method in that space, and reconstructs the merged update\.

## Appendix BMotivation for Error\-Subspace Calibration

We examine the discrepancy between a task\-specific model and the merged model across consecutive transformations\. We consider two properties of the merging error: the subspace containing the locally induced discrepancy at the current transformation, and the input directions relevant to the merging error of the following transformation\. Here, the transformations are the selected linear maps used for the error decompositions, not entire nonlinear Transformer blocks\. We use a row\-vector convention in which a linear transformation maps𝒙\{\\bm\{x\}\}to𝒙​𝑾\{\\bm\{x\}\}\{\\bm\{W\}\}, with all weight matrices written in input\-by\-output form\.

We assume that the expert and merged model share the same architecture, with potentially different weights\. We fix a task indext∈\[T\]t\\in\[T\]and omit it when clear from context\.

#### Local merging error lies in the output error subspace\.

For a blockℓ∈\[L\]\\ell\\in\[L\], choose the same linear layer and take its weight matrix:𝑾ℓt\{\\bm\{W\}\}\_\{\\ell\}^\{t\}for the expert and𝑾¯ℓ\\overline\{\{\\bm\{W\}\}\}\_\{\\ell\}for the merged model\. Recall that the corresponding merging error is𝑬ℓt=𝑾ℓt−𝑾¯ℓ\{\\bm\{E\}\}\_\{\\ell\}^\{t\}=\{\\bm\{W\}\}\_\{\\ell\}^\{t\}\-\\overline\{\{\\bm\{W\}\}\}\_\{\\ell\}\. Write its compact singular value decomposition \(SVD\) as𝑬ℓt=𝑼ℓt​𝚺ℓt​𝑽ℓt⊤\{\\bm\{E\}\}\_\{\\ell\}^\{t\}=\{\\bm\{U\}\}\_\{\\ell\}^\{t\}\{\\bm\{\\Sigma\}\}\_\{\\ell\}^\{t\}\{\\bm\{V\}\}\_\{\\ell\}^\{t\\top\}, where𝚺ℓt\{\\bm\{\\Sigma\}\}\_\{\\ell\}^\{t\}contains all nonzero singular values in descending order\. Under our row\-vector convention, the columns of𝑼ℓt\{\\bm\{U\}\}\_\{\\ell\}^\{t\}and𝑽ℓt\{\\bm\{V\}\}\_\{\\ell\}^\{t\}span the input\-side and output\-side error subspaces of this transformation, respectively\.

For a shared input representation𝒙\{\\bm\{x\}\}, the discrepancy between the task\-specific and merged transformations is

𝒙​𝑾ℓt−𝒙​𝑾¯ℓ\\displaystyle\{\\bm\{x\}\}\{\\bm\{W\}\}\_\{\\ell\}^\{t\}\-\{\\bm\{x\}\}\\overline\{\{\\bm\{W\}\}\}\_\{\\ell\}=𝒙​𝑬ℓt\\displaystyle=\{\\bm\{x\}\}\{\\bm\{E\}\}\_\{\\ell\}^\{t\}\(8\)=\(𝒙​𝑼ℓt\)​𝚺ℓt​𝑽ℓt⊤\.\\displaystyle=\\left\(\{\\bm\{x\}\}\{\\bm\{U\}\}\_\{\\ell\}^\{t\}\\right\)\{\\bm\{\\Sigma\}\}\_\{\\ell\}^\{t\}\{\\bm\{V\}\}\_\{\\ell\}^\{t\\top\}\.Therefore,

\(𝒙​𝑬ℓt\)⊤∈span⁡\(𝑽ℓt\)\\left\(\{\\bm\{x\}\}\{\\bm\{E\}\}\_\{\\ell\}^\{t\}\\right\)^\{\\top\}\\in\\operatorname\{span\}\(\{\\bm\{V\}\}\_\{\\ell\}^\{t\}\)\(9\)for any shared input𝒙\{\\bm\{x\}\}\. Thus, the discrepancy directly introduced by the weight difference at transformationℓ\\ellis confined to the output singular subspace of the merging error\.

This observation also implies that a correction outside this subspace cannot reduce the locally induced discrepancy\. Let a correction be decomposed as

Δ𝒉ℓ=Δ𝒉ℓ,∥\+Δ𝒉ℓ,⟂,\\Delta\{\\bm\{h\}\}\_\{\\ell\}=\\Delta\{\\bm\{h\}\}\_\{\\ell,\\parallel\}\+\\Delta\{\\bm\{h\}\}\_\{\\ell,\\perp\},\(10\)where

Δ𝒉ℓ,∥⊤∈span\(𝑽ℓt\),Δ𝒉ℓ,⟂⊤⟂span\(𝑽ℓt\)\.\\Delta\{\\bm\{h\}\}\_\{\\ell,\\parallel\}^\{\\top\}\\in\\operatorname\{span\}\(\{\\bm\{V\}\}\_\{\\ell\}^\{t\}\),\\qquad\\Delta\{\\bm\{h\}\}\_\{\\ell,\\perp\}^\{\\top\}\\perp\\operatorname\{span\}\(\{\\bm\{V\}\}\_\{\\ell\}^\{t\}\)\.\(11\)Since\(𝒙​𝑬ℓt\)⊤\\left\(\{\\bm\{x\}\}\{\\bm\{E\}\}\_\{\\ell\}^\{t\}\\right\)^\{\\top\}lies inspan⁡\(𝑽ℓt\)\\operatorname\{span\}\(\{\\bm\{V\}\}\_\{\\ell\}^\{t\}\),

‖𝒙​𝑬ℓt−Δ​𝒉ℓ‖22\\displaystyle\\left\\\|\{\\bm\{x\}\}\{\\bm\{E\}\}\_\{\\ell\}^\{t\}\-\\Delta\{\\bm\{h\}\}\_\{\\ell\}\\right\\\|\_\{2\}^\{2\}=‖𝒙𝑬ℓt−Δ𝒉ℓ,∥‖22\+‖Δ𝒉ℓ,⟂‖22\.\\displaystyle=\\left\\\|\{\\bm\{x\}\}\{\\bm\{E\}\}\_\{\\ell\}^\{t\}\-\\Delta\{\\bm\{h\}\}\_\{\\ell,\\parallel\}\\right\\\|\_\{2\}^\{2\}\+\\left\\\|\\Delta\{\\bm\{h\}\}\_\{\\ell,\\perp\}\\right\\\|\_\{2\}^\{2\}\.\(12\)Hence, the orthogonal componentΔ​𝒉ℓ,⟂\\Delta\{\\bm\{h\}\}\_\{\\ell,\\perp\}cannot improve the correction of the discrepancy directly induced by𝑬ℓt\{\\bm\{E\}\}\_\{\\ell\}^\{t\}\. This motivates restricting the correction tospan⁡\(𝑽ℓt\)\\operatorname\{span\}\(\{\\bm\{V\}\}\_\{\\ell\}^\{t\}\)\.

This argument concerns the local discrepancy induced by the weight difference under a shared input\. The total representation discrepancy may also contain errors propagated from preceding layers\.

#### The following transformation identifies error\-relevant input directions\.

We next consider the merging error of the transformation following the calibration location\. For an input representation𝒉\{\\bm\{h\}\},

𝒉​𝑬ℓ\+1t=\(𝒉​𝑼ℓ\+1t\)​𝚺ℓ\+1t​𝑽ℓ\+1t⊤\.\{\\bm\{h\}\}\{\\bm\{E\}\}\_\{\\ell\+1\}^\{t\}=\\left\(\{\\bm\{h\}\}\{\\bm\{U\}\}\_\{\\ell\+1\}^\{t\}\\right\)\{\\bm\{\\Sigma\}\}\_\{\\ell\+1\}^\{t\}\{\\bm\{V\}\}\_\{\\ell\+1\}^\{t\\top\}\.\(13\)The discrepancy induced by the following transformation therefore depends on𝒉\{\\bm\{h\}\}through its coordinates along𝑼ℓ\+1t\{\\bm\{U\}\}\_\{\\ell\+1\}^\{t\}\.

More specifically, let𝒖ℓ\+1,it\{\\bm\{u\}\}\_\{\\ell\+1,i\}^\{t\}and𝒗ℓ\+1,it\{\\bm\{v\}\}\_\{\\ell\+1,i\}^\{t\}denote theii\-th left and right singular vectors of𝑬ℓ\+1t\{\\bm\{E\}\}\_\{\\ell\+1\}^\{t\}\. Then

𝒖ℓ\+1,it⊤​𝑬ℓ\+1t=σℓ\+1,it​𝒗ℓ\+1,it⊤,\{\\bm\{u\}\}\_\{\\ell\+1,i\}^\{t\\top\}\{\\bm\{E\}\}\_\{\\ell\+1\}^\{t\}=\\sigma\_\{\\ell\+1,i\}^\{t\}\{\\bm\{v\}\}\_\{\\ell\+1,i\}^\{t\\top\},\(14\)and, since𝒗ℓ\+1,it\{\\bm\{v\}\}\_\{\\ell\+1,i\}^\{t\}has unit norm,

‖𝒖ℓ\+1,it⊤​𝑬ℓ\+1t‖2=σℓ\+1,it\.\\left\\\|\{\\bm\{u\}\}\_\{\\ell\+1,i\}^\{t\\top\}\{\\bm\{E\}\}\_\{\\ell\+1\}^\{t\}\\right\\\|\_\{2\}=\\sigma\_\{\\ell\+1,i\}^\{t\}\.\(15\)Thus,σℓ\+1,it\\sigma\_\{\\ell\+1,i\}^\{t\}measures the magnitude of the discrepancy induced by the error operator for a unit input along𝒖ℓ\+1,it\{\\bm\{u\}\}\_\{\\ell\+1,i\}^\{t\}\. Larger singular values therefore identify input directions along which the following transformation’s weight difference can induce larger local discrepancies\. This motivates using𝑼ℓ\+1t\{\\bm\{U\}\}\_\{\\ell\+1\}^\{t\}as the read subspace ofBridge\.

#### Scope of the argument\.

The quantity𝒉​𝑬ℓ\+1t\{\\bm\{h\}\}\{\\bm\{E\}\}\_\{\\ell\+1\}^\{t\}is a local discrepancy caused by the following transformation’s weight difference\. It is distinct from the propagation of an existing representation perturbationΔ​𝒉\\Delta\{\\bm\{h\}\}through the merged linear transformation, which isΔ​𝒉​𝑾¯ℓ\+1\\Delta\{\\bm\{h\}\}\\overline\{\{\\bm\{W\}\}\}\_\{\\ell\+1\}\. For a nonlinear block, such propagation depends on the block’s local Jacobian\. Thus, the argument motivates the read and write subspaces ofBridge, but does not imply optimality with respect to propagated error or the total discrepancy of the nonlinear network\.

#### Why retain the leading rank\-rrsubspaces?

The arguments above concern the error subspaces spanned by all singular vectors associated with nonzero singular values\. In practice, we retain only the leadingrrdirections\. For any transformationℓ\\ell, the rank\-rrtruncated SVD satisfies

‖𝑬ℓt−𝑼ℓ,rt​𝚺ℓ,rt​𝑽ℓ,rt⊤‖F2=∑i\>r\(σℓ,it\)2\.\\left\\\|\{\\bm\{E\}\}\_\{\\ell\}^\{t\}\-\{\\bm\{U\}\}\_\{\\ell,r\}^\{t\}\{\\bm\{\\Sigma\}\}\_\{\\ell,r\}^\{t\}\{\\bm\{V\}\}\_\{\\ell,r\}^\{t\\top\}\\right\\\|\_\{F\}^\{2\}=\\sum\_\{i\>r\}\\left\(\\sigma\_\{\\ell,i\}^\{t\}\\right\)^\{2\}\.\(16\)Truncated SVD minimizes this residual among rank\-rrapproximations of the merging\-error operator\. For a fixed rank budget, it retains the dominant merging\-error directions and minimizes the discarded error energy in the Frobenius norm\. This optimality concerns approximation of the error matrix, not minimization of a data\-weighted feature loss or end\-to\-end prediction error\.

These observations motivate using𝑽ℓ,rt\{\\bm\{V\}\}\_\{\\ell,r\}^\{t\}as the output space for the correction at layerℓ\\elland𝑼ℓ\+1,rt\{\\bm\{U\}\}\_\{\\ell\+1,r\}^\{t\}as the input space from which the correction is determined\. Accordingly, ourBridgemodule is parameterized as

𝑩ℓt=𝑼ℓ\+1,rt​𝑪ℓt​𝑽ℓ,rt⊤,𝑪ℓt∈ℝr×r\.\{\\bm\{B\}\}\_\{\\ell\}^\{t\}=\{\\bm\{U\}\}\_\{\\ell\+1,r\}^\{t\}\{\\bm\{C\}\}\_\{\\ell\}^\{t\}\{\\bm\{V\}\}\_\{\\ell,r\}^\{t\\top\},\\qquad\{\\bm\{C\}\}\_\{\\ell\}^\{t\}\\in\\mathbb\{R\}^\{r\\times r\}\.\(17\)
The error subspaces are fixed by the task\-specific and merged weights, while calibration learns only the mapping𝑪ℓt\{\\bm\{C\}\}\_\{\\ell\}^\{t\}between them\.

The resulting feature correction can then be written as

Δ​𝒉ℓt=𝒉ℓ​𝑩ℓt=\(𝒉ℓ​𝑼ℓ\+1,rt\)​𝑪ℓt​𝑽ℓ,rt⊤\.\\Delta\{\\bm\{h\}\}\_\{\\ell\}^\{t\}=\{\\bm\{h\}\}\_\{\\ell\}\{\\bm\{B\}\}\_\{\\ell\}^\{t\}=\\left\(\{\\bm\{h\}\}\_\{\\ell\}\{\\bm\{U\}\}\_\{\\ell\+1,r\}^\{t\}\\right\)\{\\bm\{C\}\}\_\{\\ell\}^\{t\}\{\\bm\{V\}\}\_\{\\ell,r\}^\{t\\top\}\.\(18\)Hence,Bridgereads coordinates from the following transformation’s input error subspace, maps them through𝑪ℓt\{\\bm\{C\}\}\_\{\\ell\}^\{t\}, and writes the resulting correction into the current transformation’s output error subspace\.

## Appendix CExperimental Details and Additional Results

We report results for additional model architectures, base merging methods, and NLG task combinations, along with full per\-task results and variation across random seeds\. Unless otherwise noted, post\-hoc calibration results are reported as the mean±\\pmsample standard deviation over five random seeds\. For stochastic base merging methods, we also report five\-seed statistics where available\. Results from a single evaluation omit standard deviations\.

### C\.1Experimental Settings

#### Vision\.

We consider three CLIP vision transformer architectures, ViT\-B/32, ViT\-B/16, and ViT\-L/14\([Radford et al\., 2021](https://arxiv.org/html/2610.33483#bib.bib15)\)\. For each architecture, we use task\-specific models on eight image classification datasets: SUN397\([Xiao et al\., 2010](https://arxiv.org/html/2610.33483#bib.bib22)\), Cars\([Krause et al\., 2013](https://arxiv.org/html/2610.33483#bib.bib8)\), RESISC45\([Cheng et al\., 2017](https://arxiv.org/html/2610.33483#bib.bib1)\), EuroSAT\([Helber et al\., 2019](https://arxiv.org/html/2610.33483#bib.bib6)\), SVHN\([Netzer et al\., 2011](https://arxiv.org/html/2610.33483#bib.bib13)\), GTSRB\([Stallkamp et al\., 2011](https://arxiv.org/html/2610.33483#bib.bib17)\), MNIST\([LeCun et al\., 1998](https://arxiv.org/html/2610.33483#bib.bib9)\), and DTD\([Cimpoi et al\., 2014](https://arxiv.org/html/2610.33483#bib.bib3)\)\. The eight task\-specific models are merged within each architecture, and performance is evaluated on the corresponding test sets\. We report task\-wise classification accuracy and its average across the eight tasks\.

#### NLU\.

We use RoBERTa\-base\([Liu et al\., 2019](https://arxiv.org/html/2610.33483#bib.bib10)\)as the shared pretrained model and consider eight tasks from the GLUE benchmark\([Wang et al\., 2018](https://arxiv.org/html/2610.33483#bib.bib20)\): CoLA, SST\-2, MRPC, STS\-B, QQP, MNLI, QNLI, and RTE\. We merge the eight task\-specific models and evaluate the resulting model using the corresponding task\-specific metrics\. The reported average is the arithmetic mean of the eight task scores\.

#### NLG\.

We use LLaMA\-2\-13B\([Touvron et al\., 2023](https://arxiv.org/html/2610.33483#bib.bib19)\)as the common base model and consider task experts specialized for instruction following, mathematical reasoning, and code generation\. The instruction\-following and mathematical\-reasoning experts are based on WizardLM\([Xu et al\., 2024](https://arxiv.org/html/2610.33483#bib.bib23)\)and WizardMath\([Luo et al\., 2025](https://arxiv.org/html/2610.33483#bib.bib11)\), respectively\. We evaluate all three pairwise combinations of the task experts\. Detailed evaluation settings for the NLG experiments are provided together with the corresponding results\.

### C\.2Hyperparameter Search

We select hyperparameters independently for each base merging method and benchmark setting\. Here,rrdenotes the calibration rank,η\\etathe learning rate, andEEthe number of training epochs\. We choose method\-specific rank ranges with reference to trainable parameter budgets\. In the selected configurations,ESCaluses more trainable parameters thanSurgeryand fewer thanSurgeryV2\.

#### Vision\.

ForSurgeryandSurgeryV2, we fix the calibration rank atr=16r=16\. Then we search the learning rateη∈\{1e\-4,3e\-4,1e\-3\}\\eta\\in\\\{\\text\{1e\-4\},\\text\{3e\-4\},\\text\{1e\-3\}\\\}and the number of training epochsE∈\{50,100,200,250,500\}E\\in\\\{50,100,200,250,500\\\}\. The search extends to 500 epochs \(1,000 outer iterations under our vision calibration setup\), while the selected best setting for accuracy is 250 epochs \(500 outer iterations\)\. The separate timing comparison in Table[20](https://arxiv.org/html/2610.33483#A4.T20)uses 200 epochs\. ForFeatCal, across all vision architectures and base merging methods, we fixλ=0\.05\\lambda=0\.05and searchρ∈\{1,1\.5,2,2\.5\}\\rho\\in\\\{1,1\.5,2,2\.5\\\}andα∈\{0\.1,0\.3,0\.5\}\\alpha\\in\\\{0\.1,0\.3,0\.5\\\}\. ForESCal, we fixr=128r=128and searchη∈\{1e\-4,3e\-4,1e\-3\}\\eta\\in\\\{\\text\{1e\-4\},\\text\{3e\-4\},\\text\{1e\-3\}\\\}andE∈\{50,100,200\}E\\in\\\{50,100,200\\\}\.

#### NLU\.

ForSurgery,SurgeryV2, andESCal, we searchE∈\{200,500,1000\}E\\in\\\{200,500,1000\\\}\. ForSurgeryandSurgeryV2, we searchr∈\{4,16,64\}r\\in\\\{4,16,64\\\}andη∈\{1e\-5,3e\-5,1e\-4,3e\-4,1e\-3,3e\-3,1e\-2\}\\eta\\in\\\{\\text\{1e\-5\},\\text\{3e\-5\},\\text\{1e\-4\},\\text\{3e\-4\},\\text\{1e\-3\},\\text\{3e\-3\},\\text\{1e\-2\}\\\}\. ForFeatCal, we searchλ∈\{1e\-3,3e\-3,1e\-2,0\.05,0\.2\}\\lambda\\in\\\{\\text\{1e\-3\},\\text\{3e\-3\},\\text\{1e\-2\},0\.05,0\.2\\\},ρ∈\{1,1\.5,2,2\.5\}\\rho\\in\\\{1,1\.5,2,2\.5\\\}, andα∈\{0\.1,0\.3,0\.5,0\.7,0\.9,1\.0\}\\alpha\\in\\\{0\.1,0\.3,0\.5,0\.7,0\.9,1\.0\\\}\. ForESCal, we searchr∈\{32,64,128\}r\\in\\\{32,64,128\\\}andη∈\{1e\-4,3e\-4,1e\-3,3e\-3,6e\-3,1e\-2\}\\eta\\in\\\{\\text\{1e\-4\},\\text\{3e\-4\},\\text\{1e\-3\},\\text\{3e\-3\},\\text\{6e\-3\},\\text\{1e\-2\}\\\}\.

#### NLG\.

ForSurgery, we searchr∈\{16,32\}r\\in\\\{16,32\\\},η∈\{1e\-6,1e\-5,1e\-4,1e\-3\}\\eta\\in\\\{\\text\{1e\-6\},\\text\{1e\-5\},\\text\{1e\-4\},\\text\{1e\-3\}\\\}, andE∈\{5,25,50\}E\\in\\\{5,25,50\\\}\. ForSurgeryV2, we searchr∈\{8,16\}r\\in\\\{8,16\\\},η∈\{1e\-5,1e\-4,1e\-3\}\\eta\\in\\\{\\text\{1e\-5\},\\text\{1e\-4\},\\text\{1e\-3\}\\\}, andE∈\{5,25,50\}E\\in\\\{5,25,50\\\}\. Because NLG evaluation is computationally expensive, we use a smaller hyperparameter grid forFeatCal\. We fixρ=1\\rho=1and searchλ∈\{1e\-3,1e\-1\}\\lambda\\in\\\{\\text\{1e\-3\},\\text\{1e\-1\}\\\}andα∈\{0\.1,0\.3,0\.5\}\\alpha\\in\\\{0\.1,0\.3,0\.5\\\}\. ForESCal, we searchr∈\{64,128,256\}r\\in\\\{64,128,256\\\},η∈\{1e\-6,1e\-5,1e\-4,1e\-3\}\\eta\\in\\\{\\text\{1e\-6\},\\text\{1e\-5\},\\text\{1e\-4\},\\text\{1e\-3\}\\\}, andE∈\{5,25,50\}E\\in\\\{5,25,50\\\}\.

#### IntervMerge\.

ForIntervMerge, we select a configuration independently for each architecture and base merging method using the existing seed\-0 search results\. For vision, we search ranksr∈\{1,4,16\}r\\in\\\{1,4,16\\\}, learning ratesη∈\{10−4,3×10−4,10−3\}\\eta\\in\\\{10^\{\-4\},3\\times 10^\{\-4\},10^\{\-3\}\\\}, and outer\-iteration countsK∈\{100,200,400,500,1000\}K\\in\\\{100,200,400,500,1000\\\}\. The selection score is the arithmetic mean of the eight task test accuracies on seed 0; thus, the test sets are also used for configuration selection\. For NLU, we searchr∈\{1,4,16\}r\\in\\\{1,4,16\\\},η∈\{10−4,3×10−4,10−3,3×10−3,10−2\}\\eta\\in\\\{10^\{\-4\},3\\times 10^\{\-4\},10^\{\-3\},3\\times 10^\{\-3\},10^\{\-2\}\\\}, andE∈\{100,200,500,1000\}E\\in\\\{100,200,500,1000\\\}epochs, selecting by the eight\-task mean GLUE validation score on seed 0\. After selection, each configuration is fixed for seeds 0–4\. We reuse the matching seed\-0 search result and report the mean and sample standard deviation over these five runs; seeds 1–4 are not used for configuration selection\. Both settings use 64 calibration examples per task, batch size 16, Adam with constant learning rate, no warmup, and zero weight decay\. The merged backbone remains frozen while task\-specific interventions on the attention\-output CLS representation are trained by final\-representation L1 matching\. Vision uses two batches per task per outer iteration; NLU uses full passes over the task\-specific calibration sets per epoch\.

Table 9:Selected IntervMerge configurations\. Each vision entry is\(r,η,K\)\(r,\\eta,K\), withKKmeasured in outer iterations\.Base merging methodViT\-B/32ViT\-B/16ViT\-L/14Task Arithmetic\(16,×10−4,1000\)\(16,3\\\!\\times\\\!10^\{\-4\},1000\)\(16,10−3,200\)\(16,10^\{\-3\},200\)\(4,10−3,400\)\(4,10^\{\-3\},400\)TIES\-Merging\(16,×10−4,500\)\(16,3\\\!\\times\\\!10^\{\-4\},500\)\(16,10−3,1000\)\(16,10^\{\-3\},1000\)\(16,×10−4,1000\)\(16,3\\\!\\times\\\!10^\{\-4\},1000\)AdaMerging\(tw\)\(16,10−3,400\)\(16,10^\{\-3\},400\)\(16,10−3,400\)\(16,10^\{\-3\},400\)\(4,×10−4,1000\)\(4,3\\\!\\times\\\!10^\{\-4\},1000\)AdaMerging\(lw\)\(16,×10−4,400\)\(16,3\\\!\\times\\\!10^\{\-4\},400\)\(16,10−3,100\)\(16,10^\{\-3\},100\)\(16,×10−4,100\)\(16,3\\\!\\times\\\!10^\{\-4\},100\)WUDI\-Merging\(16,10−4,500\)\(16,10^\{\-4\},500\)\(16,10−3,100\)\(16,10^\{\-3\},100\)\(16,×10−4,1000\)\(16,3\\\!\\times\\\!10^\{\-4\},1000\)ProDistill\(16,10−3,1000\)\(16,10^\{\-3\},1000\)\(16,×10−4,1000\)\(16,3\\\!\\times\\\!10^\{\-4\},1000\)\(16,×10−4,500\)\(16,3\\\!\\times\\\!10^\{\-4\},500\)Table 10:Selected IntervMerge configurations for RoBERTa\-base\.Base merging methodRankrrLearning rateη\\etaEpochsEETask Arithmetic43×10−33\\times 10^\{\-3\}500TIES\-Merging1610−310^\{\-3\}200AdaMerging\(tw\)163×10−33\\times 10^\{\-3\}500AdaMerging\(lw\)43×10−33\\times 10^\{\-3\}1000WUDI\-Merging1610−310^\{\-3\}1000ProDistill163×10−33\\times 10^\{\-3\}1000

### C\.3Vision Results

Tables[11](https://arxiv.org/html/2610.33483#A3.T11),[12](https://arxiv.org/html/2610.33483#A3.T12), and[13](https://arxiv.org/html/2610.33483#A3.T13)report the complete task\-wise test accuracies for ViT\-B/32, ViT\-B/16, and ViT\-L/14, respectively, across all six base merging methods and five post\-merging calibration methods\. TheAvg\.column reports the mean accuracy across the eight tasks\.

ESCalconsistently improves over the corresponding uncalibrated models and achieves the highest average accuracy among the compared calibration methods in most settings\.FeatCalperforms best forProDistillacross all three architectures\.

Table 11:Full task\-wise test accuracy \(%\) on ViT\-B/32\.MethodSUN397CarsRESISC45EuroSATSVHNGTSRBMNISTDTDAvg\.Task Arithmetic55\.3254\.9866\.8277\.7880\.2169\.6897\.3450\.3769\.06w/Surgery60\.39±\\pm0\.4652\.62±\\pm0\.5971\.38±\\pm1\.7688\.65±\\pm0\.7981\.67±\\pm1\.6574\.27±\\pm0\.9197\.82±\\pm0\.1652\.77±\\pm2\.7372\.45±\\pm0\.52w/SurgeryV270\.70±\\pm0\.1966\.41±\\pm0\.4990\.65±\\pm1\.2399\.12±\\pm0\.1896\.61±\\pm0\.0795\.88±\\pm0\.6299\.47±\\pm0\.0670\.90±\\pm0\.7586\.22±\\pm0\.17w/IntervMerge66\.35±\\pm0\.3858\.87±\\pm0\.8385\.88±\\pm1\.5597\.61±\\pm0\.3293\.44±\\pm0\.2287\.49±\\pm1\.0998\.62±\\pm0\.1064\.31±\\pm1\.1081\.57±\\pm0\.29w/FeatCal69\.2270\.3189\.0698\.0095\.0192\.4099\.1371\.1185\.53w/ESCal70\.33±\\pm0\.2269\.02±\\pm0\.1993\.60±\\pm0\.3699\.48±\\pm0\.0797\.10±\\pm0\.0597\.70±\\pm0\.2299\.55±\\pm0\.0373\.52±\\pm0\.4487\.54±\\pm0\.07TIES\-Merging65\.0564\.4374\.9276\.4881\.2869\.3796\.5354\.5272\.82w/Surgery67\.64±\\pm0\.3959\.57±\\pm0\.5478\.50±\\pm1\.7789\.24±\\pm1\.3081\.30±\\pm1\.4376\.07±\\pm0\.2997\.14±\\pm0\.3160\.03±\\pm1\.8176\.19±\\pm0\.39w/SurgeryV270\.87±\\pm0\.2066\.26±\\pm0\.7590\.14±\\pm1\.7198\.94±\\pm0\.2196\.54±\\pm0\.0995\.52±\\pm0\.7799\.45±\\pm0\.0770\.89±\\pm0\.7286\.08±\\pm0\.21w/IntervMerge68\.65±\\pm0\.2860\.59±\\pm1\.1286\.21±\\pm1\.7597\.39±\\pm0\.5292\.99±\\pm0\.2387\.60±\\pm0\.9798\.24±\\pm0\.1865\.14±\\pm1\.1782\.10±\\pm0\.26w/FeatCal70\.12±\\pm0\.0270\.74±\\pm0\.1789\.28±\\pm0\.3197\.93±\\pm0\.1394\.71±\\pm0\.0292\.55±\\pm0\.2899\.10±\\pm0\.0371\.13±\\pm0\.7085\.69±\\pm0\.13w/ESCal70\.93±\\pm0\.2369\.23±\\pm0\.1893\.31±\\pm0\.4899\.36±\\pm0\.1097\.09±\\pm0\.0397\.70±\\pm0\.2599\.53±\\pm0\.0473\.35±\\pm0\.4387\.56±\\pm0\.07AdaMerging\(tw\)58\.80±\\pm3\.1653\.42±\\pm1\.7664\.91±\\pm6\.6386\.51±\\pm1\.8679\.67±\\pm4\.4183\.34±\\pm4\.1390\.32±\\pm2\.6049\.55±\\pm8\.5670\.82±\\pm1\.04w/Surgery61\.90±\\pm2\.4849\.77±\\pm2\.2569\.97±\\pm7\.0392\.50±\\pm1\.0380\.29±\\pm4\.9385\.75±\\pm4\.3294\.73±\\pm1\.0452\.66±\\pm9\.3973\.45±\\pm1\.22w/SurgeryV269\.86±\\pm1\.8661\.67±\\pm2\.4388\.89±\\pm2\.8399\.25±\\pm0\.1396\.37±\\pm0\.2296\.97±\\pm0\.8299\.32±\\pm0\.1568\.57±\\pm5\.3685\.11±\\pm0\.55w/IntervMerge66\.08±\\pm2\.2153\.08±\\pm3\.7082\.23±\\pm4\.7698\.36±\\pm0\.3792\.45±\\pm1\.4692\.67±\\pm2\.4596\.77±\\pm1\.3361\.95±\\pm7\.1980\.45±\\pm0\.86w/FeatCal68\.86±\\pm2\.2166\.32±\\pm2\.1486\.92±\\pm2\.3498\.42±\\pm0\.3994\.72±\\pm0\.5894\.68±\\pm1\.1398\.51±\\pm0\.3769\.31±\\pm4\.9984\.72±\\pm0\.52w/ESCal70\.10±\\pm1\.6565\.55±\\pm1\.4392\.79±\\pm1\.3299\.59±\\pm0\.0597\.03±\\pm0\.1298\.06±\\pm0\.2799\.51±\\pm0\.0572\.35±\\pm3\.0786\.87±\\pm0\.29AdaMerging\(lw\)63\.28±\\pm2\.8966\.71±\\pm2\.1679\.15±\\pm3\.5089\.81±\\pm1\.3183\.82±\\pm3\.2590\.56±\\pm3\.9197\.87±\\pm0\.4863\.28±\\pm3\.7679\.31±\\pm0\.76w/Surgery66\.33±\\pm2\.1964\.87±\\pm2\.8884\.24±\\pm2\.5694\.24±\\pm0\.4687\.64±\\pm2\.5792\.83±\\pm3\.2498\.30±\\pm0\.4167\.12±\\pm3\.6181\.95±\\pm0\.85w/SurgeryV271\.45±\\pm1\.2670\.24±\\pm2\.2792\.79±\\pm0\.7199\.14±\\pm0\.0796\.54±\\pm0\.1298\.08±\\pm0\.3199\.54±\\pm0\.0375\.45±\\pm1\.5687\.90±\\pm0\.31w/IntervMerge68\.94±\\pm2\.0966\.72±\\pm2\.7489\.75±\\pm1\.0498\.17±\\pm0\.3192\.83±\\pm0\.7896\.63±\\pm0\.8298\.83±\\pm0\.2172\.34±\\pm2\.7885\.53±\\pm0\.37w/FeatCal70\.68±\\pm1\.7074\.08±\\pm1\.4191\.16±\\pm0\.5998\.05±\\pm0\.1394\.69±\\pm0\.2797\.03±\\pm0\.5299\.21±\\pm0\.0975\.06±\\pm1\.3187\.50±\\pm0\.31w/ESCal69\.51±\\pm1\.8970\.99±\\pm1\.4393\.40±\\pm0\.3699\.48±\\pm0\.1197\.04±\\pm0\.0698\.38±\\pm0\.2399\.58±\\pm0\.0374\.87±\\pm0\.9487\.91±\\pm0\.24WUDI\-Merging69\.1171\.3885\.6295\.6394\.2794\.5199\.4769\.4784\.93w/Surgery69\.24±\\pm0\.2269\.42±\\pm0\.2287\.70±\\pm0\.2097\.53±\\pm0\.3394\.81±\\pm0\.0795\.43±\\pm0\.3499\.48±\\pm0\.0471\.37±\\pm0\.4985\.62±\\pm0\.17w/SurgeryV273\.24±\\pm0\.0873\.57±\\pm0\.4594\.15±\\pm0\.1399\.59±\\pm0\.0697\.17±\\pm0\.0798\.38±\\pm0\.1799\.64±\\pm0\.0177\.66±\\pm0\.6189\.18±\\pm0\.09w/IntervMerge71\.72±\\pm0\.2572\.55±\\pm0\.2192\.65±\\pm0\.1599\.21±\\pm0\.2096\.33±\\pm0\.0797\.97±\\pm0\.2399\.59±\\pm0\.0375\.89±\\pm0\.7488\.24±\\pm0\.11w/FeatCal71\.98±\\pm0\.1475\.80±\\pm0\.1793\.71±\\pm0\.0899\.31±\\pm0\.0796\.53±\\pm0\.0498\.42±\\pm0\.0399\.50±\\pm0\.0177\.52±\\pm0\.2489\.10±\\pm0\.05w/ESCal73\.55±\\pm0\.1374\.68±\\pm0\.1594\.80±\\pm0\.1499\.58±\\pm0\.0497\.31±\\pm0\.0198\.68±\\pm0\.0799\.64±\\pm0\.0277\.52±\\pm0\.4089\.47±\\pm0\.06ProDistill68\.84±\\pm0\.2070\.50±\\pm0\.2990\.74±\\pm0\.8199\.06±\\pm0\.1496\.35±\\pm0\.3596\.17±\\pm0\.6399\.50±\\pm0\.0367\.53±\\pm0\.7486\.09±\\pm0\.12w/Surgery69\.55±\\pm0\.1470\.56±\\pm0\.4290\.90±\\pm0\.8099\.08±\\pm0\.1296\.33±\\pm0\.3596\.07±\\pm0\.6899\.51±\\pm0\.0368\.98±\\pm0\.7986\.37±\\pm0\.13w/SurgeryV270\.09±\\pm0\.2570\.38±\\pm0\.3591\.21±\\pm0\.9998\.93±\\pm0\.1696\.85±\\pm0\.1496\.25±\\pm0\.6799\.48±\\pm0\.0369\.29±\\pm0\.8886\.56±\\pm0\.19w/IntervMerge69\.48±\\pm0\.3170\.09±\\pm0\.3790\.97±\\pm1\.0099\.06±\\pm0\.1496\.34±\\pm0\.3696\.11±\\pm0\.6499\.51±\\pm0\.0368\.94±\\pm0\.8686\.31±\\pm0\.12w/FeatCal69\.63±\\pm0\.1469\.80±\\pm0\.4592\.60±\\pm0\.3199\.08±\\pm0\.1696\.88±\\pm0\.1196\.35±\\pm0\.5599\.51±\\pm0\.0472\.44±\\pm0\.6887\.03±\\pm0\.13w/ESCal69\.21±\\pm0\.2670\.44±\\pm0\.4091\.30±\\pm0\.8799\.19±\\pm0\.1396\.87±\\pm0\.1196\.89±\\pm0\.5199\.53±\\pm0\.0368\.59±\\pm1\.0286\.50±\\pm0\.14

Table 12:Full task\-wise test accuracy \(%\) on ViT\-B/16\.MethodSUN397CarsRESISC45EuroSATSVHNGTSRBMNISTDTDAvg\.Task Arithmetic61\.7465\.9674\.1577\.0788\.0673\.9298\.4853\.0374\.05w/Surgery65\.25±\\pm0\.4363\.49±\\pm1\.6376\.73±\\pm1\.3689\.67±\\pm0\.9988\.03±\\pm0\.4076\.13±\\pm2\.1198\.39±\\pm0\.1258\.62±\\pm2\.2177\.04±\\pm0\.40w/SurgeryV274\.17±\\pm0\.1676\.68±\\pm0\.7593\.02±\\pm0\.5998\.95±\\pm0\.2497\.21±\\pm0\.0696\.51±\\pm0\.6299\.52±\\pm0\.0474\.85±\\pm0\.9588\.86±\\pm0\.17w/IntervMerge70\.34±\\pm0\.2269\.55±\\pm0\.9688\.15±\\pm2\.1297\.50±\\pm0\.4595\.30±\\pm0\.2286\.57±\\pm2\.1499\.00±\\pm0\.1267\.57±\\pm1\.8884\.25±\\pm0\.36w/FeatCal71\.96±\\pm0\.0679\.84±\\pm0\.1990\.12±\\pm0\.3297\.95±\\pm0\.2696\.44±\\pm0\.0393\.37±\\pm0\.0399\.16±\\pm0\.0373\.41±\\pm0\.6587\.78±\\pm0\.11w/ESCal74\.22±\\pm0\.2779\.82±\\pm0\.5495\.07±\\pm0\.2699\.46±\\pm0\.0997\.61±\\pm0\.0898\.47±\\pm0\.2699\.58±\\pm0\.0277\.31±\\pm0\.6390\.19±\\pm0\.20TIES\-Merging69\.0272\.6380\.7084\.5685\.0871\.5698\.1954\.9577\.09w/Surgery71\.14±\\pm0\.2268\.61±\\pm1\.2182\.55±\\pm2\.3692\.36±\\pm1\.1384\.04±\\pm1\.3175\.18±\\pm2\.7898\.10±\\pm0\.2663\.81±\\pm2\.3679\.47±\\pm0\.40w/SurgeryV274\.30±\\pm0\.0975\.93±\\pm0\.9292\.40±\\pm1\.0298\.70±\\pm0\.3697\.18±\\pm0\.0996\.22±\\pm0\.5599\.51±\\pm0\.0274\.89±\\pm1\.2588\.64±\\pm0\.20w/IntervMerge71\.65±\\pm0\.2369\.77±\\pm0\.9787\.73±\\pm2\.3697\.22±\\pm0\.5294\.93±\\pm0\.3485\.45±\\pm2\.6098\.91±\\pm0\.1467\.83±\\pm1\.7284\.19±\\pm0\.49w/FeatCal73\.67±\\pm0\.0381\.47±\\pm0\.1090\.71±\\pm0\.1998\.01±\\pm0\.1496\.38±\\pm0\.0193\.99±\\pm0\.0999\.13±\\pm0\.0374\.04±\\pm0\.4688\.42±\\pm0\.07w/ESCal74\.50±\\pm0\.3079\.98±\\pm0\.3295\.13±\\pm0\.2699\.40±\\pm0\.0997\.53±\\pm0\.0698\.38±\\pm0\.2899\.57±\\pm0\.0277\.36±\\pm0\.8690\.23±\\pm0\.15AdaMerging\(tw\)66\.63±\\pm2\.4562\.80±\\pm4\.0171\.15±\\pm5\.5184\.53±\\pm6\.4686\.88±\\pm4\.5691\.13±\\pm1\.9396\.95±\\pm0\.5742\.65±\\pm7\.0875\.34±\\pm0\.71w/Surgery68\.31±\\pm2\.1660\.42±\\pm4\.3375\.25±\\pm5\.0591\.07±\\pm3\.6887\.31±\\pm5\.2590\.51±\\pm2\.7497\.08±\\pm0\.4649\.45±\\pm8\.8377\.43±\\pm1\.00w/SurgeryV274\.03±\\pm1\.5672\.72±\\pm3\.6991\.17±\\pm2\.1498\.97±\\pm0\.3797\.07±\\pm0\.2297\.98±\\pm0\.4699\.44±\\pm0\.0767\.66±\\pm5\.5087\.38±\\pm0\.47w/IntervMerge70\.99±\\pm1\.9064\.34±\\pm4\.0985\.25±\\pm4\.3397\.66±\\pm0\.8694\.72±\\pm1\.6494\.41±\\pm1\.8498\.35±\\pm0\.3758\.23±\\pm6\.9083\.00±\\pm0\.95w/FeatCal72\.91±\\pm1\.7275\.85±\\pm3\.4787\.93±\\pm2\.1598\.01±\\pm0\.6396\.27±\\pm0\.4996\.52±\\pm0\.5798\.87±\\pm0\.1062\.30±\\pm7\.9486\.08±\\pm0\.66w/ESCal74\.84±\\pm1\.1877\.85±\\pm2\.1894\.78±\\pm0\.5999\.47±\\pm0\.1997\.50±\\pm0\.1198\.80±\\pm0\.1299\.51±\\pm0\.0372\.20±\\pm3\.9089\.37±\\pm0\.32AdaMerging\(lw\)68\.96±\\pm1\.8580\.02±\\pm2\.7776\.80±\\pm6\.1086\.30±\\pm4\.6592\.06±\\pm0\.6394\.35±\\pm2\.4898\.22±\\pm0\.5962\.54±\\pm8\.0882\.41±\\pm1\.08w/Surgery70\.45±\\pm1\.3878\.87±\\pm2\.6481\.87±\\pm3\.0191\.79±\\pm1\.8592\.04±\\pm0\.8494\.88±\\pm1\.8698\.50±\\pm0\.4768\.65±\\pm5\.2184\.63±\\pm0\.71w/SurgeryV275\.42±\\pm1\.0883\.29±\\pm1\.1793\.90±\\pm1\.1799\.06±\\pm0\.1897\.24±\\pm0\.1798\.70±\\pm0\.1899\.55±\\pm0\.0577\.53±\\pm2\.6990\.59±\\pm0\.26w/IntervMerge72\.81±\\pm1\.4680\.42±\\pm2\.1188\.93±\\pm2\.7297\.85±\\pm0\.4195\.33±\\pm0\.5696\.95±\\pm0\.9399\.06±\\pm0\.1672\.43±\\pm5\.0587\.97±\\pm0\.39w/FeatCal72\.58±\\pm0\.9984\.67±\\pm1\.2989\.75±\\pm3\.0598\.09±\\pm0\.1596\.20±\\pm0\.2097\.11±\\pm0\.3099\.11±\\pm0\.1374\.84±\\pm3\.8589\.04±\\pm0\.45w/ESCal74\.66±\\pm0\.6983\.25±\\pm0\.8795\.06±\\pm0\.6499\.43±\\pm0\.0997\.58±\\pm0\.0698\.85±\\pm0\.1099\.57±\\pm0\.0577\.21±\\pm2\.4090\.70±\\pm0\.26WUDI\-Merging73\.2981\.7190\.0798\.1995\.3096\.5599\.4473\.7888\.54w/Surgery73\.68±\\pm0\.1881\.00±\\pm0\.3691\.34±\\pm0\.3098\.60±\\pm0\.1395\.09±\\pm0\.2196\.42±\\pm0\.2599\.45±\\pm0\.0276\.21±\\pm0\.8588\.97±\\pm0\.11w/SurgeryV276\.68±\\pm0\.1184\.13±\\pm0\.4195\.70±\\pm0\.1499\.64±\\pm0\.0897\.67±\\pm0\.0798\.90±\\pm0\.1099\.67±\\pm0\.0180\.45±\\pm0\.7091\.60±\\pm0\.10w/IntervMerge75\.01±\\pm0\.1782\.82±\\pm0\.4494\.18±\\pm0\.4499\.29±\\pm0\.1097\.13±\\pm0\.0998\.44±\\pm0\.2899\.57±\\pm0\.0279\.21±\\pm0\.5290\.71±\\pm0\.10w/FeatCal75\.64±\\pm0\.1084\.62±\\pm0\.1594\.52±\\pm0\.1098\.96±\\pm0\.0697\.27±\\pm0\.0398\.46±\\pm0\.0499\.46±\\pm0\.0279\.50±\\pm0\.2991\.06±\\pm0\.02w/ESCal76\.78±\\pm0\.0684\.28±\\pm0\.2796\.02±\\pm0\.0599\.61±\\pm0\.0797\.78±\\pm0\.0498\.95±\\pm0\.0799\.65±\\pm0\.0280\.50±\\pm0\.1991\.70±\\pm0\.05ProDistill72\.58±\\pm0\.0881\.01±\\pm0\.2893\.07±\\pm0\.8699\.19±\\pm0\.1097\.34±\\pm0\.1697\.65±\\pm0\.3399\.59±\\pm0\.0271\.35±\\pm0\.8988\.97±\\pm0\.17w/Surgery73\.26±\\pm0\.2281\.12±\\pm0\.3692\.98±\\pm0\.9499\.21±\\pm0\.1197\.34±\\pm0\.1597\.41±\\pm0\.3899\.59±\\pm0\.0372\.66±\\pm1\.0789\.20±\\pm0\.18w/SurgeryV273\.73±\\pm0\.1881\.32±\\pm0\.5193\.14±\\pm0\.9899\.24±\\pm0\.1597\.42±\\pm0\.1097\.47±\\pm0\.2899\.60±\\pm0\.0273\.30±\\pm0\.7989\.40±\\pm0\.19w/IntervMerge73\.41±\\pm0\.1780\.84±\\pm0\.5093\.14±\\pm0\.9799\.19±\\pm0\.1397\.35±\\pm0\.1697\.46±\\pm0\.3199\.61±\\pm0\.0372\.80±\\pm0\.7689\.23±\\pm0\.18w/FeatCal72\.26±\\pm0\.3580\.29±\\pm0\.2394\.65±\\pm0\.3999\.27±\\pm0\.0697\.53±\\pm0\.1097\.93±\\pm0\.2299\.68±\\pm0\.0176\.56±\\pm0\.5089\.77±\\pm0\.05w/ESCal72\.96±\\pm0\.0281\.02±\\pm0\.3793\.52±\\pm0\.7099\.19±\\pm0\.1697\.50±\\pm0\.1397\.95±\\pm0\.2799\.60±\\pm0\.0172\.47±\\pm1\.0489\.27±\\pm0\.17

Table 13:Full task\-wise test accuracy \(%\) on ViT\-L/14\.MethodSUN397CarsRESISC45EuroSATSVHNGTSRBMNISTDTDAvg\.Task Arithmetic74\.1682\.0986\.6793\.9387\.9186\.7798\.9465\.6984\.52w/Surgery74\.71±\\pm0\.2780\.50±\\pm0\.4387\.89±\\pm1\.6695\.47±\\pm0\.5788\.82±\\pm0\.5986\.87±\\pm1\.0198\.94±\\pm0\.1369\.84±\\pm1\.9485\.38±\\pm0\.19w/SurgeryV278\.61±\\pm0\.2886\.93±\\pm0\.3194\.57±\\pm1\.0099\.39±\\pm0\.0497\.73±\\pm0\.0497\.55±\\pm0\.5499\.58±\\pm0\.0378\.62±\\pm0\.9991\.62±\\pm0\.11w/IntervMerge76\.40±\\pm0\.2683\.59±\\pm0\.6891\.62±\\pm1\.5198\.53±\\pm0\.3096\.29±\\pm0\.1592\.54±\\pm1\.6199\.42±\\pm0\.0474\.19±\\pm1\.0189\.07±\\pm0\.10w/FeatCal79\.05±\\pm0\.0490\.18±\\pm0\.0994\.11±\\pm0\.1199\.30±\\pm0\.0596\.47±\\pm0\.0496\.85±\\pm0\.0899\.36±\\pm0\.0378\.96±\\pm0\.0691\.78±\\pm0\.02w/ESCal79\.12±\\pm0\.1789\.28±\\pm0\.1696\.11±\\pm0\.2899\.66±\\pm0\.0597\.98±\\pm0\.0398\.75±\\pm0\.2399\.64±\\pm0\.0280\.52±\\pm0\.2892\.63±\\pm0\.05TIES\-Merging75\.0384\.5487\.8694\.7085\.6682\.0798\.6567\.5084\.50w/Surgery75\.63±\\pm0\.1781\.36±\\pm0\.5488\.41±\\pm2\.5595\.84±\\pm0\.5486\.21±\\pm0\.9683\.44±\\pm2\.0598\.61±\\pm0\.1770\.46±\\pm2\.1584\.99±\\pm0\.18w/SurgeryV278\.26±\\pm0\.2385\.99±\\pm0\.5193\.92±\\pm1\.1199\.22±\\pm0\.1697\.63±\\pm0\.0796\.52±\\pm0\.8899\.58±\\pm0\.0477\.60±\\pm1\.0491\.09±\\pm0\.14w/IntervMerge76\.85±\\pm0\.2883\.26±\\pm0\.4891\.73±\\pm1\.8798\.27±\\pm0\.4196\.11±\\pm0\.1490\.27±\\pm1\.5299\.36±\\pm0\.0674\.40±\\pm1\.0988\.78±\\pm0\.19w/FeatCal79\.25±\\pm0\.0690\.28±\\pm0\.0794\.02±\\pm0\.1199\.42±\\pm0\.0696\.68±\\pm0\.0396\.59±\\pm0\.0699\.40±\\pm0\.0278\.67±\\pm0\.2091\.79±\\pm0\.03w/ESCal78\.95±\\pm0\.1689\.25±\\pm0\.2196\.01±\\pm0\.4199\.59±\\pm0\.0997\.90±\\pm0\.0298\.55±\\pm0\.2499\.61±\\pm0\.0380\.39±\\pm0\.3092\.53±\\pm0\.06AdaMerging\(tw\)73\.63±\\pm2\.8881\.74±\\pm2\.9279\.62±\\pm4\.9790\.39±\\pm2\.7487\.37±\\pm1\.6096\.10±\\pm0\.3197\.48±\\pm0\.3470\.63±\\pm4\.0584\.62±\\pm0\.66w/Surgery74\.57±\\pm2\.5381\.35±\\pm3\.4582\.71±\\pm4\.3694\.08±\\pm2\.2388\.03±\\pm1\.4296\.21±\\pm0\.5297\.83±\\pm0\.3273\.29±\\pm4\.1386\.01±\\pm0\.64w/SurgeryV279\.77±\\pm1\.4988\.15±\\pm2\.0593\.48±\\pm1\.8099\.44±\\pm0\.1297\.69±\\pm0\.0599\.07±\\pm0\.1099\.54±\\pm0\.0481\.93±\\pm1\.0892\.38±\\pm0\.28w/IntervMerge77\.74±\\pm1\.3185\.16±\\pm2\.9988\.83±\\pm3\.0098\.64±\\pm0\.5295\.75±\\pm0\.3098\.37±\\pm0\.2099\.04±\\pm0\.1178\.41±\\pm3\.4690\.24±\\pm0\.53w/FeatCal79\.90±\\pm1\.6090\.73±\\pm1\.1592\.30±\\pm1\.9099\.35±\\pm0\.2296\.20±\\pm0\.2698\.80±\\pm0\.1099\.15±\\pm0\.1081\.97±\\pm1\.2292\.30±\\pm0\.20w/ESCal79\.97±\\pm0\.7189\.69±\\pm1\.0795\.42±\\pm0\.5799\.69±\\pm0\.0497\.93±\\pm0\.0699\.15±\\pm0\.0599\.61±\\pm0\.0382\.16±\\pm0\.9392\.95±\\pm0\.11AdaMerging\(lw\)76\.97±\\pm1\.2988\.59±\\pm1\.4888\.31±\\pm3\.4395\.52±\\pm1\.1891\.38±\\pm2\.0297\.60±\\pm0\.4998\.79±\\pm0\.4078\.39±\\pm1\.4689\.44±\\pm0\.54w/Surgery77\.79±\\pm1\.2588\.58±\\pm0\.9990\.96±\\pm3\.2196\.81±\\pm0\.4591\.97±\\pm2\.4897\.74±\\pm0\.3598\.82±\\pm0\.3279\.72±\\pm1\.8390\.30±\\pm0\.60w/SurgeryV281\.11±\\pm0\.4291\.15±\\pm0\.4195\.90±\\pm0\.6899\.59±\\pm0\.1597\.65±\\pm0\.1099\.13±\\pm0\.0999\.61±\\pm0\.0783\.40±\\pm0\.4693\.44±\\pm0\.12w/IntervMerge80\.38±\\pm0\.6490\.42±\\pm0\.7893\.61±\\pm2\.2398\.94±\\pm0\.2896\.31±\\pm0\.3598\.46±\\pm0\.2199\.37±\\pm0\.0781\.77±\\pm0\.6192\.41±\\pm0\.27w/FeatCal80\.62±\\pm0\.1591\.42±\\pm0\.1394\.50±\\pm0\.7199\.39±\\pm0\.0995\.95±\\pm0\.3298\.69±\\pm0\.0999\.32±\\pm0\.1083\.27±\\pm0\.5192\.89±\\pm0\.11w/ESCal80\.91±\\pm0\.3791\.34±\\pm0\.2996\.21±\\pm0\.3099\.73±\\pm0\.0897\.90±\\pm0\.0399\.17±\\pm0\.0399\.61±\\pm0\.0482\.83±\\pm0\.5193\.46±\\pm0\.08WUDI\-Merging79\.6590\.9593\.9899\.4196\.2198\.0499\.6380\.6492\.31w/Surgery79\.81±\\pm0\.1290\.82±\\pm0\.1294\.65±\\pm0\.4499\.49±\\pm0\.0396\.30±\\pm0\.1298\.11±\\pm0\.1899\.69±\\pm0\.0280\.90±\\pm0\.6092\.47±\\pm0\.11w/SurgeryV281\.13±\\pm0\.1291\.36±\\pm0\.2296\.64±\\pm0\.1499\.74±\\pm0\.0498\.06±\\pm0\.0399\.18±\\pm0\.0499\.73±\\pm0\.0183\.21±\\pm0\.3793\.63±\\pm0\.02w/IntervMerge80\.62±\\pm0\.0991\.18±\\pm0\.1096\.15±\\pm0\.4799\.65±\\pm0\.0797\.69±\\pm0\.0699\.05±\\pm0\.0999\.71±\\pm0\.0282\.72±\\pm0\.1493\.35±\\pm0\.06w/FeatCal79\.92±\\pm0\.0390\.64±\\pm0\.0595\.91±\\pm0\.0799\.74±\\pm0\.0497\.73±\\pm0\.0299\.17±\\pm0\.0299\.70±\\pm0\.0183\.91±\\pm0\.1193\.34±\\pm0\.02w/ESCal81\.08±\\pm0\.1391\.56±\\pm0\.1196\.76±\\pm0\.1399\.79±\\pm0\.0598\.14±\\pm0\.0299\.21±\\pm0\.0399\.69±\\pm0\.0282\.86±\\pm0\.2193\.64±\\pm0\.04ProDistill77\.70±\\pm0\.1789\.96±\\pm0\.2195\.16±\\pm0\.6799\.56±\\pm0\.0797\.79±\\pm0\.1198\.40±\\pm0\.3499\.60±\\pm0\.0378\.11±\\pm0\.4992\.04±\\pm0\.07w/Surgery77\.89±\\pm0\.1689\.72±\\pm0\.1495\.18±\\pm0\.7699\.59±\\pm0\.1097\.79±\\pm0\.1098\.39±\\pm0\.3599\.61±\\pm0\.0178\.40±\\pm0\.1892\.07±\\pm0\.04w/SurgeryV277\.98±\\pm0\.1589\.74±\\pm0\.2195\.22±\\pm0\.6999\.56±\\pm0\.1097\.80±\\pm0\.0998\.38±\\pm0\.3699\.62±\\pm0\.0178\.53±\\pm0\.1892\.11±\\pm0\.06w/IntervMerge77\.95±\\pm0\.1589\.78±\\pm0\.1995\.20±\\pm0\.7799\.58±\\pm0\.0997\.80±\\pm0\.1198\.32±\\pm0\.3699\.63±\\pm0\.0178\.64±\\pm0\.3692\.11±\\pm0\.08w/FeatCal79\.14±\\pm0\.1789\.71±\\pm0\.2296\.17±\\pm0\.2499\.62±\\pm0\.0497\.91±\\pm0\.0898\.66±\\pm0\.2299\.60±\\pm0\.0381\.11±\\pm0\.4292\.74±\\pm0\.10w/ESCal77\.88±\\pm0\.1489\.91±\\pm0\.2695\.32±\\pm0\.6199\.56±\\pm0\.0697\.84±\\pm0\.0998\.46±\\pm0\.3399\.61±\\pm0\.0278\.37±\\pm0\.1892\.12±\\pm0\.03

### C\.4NLU Results

Table[14](https://arxiv.org/html/2610.33483#A3.T14)reports the complete task\-wise results for RoBERTa\-base across all six base merging methods and the corresponding post\-merging calibration methods\. TheAvg\.column reports the arithmetic mean across the eight NLU tasks, with the pretrained and expert models included as reference points\.

ESCalimproves over all uncalibrated models and achieves the highest average score among the compared calibration methods in most merging settings\. The exception isTIES\-Merging, whereESCalandSurgeryV2 obtain comparable average scores\.

Table 14:Full task\-wise performance on the NLU benchmark using RoBERTa\-base\.MethodCoLASST\-2MRPCSTS\-BQQPMNLIQNLIRTEAvg\.Pretrained0\.00000\.59750\.1581\-0\.07580\.46330\.31830\.50780\.52710\.3120Expert0\.58950\.94380\.90910\.89970\.89940\.87180\.91980\.77260\.8507Task Arithmetic0\.12320\.61350\.55000\.53560\.58810\.62700\.53650\.53430\.5135w/Surgery0\.2553±\\pm0\.02530\.8532±\\pm0\.01240\.7657±\\pm0\.01550\.7877±\\pm0\.00890\.7793±\\pm0\.01920\.7215±\\pm0\.01180\.7839±\\pm0\.02240\.7184±\\pm0\.00990\.7081±\\pm0\.0088w/SurgeryV20\.5358±\\pm0\.01060\.9326±\\pm0\.00290\.8908±\\pm0\.00370\.8978±\\pm0\.00180\.8002±\\pm0\.01370\.8404±\\pm0\.00160\.9084±\\pm0\.00190\.7408±\\pm0\.01100\.8184±\\pm0\.0016w/IntervMerge0\.4647±\\pm0\.02730\.9135±\\pm0\.00490\.8477±\\pm0\.00610\.8706±\\pm0\.00750\.7779±\\pm0\.01350\.7625±\\pm0\.00800\.8524±\\pm0\.00890\.7199±\\pm0\.00980\.7762±\\pm0\.0067w/FeatCal0\.3879±\\pm0\.01110\.9140±\\pm0\.00950\.8162±\\pm0\.01160\.4900±\\pm0\.03530\.7680±\\pm0\.02220\.7569±\\pm0\.00260\.8499±\\pm0\.01160\.7011±\\pm0\.00650\.7105±\\pm0\.0024w/ESCal0\.5588±\\pm0\.00900\.9319±\\pm0\.00260\.8937±\\pm0\.00390\.8998±\\pm0\.00190\.8061±\\pm0\.02040\.8469±\\pm0\.00230\.9117±\\pm0\.00290\.7415±\\pm0\.01370\.8238±\\pm0\.0037TIES\-Merging0\.19720\.76260\.50750\.43900\.60390\.66700\.53800\.49460\.5262w/Surgery0\.2680±\\pm0\.04010\.8635±\\pm0\.02130\.7570±\\pm0\.00820\.7770±\\pm0\.02220\.7770±\\pm0\.02310\.7347±\\pm0\.01300\.7988±\\pm0\.00670\.6989±\\pm0\.02570\.7094±\\pm0\.0126w/SurgeryV20\.5420±\\pm0\.01270\.9319±\\pm0\.00210\.8848±\\pm0\.00690\.8965±\\pm0\.00300\.8205±\\pm0\.01170\.8474±\\pm0\.00160\.9079±\\pm0\.00270\.7394±\\pm0\.01180\.8213±\\pm0\.0027w/IntervMerge0\.4382±\\pm0\.02550\.9174±\\pm0\.00500\.8368±\\pm0\.01550\.8702±\\pm0\.00960\.8023±\\pm0\.01120\.7805±\\pm0\.01340\.8660±\\pm0\.00490\.7249±\\pm0\.01310\.7796±\\pm0\.0052w/FeatCal0\.4055±\\pm0\.01300\.9106±\\pm0\.01100\.8111±\\pm0\.01300\.4765±\\pm0\.04980\.7702±\\pm0\.01830\.7569±\\pm0\.00390\.8433±\\pm0\.01150\.7061±\\pm0\.01210\.7100±\\pm0\.0046w/ESCal0\.5552±\\pm0\.02530\.9339±\\pm0\.00440\.8769±\\pm0\.01130\.8935±\\pm0\.00200\.8144±\\pm0\.02310\.8443±\\pm0\.00240\.9070±\\pm0\.00430\.7430±\\pm0\.01030\.8210±\\pm0\.0052AdaMerging\(tw\)0\.02830\.48050\.26260\.70310\.22040\.81030\.78180\.70400\.4989w/Surgery0\.1338±\\pm0\.04700\.8234±\\pm0\.02930\.7918±\\pm0\.00950\.7757±\\pm0\.01880\.7438±\\pm0\.00720\.8336±\\pm0\.00340\.8711±\\pm0\.00660\.7401±\\pm0\.00680\.7142±\\pm0\.0076w/SurgeryV20\.5227±\\pm0\.02590\.9339±\\pm0\.00560\.8850±\\pm0\.00790\.8910±\\pm0\.00310\.7570±\\pm0\.01950\.8651±\\pm0\.00150\.9151±\\pm0\.00150\.7783±\\pm0\.02530\.8185±\\pm0\.0071w/IntervMerge0\.2981±\\pm0\.05910\.8945±\\pm0\.02860\.8013±\\pm0\.02850\.8272±\\pm0\.03680\.7287±\\pm0\.01930\.8507±\\pm0\.00580\.8763±\\pm0\.01710\.7422±\\pm0\.01460\.7524±\\pm0\.0096w/FeatCal0\.4004±\\pm0\.01930\.9183±\\pm0\.00750\.8189±\\pm0\.00570\.4628±\\pm0\.04790\.7737±\\pm0\.01640\.7525±\\pm0\.00320\.8194±\\pm0\.01060\.6606±\\pm0\.01420\.7008±\\pm0\.0042w/ESCal0\.5383±\\pm0\.01630\.9344±\\pm0\.00360\.8892±\\pm0\.00740\.8905±\\pm0\.00230\.7594±\\pm0\.02430\.8648±\\pm0\.00170\.9110±\\pm0\.00280\.7740±\\pm0\.00830\.8202±\\pm0\.0048AdaMerging\(lw\)0\.01370\.51150\.30690\.72050\.18680\.80380\.60770\.70760\.4823w/Surgery0\.1864±\\pm0\.05940\.8314±\\pm0\.01490\.7948±\\pm0\.00990\.7699±\\pm0\.03380\.7763±\\pm0\.01110\.8182±\\pm0\.00770\.8242±\\pm0\.01060\.7372±\\pm0\.01810\.7173±\\pm0\.0122w/SurgeryV20\.5145±\\pm0\.02010\.9333±\\pm0\.00740\.8867±\\pm0\.00550\.8930±\\pm0\.00310\.8073±\\pm0\.02040\.8585±\\pm0\.00080\.9080±\\pm0\.00280\.7711±\\pm0\.01970\.8215±\\pm0\.0059w/IntervMerge0\.4303±\\pm0\.02700\.9147±\\pm0\.01000\.8324±\\pm0\.02800\.8544±\\pm0\.01340\.7620±\\pm0\.01460\.8355±\\pm0\.01190\.8510±\\pm0\.01960\.7357±\\pm0\.00860\.7770±\\pm0\.0052w/FeatCal0\.4052±\\pm0\.02880\.9092±\\pm0\.01360\.8297±\\pm0\.01330\.4788±\\pm0\.03060\.7767±\\pm0\.01810\.7368±\\pm0\.00270\.8354±\\pm0\.00920\.6801±\\pm0\.02610\.7065±\\pm0\.0030w/ESCal0\.5391±\\pm0\.01400\.9339±\\pm0\.00290\.8926±\\pm0\.00570\.8968±\\pm0\.00100\.8204±\\pm0\.01750\.8592±\\pm0\.00150\.9082±\\pm0\.00350\.7704±\\pm0\.01190\.8276±\\pm0\.0040WUDI\-Merging0\.06250\.80730\.25650\.09270\.76800\.44310\.46220\.48740\.4225w/Surgery0\.2226±\\pm0\.02960\.9025±\\pm0\.00950\.7860±\\pm0\.01110\.7758±\\pm0\.02640\.8270±\\pm0\.00540\.7755±\\pm0\.01790\.8118±\\pm0\.01650\.7206±\\pm0\.01390\.7277±\\pm0\.0059w/SurgeryV20\.5529±\\pm0\.01300\.9392±\\pm0\.00490\.8908±\\pm0\.00870\.8868±\\pm0\.00270\.8798±\\pm0\.00320\.8620±\\pm0\.00170\.9136±\\pm0\.00160\.7603±\\pm0\.01240\.8357±\\pm0\.0018w/IntervMerge0\.4766±\\pm0\.00870\.9282±\\pm0\.00500\.8303±\\pm0\.01350\.8668±\\pm0\.01020\.8562±\\pm0\.00270\.8347±\\pm0\.00540\.8884±\\pm0\.00210\.7516±\\pm0\.01120\.8041±\\pm0\.0027w/FeatCal0\.4387±\\pm0\.02450\.9190±\\pm0\.00840\.8215±\\pm0\.00820\.4953±\\pm0\.04270\.8234±\\pm0\.01120\.7748±\\pm0\.00330\.8358±\\pm0\.01320\.6888±\\pm0\.02130\.7247±\\pm0\.0069w/ESCal0\.5663±\\pm0\.02170\.9397±\\pm0\.00550\.8950±\\pm0\.00510\.8953±\\pm0\.00270\.8744±\\pm0\.00610\.8616±\\pm0\.00130\.9144±\\pm0\.00250\.7545±\\pm0\.01200\.8376±\\pm0\.0024ProDistill\(ew\)0\.40760\.91060\.84630\.52880\.79380\.77590\.86090\.71480\.7298w/Surgery0\.4470±\\pm0\.02250\.9158±\\pm0\.00710\.8371±\\pm0\.00750\.7119±\\pm0\.04940\.7756±\\pm0\.02530\.7851±\\pm0\.00610\.8647±\\pm0\.00320\.7249±\\pm0\.01640\.7578±\\pm0\.0049w/SurgeryV20\.5006±\\pm0\.02190\.9257±\\pm0\.00420\.8717±\\pm0\.00740\.8001±\\pm0\.02740\.8089±\\pm0\.02330\.8283±\\pm0\.00320\.9012±\\pm0\.00230\.7256±\\pm0\.01370\.7953±\\pm0\.0051w/IntervMerge0\.4664±\\pm0\.01190\.9202±\\pm0\.00840\.8582±\\pm0\.00560\.7711±\\pm0\.01880\.7986±\\pm0\.02340\.7965±\\pm0\.00650\.8840±\\pm0\.00480\.7256±\\pm0\.01350\.7776±\\pm0\.0030w/FeatCal0\.4384±\\pm0\.02440\.9122±\\pm0\.00450\.8246±\\pm0\.01480\.4523±\\pm0\.07160\.8062±\\pm0\.01600\.7914±\\pm0\.00740\.8493±\\pm0\.01380\.7141±\\pm0\.01600\.7235±\\pm0\.0116w/ESCal0\.4880±\\pm0\.02970\.9239±\\pm0\.00450\.8759±\\pm0\.00690\.8734±\\pm0\.01060\.7913±\\pm0\.03880\.8292±\\pm0\.00230\.9017±\\pm0\.00270\.7155±\\pm0\.01790\.7999±\\pm0\.0062

### C\.5NLG Results

We further evaluate post\-merging calibration on natural language generation \(NLG\) using LLaMA\-2\-13B experts specialized for instruction following, mathematical reasoning, and code generation, considering all three pairwise merges\.

#### Aggregation and expert references\.

For each expert pair, Avg\. is the arithmetic mean over all benchmarks shown in the corresponding table, including AlpacaEval when present\. Norm Avg\. divides each benchmark score by the score of the corresponding calibration target expert and then averages these ratios\. For benchmark setℬ\\mathcal\{B\}, we report

Avg\.\(m\)=1\|ℬ\|​∑b∈ℬsm,b,Norm​Avg\.\(m\)=1\|ℬ\|​∑b∈ℬsm,bse⁡\(b\),b,\\mathrm\{Avg\.\}\(m\)=\\frac\{1\}\{\|\\mathcal\{B\}\|\}\\sum\_\{b\\in\\mathcal\{B\}\}s\_\{m,b\},\\qquad\\mathrm\{Norm\\ Avg\.\}\(m\)=\\frac\{1\}\{\|\\mathcal\{B\}\|\}\\sum\_\{b\\in\\mathcal\{B\}\}\\frac\{s\_\{m,b\}\}\{s\_\{e\(b\),b\}\},\(19\)wheresm,bs\_\{m,b\}is the score of modelmmon benchmarkbb, ande⁡\(b\)e\(b\)is the designated calibration target expert for that benchmark\. The reference is fixed across all rows within each table\. A per\-benchmark normalized score above one indicates performance above the corresponding target expert; a normalized average above one need not indicate improvements on every benchmark\.

For math–code merging, WizardMath\-13B is the reference for GSM8K and MATH, while Llama\-2\-13b\-code\-alpaca is the reference for HumanEval and MBPP\. For instruction–code merging, Llama\-2\-13b\-code\-alpaca is the reference for HumanEval and MBPP, and WizardLM\-13B\-V1\.2 is the reference for AlpacaEval\. For instruction–math merging, WizardMath\-13B is the reference for GSM8K and MATH, while WizardLM\-13B\-V1\.2 is the reference for HumanEval, MBPP, and AlpacaEval\. The reference therefore follows the calibration target rather than whichever constituent expert obtains the higher benchmark score\. The reported aggregates are computed from the displayed benchmark scores and rounded to four decimal places\.

Table 15:Post\-merging calibration for LLaMA\-2\-13B math–code merging\.MethodGSM8KMATHHumanEvalMBPPAvg\.Norm Avg\.WizardMath\-13B0\.63430\.14680\.07320\.21560\.26750\.7655Llama\-2\-13b\-code\-alpaca0\.00000\.00000\.21950\.29590\.12890\.5000Task Arithmetic0\.64940\.14460\.01220\.09170\.22450\.5936w/ Surgery0\.65420\.14680\.04880\.15140\.25030\.6913w/SurgeryV20\.62550\.14300\.07930\.21330\.26530\.7606w/FeatCal0\.64060\.14520\.20730\.26830\.31540\.9625w/ESCal0\.63510\.15440\.18290\.25230\.30620\.9347TIES\-Merging0\.59120\.11940\.09760\.13300\.23530\.6599w/ Surgery0\.61590\.12020\.09760\.16280\.24910\.6962w/SurgeryV20\.59760\.11800\.18900\.28670\.29780\.8940w/FeatCal0\.62630\.13520\.18290\.25690\.30030\.9025w/ESCal0\.61120\.12980\.19510\.27980\.30400\.9206

Table 16:Post\-merging calibration for LLaMA\-2\-13B instruction–code merging\.MethodHumanEvalMBPPAlpacaEvalAvg\.Norm Avg\.WizardLM\-13B\-V1\.20\.32930\.35090\.16460\.28161\.2287Llama\-2\-13b\-code\-alpaca0\.21950\.29590\.01960\.17830\.7064Task Arithmetic0\.32320\.34400\.15760\.27491\.1975w/ Surgery0\.31710\.34860\.16040\.27541\.1991w/SurgeryV20\.32930\.33490\.16150\.27521\.2044w/FeatCal0\.31100\.33720\.15200\.26671\.1600w/ESCal0\.31100\.34170\.15700\.26991\.1752TIES\-Merging0\.23170\.33260\.05010\.20480\.8280w/ Surgery0\.25610\.33940\.05270\.21610\.8780w/SurgeryV20\.26220\.33720\.06010\.21980\.8997w/FeatCal0\.27440\.32110\.06220\.21920\.9044w/ESCal0\.28050\.33260\.08190\.23170\.9665

Table 17:Post\-merging calibration for LLaMA\-2\-13B instruction–math merging\.MethodGSM8KMATHHumanEvalMBPPAlpacaEvalAvg\.Norm Avg\.WizardLM\-13B\-V1\.20\.02230\.00040\.32930\.32800\.16460\.16890\.6076WizardMath\-13B0\.63430\.14680\.07320\.21000\.04870\.22260\.6317Task Arithmetic0\.66140\.14400\.26220\.30000\.10530\.29460\.8749w/ Surgery0\.66930\.13580\.26830\.33000\.11010\.30270\.8940w/SurgeryV20\.65660\.14340\.25000\.31600\.11490\.29620\.8865w/FeatCal0\.64540\.14040\.24390\.30400\.10410\.28760\.8548w/ESCal0\.66530\.14500\.25610\.29800\.10730\.29430\.8749TIES\-Merging0\.57610\.10420\.09150\.30800\.06090\.22810\.6410w/ Surgery0\.60080\.11420\.19510\.32400\.08990\.26480\.7703w/SurgeryV20\.60640\.11560\.23170\.32200\.09200\.27350\.7975w/FeatCal0\.63750\.13660\.18900\.30200\.10950\.27490\.8191w/ESCal0\.61040\.12520\.23780\.33400\.08590\.27870\.8155

#### Calibration when the merged model already performs well\.

Task Arithmetic already outperforms its calibration targets on some benchmarks\. For example, Task Arithmetic achieves a GSM8K score of 0\.6614 in the instruction–math merge, exceeding WizardMath\-13B’s 0\.6343\. In instruction–code merging, its HumanEval and MBPP scores also exceed those of the code expert, although they remain below those of WizardLM\-13B\-V1\.2\. Its normalized average is 1\.1975 relative to the designated targets, although its AlpacaEval score remains below that of the instruction expert\. A merged model can also exceed both experts’ overall averages by combining their strengths, even if it does not outperform the stronger expert on every benchmark\.

Surgery,SurgeryV2,FeatCal, andESCalaim to reduce merging\-induced discrepancies by using the constituent experts as calibration targets or references\. When merging already improves a benchmark score over the corresponding expert, closer alignment with that expert need not yield further performance gains\. Calibration may still improve other benchmarks\. For Task Arithmetic, its effect on the overall average varies across calibration methods\. In instruction–code merging, the full\-benchmark average changes from 0\.2749 to 0\.2754 withSurgeryand 0\.2752 withSurgeryV2, but to 0\.2667 withFeatCaland 0\.2699 withESCal\.

#### Calibration when merging leaves an expert\-performance gap\.

The TIES merges remain below their designated calibration targets on average across all three expert pairs\.ESCalimproves both Avg\. and Norm Avg\. for all three TIES merges, and achieves the highest values for both metrics among the evaluated calibration variants in the math–code and instruction–code settings\. For math–code merging, it increases Avg\. from 0\.2353 to 0\.3040 and Norm Avg\. from 0\.6599 to 0\.9206\. For instruction–code merging, the corresponding improvements are 0\.2048 to 0\.2317 and 0\.8280 to 0\.9665\. These results show that corrections between fixed error subspaces can improve performance when the merged model remains below its calibration targets\.

## Appendix DAdditional Analyses

### D\.1Error\-Subspace Construction

Table[18](https://arxiv.org/html/2610.33483#A4.T18)provides the full per\-task results for the basis\-source comparison in Table[3](https://arxiv.org/html/2610.33483#S5.T3), together with the bottom\-rrmerging\-error baseline\. Bases derived from the top\-rrsingular directions of the merging error achieve the highest mean accuracy on all eight tasks\. Their average accuracy is87\.54%87\.54\\%, compared with87\.01%87\.01\\%for task\-vector bases and84\.80%84\.80\\%for fine\-tuned\-weight bases\. Random orthonormal bases and bottom\-rrmerging\-error bases yield84\.08%84\.08\\%and80\.86%80\.86\\%, respectively\. These results support using the dominant singular subspaces of the expert–merged weight discrepancy to constrain calibration\.

Table 18:Full results for the error\-subspace construction ablation\.We report per\-task and average accuracy \(%; mean±\\pmstandard deviation\) for the basis\-source comparison in Table[3](https://arxiv.org/html/2610.33483#S5.T3), together with the bottom\-rrmerging\-error baseline\.Basis constructionSUN397CarsRESISC45EuroSATSVHNGTSRBMNISTDTDAvg\.Random orthonormal65\.42±\\pm0\.2763\.88±\\pm0\.5588\.54±\\pm0\.5798\.61±\\pm0\.2396\.06±\\pm0\.1095\.81±\\pm0\.2299\.32±\\pm0\.0564\.98±\\pm1\.0284\.08±\\pm0\.15Fine\-tuned weights, top\-rr66\.94±\\pm0\.2265\.02±\\pm0\.4889\.85±\\pm0\.6199\.03±\\pm0\.1596\.17±\\pm0\.0496\.32±\\pm0\.4599\.36±\\pm0\.0665\.70±\\pm0\.8484\.80±\\pm0\.24Task vector, top\-rr69\.29±\\pm0\.1567\.72±\\pm0\.6793\.01±\\pm0\.4599\.44±\\pm0\.1197\.08±\\pm0\.0497\.47±\\pm0\.2899\.53±\\pm0\.0472\.53±\\pm0\.3987\.01±\\pm0\.09Merging error, bottom\-rr62\.11±\\pm0\.2162\.17±\\pm0\.3082\.26±\\pm0\.3697\.20±\\pm0\.3594\.02±\\pm0\.1991\.05±\\pm0\.1798\.83±\\pm0\.0759\.21±\\pm0\.5880\.86±\\pm0\.13Merging error, top\-rr70\.33±\\pm0\.2269\.02±\\pm0\.1993\.60±\\pm0\.3699\.48±\\pm0\.0797\.10±\\pm0\.0597\.70±\\pm0\.2299\.55±\\pm0\.0373\.52±\\pm0\.4487\.54±\\pm0\.07

### D\.2Computational Cost

We evaluate trainable parameter counts, additional inference\-time storage, and computation time on ViT\-B/32 withT=8T=8tasks and 64 calibration examples per task\. Storage and inference measurements assume a single active task\.

#### Parameters and storage\.

Table[19](https://arxiv.org/html/2610.33483#A4.T19)summarizes the trainable parameters across all eight tasks and the additional tensor storage required for single\-task inference, relative to the shared image encoder\. We useL=12L=12,d=768d=768,dout=512d\_\{\\mathrm\{out\}\}=512,rs=16r\_\{s\}=16, andrb=128r\_\{b\}=128\.

Table 19:Trainable parameters across eight tasks and additional storage for single\-task inference on ViT\-B/32\.Training \(T=8T=8tasks\)Inference \(one task\)MethodTrainable Parameters\# TrainableStored Entries\# StoredBF16 StorageOverheadShared ViT\-B/32 image encoder––PimgP\_\{\\mathrm\{img\}\}87,849,216167\.56 MiBReferenceSurgery2​T​dout​rs2Td\_\{\\mathrm\{out\}\}r\_\{s\}131,0722​dout​rs2d\_\{\\mathrm\{out\}\}r\_\{s\}16,3840\.031 MiB0\.02%SurgeryV2T⁡\(2​L​d​rs\+2​dout​rs\)T\(2Ldr\_\{s\}\+2d\_\{\\mathrm\{out\}\}r\_\{s\}\)2,490,3682​L​d​rs\+2​dout​rs2Ldr\_\{s\}\+2d\_\{\\mathrm\{out\}\}r\_\{s\}311,2960\.594 MiB0\.35%FeatCal00\(closed form\)00000\.000 MiB0\.00%ESCalT​L​rb2TLr\_\{b\}^\{2\}1,572,864L​rb2\+2​L​d​rbLr\_\{b\}^\{2\}\+2Ldr\_\{b\}2,555,9044\.875 MiB2\.91%

ESCallearnsT​L​rb2=1,572,864TLr\_\{b\}^\{2\}=1\{,\}572\{,\}864core parameters across eight tasks, which is36\.8%36\.8\\%fewer thanSurgeryV2\. For single\-task inference, it storesL​rb2=196,608Lr\_\{b\}^\{2\}=196\{,\}608core entries and2​L​d​rb=2,359,2962Ldr\_\{b\}=2\{,\}359\{,\}296fixed read/write basis entries\. The resulting 2,555,904 additional entries require 4\.875 MiB at BF16, equivalent to2\.91%2\.91\\%of the shared image encoder’s storage\. Thus,ESCalreduces the number of trainable parameters relative toSurgeryV2, while requiring more inference\-time storage for its fixed bases\.

Surgeryapplies low\-rank calibration at the output embedding, whileSurgeryV2 additionally applies calibration at each residual block\.FeatCalupdates the merged encoder in closed form and stores the resulting weights in place, requiring no additional inference\-time tensors for the active task\. This accounting excludes activations, optimizer states, temporary buffers, and serialization metadata; it does not quantify peak training memory or total training computation\.

#### Construction and calibration time\.

Table[20](https://arxiv.org/html/2610.33483#A4.T20)reports base\-model construction and calibration times separately\. We compare the time needed for base merging followed by calibration with that of more expensive merging procedures\. The total time is the sum of base\-model construction and calibration\. Stage timings include setup, training or closed\-form updates, and saving model artifacts, but exclude test evaluation, library imports, and reporting\.

Table 20:Base\-model construction and calibration times in minutes on ViT\-B/32 with eight tasks and 64 calibration examples per task \(single run on an NVIDIA RTX A6000\)\. Calibration times exclude base\-model construction; gradient\-based calibration uses 200 epochs\. The task\-wise and layer\-wise variants ofAdaMergingare denoted by tw and lw, respectively\.Base constructionCalibrationBase merging methodTimeSurgerySurgeryV2FeatCalESCalTask Arithmetic0\.096\.2813\.410\.6313\.22TIES\-Merging0\.395\.4812\.050\.5812\.92WUDI\-Merging5\.615\.5312\.180\.5313\.05AdaMerging\(tw\)39\.466\.8713\.690\.6514\.06AdaMerging\(lw\)40\.046\.9613\.690\.8914\.05ProDistill23\.266\.7313\.380\.6114\.05

For this timing comparison, we use 200 calibration epochs for all gradient\-based methods: 400 outer iterations forSurgeryandSurgeryV2, and 200 epochs forESCal\. With 64 calibration examples per task and a batch size of 16, each method performs 800 updates per task\. This timing protocol is separate from the best\-setting accuracy evaluation\.FeatCaluses the same calibration examples for its closed\-form update\. Base\-model construction uses 500 epochs forAdaMergingand 100 epochs per block forProDistill; forTask Arithmetic, it includes loading, merging, and saving the model\.

#### Inference latency and throughput\.

We compare inference cost with and without calibration modules on aTask ArithmeticViT\-B/32 model\. As shown in Table[21](https://arxiv.org/html/2610.33483#A4.T21),ESCalincreases per\-image latency by28\.53%28\.53\\%at batch size 1 and reduces throughput by5\.29%5\.29\\%at batch size 64\. These overheads are similar to those ofSurgeryV2 \(31\.91%31\.91\\%and5\.04%5\.04\\%, respectively\) in this setting\.

We benchmark a model merged with coefficient 0\.3, using SUN397 as the active task, on an NVIDIA RTX A6000 with BF16 autocast\. Timing covers the image encoder, calibration modules, and classification head, with preprocessed inputs already on the GPU\. We perform 10 paired repetitions of each calibrated model and its uncalibrated counterpart, alternating their execution order\. Each repetition uses 100 warm\-up passes and 200 timed passes with CUDA synchronization\.

Table 21:Inference cost relative to the uncalibratedTask ArithmeticViT\-B/32 model\.None denotes the model without calibration\. Absolute values are mean±\\pmstandard deviation\.Δ​t\\Delta tdenotes additional per\-image latency\. Relative changes are computed against paired uncalibrated runs and reported with 95% bootstrap confidence intervals; they may therefore differ from ratios based on the pooled None row\.Batch size 1Batch size 64CalibrationImage latency \(ms\)Δ​t\\Delta t\(ms\)Overhead \(%\)Batch latency \(ms\)Throughput \(img/s\)Throughput reduction \(%\)None5\.51±\\pm0\.02Ref\.Ref\.14\.83±\\pm0\.054316\.5±\\pm15\.6Ref\.Surgery5\.60±\\pm0\.03\+0\.095\+1\.72 \[\+1\.51, \+1\.95\]14\.81±\\pm0\.074320\.1±\\pm20\.0\+0\.15 \[\+0\.06, \+0\.25\]SurgeryV27\.27±\\pm0\.02\+1\.758\+31\.91 \[\+31\.60, \+32\.27\]15\.63±\\pm0\.044095\.0±\\pm11\.6\+5\.04 \[\+4\.99, \+5\.08\]FeatCal5\.50±\\pm0\.02\+0\.004\+0\.07 \[\-0\.30, \+0\.48\]14\.84±\\pm0\.034311\.4±\\pm8\.3\+0\.01 \[\-0\.01, \+0\.04\]ESCal7\.09±\\pm0\.02\+1\.575\+28\.53 \[\+28\.23, \+28\.84\]15\.66±\\pm0\.044086\.9±\\pm10\.3\+5\.29 \[\+5\.22, \+5\.35\]

### D\.3Calibration Data Budget

We evaluate howESCalperformance changes with the number of calibration examples per task on ViT\-B/32 withTask Arithmeticas the base merging method, using 64 examples as the reference\. For each data budget, we use the same merged checkpoint and vary only the task\-specific calibration examples\. We report task\-wise accuracy and its average across tasks\.

Table 22:Effect of the per\-task support size onESCal, ViT\-B/32 with a task\-arithmetic base\. Epochs are held at 200, so the optimizer’s update count scales with the support size\. Accuracy is on the full test set; the default setting \(6464\) is set in bold\.Support / taskSUN397CarsRESISC45EuroSATSVHNGTSRBMNISTDTDAvg\.466\.64±\\pm0\.2163\.17±\\pm1\.2085\.49±\\pm1\.0891\.32±\\pm3\.0793\.87±\\pm0\.3392\.01±\\pm1\.3797\.47±\\pm0\.4960\.33±\\pm1\.4481\.29±\\pm0\.36867\.83±\\pm0\.4265\.05±\\pm1\.2188\.01±\\pm0\.9194\.77±\\pm3\.2895\.47±\\pm0\.3393\.60±\\pm0\.7598\.27±\\pm0\.6463\.48±\\pm1\.6183\.31±\\pm0\.491668\.71±\\pm0\.2766\.50±\\pm0\.8490\.03±\\pm0\.5696\.72±\\pm0\.9196\.02±\\pm0\.5194\.87±\\pm0\.6998\.79±\\pm0\.3465\.49±\\pm0\.8684\.64±\\pm0\.343269\.68±\\pm0\.1467\.77±\\pm0\.4192\.35±\\pm0\.2899\.09±\\pm0\.2396\.79±\\pm0\.1196\.90±\\pm0\.6399\.38±\\pm0\.0769\.89±\\pm1\.4286\.48±\\pm0\.216470\.33±\\pm0\.2269\.02±\\pm0\.1993\.60±\\pm0\.3699\.48±\\pm0\.0797\.10±\\pm0\.0597\.70±\\pm0\.2299\.55±\\pm0\.0373\.52±\\pm0\.4487\.54±\\pm0\.0712870\.66±\\pm0\.0869\.70±\\pm0\.2494\.09±\\pm0\.1999\.57±\\pm0\.0497\.22±\\pm0\.0498\.18±\\pm0\.1899\.64±\\pm0\.0275\.46±\\pm0\.2188\.07±\\pm0\.0625670\.76±\\pm0\.1870\.11±\\pm0\.2794\.33±\\pm0\.1599\.64±\\pm0\.0397\.29±\\pm0\.0398\.37±\\pm0\.0299\.67±\\pm0\.0176\.19±\\pm0\.3488\.29±\\pm0\.07