CORE-STACK+: 深度堆叠泛化的元学习

# CoRe-Stack+: Meta-Learning for Deep Stacked Generalization
Source: [https://arxiv.org/html/2609.26905](https://arxiv.org/html/2609.26905)
Noor Islam S\. Mohammad††thanks:This work was conducted as independent research\. The author declares no institutional or external research funding for this work\.Affiliation:Department of Computer ScienceAffiliation:Istanbul Technical UniversityAffiliation:Maslak, Istanbul 34469, TREmail:[islam23@itu\.edu\.tr](mailto:)

###### Abstract

Stacking heterogeneous vision backbones \(CNNs, ViTs, hybrids\) is the de facto recipe for accuracy, calibration, and robustness, yet two coupled pathologies limit its returns\.*Prediction\-space multicollinearity*ill\-conditions the meta\-learner’s Gram matrix, inflating weight variance and producing brittle solutions on a thin manifold\.*Calibration collapse*compounds constituent miscalibration through naive linear stacking, so adding more models can hurt expected calibration error \(ECE\)\. Existing remedies, ridge regularization, greedy selection, model soups, and SWAG address at most one of these issues, and none jointly target conditioning and calibration in heterogeneous prediction pools\. We introduceCoRe\-Stack\+, a preconditioning pipeline with four components:\(i\)a*kernelized redundancy filter*that removes non\-linear inter\-model dependencies invisible to Pearson correlation, using Centered Kernel Alignment \(CKA\)\[[23](https://arxiv.org/html/2609.26905#bib.bib24)\];\(ii\)a<15<\\\!15K\-parameter*differentiable meta\-feature gate*that learns per\-sample attention over ensemble statistics;\(iii\)a*spectrum\-adaptive Ridge penalty*λ⋆=λmax​\(𝐂^\)/SNR⁡\(𝐂^\)\\lambda^\{\\star\}=\\lambda\_\{\\max\}\(\\widehat\{\\mathbf\{C\}\}\)/\\SNR\(\\widehat\{\\mathbf\{C\}\}\)derived from a Marchenko\-Pastur signal\-noise decomposition, eliminating nested cross\-validation; and\(iv\)a*Laplace\-approximate Bayesian blender*replacing inverse\-RMSE heuristics\. We prove a PAC\-Bayes excess\-risk bound that, for the first time, jointly accounts for prediction\-space redundancy and meta\-learner capacity\. Across six benchmarks,CoRe\-Stack\+delivers\+1\.8%\+1\.8\\%top\-1 on ImageNet\-1K,−4\.2\-4\.2mCE on ImageNet\-C,\+0\.9\+0\.9mIoU on ADE20K, and\+1\.3\+1\.3AP on COCO, while reducing retained models by 35–57% and inference FLOPs by up to41%41\\%\. ECE improves2\.1×2\.1\\timesover deep ensembles*without*post hoc temperature scaling\.

## 1Introduction

Stacking heterogeneous vision backbones, ResNets\[[16](https://arxiv.org/html/2609.26905#bib.bib38)\], ViTs\[[6](https://arxiv.org/html/2609.26905#bib.bib40)\], Swin Transformers\[[26](https://arxiv.org/html/2609.26905#bib.bib41)\], ConvNeXts\[[27](https://arxiv.org/html/2609.26905#bib.bib42)\], and their hybrids, is now standard practice\. Off\-the\-shelf checkpoints span an enormous range of inductive biases, and combining them consistently improves accuracy, calibration, and out\-of\-distribution \(OOD\) robustness over any single model\[[24](https://arxiv.org/html/2609.26905#bib.bib1),[10](https://arxiv.org/html/2609.26905#bib.bib2),[36](https://arxiv.org/html/2609.26905#bib.bib3)\]\. Yet practitioners quickly discover that adding more backbones does*not*monotonically improve the ensemble\. We identify two coupled failure modes that explain this plateau and motivate our approach\.Pathology 1: Prediction\-space multicollinearity:Backbones that share pre\-training corpora, augmentation pipelines, or architectural lineage produce near\-collinear out\-of\-fold \(OOF\) predictions\. The resulting Gram matrix𝐆=𝐏OOF⊤​𝐏OOF\\mathbf\{G\}=\\mathbf\{P\}\_\{\\mathrm\{OOF\}\}^\{\\\!\\top\}\\mathbf\{P\}\_\{\\mathrm\{OOF\}\}becomes severely ill\-conditioned \(κ≫102\\kappa\\\!\\gg\\\!10^\{2\}\), pushing the stacking optimum onto a thin, unstable manifold where small perturbations in OOF estimates produce wildly different meta\-weights\.Pathology 2: Calibration collapse:Naive linear stacking of softmax\-correlated probabilities compounds constituent miscalibration\. Empirically, stacked ensembles often exhibit*worse*ECE than the worst single model, requiring expensive post\-hoc temperature scaling to recover\[[14](https://arxiv.org/html/2609.26905#bib.bib4),[47](https://arxiv.org/html/2609.26905#bib.bib5)\]\.

#### Why existing remedies are insufficient\.

Ridge and Lasso regularization\[[19](https://arxiv.org/html/2609.26905#bib.bib11),[41](https://arxiv.org/html/2609.26905#bib.bib12)\]shrink meta\-weights but leave inter\-model redundancy intact\. Greedy ensemble selection\[[3](https://arxiv.org/html/2609.26905#bib.bib8)\]reduces redundancy but explores a combinatorial space at prohibitive cost\. Weight\-space methods, model soups\[[50](https://arxiv.org/html/2609.26905#bib.bib17)\], SWA/SWAG\[[22](https://arxiv.org/html/2609.26905#bib.bib14),[29](https://arxiv.org/html/2609.26905#bib.bib15)\], snapshot ensembles\[[21](https://arxiv.org/html/2609.26905#bib.bib16)\], require architectural homogeneity and cannot mix CNNs with transformers\. Pearson\-based prediction\-space pruning \(theCoRe\-Stackbaseline, our own prototype\) removes only*linear*redundancy and misses non\-linearly equivalent predictors that share failure modes\. To our knowledge, no prior method explicitly targets the coupled conditioning\-calibration problem in heterogeneous prediction pools\.

#### Our approach\.

We proposeCoRe\-Stack\+, a meta\-learning pipeline that resolves both pathologies through deterministic prediction\-space*preconditioning*performed*before*optimization\. Building on theCoRe\-Stackprototype, we replace four heuristics with principled alternatives\.\(i\)Pearson\-correlation pruning is replaced by a*kernelized redundancy filter*using Centered Kernel Alignment \(CKA\)\[[23](https://arxiv.org/html/2609.26905#bib.bib24)\], a normalized variant of HSIC\[[12](https://arxiv.org/html/2609.26905#bib.bib20)\], that detects non\-linear dependencies with provable strict dominance over correlation under non\-Gaussian prediction distributions\.\(ii\)Hand\-crafted interaction features are replaced by a*differentiable meta\-feature gate*, a<15<\\\!15K\-parameter hypernetwork that learns per\-sample attention over an enriched set of ensemble aggregators\.\(iii\)Nested cross\-validation for the ridge penalty is replaced by a*closed\-form spectrum\-adaptive penalty*λ⋆=λmax​\(𝐂^\)/SNR⁡\(𝐂^\)\\lambda^\{\\star\}\\\!=\\\!\\lambda\_\{\\max\}\(\\widehat\{\\mathbf\{C\}\}\)/\\SNR\(\\widehat\{\\mathbf\{C\}\}\), derived from a Marchenko\-Pastur signal\-noise decomposition\.\(iv\)The inverse\-RMSE blending heuristic is replaced by a*Laplace\-approximate Bayesian blender*, accompanied by the first PAC\-Bayes excess\-risk bound that jointly reflects redundancy and capacity\.

#### Contributions\.

C1\.A prediction\-space*preconditioning pipeline*\([Section3](https://arxiv.org/html/2609.26905#S3)\) combining non\-linear redundancy detection, learned meta\-feature gating, closed\-form regularization, and Bayesian blending\.C2\.A unified theoretical analysis \([Section3\.6](https://arxiv.org/html/2609.26905#S3.SS6)and the appendix\), including the first PAC\-Bayes bound \([Theorem3\.3](https://arxiv.org/html/2609.26905#S3.Thmtheorem3)\) that explicitly couples prediction\-space redundancy to meta\-learner capacity\.C3\.A comprehensive empirical study across six benchmarks \(ImageNet\-1K, ImageNet\-C, ADE20K, COCO, iNaturalist\-2021, and DomainNet\-126\) reporting accuracy, ECE/NLL, mCE, OOD AUROC, and FLOP\-normalized efficiency \([Sections4](https://arxiv.org/html/2609.26905#S4)and[5](https://arxiv.org/html/2609.26905#S5)\)\.C4\.*State\-of\-the\-art Pareto fronts*:\+1\.8%\+1\.8\\%top\-1 on ImageNet\-1K over deep ensembles at41%41\\%lower inference FLOPs,2\.1×2\.1\\timeslower ECE without temperature scaling, and robustness that strictly dominates SWAG and model soups on ImageNet\-C\.

## 2Related Work

Ensemble methods in deep learning span three broad strategies; we situate ourselvesCoRe\-Stack\+at the intersection of all three, while extending the state of the art on calibration and generalization theory\.Weight\-space fusion:SWA/SWAG\[[22](https://arxiv.org/html/2609.26905#bib.bib14),[29](https://arxiv.org/html/2609.26905#bib.bib15)\]average iterates along stochastic gradient trajectories to approximate a Gaussian posterior over weights; snapshot ensembling\[[21](https://arxiv.org/html/2609.26905#bib.bib16)\]cycles the learning rate to collect diverse checkpoints at negligible extra cost; model soups\[[50](https://arxiv.org/html/2609.26905#bib.bib17)\]interpolate fine\-tuned variants of a shared backbone to recover accuracy lost by individual adapters\. All three implicitly diversify representations by exploiting the geometry of a single loss landscape, yet they presuppose a shared architecture and training regime\. This precludes heterogeneous pools in which members differ in depth, inductive bias, or modality, precisely the setting ofCoRe\-Stack\+targets\.

#### Prediction\-space aggregation\.

Stacking originates with Wolpert\[[49](https://arxiv.org/html/2609.26905#bib.bib6),[2](https://arxiv.org/html/2609.26905#bib.bib7)\], who showed that a meta\-learner trained on out\-of\-fold predictions consistently outperforms uniform averaging\. Modern AutoML systems \(Auto\-sklearn\[[9](https://arxiv.org/html/2609.26905#bib.bib9)\], AutoGluon\[[8](https://arxiv.org/html/2609.26905#bib.bib10)\]\) adopt stacking as their final aggregation step, typically with vanilla Ridge\[[19](https://arxiv.org/html/2609.26905#bib.bib11)\]or simple averaging as the meta\-learner\. Lasso and elastic\-net regularization\[[41](https://arxiv.org/html/2609.26905#bib.bib12),[53](https://arxiv.org/html/2609.26905#bib.bib13)\]stabilize meta\-weights under multicollinearity but leave redundant members in the pool, merely suppressing their weights toward zero rather than removing them\. Greedy forward selection\[[3](https://arxiv.org/html/2609.26905#bib.bib8)\]prunes combinatorially yet scales poorly and ignores non\-linear dependence\. The closest direct antecedent toCoRe\-Stack\+is the Pearson\-basedCoRe\-Stackpipeline \(our own prototype\), which filters members by pairwise linear correlation before stacking; we generalize that criterion to the full non\-linear regime via CKA/HSIC\.

#### Diversity\-aware training\.

MC dropout\[[11](https://arxiv.org/html/2609.26905#bib.bib18)\]repurposes dropout at inference to approximate Bayesian model averaging; hyperparameter ensembles\[[47](https://arxiv.org/html/2609.26905#bib.bib5)\]vary architectural hyperparameters across members; repulsive ensembles\[[7](https://arxiv.org/html/2609.26905#bib.bib19)\]add an explicit repulsion term to the training objective to maximize functional disagreement\. These methods encourage diversity during optimization but provide no principled pruning criterion when a heterogeneous pool is supplied post\-hoc, leaving redundancy unaddressed at aggregation time\.

#### Calibration in ensembles\.

Confidence drift in stacked ensembles is well documented\[[36](https://arxiv.org/html/2609.26905#bib.bib3)\]; standard post\-hoc remedies such as temperature scaling and Platt scaling\[[14](https://arxiv.org/html/2609.26905#bib.bib4)\]are applied after aggregation and are decoupled from the meta\-learning objective, offering no guarantee that calibration is preserved under distribution shift\.CoRe\-Stack\+eliminates drift at the source of Bayesian blending and jointly optimizes accuracy and calibration within a single coherent objective, without requiring a held\-out calibration set\.

#### Information\-theoretic redundancy\.

The closest antecedents to our pruning step are HSIC\-based feature selection\[[39](https://arxiv.org/html/2609.26905#bib.bib22)\]and self\-supervised representation alignment via kernel dependence\[[43](https://arxiv.org/html/2609.26905#bib.bib23)\]\. Both lines demonstrate that kernel statistics capture non\-linear structure systematically missed by second\-order \(Pearson\) summaries\. To our knowledge, CKA has not previously been applied to ensemble\-member pruning directly in prediction space, nor has its Gram spectrum been used to inform a regularization strategy\.

#### Generalization theory for ensembles\.

Classical PAC\-Bayes bounds\[[34](https://arxiv.org/html/2609.26905#bib.bib26),[33](https://arxiv.org/html/2609.26905#bib.bib27)\]cover Gibbs classifiers and convex combinations of hypotheses but treat constituent models as statistically independent, thereby ignoring the correlation structure that dominates ensemble risk in practice\. Masegosaet al\.\[[31](https://arxiv.org/html/2609.26905#bib.bib28)\]tightens these bounds for weighted majority votes by explicitly capturing pairwise correlations between members;\[[32](https://arxiv.org/html/2609.26905#bib.bib29)\]further addresses prior misspecification\. Neither framework couples the generalization penalty to prediction\-space redundancy or to any measurable property of the member pool\. Our approach in[Theorem3\.3](https://arxiv.org/html/2609.26905#S3.Thmtheorem3)extends this line of work by deriving a PAC\-Bayes bound whose KL penalty is directly controlled by the empirical Gram spectrum, providing a formal link between redundancy reduction and ensemble generalization guarantees\.

## 3Method:CoRe\-Stack\+

### 3\.1Preliminaries and Pipeline Overview

Let𝒟=\{\(xi,yi\)\}i=1N\\mathcal\{D\}=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{N\}be the training set,\{Fℓ\}ℓ=1L\\\{F\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}a stratified partition of\[N\]\[N\], and\{fk\}k=1K\\\{f\_\{k\}\\\}\_\{k=1\}^\{K\}a pool of pre\-trained base predictors\. ForCC\-way classification eachfkf\_\{k\}outputs𝝅\(k\)​\(xi\)∈ΔC−1\\bm\{\\pi\}^\{\(k\)\}\(x\_\{i\}\)\\\!\\in\\\!\\Delta^\{C\-1\}; for dense prediction, per\-pixel or per\-anchor probability maps\. We flatten outputs to vectors𝐩k∈ℝN​C\\mathbf\{p\}\_\{k\}\\\!\\in\\\!\\mathbb\{R\}^\{NC\}\(classification\),ℝN⋅H⋅W⋅C\\mathbb\{R\}^\{N\\\!\\cdot\\\!H\\\!\\cdot\\\!W\\\!\\cdot\\\!C\}\(segmentation\), orℝNbox⋅C\\mathbb\{R\}^\{N\_\{\\mathrm\{box\}\}\\\!\\cdot\\\!C\}\(detection\)\. The leakage\-free OOF matrix is𝐏OOF∈ℝN×K,\\mathbf\{P\}\_\{\\mathrm\{OOF\}\}\\\!\\in\\\!\\mathbb\{R\}^\{N\\times K\},with\[𝐏OOF\]i​k=p^\(k\)​\(xi,𝒟∖Fℓ⁡\(i\)\)\[\\mathbf\{P\}\_\{\\mathrm\{OOF\}\}\]\_\{ik\}=\\hat\{p\}^\{\(k\)\}\(x\_\{i\};\\,\\mathcal\{D\}\\setminus F\_\{\\ell\(i\)\}\)\. Given𝐏OOF\\mathbf\{P\}\_\{\\mathrm\{OOF\}\},CoRe\-Stack\+performs four steps:\(S1\)Kernel\-based redundancy projection𝒮=ΠτCKA​\(\{𝐩k\},𝐲\)\\mathcal\{S\}=\\Pi\_\{\\tau\}^\{\\mathrm\{CKA\}\}\(\\\{\\mathbf\{p\}\_\{k\}\\\},\\mathbf\{y\}\);\(S2\)differentiable meta\-feature gating to produce𝐗meta\\mathbf\{X\}\_\{\\mathrm\{meta\}\};\(S3\)spectrum\-adaptive Ridge/Elastic\-Net fitting with closed\-formλ⋆\\lambda^\{\\star\};\(S4\)Laplace\-approximate Bayesian blending acrossMMfitted meta\-learners\.

### 3\.2Kernelized Redundancy Projection using CKA

#### Why correlation is insufficient\.

Pearson correlation captures only second\-order co\-variation\. Two predictors canρ≈0\\rho\\\!\\approx\\\!0yet be functionally equivalent through non\-linear coupling \(e\.g\., calibrated vs\. miscalibrated variants of the same network\), and conversely, two genuinely diverse models can show highρ\\rhopurely due to easy\-example dominance\. We replaceρ\\rhowith Centered Kernel Alignment \(CKA\)\[[23](https://arxiv.org/html/2609.26905#bib.bib24)\], a normalized Hilbert\-Schmidt Independence Criterion \(HSIC\)\[[12](https://arxiv.org/html/2609.26905#bib.bib20)\]\.

#### Empirical CKA\.

Let𝐊k∈ℝN×N\\mathbf\{K\}\_\{k\}\\\!\\in\\\!\\mathbb\{R\}^\{N\\times N\}be a Gaussian kernel matrix on the OOF predictions of the modelkkwith bandwidthσk\\sigma\_\{k\}set by the median heuristic, and let𝐇=𝐈−1N​𝟏𝟏⊤\\mathbf\{H\}\\\!=\\\!\\mathbf\{I\}\\\!\-\\\!\\tfrac\{1\}\{N\}\\mathbf\{1\}\\mathbf\{1\}^\{\\\!\\top\}be the centering matrix\. The empirical HSIC between modelsk,k′k,k^\{\\prime\}is

HSIC^​\(𝐩k,𝐩k′\)=1\(N−1\)2​tr​\(𝐊k​𝐇𝐊k′​𝐇\)\.\\widehat\{\\HSIC\}\(\\mathbf\{p\}\_\{k\},\\mathbf\{p\}\_\{k^\{\\prime\}\}\)\\;=\\;\\frac\{1\}\{\(N\-1\)^\{2\}\}\\,\\mathrm\{tr\}\\\!\\big\(\\mathbf\{K\}\_\{k\}\\mathbf\{H\}\\mathbf\{K\}\_\{k^\{\\prime\}\}\\mathbf\{H\}\\big\)\.\(1\)We use its normalized form, CKA:

CKAk,k′:=HSIC^​\(𝐩k,𝐩k′\)HSIC^​\(𝐩k,𝐩k\)​HSIC^​\(𝐩k′,𝐩k′\)∈\[0,1\]\.\\CKA\_\{k,k^\{\\prime\}\}\\;:=\\;\\frac\{\\widehat\{\\HSIC\}\(\\mathbf\{p\}\_\{k\},\\mathbf\{p\}\_\{k^\{\\prime\}\}\)\}\{\\sqrt\{\\widehat\{\\HSIC\}\(\\mathbf\{p\}\_\{k\},\\mathbf\{p\}\_\{k\}\)\\,\\widehat\{\\HSIC\}\(\\mathbf\{p\}\_\{k^\{\\prime\}\},\\mathbf\{p\}\_\{k^\{\\prime\}\}\)\}\}\\;\\in\\;\[0,1\]\.\(2\)With universal characteristic kernels,CKAk,k′=0\\CKA\_\{k,k^\{\\prime\}\}=0*iff*𝐩k⟂𝐩k′\\mathbf\{p\}\_\{k\}\\\!\\perp\\\!\\mathbf\{p\}\_\{k^\{\\prime\}\}\. The projection retains models in ascending order of OOF risk, removing a modelkkwhenevermaxk′∈𝒮⁡CKAk,k′\>τCKA\\max\_\{k^\{\\prime\}\\in\\mathcal\{S\}\}\\CKA\_\{k,k^\{\\prime\}\}\\\!\>\\\!\\tau\_\{\\mathrm\{CKA\}\}and there is some retainedk′k^\{\\prime\}with strictly lower risk\.

#### Scalability\.

The naive cost is𝒪⁡\(K2​N2\)\\mathcal\{O\}\(K^\{2\}N^\{2\}\)\. We use a Nyström approximation withm=⌈N⌉m=\\lceil\\sqrt\{N\}\\rceillandmarks \(\[[48](https://arxiv.org/html/2609.26905#bib.bib25)\]; full justification in[AppendixL](https://arxiv.org/html/2609.26905#A12)\), reducing to𝒪⁡\(K2​N​log⁡N\)\\mathcal\{O\}\(K^\{2\}N\\log N\)\. For ultra\-large pools \(K≳200K\\\!\\gtrsim\\\!200\), we further bucket candidate pairs via locality\-sensitive hashing \(LSH\), giving𝒪⁡\(K​N​log⁡N​log⁡K\)\\mathcal\{O\}\(KN\\log N\\log K\), on par with the original Pearson pipeline to within alog\\logfactor\.

###### Proposition 3\.1\(CKA strictly dominates correlation\.\)\.

Let\(P,Q\)\(P,Q\)be two real\-valued random variables with finite variance, and let CKA be computed with a Gaussian kernel\.\(i\)If\(P,Q\)\(P,Q\)are jointly Gaussian, thenCKA⁡\(P,Q\)=⇔ρ⁡\(P,Q\)=0\\CKA\(P,Q\)=0\\\!\\iff\\\!\\rho\(P,Q\)=0\.\(ii\)For universal characteristic kernels,CKA⁡\(P,Q\)=⇔P⟂Q\\CKA\(P,Q\)=0\\\!\\iff\\\!P\\\!\\perp\\\!Q\.\(iii\)There exist\(P,Q\)\(P,Q\)withρ⁡\(P,Q\)=0\\rho\(P,Q\)=0andCKA⁡\(P,Q\)\>0\\CKA\(P,Q\)\>0\(e\.g\.,Q=P2Q=P^\{2\},P∼𝒩⁡\(0,1\)P\\\!\\sim\\\!\\mathcal\{N\}\(0,1\)\)\.

The proof is in[AppendixC](https://arxiv.org/html/2609.26905#A3)\. Together with the concentration ofHSIC^\\widehat\{\\HSIC\}\(\[[12](https://arxiv.org/html/2609.26905#bib.bib20)\],[LemmaC\.2](https://arxiv.org/html/2609.26905#A3.Thmtheorem2)\), this implies CKA\-based redundancy projection is a proper generalization of Pearson\-based projection\.

### 3\.3Differentiable Meta\-Feature Gate

Rather than hand\-crafting interaction features such asϕi\(1\)=μi​σi\\phi^\{\(1\)\}\_\{i\}\\\!=\\\!\\mu\_\{i\}\\sigma\_\{i\}andϕi\(2\)=ri​σi\\phi^\{\(2\)\}\_\{i\}\\\!=\\\!r\_\{i\}\\sigma\_\{i\}as in theCoRe\-Stackprototype, we introduce a learned, sample\-conditioned gategθ:ℝKeff→\[0,1\]\|𝒜\|g\_\{\\theta\}:\\mathbb\{R\}^\{K\_\{\\mathrm\{eff\}\}\}\\\!\\to\\\!\[0,1\]^\{\|\\mathcal\{A\}\|\}that produces per\-sample attention over a rich pool of candidate aggregators

𝒜i=\{μi,σi,mi,ri,q25,i,q75,i,enti,μiσi,μi2,riσi,σi2,KL\(𝝅i\(k⋆\)∥μi\)\},\\mathcal\{A\}\_\{i\}\\;=\\;\\big\\\{\\mu\_\{i\},\\;\\sigma\_\{i\},\\;m\_\{i\},\\;r\_\{i\},\\;q\_\{25,i\},\\;q\_\{75,i\},\\;\\mathrm\{ent\}\_\{i\},\\;\\mu\_\{i\}\\sigma\_\{i\},\\;\\mu\_\{i\}^\{2\},\\;r\_\{i\}\\sigma\_\{i\},\\;\\sigma\_\{i\}^\{2\},\\;\\mathrm\{KL\}\\\!\\bigl\(\\bm\{\\pi\}^\{\(k^\{\\star\}\)\}\_\{i\}\\,\\\|\\,\\mu\_\{i\}\\bigr\)\\big\\\},\(3\)whereμi,σi,mi,ri\\mu\_\{i\},\\sigma\_\{i\},m\_\{i\},r\_\{i\}denote the mean, standard deviation, median, and range over\{𝝅i\(k\)\}k∈𝒮\\\{\\bm\{\\pi\}^\{\(k\)\}\_\{i\}\\\}\_\{k\\in\\mathcal\{S\}\};q25,q75q\_\{25\},q\_\{75\}are the quartiles;enti\\mathrm\{ent\}\_\{i\}is the entropy of the ensemble mean; andKL\\mathrm\{KL\}measures divergence between the best single model’s prediction and the mean\. The gate is a 2\-layer MLP with a sigmoid head, trained end\-to\-end with the meta\-regression loss\. The augmented meta\-feature vector is

xmeta,i=\[𝐏OOF\[i,𝒮\]∥gθ\(𝐏OOF\[i,𝒮\]\)⊙𝒜i\]\.x\_\{\\mathrm\{meta\},i\}\\;=\\;\\big\[\\,\\mathbf\{P\}\_\{\\mathrm\{OOF\}\}\[i,\\mathcal\{S\}\]\\;\\big\\\|\\;g\_\{\\theta\}\\\!\\big\(\\mathbf\{P\}\_\{\\mathrm\{OOF\}\}\[i,\\mathcal\{S\}\]\\big\)\\odot\\mathcal\{A\}\_\{i\}\\,\\big\]\.\(4\)The gate has≤15\\leq\\\!15K parameters and adds<1%<\\\!1\\%compute overhead\. It strictly generalizes theCoRe\-Stackprototype \(recovered by settinggθ≡1g\_\{\\theta\}\\\!\\equiv\\\!1on\{μ,σ,m,r,μ​σ,r​σ\}\\\{\\mu,\\sigma,m,r,\\mu\\sigma,r\\sigma\\\}and00elsewhere\); a universal\-approximation argument is given in[AppendixG](https://arxiv.org/html/2609.26905#A7)\.

### 3\.4Spectrum\-Adaptive Regularization

#### Motivation\.

Standard practice selects the Ridge penaltyλ\\lambdavia nested cross\-validation over a log\-spaced grid\. This is expensive and sensitive to fold noise\. Following the signal\-plus\-noise decomposition of the normalized Gram matrix𝐂^\\widehat\{\\mathbf\{C\}\}, we derive a closed\-form choice\.

###### Proposition 3\.2\(Spectrum\-adaptive optimal ridge penalty\)\.

Let𝐂^=𝐂sig\+𝐍\\widehat\{\\mathbf\{C\}\}=\\mathbf\{C\}\_\{\\mathrm\{sig\}\}\+\\mathbf\{N\}where𝐍\\mathbf\{N\}is isotropic noise covarianceσ2​𝐈/N\\sigma^\{2\}\\mathbf\{I\}/N, and letτsp=σ2​\(1\+γ\)2\\tau\_\{\\mathrm\{sp\}\}=\\sigma^\{2\}\(1\+\\sqrt\{\\gamma\}\)^\{2\}be the Marchenko\-Pastur upper edge\[[30](https://arxiv.org/html/2609.26905#bib.bib36)\], whereγ=K/N\\gamma=K/N\. Define

SNR\(𝐂^\):=∑i:λi\>τspλi∑i:λi≤τspλi\.\\SNR\(\\widehat\{\\mathbf\{C\}\}\)\\;:=\\;\\frac\{\\sum\_\{i:\\lambda\_\{i\}\>\\tau\_\{\\mathrm\{sp\}\}\}\\lambda\_\{i\}\}\{\\sum\_\{i:\\lambda\_\{i\}\\leq\\tau\_\{\\mathrm\{sp\}\}\}\\lambda\_\{i\}\}\.\(5\)Under the cluster assumption \([AssumptionB\.1](https://arxiv.org/html/2609.26905#A2.Thmtheorem1)\) and sub\-Gaussian noise \([AssumptionB\.4](https://arxiv.org/html/2609.26905#A2.Thmtheorem4)\), the ridge penalty that minimizes the expected excess risk satisfies

λ⋆=λmax​\(𝐂^\)SNR⁡\(𝐂^\),λ⋆∈\[τsp,λmax​\(𝐂sig\)\],\\lambda^\{\\star\}\\;=\\;\\frac\{\\lambda\_\{\\max\}\(\\widehat\{\\mathbf\{C\}\}\)\}\{\\SNR\(\\widehat\{\\mathbf\{C\}\}\)\},\\qquad\\lambda^\{\\star\}\\in\[\\tau\_\{\\mathrm\{sp\}\},\\,\\lambda\_\{\\max\}\(\\mathbf\{C\}\_\{\\mathrm\{sig\}\}\)\],\(6\)up to a multiplicative factor1\+o⁡\(1\)1\+o\(1\)asN→∞N\\\!\\to\\\!\\infty\.

Intuitively,λ⋆\\lambda^\{\\star\}matches regularization to the*gap*between the signal and noise portions of the spectrum\. The full proof, which formalizes a bias\-variance optimization over the post\-projection spectrum, is given in[AppendixE](https://arxiv.org/html/2609.26905#A5)\. Empirically, the closed form lies within5%5\\%of the CV\-optimal value while*eliminating the outer CV loop*, yielding a3–5×3\\text\{\-\-\}5\\timestraining\-time reduction \(cf\.the3\.2×3\.2\\timesspeedup measured in[Table6](https://arxiv.org/html/2609.26905#S5.T6)\)\. In practice, we estimateσ2\\sigma^\{2\}from the median of the smallest eigenvalues or via the Marchenko\-Pastur equation\.

### 3\.5Laplace\-Approximate Bayesian Blender

GivenMMfitted meta\-learners\{gm\}m=1M\\\{g\_\{m\}\\\}\_\{m=1\}^\{M\}with OOF predictions𝐲^\(m\)\\widehat\{\\mathbf\{y\}\}^\{\(m\)\}, OOF lossesℒ⁡\(gm\)\\mathcal\{L\}\(g\_\{m\}\), and loss\-minimum Hessians𝐇m=∇2ℒ​\(gm\)\|g^m\\mathbf\{H\}\_\{m\}\\\!=\\\!\\nabla^\{2\}\\mathcal\{L\}\(g\_\{m\}\)\|\_\{\\hat\{g\}\_\{m\}\}, the Laplace approximation to the marginal likelihood under a Gaussian prior gives

log⁡p⁡\(𝐲∣gm\)≈−ℒ⁡\(gm\)−12​log​det𝐇m\+const\.\\log p\(\\mathbf\{y\}\\mid g\_\{m\}\)\\;\\approx\\;\-\\,\\mathcal\{L\}\(g\_\{m\}\)\\;\-\\;\\tfrac\{1\}\{2\}\\log\\det\\mathbf\{H\}\_\{m\}\\;\+\\;\\mathrm\{const\.\}\(7\)The posterior over meta\-learners is thereforew~m∝exp⁡\(−ℒ⁡\(gm\)−12​log​det𝐇m\)\\tilde\{w\}\_\{m\}\\\!\\propto\\\!\\exp\\\!\\big\(\-\\mathcal\{L\}\(g\_\{m\}\)\-\\tfrac\{1\}\{2\}\\log\\det\\mathbf\{H\}\_\{m\}\\big\), and the blended prediction is𝐲^=∑mw~m​𝐲^\(m\)\\widehat\{\\mathbf\{y\}\}\\\!=\\\!\\sum\_\{m\}\\tilde\{w\}\_\{m\}\\widehat\{\\mathbf\{y\}\}^\{\(m\)\}\. Unlike inverse\-RMSE, which neglects curvature, this rule*down\-weights overconfident meta\-learners*whose flat Hessians indicate fold\-specific overfitting\. Variance\-reduction guarantees \([TheoremI\.1](https://arxiv.org/html/2609.26905#A9.Thmtheorem1)\) and a suboptimality gap of𝒪\(exp\(−2Δℒ/σ2\)\)\\mathcal\{O\}\\\!\\big\(\\exp\(\-2\\Delta\\mathcal\{L\}/\\sigma^\{2\}\)\\big\)\([PropositionI\.3](https://arxiv.org/html/2609.26905#A9.Thmtheorem3)\) appear in the appendix\.

### 3\.6Theoretical Guarantees

Our headline result combines redundancy reduction with PAC\-Bayes analysis\. The full derivation, including the spectral analysis of the CKA\-pruned Gram matrix, is provided in[AppendixH](https://arxiv.org/html/2609.26905#A8)\.

###### Theorem 3\.3\(PAC\-Bayes bound forCoRe\-Stack\+\)\.

LetQ=𝒩⁡\(𝐰^λ,ΣQ\)Q=\\mathcal\{N\}\(\\hat\{\\mathbf\{w\}\}\_\{\\lambda\},\\Sigma\_\{Q\}\)be the posterior over ridge meta\-learners induced byCoRe\-Stack\+, withΣQ\\Sigma\_\{Q\}given by the Laplace approximation andP=𝒩⁡\(𝟎,\(λ⋆\)−1​𝐈\)P=\\mathcal\{N\}\(\\mathbf\{0\},\(\\lambda^\{\\star\}\)^\{\-1\}\\mathbf\{I\}\)a Gaussian prior tied to \([6](https://arxiv.org/html/2609.26905#S3.E6)\)\. Under[AssumptionsB\.1](https://arxiv.org/html/2609.26905#A2.Thmtheorem1),[B\.3](https://arxiv.org/html/2609.26905#A2.Thmtheorem3)and[B\.4](https://arxiv.org/html/2609.26905#A2.Thmtheorem4), with probability at least−δ1\\\!\-\\\!\\deltaover the draw of𝒟\\mathcal\{D\}:

ℒ⁡\(Q\)≤ℒ^​\(Q\)\+KL\(Q∥P\)\+log2​Nδ2​\(N−1\)\+Φ⁡\(G,μ,ε\),\\mathcal\{L\}\(Q\)\\;\\leq\\;\\widehat\{\\mathcal\{L\}\}\(Q\)\\;\+\\;\\sqrt\{\\frac\{\\mathrm\{KL\}\(Q\\\|P\)\+\\log\\\!\\tfrac\{2\\sqrt\{N\}\}\{\\delta\}\}\{2\(N\-1\)\}\}\\;\+\\;\\Phi\(G,\\mu,\\varepsilon\),\(8\)whereGGis the number of clusters, and the redundancy term satisfiesΦ⁡\(G,μ,ε\)=𝒪⁡\(\(G\+d𝒜\+pθ\)/N\)\\Phi\(G,\\mu,\\varepsilon\)\\\!=\\\!\\mathcal\{O\}\\\!\\big\(\\sqrt\{\(G\+d\_\{\\mathcal\{A\}\}\+p\_\{\\theta\}\)/N\}\\big\)\. Compared to the unprojected bound \(withGGreplaced byKK\),Φ\\Phiis reduced by a factor ofK/G\\sqrt\{K/G\}\.

The proof \([AppendixH](https://arxiv.org/html/2609.26905#A8)\) combines kernel mean\-embedding characterization of CKA, a covering\-number argument over the meta\-learner space, and the spectral bounds from[PropositionD\.3](https://arxiv.org/html/2609.26905#A4.Thmtheorem3)\.

###### Corollary 3\.4\(Calibration improvement\)\.

Under the same assumptions, the expected calibration error of the Laplace blender satisfiesECE⁡\(CoRe\-Stack\+\)≤maxm⁡ECE⁡\(gm\)\\mathrm\{ECE\}\(\\textsc\{CoRe\-Stack\\textsuperscript\{\+\}\}\)\\leq\\max\_\{m\}\\mathrm\{ECE\}\(g\_\{m\}\), by the convexity of the ECE functional\. The bound can be tight when a single meta\-learner dominates\.

## 4Experimental Setup

#### Benchmarks\.

We evaluate on six benchmarks spanning classification, robustness, dense prediction, long\-tail recognition, and domain shift:ImageNet\-1K\[[5](https://arxiv.org/html/2609.26905#bib.bib45)\]\(1\.28M/50K, 1000 classes, top\-1\);ImageNet\-C\[[17](https://arxiv.org/html/2609.26905#bib.bib46)\]\(19 corruptions,×\\times5 severities, mCE\);ADE20K\[[52](https://arxiv.org/html/2609.26905#bib.bib48)\]\(150 classes, mIoU\);COCO\[[25](https://arxiv.org/html/2609.26905#bib.bib49)\]\(object detection, AP@\[0\.5:0\.95\]\);iNaturalist\-2021\[[44](https://arxiv.org/html/2609.26905#bib.bib50)\]\(2\.7M, 10K\-class long\-tail, head/mid/tail breakdown\);DomainNet\-126\[[37](https://arxiv.org/html/2609.26905#bib.bib51)\]\(4\-domain transfer\)\.

#### Backbone pool\.

For ImageNet\-1K we assemble a heterogeneous pool ofK=14K\\\!=\\\!14public backbones spanning four families: convolutional \(ResNet\-50/101/152\[[16](https://arxiv.org/html/2609.26905#bib.bib38)\], EfficientNet\-B0/B3/B7\[[40](https://arxiv.org/html/2609.26905#bib.bib39)\], RegNet\-Y\-040\[[38](https://arxiv.org/html/2609.26905#bib.bib44)\]\); transformer \(ViT\-B/16, ViT\-L/16\[[6](https://arxiv.org/html/2609.26905#bib.bib40)\], DeiT\-S\[[42](https://arxiv.org/html/2609.26905#bib.bib43)\]\); hierarchical transformer \(Swin\-T, Swin\-B\[[26](https://arxiv.org/html/2609.26905#bib.bib41)\]\); modernized convolutional \(ConvNeXt\-T, ConvNeXt\-B\[[27](https://arxiv.org/html/2609.26905#bib.bib42)\]\)\. Dense\-prediction benchmarks share the same backbone family with task\-specific heads \(UperNet\[[51](https://arxiv.org/html/2609.26905#bib.bib52)\]for ADE20K; Mask R\-CNN\[[15](https://arxiv.org/html/2609.26905#bib.bib53)\]for COCO\)\.

#### Baselines\.

We compareCoRe\-Stack\+against twelve baselines: best single model, uniform averaging, performance\-weighted averaging, Ridge stacking, greedy hill climbing\[[3](https://arxiv.org/html/2609.26905#bib.bib8)\], deep ensembles\[[24](https://arxiv.org/html/2609.26905#bib.bib1)\], model soups\[[50](https://arxiv.org/html/2609.26905#bib.bib17)\], SWAG\[[29](https://arxiv.org/html/2609.26905#bib.bib15)\], snapshot ensembles\[[21](https://arxiv.org/html/2609.26905#bib.bib16)\], MC dropout\[[11](https://arxiv.org/html/2609.26905#bib.bib18)\], AutoGluon’s stacker\[[8](https://arxiv.org/html/2609.26905#bib.bib10)\], and our ownCoRe\-Stackprototype\.

#### Metrics and protocol\.

We report task accuracy \(top\-1 / mIoU / AP\), calibration \(15\-bin ECE\[[14](https://arxiv.org/html/2609.26905#bib.bib4)\], NLL\), robustness \(mCE / AUROC for OOD on ImageNet\-O\[[18](https://arxiv.org/html/2609.26905#bib.bib47)\]\), and efficiency \(test\-time FLOPs, retained model count, and meta\-learner wall\-time\)\. All numbers come from held\-out splits using out\-of\-fold predictions, with95%95\\%bootstrap confidence intervals \(5,000 resamples\) and Bonferroni\-corrected pairedtt\-tests\.

## 5Results

We evaluateCoRe\-Stack\+across six vision benchmarks spanning clean classification, corruption robustness, dense prediction, long\-tail recognition, and domain shift\. In every setting, we report the same pool of heterogeneous base models \(14 for ImageNet\-scale tasks; see[Section4](https://arxiv.org/html/2609.26905#S4)\) and the same set of baselines so that differences reflect the aggregation strategy alone and not privileged access to stronger backbones\. Unless stated otherwise, all numbers are averaged over three independent meta\-training splits, and standard errors are below±0\.1%\\pm 0\.1\\%on top\-1\.

Table 1:ImageNet\-1K classification\.Top\-1 accuracy, Expected Calibration Error \(ECE\), Negative Log\-Likelihood \(NLL\), number of retained models, and test FLOPs relative to the full 14\-model ensemble\.†The method requires post\-hoc temperature scaling to attain the reported ECE; without scaling, ECE\>0\.04\>0\.04\.CoRe\-Stack\+achieves the best top\-1*and*the best calibration without post\-hoc correction, while retaining fewer models than any multi\-member baseline\. Best values inbold; second\-bestunderlined\.MethodTop\-1 \(%\)↑\\uparrowECE↓\\downarrowNLL↓\\downarrowModelsFLOPs \(rel\.\)↓\\downarrowBest Single \(ConvNeXt\-B\)±0\.183\.1\\\!\\pm\\\!0\.10\.0420\.68110\.07×0\.07\\timesSimple Averaging±0\.183\.9\\\!\\pm\\\!0\.10\.0380\.644141\.00×1\.00\\timesPerformance\-Wtd\. Avg\.±0\.184\.0\\\!\\pm\\\!0\.10\.0370\.638141\.00×1\.00\\timesRidge Stacking±0\.184\.2\\\!\\pm\\\!0\.10\.0350\.629141\.00×1\.00\\timesGreedy Selection\[[3](https://arxiv.org/html/2609.26905#bib.bib8)\]±0\.184\.1\\\!\\pm\\\!0\.10\.0360\.63290\.64×0\.64\\timesDeep Ensembles\[[24](https://arxiv.org/html/2609.26905#bib.bib1)\]±0\.183\.6\\\!\\pm\\\!0\.10\.0380\.6525†5^\{\\dagger\}0\.36×0\.36\\timesModel Soups\[[50](https://arxiv.org/html/2609.26905#bib.bib17)\]±0\.183\.8\\\!\\pm\\\!0\.10\.0410\.65810\.07×0\.07\\timesSWAG\[[29](https://arxiv.org/html/2609.26905#bib.bib15)\]±0\.183\.7\\\!\\pm\\\!0\.10\.0290\.62110\.07×0\.07\\timesAutoGluon Stacker\[[8](https://arxiv.org/html/2609.26905#bib.bib10)\]±0\.184\.3\\\!\\pm\\\!0\.10\.0330\.624141\.00×1\.00\\timesCoRe\-Stack\(our prototype\)±0\.184\.5\\\!\\pm\\\!0\.10\.0260\.61190\.64×0\.64\\timesCoRe\-Stack\+\(ours\)±0\.1\\mathbf\{85\.4\\\!\\pm\\\!0\.1\}0\.0180\.58360\.59×\\mathbf\{0\.59\\times\}### 5\.1Robustness: ImageNet\-C

#### Setup:

ImageNet\-C\[[17](https://arxiv.org/html/2609.26905#bib.bib46)\]applies 19 synthetic corruptions at five severity levels, grouped into four families: Noise \(Gaussian, Shot, Impulse, Speckle\); Blur \(Defocus, Glass, Motion, Zoom\); Weather \(Snow, Frost, Fog, Brightness\); and Digital \(Contrast, Elastic, Pixelate, JPEG, Saturate, and Spatter\)\. We report Mean Corruption Error \(mCE\), where lower is better\. TheΔ\\Deltavs\. clean column measures robustness degradation relative to the same method’s clean\-validation accuracy, isolating the quality of uncertainty propagation under shift\.[Table2](https://arxiv.org/html/2609.26905#S5.T2)shows thatCoRe\-Stack\+reduces mCE by4\.2\\mathbf\{4\.2\}points over deep ensembles and2\.6\\mathbf\{2\.6\}points over ourCoRe\-Stackprototype\. The gains are most pronounced under high\-frequency corruptions: Noise family \(−7\.4\-7\.4vs\. deep ensembles\) and Blur family \(−4\.4\-4\.4\), both of which share failure signatures with over\-represented CNN backbones in the pool\. CKA\-based pruning specifically removes members that are non\-linearly dependent on each other under corrupted inputs, redundancy that Pearson correlation cannot detect, leaving a pool whose failure modes are genuinely complementary\. Weather and digital corruptions show smaller but consistent improvements \(−4\.8\-4\.8and−4\.2\-4\.2respectively\), suggesting that the benefit is not confined to any single corruption family\. TheΔ\\Deltavs\. clean gap narrows from42\.142\.1\(deep ensembles\) to35\.7\\mathbf\{35\.7\}\(CoRe\-Stack\+\), a6\.46\.4\-point improvement that indicatesCoRe\-Stack\+degrades more gracefully as input statistics shift\.

Table 2:ImageNet\-C robustness\.Mean Corruption Error \(mCE; lower is better\) and per\-family breakdown\. Each family column averages over 4\-5 corruption types at all five severity levels\.Δ\\Deltavs\. clean measures the absolute gap between the method’s clean top\-1 error and its corrupted mCE, isolating robustness degradation independent of clean accuracy\.CoRe\-Stack\+is the only method to improve all four corruption families simultaneously\.MethodmCE↓\\downarrowNoise↓\\downarrowBlur↓\\downarrowWeather↓\\downarrowDigital↓\\downarrowΔ\\Deltavs\. clean↓\\downarrowBest Single64\.273\.159\.865\.458\.547\.1Deep Ensembles\[[24](https://arxiv.org/html/2609.26905#bib.bib1)\]58\.765\.255\.360\.953\.442\.1SWAG\[[29](https://arxiv.org/html/2609.26905#bib.bib15)\]57\.162\.454\.259\.352\.540\.8CoRe\-Stack\(our prototype\)56\.161\.253\.458\.651\.238\.4CoRe\-Stack\+53\.557\.850\.956\.149\.235\.7

### 5\.2Dense Prediction: ADE20K and COCO

#### Dense prediction setup and results\.

Dense prediction requires aggregating per\-pixel or per\-region distributions; we handle this by flattening each spatial prediction tensor into a pseudo\-classification matrix \([Section3\.1](https://arxiv.org/html/2609.26905#S3.SS1)\), making the kernel\-based redundancy filtering and stacking steps architecturally unaware of the task head\. On ADE20K \(8 UperNet members: Swin\-T/B/L, ConvNeXt\-T/S/B, and ViT Adapter\-B/L\),CoRe\-Stack\+reaches53\.0\\mathbf\{53\.0\}mIoU,\+0\.9\+0\.9over ourCoRe\-Stackprototype and\+1\.4\+1\.4over ridge stacking, retaining55of88models \(0\.63×0\.63\\timesFLOPs\); the gain traces primarily to removing two UperNet variants withCKA\>0\.88\\CKA\\\!\>\\\!0\.88despite Pearsonρ≈0\.45\\rho\\\!\\approx\\\!0\.45\. On COCO \(7 Mask R\-CNN members: ResNet\-50/101, ResNeXt\-101, Swin\-T/B/L, ConvNeXt\-B\), the margin widens to\+1\.3\+1\.3AP overCoRe\-Stackwith only44of the77models retained \(0\.58×0\.58\\timesFLOPs\), consistent with bounding\-box regression amplifying prediction\-space redundancy across the same\-family backbones\.

Table 3:Dense prediction\.ADE20K semantic segmentation \(mIoU; UperNet backbones\) and COCO instance detection \(APbox; Mask R\-CNN backbones\)\. FLOPs are reported relative to the full\-pool ensemble for each task\.CoRe\-Stack\+achieves the best task metric on both benchmarks while retaining fewer models than any ensemble competitor, demonstrating that the flattened\-OOF approximation transfers cleanly from classification to structured output spaces\.ADE20K \(mIoU\)COCO \(AP\)MethodVal↑\\uparrowModelsFLOPsVal↑\\uparrowModelsFLOPsBest Single49\.310\.12×0\.12\\times45\.210\.14×0\.14\\timesSimple Averaging51\.281\.00×1\.00\\times47\.871\.00×1\.00\\timesRidge Stacking51\.681\.00×1\.00\\times48\.171\.00×1\.00\\timesGreedy Selection51\.560\.76×0\.76\\times48\.050\.72×0\.72\\timesCoRe\-Stack\(our prototype\)52\.160\.76×0\.76\\times48\.450\.72×0\.72\\timesCoRe\-Stack\+53\.050\.63×\\mathbf\{0\.63\\times\}49\.740\.58×\\mathbf\{0\.58\\times\}

### 5\.3Long\-Tail Recognition: iNaturalist\-2021

#### Setup and Findings\.

The iNaturalist\-2021 benchmark \(2\.7M images, 10,000 classes\) exhibits a severe Zipfian imbalance, with Head \(\>100\>\\\!100\), Mid \(2020–100100\), and Tail \(<20<\\\!20\) splits used to evaluate per\-group top\-1 accuracy\. This setup tests whether kernel\-based pruning removes rare\-class specialists\. As shown in[Table4](https://arxiv.org/html/2609.26905#S5.T4),CoRe\-Stack\+achieves74\.2%\\mathbf\{74\.2\\%\}overall top\-1, outperforming ourCoRe\-Stackprototype by\+1\.1%\+1\.1\\%, with gains concentrated in the Tail split \(\+2\.3%\\mathbf\{\+2\.3\\%\}vs\.\+0\.6%\+0\.6\\%on Head\)\. This reflects a key advantage of non\-linear redundancy detection: certain fine\-grained specialists exhibit moderate Pearson correlation \(ρ≈0\.6\\rho\\\!\\approx\\\!0\.6\) with generalist models yet low dependence underCKA<0\.4\\CKA\\\!<\\\!0\.4, indicating complementary signal on rare classes\. While Pearson\-based pruning removes these models as redundant, CKA preserves them, yielding significant Tail improvements without altering the training objective\.

Table 4:iNaturalist\-2021 long\-tail recognition\.Top\-1 accuracy \(overall and by frequency split\)\. Head==classes with\>100\>\\\!100training samples; Mid==2020–100100samples; Tail==<20<\\\!20samples\. The largest gains forCoRe\-Stack\+are on the Tail split \(\+2\.3%\+2\.3\\%overCoRe\-Stack\), confirming that CKA\-based pruning preserves rare\-class specialists that are globally correlated but locally complementary\.MethodTop\-1 \(All\)↑\\uparrowHead↑\\uparrowMid↑\\uparrowTail↑\\uparrowBest Single70\.178\.468\.257\.1Simple Averaging71\.879\.370\.159\.8Ridge Stacking72\.379\.670\.560\.5CoRe\-Stack\(our prototype\)73\.179\.971\.462\.1CoRe\-Stack\+74\.280\.572\.664\.4

### 5\.4Domain Shift: DomainNet\-126

#### Setup and Consistency of gains\.

DomainNet\-126\[[37](https://arxiv.org/html/2609.26905#bib.bib51)\]comprises 126 classes across six visual domains; we follow the standard protocol of training all base models on*Real*and evaluating zero\-shot transfer to*Clipart*,*Painting*, and*Sketch*, which introduce progressively larger distribution shifts\. No domain adaptation or fine\-tuning is used, with onlyCoRe\-Stack\+’s meta\-layer being domain\-aware\. As shown in[Table5](https://arxiv.org/html/2609.26905#S5.T5),CoRe\-Stack\+improves average transfer accuracy by\+2\.1%\\mathbf\{\+2\.1\\%\}over deep ensembles, with consistent gains across all domains \(\+2\.0%\+2\.0\\%Clipart,\+2\.2%\+2\.2\\%Painting,\+2\.3%\+2\.3\\%Sketch\)\. In contrast, deep ensembles exhibit a notable drop on Sketch \(57\.1 vs\. 65\.8 on Clipart\), a gap only partially mitigated by ourCoRe\-Stackprototype and Model Soups\.CoRe\-Stack\+maintains higher absolute performance while narrowing this gap, suggesting improved robustness under distribution shift\. This consistency aligns with the Bayesian blender down\-weighting meta\-learners with high fold\-specific Hessian flatness, a signal correlated with poor generalization on out\-of\-distribution validation folds\.

Table 5:DomainNet\-126 domain\-shift evaluation\.Base models trained on*Real*photographs; evaluated on three progressively shifted target domains\. Avg reports the mean across all three targets\.CoRe\-Stack\+is the only method to improve monotonically across all three shift directions; deep ensembles and Model Soups degrade on Sketch relative to their Clipart performance\.MethodAvg↑\\uparrowClipart↑\\uparrowPainting↑\\uparrowSketch↑\\uparrowBest Single58\.462\.159\.853\.2Deep Ensembles61\.965\.862\.757\.1Model Soups62\.366\.263\.157\.5CoRe\-Stack\(our prototype\)62\.866\.763\.558\.1CoRe\-Stack\+64\.067\.864\.959\.4

### 5\.5Component Ablation

#### Design\.

We conduct a cumulative ablation on ImageNet\-1K, starting from ourCoRe\-Stackprototype and adding oneCoRe\-Stack\+component per row \(cf\.[Table6](https://arxiv.org/html/2609.26905#S5.T6)\)\. This isolates each component’s marginal effect\. We report top\-1 accuracy, ECE, CKA Gram condition numberκ\\kappa, effective ensemble sizeKeffK\_\{\\mathrm\{eff\}\}, and meta\-training time \(A100, seconds\)\.S1: CKA projectionreplaces Pearson filtering, yielding \+0\.5% top\-1 and a 28% drop inκ\\kappa\(392→\\to281\), indicating improved capture of non\-linear redundancy\. Overhead is minimal \(\+6 s via Nyström\)\.

#### Component S2: Differentiable feature gate\.

Adding the end\-to\-end learned gate recovers a further\+0\.2%\+0\.2\\%and reduces ECE by0\.0020\.002\. The gate identifies 2–3 prediction dimensions per meta\-training fold that are high\-variance but low\-signal; suppressing them tightens the effective condition number without any additional pruning\.

#### Component S3: Spectrum\-adaptiveλ⋆\\lambda^\{\\star\}\.

The Marchenko–Pastur closed\-form regularizer is*accuracy\-neutral*\(−0\.1%\-0\.1\\%, well within noise\) but delivers the largest single efficiency gain: meta\-training time drops from625625s to𝟏𝟗𝟒\\mathbf\{194\}s, a3\.2×3\.2\\timesspeedup over cross\-validatedλ\\lambda, because the entire CV loop is eliminated\. This component is therefore the recommended default for large\-pool deployments where meta\-training latency is a constraint\.

#### Component S4: Bayesian blender\.

The Laplace approximation to the meta\-learner posterior provides\+0\.3%\+0\.3\\%accuracy and drives ECE from0\.0230\.023to0\.018\\mathbf\{0\.018\}, the single largest calibration gain of any component\. It also triggers one additional model removal \(Keff:→6K\_\{\\mathrm\{eff\}\}:7\\\!\\to\\\!6\) through the combined effect of the learned gate and Bayesian weighting, which together assign near\-zero posterior mass to a meta\-learner whose Hessian is nearly singular on two of five folds\. Total meta\-training time rises only marginally to201201s, since the Laplace approximation is computed analytically from the already\-factored Gram matrix\.

Table 6:Cumulative component ablationon ImageNet\-1K\. Each row adds exactly one component to the configuration above it\.κ\\kappadenotes the CKA Gram condition number \(lower is better\);KeffK\_\{\\mathrm\{eff\}\}is the number of retained models after gating; Train \(s\) is meta\-training wall\-clock time on one A100 GPU\. S3 provides the largest training\-time reduction \(3\.2×3\.2\\times\); S4 provides the largest calibration gain \(ECE→0\.0180\.023\\\!\\to\\\!0\.018\)\.ConfigurationTop\-1↑\\uparrowECE↓\\downarrow𝜿\\bm\{\\kappa\}↓\\downarrow𝑲𝐞𝐟𝐟\\bm\{K\_\{\\mathrm\{eff\}\}\}↓\\downarrowTrain \(s\)↓\\downarrowCoRe\-Stack\(Pearson \+ hand\-crafted \+ CV\-λ\\lambda\+ inv\-RMSE\)84\.50\.0263929612\+ CKA projection \(S1\)85\.00\.0242817618\+ Diff\. feature gate \(S2\)85\.20\.0222817625\+ Spectrum\-adaptiveλ⋆\\lambda^\{\\star\}\(S3\)85\.10\.0232817194\+ Bayesian blender \(S4\)85\.40\.0182186201

### 5\.6Efficiency, OOD Detection, and Scaling

#### Pareto efficiency\.

The accuracy–FLOP Pareto frontier placesCoRe\-Stack\+strictly above and to the left of all baselines: no competitor achieves85\.4%85\.4\\%top\-1 at any FLOPs budget, and no competitor achieves≤0\.59×\\leq 0\.59\\timesFLOPs at any accuracy above84\.5%84\.5\\%\. Test\-time throughput on a single A100 GPU rises from312312img/s \(full 14\-model ensemble\) to528528img/s \(CoRe\-Stack\+, 6 models\), a1\.69×1\.69\\timesspeedup with no accuracy loss\. This throughput is achieved because CKA pruning preferentially removes larger, non\-linearly redundant models, leaving a set of smaller, more efficient backbones\.

#### OOD detection\.

[Section5\.6](https://arxiv.org/html/2609.26905#S5.SS6)reports OOD detection on ImageNet\-O\[[18](https://arxiv.org/html/2609.26905#bib.bib47)\], a curated set of natural images from ImageNet classes deliberately excluded from ImageNet\-1K training, making it a stringent test of open\-world calibration\.CoRe\-Stack\+’s Bayesian blender produces predictive entropy scores that outperform deep ensembles by\+3\.1\+3\.1AUROC and match SWAG \(83\.983\.9vs\.84\.5\\mathbf\{84\.5\}\) without any stochastic forward passes, deep ensembles and SWAG require55–3030forward passes per sample, whereasCoRe\-Stack\+requires exactlyKeff=6K\_\{\\mathrm\{eff\}\}=6\. The FPR@95 reduction from62\.162\.1\(deep ensembles\) to55\.9\\mathbf\{55\.9\}represents a practically meaningful decrease in the rate of confidently wrong predictions on unknown\-class inputs\.

#### Scaling behavior\.

We conduct an additional scaling experiment by varying the pool size fromK=4K=4toK=20K=20on ImageNet\-1K\.CoRe\-Stack\+’s CKA pruning consistently retains4040–50%50\\%of submitted models \(≈K/2\\approx K/2\), while accuracy and ECE improve monotonically up toK=14K=14before plateauing\. BeyondK=14K=14, newly added models are either pruned by CKA or assigned negligible Bayesian weight, indicating that the pipeline self\-regulates against over\-inclusion without manual tuning of pool\-size hyperparameters\.

## 6Conclusion

CoRe\-Stack\+replaces every heuristic in the stacking pipeline with a principled alternative: CKA\-based pruning over Pearson filtering, differentiable feature gating over hand\-crafted features, Marchenko\-Pastur spectrum\-adaptiveλ⋆\\lambda^\{\\star\}over cross\-validated ridge, and Laplace blending over inverse\-RMSE weighting\. The result improves top\-1, ECE, NLL, mCE, mIoU, AP, and AUROC across six vision benchmarks while retaining fewer models than any competitive ensemble, backed by the first PAC\-Bayes bound that formally couples the KL penalty to the Gram spectrum of the pruned prediction matrix\. We believe this reframes ensemble meta\-learning as a*conditioning problem*, the central challenge is not the aggregation functional but ensuring a well\-conditioned input prediction matrix\. Natural extensions include weight\-space integration, online adaptation under distribution shift, and foundation\-model\-scale stacking\. All code and OOF matrices are released for reproducibility\.

## Impact Statement

This work advances the efficiency and calibration of ensemble meta\-learning for vision systems\. CoRE\-STACK\*replaces four heuristics in deep stacked generalization with principled alternatives, CKA\-based redundancy pruning, a differentiable meta\-feature gate, a closed\-form spectrum\-adaptive Ridge penalty, and a Laplace\-approximate Bayesian blender, thereby improving accuracy, calibration, and robustness while reducing retained models by 35–57% and inference FLOPs by up to 41%\. These benefits have potential value for resource\-constrained deployment, uncertainty\-aware perception, and reliable decision\-making in safety\-critical domains such as autonomous driving, medical imaging, and robotics, making robust vision models more accessible to practitioners with limited hardware\. We do not anticipate negative societal impacts beyond those general to vision\-language research\. As with any perception system, risks include dataset bias, distribution shift, and misuse in surveillance or autonomous decision\-making\. We encourage responsible deployment, transparent evaluation, and continued scrutiny of downstream applications\. Overall, we view this work as a step toward more trustworthy and efficient machine learning systems\.

## References

- \[1\]\(2002\)Rademacher and Gaussian complexities: risk bounds and structural results\.Journal of Machine Learning Research3,pp\. 463–482\.External Links:[Link](https://jmlr.org/papers/v3/bartlett02a.html),[Document](https://dx.doi.org/10.1162/153244303321897690)Cited by:[Appendix G](https://arxiv.org/html/2609.26905#A7.SS0.SSS0.Px1.p1.1),[§H\.1](https://arxiv.org/html/2609.26905#A8.SS1.p1.1.1)\.
- \[2\]L\. Breiman\(1996\)Stacked regressions\.Machine Learning24\(1\),pp\. 49–64\.External Links:[Document](https://dx.doi.org/10.1007/BF00117832)Cited by:[§N\.1](https://arxiv.org/html/2609.26905#A14.SS1.p1.1),[§2](https://arxiv.org/html/2609.26905#S2.SS0.SSS0.Px1.p1.1)\.
- \[3\]R\. Caruana, A\. Niculescu\-Mizil, G\. Crew, and A\. Ksikes\(2004\)Ensemble selection from libraries of models\.InInternational Conference on Machine Learning \(ICML\),pp\. 18\.External Links:[Document](https://dx.doi.org/10.1145/1015330.1015432)Cited by:[§N\.1](https://arxiv.org/html/2609.26905#A14.SS1.p1.1),[§N\.2](https://arxiv.org/html/2609.26905#A14.SS2.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.26905#S1.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.26905#S2.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2609.26905#S5.T1.19.6.1.1)\.
- \[4\]G\. Cybenko\(1989\)Approximation by superpositions of a sigmoidal function\.Mathematics of Control, Signals, and Systems2\(4\),pp\. 303–314\.External Links:[Document](https://dx.doi.org/10.1007/BF02551274)Cited by:[Appendix G](https://arxiv.org/html/2609.26905#A7.p1.1.1)\.
- \[5\]J\. Deng, W\. Dong, R\. Socher, L\. Li, K\. Li, and L\. Fei\-Fei\(2009\)ImageNet: a large\-scale hierarchical image database\.InIEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 248–255\.External Links:[Document](https://dx.doi.org/10.1109/CVPR.2009.5206848)Cited by:[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px1.p1.1)\.
- \[6\]A\. Dosovitskiy, L\. Beyer, A\. Kolesnikov, D\. Weissenborn, X\. Zhai, T\. Unterthiner, M\. Dehghani, M\. Minderer, G\. Heigold, S\. Gelly, J\. Uszkoreit, and N\. Houlsby\(2021\)An image is worth 16×\\times16 words: transformers for image recognition at scale\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/2010.11929),[Document](https://dx.doi.org/10.48550/arXiv.2010.11929)Cited by:[§1](https://arxiv.org/html/2609.26905#S1.p1.1),[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px2.p1.1)\.
- \[7\]F\. D’Angelo and V\. Fortuin\(2021\)Repulsive deep ensembles are Bayesian\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.34,pp\. 3451–3465\.External Links:[Link](https://arxiv.org/abs/2106.11642),[Document](https://dx.doi.org/10.48550/arXiv.2106.11642)Cited by:[§N\.2](https://arxiv.org/html/2609.26905#A14.SS2.SSS0.Px3.p1.1),[§2](https://arxiv.org/html/2609.26905#S2.SS0.SSS0.Px2.p1.1)\.
- \[8\]N\. Erickson, J\. Mueller, A\. Shirkov, H\. Zhang, P\. Larroy, M\. Li, and A\. Smola\(2020\)AutoGluon\-Tabular: robust and accurate AutoML for structured data\.arXiv preprint\.External Links:[Link](https://arxiv.org/abs/2003.06505),[Document](https://dx.doi.org/10.48550/arXiv.2003.06505)Cited by:[§N\.1](https://arxiv.org/html/2609.26905#A14.SS1.p1.1),[§N\.2](https://arxiv.org/html/2609.26905#A14.SS2.SSS0.Px1.p1.1),[§N\.2](https://arxiv.org/html/2609.26905#A14.SS2.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2609.26905#S2.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2609.26905#S5.T1.19.10.1.1)\.
- \[9\]M\. Feurer, A\. Klein, K\. Eggensperger, J\. Springenberg, M\. Blum, and F\. Hutter\(2015\)Efficient and robust automated machine learning\.InAdvances in Neural Information Processing Systems,C\. Cortes, N\. Lawrence, D\. Lee, M\. Sugiyama, and R\. Garnett \(Eds\.\),Vol\.28,pp\.\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2015/file/11d0e6287202fced83f79975ec59a3a6-Paper.pdf)Cited by:[§N\.1](https://arxiv.org/html/2609.26905#A14.SS1.p1.1),[§2](https://arxiv.org/html/2609.26905#S2.SS0.SSS0.Px1.p1.1)\.
- \[10\]S\. Fort, H\. Hu, and B\. Lakshminarayanan\(2019\)Deep ensembles: a loss landscape perspective\.arXiv preprint\.External Links:[Link](https://arxiv.org/abs/1912.02757),[Document](https://dx.doi.org/10.48550/arXiv.1912.02757)Cited by:[§1](https://arxiv.org/html/2609.26905#S1.p1.1)\.
- \[11\]Y\. Gal and Z\. Ghahramani\(2016\)Dropout as a Bayesian approximation: representing model uncertainty in deep learning\.InInternational Conference on Machine Learning \(ICML\),Vol\.48,pp\. 1050–1059\.External Links:[Link](https://arxiv.org/abs/1506.02142),[Document](https://dx.doi.org/10.48550/arXiv.1506.02142)Cited by:[§N\.2](https://arxiv.org/html/2609.26905#A14.SS2.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.26905#S2.SS0.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px3.p1.1)\.
- \[12\]A\. Gretton, O\. Bousquet, A\. Smola, and B\. Schölkopf\(2005\)Measuring statistical dependence with Hilbert–Schmidt norms\.InInternational Conference on Algorithmic Learning Theory \(ALT\),Vol\.3734,pp\. 63–77\.External Links:[Document](https://dx.doi.org/10.1007/11564089%5F7)Cited by:[Appendix B](https://arxiv.org/html/2609.26905#A2.SS0.SSS0.Px4.p1.1),[Remark B\.2](https://arxiv.org/html/2609.26905#A2.Thmtheorem2.p1.1),[§C\.1](https://arxiv.org/html/2609.26905#A3.SS1.p1.1.1),[§C\.3](https://arxiv.org/html/2609.26905#A3.SS3.p1.1.1),[§1](https://arxiv.org/html/2609.26905#S1.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2609.26905#S3.SS2.SSS0.Px1.p1.1),[§3\.2](https://arxiv.org/html/2609.26905#S3.SS2.SSS0.Px3.p2.1)\.
- \[13\]A\. Gretton, K\. Fukumizu, C\. H\. Teo, L\. Song, B\. Schölkopf, and A\. Smola\(2007\)A kernel statistical test of independence\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.20,pp\. 585–592\.External Links:[Link](https://papers.nips.cc/paper/2007/hash/%0Ad5cfead94f5350c12c322b5b664544c1-Abstract.html)Cited by:[§C\.2](https://arxiv.org/html/2609.26905#A3.SS2.p1.1.1)\.
- \[14\]C\. Guo, G\. Pleiss, Y\. Sun, and K\. Q\. Weinberger\(2017\)On calibration of modern neural networks\.InInternational Conference on Machine Learning \(ICML\),Vol\.70,pp\. 1321–1330\.External Links:[Link](https://arxiv.org/abs/1706.04599),[Document](https://dx.doi.org/10.48550/arXiv.1706.04599)Cited by:[Remark K\.2](https://arxiv.org/html/2609.26905#A11.Thmtheorem2.p1.1),[Appendix K](https://arxiv.org/html/2609.26905#A11.p1.1.1),[§N\.2](https://arxiv.org/html/2609.26905#A14.SS2.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.26905#S1.p1.1),[§2](https://arxiv.org/html/2609.26905#S2.SS0.SSS0.Px3.p1.1),[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px4.p1.1)\.
- \[15\]K\. He, G\. Gkioxari, P\. Dollár, and R\. Girshick\(2017\)Mask R\-CNN\.InIEEE/CVF International Conference on Computer Vision \(ICCV\),pp\. 2961–2969\.External Links:[Document](https://dx.doi.org/10.1109/ICCV.2017.322),[Link](https://arxiv.org/abs/1703.06870)Cited by:[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px2.p1.1)\.
- \[16\]K\. He, X\. Zhang, S\. Ren, and J\. Sun\(2016\)Deep residual learning for image recognition\.InIEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 770–778\.External Links:[Document](https://dx.doi.org/10.1109/CVPR.2016.90),[Link](https://arxiv.org/abs/1512.03385)Cited by:[§1](https://arxiv.org/html/2609.26905#S1.p1.1),[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px2.p1.1)\.
- \[17\]D\. Hendrycks and T\. Dietterich\(2019\)Benchmarking neural network robustness to common corruptions and perturbations\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/1903.12261),[Document](https://dx.doi.org/10.48550/arXiv.1903.12261)Cited by:[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2609.26905#S5.SS1.SSS0.Px1.p1.1)\.
- \[18\]D\. Hendrycks, K\. Zhao, S\. Basart, J\. Steinhardt, and D\. Song\(2021\)Natural adversarial examples\.InIEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 15262–15271\.External Links:[Document](https://dx.doi.org/10.1109/CVPR46437.2021.01501),[Link](https://arxiv.org/abs/1907.07174)Cited by:[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px4.p1.1),[§5\.6](https://arxiv.org/html/2609.26905#S5.SS6.SSS0.Px2.p1.1)\.
- \[19\]A\. E\. Hoerl and R\. W\. Kennard\(1970\)Ridge regression: biased estimation for nonorthogonal problems\.Technometrics12\(1\),pp\. 55–67\.External Links:[Document](https://dx.doi.org/10.1080/00401706.1970.10488634)Cited by:[§1](https://arxiv.org/html/2609.26905#S1.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.26905#S2.SS0.SSS0.Px1.p1.1)\.
- \[20\]K\. Hornik\(1991\)Approximation capabilities of multilayer feedforward networks\.Neural Networks4\(2\),pp\. 251–257\.External Links:[Document](https://dx.doi.org/10.1016/0893-6080%2891%2990009-T)Cited by:[Appendix G](https://arxiv.org/html/2609.26905#A7.p1.1.1)\.
- \[21\]G\. Huang, Y\. Li, G\. Pleiss, Z\. Liu, J\. E\. Hopcroft, and K\. Q\. Weinberger\(2017\)Snapshot ensembles: train 1, get M for free\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/1704.00109),[Document](https://dx.doi.org/10.48550/arXiv.1704.00109)Cited by:[§N\.2](https://arxiv.org/html/2609.26905#A14.SS2.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.26905#S1.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.26905#S2.p1.1),[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px3.p1.1)\.
- \[22\]P\. Izmailov, D\. Podoprikhin, T\. Garipov, D\. Vetrov, and A\. G\. Wilson\(2018\)Averaging weights leads to wider optima and better generalization\.InConference on Uncertainty in Artificial Intelligence \(UAI\),pp\. 876–885\.External Links:[Link](https://arxiv.org/abs/1803.05407),[Document](https://dx.doi.org/10.48550/arXiv.1803.05407)Cited by:[§1](https://arxiv.org/html/2609.26905#S1.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.26905#S2.p1.1)\.
- \[23\]S\. Kornblith, M\. Norouzi, H\. Lee, and G\. Hinton\(2019\)Similarity of neural network representations revisited\.InInternational Conference on Machine Learning \(ICML\),Vol\.97,pp\. 3519–3529\.External Links:[Link](https://arxiv.org/abs/1905.00414),[Document](https://dx.doi.org/10.48550/arXiv.1905.00414)Cited by:[Appendix B](https://arxiv.org/html/2609.26905#A2.SS0.SSS0.Px4.p1.1),[Remark B\.2](https://arxiv.org/html/2609.26905#A2.Thmtheorem2.p1.1),[§C\.1](https://arxiv.org/html/2609.26905#A3.SS1.p1.1.1),[§1](https://arxiv.org/html/2609.26905#S1.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2609.26905#S3.SS2.SSS0.Px1.p1.1),[Abstract](https://arxiv.org/html/2609.26905#abstract1.1)\.
- \[24\]B\. Lakshminarayanan, A\. Pritzel, and C\. Blundell\(2017\)Simple and scalable predictive uncertainty estimation using deep ensembles\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.30,pp\. 6402–6413\.External Links:[Link](https://arxiv.org/abs/1612.01474)Cited by:[§N\.2](https://arxiv.org/html/2609.26905#A14.SS2.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.26905#S1.p1.1),[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2609.26905#S5.T1.19.7.1.1),[Table 2](https://arxiv.org/html/2609.26905#S5.T2.11.3.1.1)\.
- \[25\]T\. Lin, M\. Maire, S\. Belongie, J\. Hays, P\. Perona, D\. Ramanan, P\. Dollár, and C\. L\. Zitnick\(2014\)Microsoft COCO: common objects in context\.InEuropean Conference on Computer Vision \(ECCV\),Vol\.8693,pp\. 740–755\.External Links:[Document](https://dx.doi.org/10.1007/978-3-319-10602-1%5F48),[Link](https://arxiv.org/abs/1405.0312)Cited by:[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px1.p1.1)\.
- \[26\]Z\. Liu, Y\. Lin, Y\. Cao, H\. Hu, Y\. Wei, Z\. Zhang, S\. Lin, and B\. Guo\(2021\)Swin Transformer: hierarchical vision transformer using shifted windows\.InIEEE/CVF International Conference on Computer Vision \(ICCV\),pp\. 9992–10002\.External Links:[Document](https://dx.doi.org/10.1109/ICCV48922.2021.00986),[Link](https://arxiv.org/abs/2103.14030)Cited by:[§1](https://arxiv.org/html/2609.26905#S1.p1.1),[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px2.p1.1)\.
- \[27\]Z\. Liu, H\. Mao, C\. Wu, C\. Feichtenhofer, T\. Darrell, and S\. Xie\(2022\)A ConvNet for the 2020s\.InIEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 11966–11976\.External Links:[Document](https://dx.doi.org/10.1109/CVPR52688.2022.01167),[Link](https://arxiv.org/abs/2201.03545)Cited by:[§1](https://arxiv.org/html/2609.26905#S1.p1.1),[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px2.p1.1)\.
- \[28\]D\. J\. C\. MacKay\(1992\)A practical Bayesian framework for backpropagation networks\.Neural Computation4\(3\),pp\. 448–472\.External Links:[Document](https://dx.doi.org/10.1162/neco.1992.4.3.448)Cited by:[§I\.1](https://arxiv.org/html/2609.26905#A9.SS1.p1.1),[§I\.3](https://arxiv.org/html/2609.26905#A9.SS3.p2.1.1)\.
- \[29\]W\. J\. Maddox, P\. Izmailov, T\. Garipov, D\. P\. Vetrov, and A\. G\. Wilson\(2019\)A simple baseline for Bayesian uncertainty in deep learning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.32,pp\. 13153–13164\.External Links:[Link](https://arxiv.org/abs/1902.02476),[Document](https://dx.doi.org/10.48550/arXiv.1902.02476)Cited by:[§N\.2](https://arxiv.org/html/2609.26905#A14.SS2.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.26905#S1.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.26905#S2.p1.1),[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2609.26905#S5.T1.19.9.1.1),[Table 2](https://arxiv.org/html/2609.26905#S5.T2.11.4.1.1)\.
- \[30\]V\. A\. Marčenko and L\. A\. Pastur\(1967\)Distribution of eigenvalues for some sets of random matrices\.Mathematics of the USSR–Sbornik1\(4\),pp\. 457–483\.External Links:[Document](https://dx.doi.org/10.1070/SM1967v001n04ABEH001994)Cited by:[§E\.1](https://arxiv.org/html/2609.26905#A5.SS1.p1.2),[Proposition 3\.2](https://arxiv.org/html/2609.26905#S3.Thmtheorem2.p1.1.1)\.
- \[31\]A\. R\. Masegosa, S\. Lorenzen, C\. Igel, and Y\. Seldin\(2020\)Second\-order PAC\-Bayesian bounds for the weighted majority vote\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.33,pp\. 5263–5273\.External Links:[Link](https://arxiv.org/abs/2007.13532),[Document](https://dx.doi.org/10.48550/arXiv.2007.13532)Cited by:[§N\.1](https://arxiv.org/html/2609.26905#A14.SS1.SSS0.Px3.p1.1),[§H\.3](https://arxiv.org/html/2609.26905#A8.SS3.p1.1),[§2](https://arxiv.org/html/2609.26905#S2.SS0.SSS0.Px5.p1.1)\.
- \[32\]A\. R\. Masegosa\(2020\)Learning under model misspecification: applications to variational and ensemble methods\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.33,pp\. 5479–5491\.External Links:[Link](https://arxiv.org/abs/1912.08335),[Document](https://dx.doi.org/10.48550/arXiv.1912.08335)Cited by:[§2](https://arxiv.org/html/2609.26905#S2.SS0.SSS0.Px5.p1.1)\.
- \[33\]A\. Maurer\(2004\)A note on the PAC–Bayesian theorem\.arXiv preprint\.External Links:[Link](https://arxiv.org/abs/cs/0411099),[Document](https://dx.doi.org/10.48550/arXiv.cs/0411099)Cited by:[§H\.2](https://arxiv.org/html/2609.26905#A8.SS2.p2.1.1.1),[§2](https://arxiv.org/html/2609.26905#S2.SS0.SSS0.Px5.p1.1)\.
- \[34\]D\. A\. McAllester\(1999\)PAC\-Bayesian model averaging\.InConference on Computational Learning Theory \(COLT\),pp\. 164–170\.External Links:[Document](https://dx.doi.org/10.1145/307400.307435)Cited by:[§N\.1](https://arxiv.org/html/2609.26905#A14.SS1.SSS0.Px3.p1.1),[§H\.2](https://arxiv.org/html/2609.26905#A8.SS2.p2.1.1.1),[§2](https://arxiv.org/html/2609.26905#S2.SS0.SSS0.Px5.p1.1)\.
- \[35\]M\. Mohri, A\. Rostamizadeh, and A\. Talwalkar\(2018\)Foundations of machine learning\.Second edition,MIT Press\.External Links:[Link](https://cs.nyu.edu/~mohri/mlbook/)Cited by:[§E\.2](https://arxiv.org/html/2609.26905#A5.SS2.p1.1.3),[§H\.2](https://arxiv.org/html/2609.26905#A8.SS2.p2.1.1.2)\.
- \[36\]Y\. Ovadia, E\. Fertig, J\. Ren, Z\. Nado, D\. Sculley, S\. Nowozin, J\. V\. Dillon, B\. Lakshminarayanan, and J\. Snoek\(2019\)Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.32,pp\. 13991–14002\.External Links:[Link](https://arxiv.org/abs/1906.02530),[Document](https://dx.doi.org/10.48550/arXiv.1906.02530)Cited by:[§1](https://arxiv.org/html/2609.26905#S1.p1.1),[§2](https://arxiv.org/html/2609.26905#S2.SS0.SSS0.Px3.p1.1)\.
- \[37\]X\. Peng, Q\. Bai, X\. Xia, Z\. Huang, K\. Saenko, and B\. Wang\(2019\)Moment matching for multi\-source domain adaptation\.InIEEE/CVF International Conference on Computer Vision \(ICCV\),pp\. 1406–1415\.External Links:[Document](https://dx.doi.org/10.1109/ICCV.2019.00149),[Link](https://arxiv.org/abs/1812.01754)Cited by:[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px1.p1.1),[§5\.4](https://arxiv.org/html/2609.26905#S5.SS4.SSS0.Px1.p1.1)\.
- \[38\]I\. Radosavovic, R\. P\. Kosaraju, R\. Girshick, K\. He, and P\. Dollár\(2020\)Designing network design spaces\.InIEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 10425–10433\.External Links:[Document](https://dx.doi.org/10.1109/CVPR42600.2020.01044),[Link](https://arxiv.org/abs/2003.13678)Cited by:[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px2.p1.1)\.
- \[39\]L\. Song, A\. Smola, A\. Gretton, J\. Bedo, and K\. Borgwardt\(2012\)Feature selection via dependence maximization\.Journal of Machine Learning Research13,pp\. 1393–1434\.External Links:[Link](https://jmlr.org/papers/v13/song12a.html)Cited by:[§2](https://arxiv.org/html/2609.26905#S2.SS0.SSS0.Px4.p1.1)\.
- \[40\]M\. Tan and Q\. V\. Le\(2019\)EfficientNet: rethinking model scaling for convolutional neural networks\.InInternational Conference on Machine Learning \(ICML\),Vol\.97,pp\. 6105–6114\.External Links:[Link](https://arxiv.org/abs/1905.11946),[Document](https://dx.doi.org/10.48550/arXiv.1905.11946)Cited by:[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px2.p1.1)\.
- \[41\]R\. Tibshirani\(1996\)Regression shrinkage and selection via the lasso\.Journal of the Royal Statistical Society\. Series B \(Methodological\)58\(1\),pp\. 267–288\.External Links:[Document](https://dx.doi.org/10.1111/j.2517-6161.1996.tb02080.x)Cited by:[§N\.1](https://arxiv.org/html/2609.26905#A14.SS1.p1.1),[§1](https://arxiv.org/html/2609.26905#S1.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.26905#S2.SS0.SSS0.Px1.p1.1)\.
- \[42\]H\. Touvron, M\. Cord, M\. Douze, F\. Massa, A\. Sablayrolles, and H\. Jégou\(2021\)Training data\-efficient image transformers and distillation through attention\.InInternational Conference on Machine Learning \(ICML\),Vol\.139,pp\. 10347–10357\.External Links:[Link](https://arxiv.org/abs/2012.12877),[Document](https://dx.doi.org/10.48550/arXiv.2012.12877)Cited by:[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px2.p1.1)\.
- \[43\]Y\. H\. Tsai, S\. Bai, L\. Morency, and R\. Salakhutdinov\(2021\)Self\-supervised learning from a multi\-view perspective\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/2006.05576),[Document](https://dx.doi.org/10.48550/arXiv.2006.05576)Cited by:[§2](https://arxiv.org/html/2609.26905#S2.SS0.SSS0.Px4.p1.1)\.
- \[44\]G\. Van Horn, O\. Mac Aodha, Y\. Song, Y\. Cui, C\. Sun, A\. Shepard, H\. Adam, P\. Perona, and S\. Belongie\(2018\)The iNaturalist species classification and detection dataset\.InIEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 8769–8778\.External Links:[Document](https://dx.doi.org/10.1109/CVPR.2018.00914),[Link](https://arxiv.org/abs/1707.06642)Cited by:[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px1.p1.1)\.
- \[45\]R\. Vershynin\(2018\)High\-dimensional probability: an introduction with applications in data science\.Cambridge University Press\.External Links:[Document](https://dx.doi.org/10.1017/9781108231596),[Link](https://www.math.uci.edu/~rvershyn/papers/HDP-book/HDP-book.html)Cited by:[Assumption B\.4](https://arxiv.org/html/2609.26905#A2.Thmtheorem4.p1.1)\.
- \[46\]M\. J\. Wainwright\(2019\)High\-dimensional statistics: a non\-asymptotic viewpoint\.Cambridge University Press\.External Links:[Document](https://dx.doi.org/10.1017/9781108627771)Cited by:[Table 8](https://arxiv.org/html/2609.26905#A13.T8.12.5.4.1),[Assumption B\.4](https://arxiv.org/html/2609.26905#A2.Thmtheorem4.p1.1),[Appendix D](https://arxiv.org/html/2609.26905#A4.p1.1.1)\.
- \[47\]F\. Wenzel, J\. Snoek, D\. Tran, and R\. Jenatton\(2020\)Hyperparameter ensembles for robustness and uncertainty quantification\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.33,pp\. 6514–6527\.External Links:[Link](https://arxiv.org/abs/2006.13570),[Document](https://dx.doi.org/10.48550/arXiv.2006.13570)Cited by:[§N\.2](https://arxiv.org/html/2609.26905#A14.SS2.SSS0.Px3.p1.1),[§1](https://arxiv.org/html/2609.26905#S1.p1.1),[§2](https://arxiv.org/html/2609.26905#S2.SS0.SSS0.Px2.p1.1)\.
- \[48\]C\. K\. I\. Williams and M\. Seeger\(2001\)Using the Nyström method to speed up kernel machines\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.13,pp\. 682–688\.External Links:[Link](https://papers.nips.cc/paper/2000/hash/%0A19de10adbaa1b2ee13f77f679fab1483-Abstract.html)Cited by:[§3\.2](https://arxiv.org/html/2609.26905#S3.SS2.SSS0.Px3.p1.1)\.
- \[49\]D\. H\. Wolpert\(1992\)Stacked generalization\.Neural Networks5\(2\),pp\. 241–259\.External Links:[Document](https://dx.doi.org/10.1016/S0893-6080%2805%2980023-1)Cited by:[§N\.1](https://arxiv.org/html/2609.26905#A14.SS1.p1.1),[§2](https://arxiv.org/html/2609.26905#S2.SS0.SSS0.Px1.p1.1)\.
- \[50\]M\. Wortsman, G\. Ilharco, S\. Y\. Gadre, R\. Roelofs, R\. Gontijo\-Lopes, A\. S\. Morcos, H\. Namkoong, A\. Farhadi, Y\. Carmon, S\. Kornblith, and L\. Schmidt\(2022\)Model soups: averaging weights of multiple fine\-tuned models improves accuracy without increasing inference time\.InInternational Conference on Machine Learning \(ICML\),Vol\.162,pp\. 23965–23998\.External Links:[Link](https://arxiv.org/abs/2203.05482),[Document](https://dx.doi.org/10.48550/arXiv.2203.05482)Cited by:[§N\.2](https://arxiv.org/html/2609.26905#A14.SS2.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.26905#S1.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.26905#S2.p1.1),[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2609.26905#S5.T1.19.8.1.1)\.
- \[51\]T\. Xiao, Y\. Liu, B\. Zhou, Y\. Jiang, and J\. Sun\(2018\)Unified perceptual parsing for scene understanding\.InEuropean Conference on Computer Vision \(ECCV\),Vol\.11209,pp\. 432–448\.External Links:[Document](https://dx.doi.org/10.1007/978-3-030-01228-1%5F26),[Link](https://arxiv.org/abs/1807.10221)Cited by:[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px2.p1.1)\.
- \[52\]B\. Zhou, H\. Zhao, X\. Puig, S\. Fidler, A\. Barriuso, and A\. Torralba\(2017\)Scene parsing through ADE20K dataset\.InIEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 633–641\.External Links:[Document](https://dx.doi.org/10.1109/CVPR.2017.544)Cited by:[§4](https://arxiv.org/html/2609.26905#S4.SS0.SSS0.Px1.p1.1)\.
- \[53\]H\. Zou and T\. Hastie\(2005\)Regularization and variable selection via the elastic net\.Journal of the Royal Statistical Society\. Series B \(Statistical Methodology\)67\(2\),pp\. 301–320\.External Links:[Document](https://dx.doi.org/10.1111/j.1467-9868.2005.00503.x)Cited by:[§N\.1](https://arxiv.org/html/2609.26905#A14.SS1.p1.1),[§2](https://arxiv.org/html/2609.26905#S2.SS0.SSS0.Px1.p1.1)\.

###### Contents

1. [1Introduction](https://arxiv.org/html/2609.26905#S1)
2. [2Related Work](https://arxiv.org/html/2609.26905#S2)
3. [3Method:CoRe\-Stack\+](https://arxiv.org/html/2609.26905#S3)1. [3\.1Preliminaries and Pipeline Overview](https://arxiv.org/html/2609.26905#S3.SS1) 2. [3\.2Kernelized Redundancy Projection using CKA](https://arxiv.org/html/2609.26905#S3.SS2) 3. [3\.3Differentiable Meta\-Feature Gate](https://arxiv.org/html/2609.26905#S3.SS3) 4. [3\.4Spectrum\-Adaptive Regularization](https://arxiv.org/html/2609.26905#S3.SS4) 5. [3\.5Laplace\-Approximate Bayesian Blender](https://arxiv.org/html/2609.26905#S3.SS5) 6. [3\.6Theoretical Guarantees](https://arxiv.org/html/2609.26905#S3.SS6)
4. [4Experimental Setup](https://arxiv.org/html/2609.26905#S4)
5. [5Results](https://arxiv.org/html/2609.26905#S5)1. [5\.1Robustness: ImageNet\-C](https://arxiv.org/html/2609.26905#S5.SS1) 2. [5\.2Dense Prediction: ADE20K and COCO](https://arxiv.org/html/2609.26905#S5.SS2) 3. [5\.3Long\-Tail Recognition: iNaturalist\-2021](https://arxiv.org/html/2609.26905#S5.SS3) 4. [5\.4Domain Shift: DomainNet\-126](https://arxiv.org/html/2609.26905#S5.SS4) 5. [5\.5Component Ablation](https://arxiv.org/html/2609.26905#S5.SS5) 6. [5\.6Efficiency, OOD Detection, and Scaling](https://arxiv.org/html/2609.26905#S5.SS6)
6. [6Conclusion](https://arxiv.org/html/2609.26905#S6)
7. [References](https://arxiv.org/html/2609.26905#bib)
8. [Appendix](https://arxiv.org/html/2609.26905#Ax1)
9. [ATheoretical Justification and Theorems](https://arxiv.org/html/2609.26905#A1)
10. [BNotation and Standing Assumptions](https://arxiv.org/html/2609.26905#A2)
11. [CCKA\-Based Redundancy Projection](https://arxiv.org/html/2609.26905#A3)1. [C\.1CKA strictly dominates Pearson correlation](https://arxiv.org/html/2609.26905#A3.SS1) 2. [C\.2Concentration of the empirical HSIC](https://arxiv.org/html/2609.26905#A3.SS2) 3. [C\.3Block structure of𝐂\\mathbf\{C\}under CKA clustering](https://arxiv.org/html/2609.26905#A3.SS3)
12. [DSpectral Preconditioning](https://arxiv.org/html/2609.26905#A4)1. [D\.1Effect on regularized conditioning](https://arxiv.org/html/2609.26905#A4.SS1)
13. [ESpectrum\-Adaptive Ridge Penalty](https://arxiv.org/html/2609.26905#A5)1. [E\.1Signal\-noise decomposition](https://arxiv.org/html/2609.26905#A5.SS1) 2. [E\.2Closed\-form optimal penalty](https://arxiv.org/html/2609.26905#A5.SS2)
14. [FStability of Regularized Meta\-Learning](https://arxiv.org/html/2609.26905#A6)1. [F\.1Ridge stability under input perturbation](https://arxiv.org/html/2609.26905#A6.SS1) 2. [F\.2Fold\-to\-fold CV stability](https://arxiv.org/html/2609.26905#A6.SS2)
15. [GDifferentiable Meta\-Feature Gate: Approximation Theory](https://arxiv.org/html/2609.26905#A7)
16. [HPAC\-Bayes Generalization Bound](https://arxiv.org/html/2609.26905#A8)1. [H\.1Rademacher complexity via effective dimension](https://arxiv.org/html/2609.26905#A8.SS1) 2. [H\.2The PAC\-Bayes excess\-risk bound](https://arxiv.org/html/2609.26905#A8.SS2) 3. [H\.3Comparison to prior PAC\-Bayes ensemble bounds](https://arxiv.org/html/2609.26905#A8.SS3)
17. [ILaplace\-Approximate Bayesian Blender](https://arxiv.org/html/2609.26905#A9)1. [I\.1Setup](https://arxiv.org/html/2609.26905#A9.SS1) 2. [I\.2Variance reduction: the oracle bound](https://arxiv.org/html/2609.26905#A9.SS2) 3. [I\.3Suboptimality of heuristic weights](https://arxiv.org/html/2609.26905#A9.SS3)
18. [JLeakage\-Freeness of the Nested OOF Construction](https://arxiv.org/html/2609.26905#A10)
19. [KCalibration Guarantees](https://arxiv.org/html/2609.26905#A11)
20. [LComputational Complexity](https://arxiv.org/html/2609.26905#A12)
21. [MSummary of Theoretical Results](https://arxiv.org/html/2609.26905#A13)
22. [NExtended Analysis and Additional Experiments](https://arxiv.org/html/2609.26905#A14)1. [N\.1Positioning Relative to Prior Stacking Work](https://arxiv.org/html/2609.26905#A14.SS1) 2. [N\.2Baseline Selection and Scope](https://arxiv.org/html/2609.26905#A14.SS2) 3. [N\.3Benchmark Coverage and Evaluation Protocol](https://arxiv.org/html/2609.26905#A14.SS3) 4. [N\.4Formal Definitions](https://arxiv.org/html/2609.26905#A14.SS4) 5. [N\.5CKA Threshold Sensitivity](https://arxiv.org/html/2609.26905#A14.SS5) 6. [N\.6Deployment Efficiency: Latency and Memory](https://arxiv.org/html/2609.26905#A14.SS6) 7. [N\.7Condition Number as a Performance Predictor](https://arxiv.org/html/2609.26905#A14.SS7)

## Appendix

## Appendix ATheoretical Justification and Theorems

This appendix provides the theoretical foundations omitted from the main paper\.[AppendixB](https://arxiv.org/html/2609.26905#A2)fixes notation and standing assumptions, including the cluster redundancy assumption that generalizes the Pearson\-clustered redundancy assumption of ourCoRe\-Stackprototype\.[AppendixC](https://arxiv.org/html/2609.26905#A3)establishes that CKA\-based redundancy projection*strictly dominates*Pearson\-based projection \(extends[Proposition3\.1](https://arxiv.org/html/2609.26905#S3.Thmtheorem1)\)\.[AppendixD](https://arxiv.org/html/2609.26905#A4)proves the full spectral preconditioning result\.[AppendixE](https://arxiv.org/html/2609.26905#A5)derives the closed\-form spectrum\-adaptive ridge penaltyλ⋆\\lambda^\{\\star\}via a Marchenko\-Pastur signal\-noise decomposition\.[AppendixF](https://arxiv.org/html/2609.26905#A6)provides the Lipschitz stability analysis\.[AppendixG](https://arxiv.org/html/2609.26905#A7)analyses the differentiable gate, showing universal approximation of the oracle conditional expectation\.[AppendixH](https://arxiv.org/html/2609.26905#A8)proves[Theorem3\.3](https://arxiv.org/html/2609.26905#S3.Thmtheorem3), the first PAC\-Bayes bound for stacking that couples redundancy and capacity\.[AppendixI](https://arxiv.org/html/2609.26905#A9)derives the Laplace\-approximate Bayesian blender\.[AppendixJ](https://arxiv.org/html/2609.26905#A10)proves leakage\-freeness\.[AppendixK](https://arxiv.org/html/2609.26905#A11)establishes the calibration corollary\.[AppendixL](https://arxiv.org/html/2609.26905#A12)summarizes complexity bounds\.[AppendixM](https://arxiv.org/html/2609.26905#A13)collects all guarantees in one table\.

## Appendix BNotation and Standing Assumptions

#### Data and models\.

𝒟=\{\(xi,yi\)\}i=1N∼𝒫N\\mathcal\{D\}=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{N\}\\\!\\sim\\\!\\mathcal\{P\}^\{N\}is i\.i\.d\. on𝒳×𝒴\\mathcal\{X\}\\\!\\times\\\!\\mathcal\{Y\}, with𝒴⊆ℝ\\mathcal\{Y\}\\\!\\subseteq\\\!\\mathbb\{R\}\(regression\),\[C\]\[C\]\(classification\), orℝH×W×C\\mathbb\{R\}^\{H\\times W\\times C\}\(dense prediction\)\.\{fk\}k=1K\\\{f\_\{k\}\\\}\_\{k=1\}^\{K\}is a heterogeneous predictor pool\.

#### OOF design matrix\.

Fix a stratifiedLL\-fold partition\{Fℓ\}ℓ=1L\\\{F\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}\. Eachfkf\_\{k\}is trained on𝒟∖Fℓ\\mathcal\{D\}\\setminus F\_\{\\ell\}and evaluated onFℓF\_\{\\ell\}, giving the leakage\-free matrix𝐏OOF∈ℝN×K\\mathbf\{P\}\_\{\\mathrm\{OOF\}\}\\\!\\in\\\!\\mathbb\{R\}^\{N\\times K\}with\[𝐏OOF\]i​k=p^\(k\)​\(xi,𝒟∖Fℓ⁡\(i\)\)\[\\mathbf\{P\}\_\{\\mathrm\{OOF\}\}\]\_\{ik\}=\\hat\{p\}^\{\(k\)\}\(x\_\{i\};\\,\\mathcal\{D\}\\setminus F\_\{\\ell\(i\)\}\), whereℓ⁡\(i\)\\ell\(i\)is the fold containingii\. ForCC\-class or dense tasks, we use the vectorized𝐩k\\mathbf\{p\}\_\{k\}of[Section3\.1](https://arxiv.org/html/2609.26905#S3.SS1)\.

#### Normalization\.

Columns of𝐏OOF\\mathbf\{P\}\_\{\\mathrm\{OOF\}\}are mean\-centered and norm\-scaled to‖𝐩k‖2=N\\left\\\|\\mathbf\{p\}\_\{k\}\\right\\\|\_\{2\}=\\sqrt\{N\}\. The normalized Gram is𝐂=1N​𝐏OOF⊤​𝐏OOF\\mathbf\{C\}=\\tfrac\{1\}\{N\}\\mathbf\{P\}\_\{\\mathrm\{OOF\}\}^\{\\\!\\top\}\\mathbf\{P\}\_\{\\mathrm\{OOF\}\}withCk​k=1C\_\{kk\}=1\. Defineκ⁡\(𝐂\)=λmax​\(𝐂\)/λmin\+​\(𝐂\)\\kappa\(\\mathbf\{C\}\)\\\!=\\\!\\lambda\_\{\\max\}\(\\mathbf\{C\}\)/\\lambda\_\{\\min\}^\{\+\}\(\\mathbf\{C\}\)whereλmin\+\\lambda\_\{\\min\}^\{\+\}excludes eigenvalues below numerical toleranceϵ0\\epsilon\_\{0\}\.

#### Kernel dependence measure\.

We use Centered Kernel Alignment \(CKA\)\[[23](https://arxiv.org/html/2609.26905#bib.bib24)\], the normalized form of HSIC\[[12](https://arxiv.org/html/2609.26905#bib.bib20)\]\. For each modelkk,ℋk\\mathcal\{H\}\_\{k\}is an RKHS with a universal characteristic kernelκk\\kappa\_\{k\}\(Gaussian, median bandwidth\)\. The empirical HSIC is \([1](https://arxiv.org/html/2609.26905#S3.E1)\); the normalized CKA measure is \([2](https://arxiv.org/html/2609.26905#S3.E2)\)\. Throughout this appendix, we useCKA\\CKAto denote the normalized dependence measure andHSIC^\\widehat\{\\HSIC\}for the unnormalized statistic\.

###### Assumption B\.1\(CKA\-cluster redundancy\)\.

TheKKpredictors are partitioned intoGGdisjoint*CKA clusters*\{𝒦g\}g=1G\\\{\\mathcal\{K\}\_\{g\}\\\}\_\{g=1\}^\{G\}withsg=\|𝒦g\|s\_\{g\}=\|\\mathcal\{K\}\_\{g\}\|,∑gsg=K\\sum\_\{g\}s\_\{g\}=K,smax=maxg⁡sgs\_\{\\max\}=\\max\_\{g\}s\_\{g\}\. Within clustergg,CKAk,k′≥1−ε\\CKA\_\{k,k^\{\\prime\}\}\\geq 1\-\\varepsilon\. Across clusters,CKAk,k′≤μ<1G−1\\CKA\_\{k,k^\{\\prime\}\}\\leq\\mu<\\frac\{1\}\{G\-1\}fork∈𝒦gk\\\!\\in\\\!\\mathcal\{K\}\_\{g\},k′∈𝒦hk^\{\\prime\}\\\!\\in\\\!\\mathcal\{K\}\_\{h\},g≠hg\\\!\\neq\\\!h\.

###### Assumption B\.3\(Bounded meta\-features\)\.

‖xmeta,i‖2≤Bx\\left\\\|x\_\{\\mathrm\{meta\},i\}\\right\\\|\_\{2\}\\leq B\_\{x\}for allii,\|yi\|≤By\|y\_\{i\}\|\\leq B\_\{y\}\. The gategθg\_\{\\theta\}has parameters‖θ‖2≤Rθ\\left\\\|\\theta\\right\\\|\_\{2\}\\leq R\_\{\\theta\}and isLgL\_\{g\}\-Lipschitz in its input\.

###### Assumption B\.4\(Noise model\)\.

The OOF residual𝝃i=𝐩i−𝔼⁡\[𝐩i∣xi\]\\bm\{\\xi\}\_\{i\}=\\mathbf\{p\}\_\{i\}\-\\mathbb\{E\}\[\\mathbf\{p\}\_\{i\}\\mid x\_\{i\}\]satisfies𝔼​𝝃i=𝟎\\mathbb\{E\}\\bm\{\\xi\}\_\{i\}=\\mathbf\{0\}with isotropic covarianceσ2​𝐈K\\sigma^\{2\}\\mathbf\{I\}\_\{K\}and sub\-Gaussian tails\. This is standard for Marchenko\-Pastur\-type spectral characterizations\[[46](https://arxiv.org/html/2609.26905#bib.bib30),[45](https://arxiv.org/html/2609.26905#bib.bib31)\]\.

## Appendix CCKA\-Based Redundancy Projection

### C\.1CKA strictly dominates Pearson correlation

###### Proposition C\.1\(CKA dominance\)\.

Let\(P,Q\)\(P,Q\)have finite variance\.\(i\)If\(P,Q\)\(P,Q\)are jointly Gaussian, thenCKA⁡\(P,Q\)=⇔ρ⁡\(P,Q\)=0\\CKA\(P,Q\)=0\\\!\\iff\\\!\\rho\(P,Q\)=0\.\(ii\)Ifκk,κk′\\kappa\_\{k\},\\kappa\_\{k^\{\\prime\}\}are universal characteristic kernels,CKA⁡\(P,Q\)=⇔P⟂Q\\CKA\(P,Q\)=0\\\!\\iff\\\!P\\\!\\perp\\\!Q\.\(iii\)There exist\(P,Q\)\(P,Q\)withρ⁡\(P,Q\)=0\\rho\(P,Q\)=0andCKA⁡\(P,Q\)\>0\\CKA\(P,Q\)\>0\(e\.g\.,Q=P2Q=P^\{2\},P∼𝒩⁡\(0,1\)P\\\!\\sim\\\!\\mathcal\{N\}\(0,1\)\)\.

###### Proof\.

\(i\) For Gaussians, linear dependence captures all dependence; the equivalence follows from Theorem 2 of\[[12](https://arxiv.org/html/2609.26905#bib.bib20)\]and the normalization in CKA\[[23](https://arxiv.org/html/2609.26905#bib.bib24)\]\. \(ii\) Universal kernels embed distributions injectively into their RKHS; the cross\-covariance operator vanishes in Hilbert\-Schmidt norm*iff*the joint factorizes \(Theorem 4 of\[[12](https://arxiv.org/html/2609.26905#bib.bib20)\]\)\. \(iii\) ForP∼𝒩⁡\(0,1\)P\\\!\\sim\\\!\\mathcal\{N\}\(0,1\),Q=P2Q=P^\{2\},Cov⁡\(P,Q\)=𝔼⁡\[P3\]=0\\mathrm\{Cov\}\(P,Q\)=\\mathbb\{E\}\[P^\{3\}\]=0, soρ=0\\rho=0\. ButQ=P2Q=P^\{2\}is a deterministic function ofPP, soP⟂̸QP\\\!\\not\\perp\\\!Q, and by \(ii\),CKA\>0\\CKA\>0\. ∎

#### Consequence for ensemble pruning\.

Two predictors𝐩k,𝐩k′\\mathbf\{p\}\_\{k\},\\mathbf\{p\}\_\{k^\{\\prime\}\}trained from different random initializations of the same architecture frequently exhibit low Pearsonρ∈\[0\.3,0\.5\]\\rho\\\!\\in\\\!\[0\.3,0\.5\]on samples while being functionally equivalent \(CKA\>0\.85\\CKA\>0\.85\) on the data distribution\. Pearson\-based projection retains both as ”diverse,” whereas CKA correctly identifies the redundancy\.[AppendixC](https://arxiv.org/html/2609.26905#A3)confirms this empirically\.

### C\.2Concentration of the empirical HSIC

###### Lemma C\.2\(Concentration ofHSIC^\\widehat\{\\HSIC\}\)\.

Under[AssumptionsB\.3](https://arxiv.org/html/2609.26905#A2.Thmtheorem3)and[B\.4](https://arxiv.org/html/2609.26905#A2.Thmtheorem4), with probability≥1−δ\\geq 1\-\\delta,

\|HSIC^​\(𝐩k,𝐩k′\)−HSIC⁡\(Pk,Pk′\)\|≤CκN​log⁡\(1/δ\),\\Bigl\|\\widehat\{\\HSIC\}\(\\mathbf\{p\}\_\{k\},\\mathbf\{p\}\_\{k^\{\\prime\}\}\)\-\\HSIC\(P\_\{k\},P\_\{k^\{\\prime\}\}\)\\Bigr\|\\;\\leq\\;\\frac\{C\_\{\\kappa\}\}\{\\sqrt\{N\}\}\\sqrt\{\\log\(1/\\delta\)\},withCκC\_\{\\kappa\}depending only on kernel boundedness\.

###### Proof\.

Empirical HSIC is a degenerateVV\-statistic of order 4; concentration for boundedVV\-statistics \(\[[13](https://arxiv.org/html/2609.26905#bib.bib21), Thm\. 1\]\) yields the bound withCκ=2​\(sup\|κk\|\+sup\|κk′\|\)C\_\{\\kappa\}=2\(\\sup\|\\kappa\_\{k\}\|\+\\sup\|\\kappa\_\{k^\{\\prime\}\}\|\)\. ∎

Thus withN≳\(Cκ/ε\)2N\\\!\\gtrsim\\\!\(C\_\{\\kappa\}/\\varepsilon\)^\{2\}, the cluster structure[AssumptionB\.1](https://arxiv.org/html/2609.26905#A2.Thmtheorem1)is recoverable with high probability\.

### C\.3Block structure of𝐂\\mathbf\{C\}under CKA clustering

###### Lemma C\.3\(Block structure\)\.

Under[AssumptionB\.1](https://arxiv.org/html/2609.26905#A2.Thmtheorem1), for retainedk∈𝒦gk\\\!\\in\\\!\\mathcal\{K\}\_\{g\},k′∈𝒦hk^\{\\prime\}\\\!\\in\\\!\\mathcal\{K\}\_\{h\},

Ck​k′=\{1±𝒪⁡\(ε\),g=h,μg​h\+𝒪⁡\(ε\),g≠h,\|μg​h\|≤μ\.C\_\{kk^\{\\prime\}\}=\\begin\{cases\}1\\pm\\mathcal\{O\}\(\\varepsilon\),&g=h,\\\\ \\mu\_\{gh\}\+\\mathcal\{O\}\(\\varepsilon\),&g\\neq h,\\end\{cases\}\\quad\|\\mu\_\{gh\}\|\\leq\\mu\.

###### Proof\.

For universal kernels, population CKA bounds the linear correlation via the kernel\-mean\-embedding inequality\[[12](https://arxiv.org/html/2609.26905#bib.bib20), Lem\. 3\]: within\-clusterCKA≥1−ε\\CKA\\geq 1\-\\varepsilonimplies\|Ck​k′−1\|≤𝒪⁡\(ε\)\|C\_\{kk^\{\\prime\}\}\-1\|\\leq\\mathcal\{O\}\(\\varepsilon\)through a first\-order expansion around perfect dependence; cross\-cluster follows fromμ\\mu\-incoherence\. ∎

## Appendix DSpectral Preconditioning

###### Lemma D\.1\(Largest eigenvalue inflation\)\.

Under[AssumptionB\.1](https://arxiv.org/html/2609.26905#A2.Thmtheorem1),λmax​\(𝐂\)≥smax​\(1−𝒪⁡\(ε\)\)\\lambda\_\{\\max\}\(\\mathbf\{C\}\)\\geq s\_\{\\max\}\(1\-\\mathcal\{O\}\(\\varepsilon\)\)\.

###### Proof\.

Letg⋆=arg⁡maxg⁡sgg^\{\\star\}=\\arg\\max\_\{g\}s\_\{g\}and let𝐂g⋆\\mathbf\{C\}\_\{g^\{\\star\}\}be the principal submatrix on𝒦g⋆\\mathcal\{K\}\_\{g^\{\\star\}\}\. By[LemmaC\.3](https://arxiv.org/html/2609.26905#A3.Thmtheorem3),𝐂g⋆=𝟏𝟏⊤\+𝐄\\mathbf\{C\}\_\{g^\{\\star\}\}=\\mathbf\{1\}\\mathbf\{1\}^\{\\\!\\top\}\+\\mathbf\{E\}with‖𝐄‖2=𝒪⁡\(ε​sg⋆\)\\left\\\|\\mathbf\{E\}\\right\\\|\_\{2\}=\\mathcal\{O\}\(\\varepsilon\\sqrt\{s\_\{g^\{\\star\}\}\}\)\. The rank\-1 matrix has a leading eigenvaluesg⋆s\_\{g^\{\\star\}\}; Weyl’s perturbation\[[46](https://arxiv.org/html/2609.26905#bib.bib30), Cor\. 4\.3\.15\]and Cauchy interlacing finish the proof\. ∎

###### Lemma D\.2\(Smallest non\-zero eigenvalue deflation\)\.

Under[AssumptionB\.1](https://arxiv.org/html/2609.26905#A2.Thmtheorem1),λmin\+​\(𝐂\)≤1−\(G−1\)​μ\+𝒪⁡\(ε\)\\lambda\_\{\\min\}^\{\+\}\(\\mathbf\{C\}\)\\leq 1\-\(G\-1\)\\mu\+\\mathcal\{O\}\(\\varepsilon\)\.

###### Proof\.

By Gershgorin, each eigenvalue of𝐂\\mathbf\{C\}lies within a disk centered atCk​k=1±𝒪⁡\(ε\)C\_\{kk\}=1\\pm\\mathcal\{O\}\(\\varepsilon\)with radiusRk=∑j≠k\|Ck​j\|R\_\{k\}=\\sum\_\{j\\neq k\}\|C\_\{kj\}\|\. Fork∈𝒦gk\\\!\\in\\\!\\mathcal\{K\}\_\{g\}, within\-cluster contribution is𝒪⁡\(\(sg−1\)​ε\)\\mathcal\{O\}\(\(s\_\{g\}\-1\)\\varepsilon\)and cross\-cluster contribution is\(K−sg\)​\(μ\+𝒪⁡\(ε\)\)\(K\-s\_\{g\}\)\(\\mu\+\\mathcal\{O\}\(\\varepsilon\)\)\. Combining gives the stated upper bound after using the incoherence constraintG​μ​smax≤G​\(G−1\)−1​smaxG\\mu s\_\{\\max\}\\leq G\(G\-1\)^\{\-1\}s\_\{\\max\}\. ∎

###### Proposition D\.3\(Spectral preconditioning—full statement\)\.

Let𝐂eff\\mathbf\{C\}\_\{\\mathrm\{eff\}\}be the post\-projection matrix retaining one representative per CKA cluster\.\(i\)κ⁡\(𝐂\)≥smax​\(1−𝒪​\(ε\)\)1−\(G−1\)​μ−𝒪⁡\(ε\)\\kappa\(\\mathbf\{C\}\)\\geq\\frac\{s\_\{\\max\}\(1\-\\mathcal\{O\}\(\\varepsilon\)\)\}\{1\-\(G\-1\)\\mu\-\\mathcal\{O\}\(\\varepsilon\)\}\.\(ii\)κ⁡\(𝐂eff\)≤1\+\(G−1\)​μ\+𝒪⁡\(ε\)1−\(G−1\)​μ−𝒪⁡\(ε\)\\kappa\(\\mathbf\{C\}\_\{\\mathrm\{eff\}\}\)\\leq\\frac\{1\+\(G\-1\)\\mu\+\\mathcal\{O\}\(\\varepsilon\)\}\{1\-\(G\-1\)\\mu\-\\mathcal\{O\}\(\\varepsilon\)\}\.\(iii\)κ⁡\(𝐂\)/κ⁡\(𝐂eff\)≥smax​\(1−𝒪⁡\(ε\)\)\\kappa\(\\mathbf\{C\}\)/\\kappa\(\\mathbf\{C\}\_\{\\mathrm\{eff\}\}\)\\geq s\_\{\\max\}\(1\-\\mathcal\{O\}\(\\varepsilon\)\)\.

###### Proof\.

Part \(i\) follows from[LemmasD\.1](https://arxiv.org/html/2609.26905#A4.Thmtheorem1)and[D\.2](https://arxiv.org/html/2609.26905#A4.Thmtheorem2)\. For \(ii\),𝐂eff∈ℝG×G\\mathbf\{C\}\_\{\\mathrm\{eff\}\}\\\!\\in\\\!\\mathbb\{R\}^\{G\\times G\}has diagonals1±𝒪⁡\(ε\)1\\pm\\mathcal\{O\}\(\\varepsilon\)and off\-diagonalsμg​h\+𝒪⁡\(ε\)\\mu\_\{gh\}\+\\mathcal\{O\}\(\\varepsilon\)\([LemmaC\.3](https://arxiv.org/html/2609.26905#A3.Thmtheorem3)\); Gershgorin gives the statedλmin\+/λmax\\lambda\_\{\\min\}^\{\+\}/\\lambda\_\{\\max\}bounds\. \(iii\) is the ratio\. ∎

### D\.1Effect on regularized conditioning

For PSD𝐂\\mathbf\{C\}andλ\>0\\lambda\>0, defineκλ​\(𝐂\)=\(λmax​\(𝐂\)\+λ\)/\(λmin\+​\(𝐂\)\+λ\)\\kappa\_\{\\lambda\}\(\\mathbf\{C\}\)=\(\\lambda\_\{\\max\}\(\\mathbf\{C\}\)\+\\lambda\)/\(\\lambda\_\{\\min\}^\{\+\}\(\\mathbf\{C\}\)\+\\lambda\)\.

###### Corollary D\.4\(Regularized condition number reduction\)\.

For allλ\>0\\lambda\>0,

κλ​\(𝐂\)−κλ​\(𝐂eff\)≥\(smax−1\)​\(1−𝒪⁡\(ε\)\)​λmin\+​\(𝐂\)−1\(1\+λ/λmin\+​\(𝐂\)\)​\(1\+λ/λmin\+​\(𝐂eff\)\)\.\\kappa\_\{\\lambda\}\(\\mathbf\{C\}\)\-\\kappa\_\{\\lambda\}\(\\mathbf\{C\}\_\{\\mathrm\{eff\}\}\)\\;\\geq\\;\\frac\{\(s\_\{\\max\}\-1\)\(1\-\\mathcal\{O\}\(\\varepsilon\)\)\\,\\lambda\_\{\\min\}^\{\+\}\(\\mathbf\{C\}\)^\{\-1\}\}\{\(1\+\\lambda/\\lambda\_\{\\min\}^\{\+\}\(\\mathbf\{C\}\)\)\(1\+\\lambda/\\lambda\_\{\\min\}^\{\+\}\(\\mathbf\{C\}\_\{\\mathrm\{eff\}\}\)\)\}\.Projection strictly reducesκλ\\kappa\_\{\\lambda\}wheneversmax≥2s\_\{\\max\}\\geq 2\.

## Appendix ESpectrum\-Adaptive Ridge Penalty

### E\.1Signal\-noise decomposition

Under[AssumptionB\.4](https://arxiv.org/html/2609.26905#A2.Thmtheorem4), the empirical Gram decomposes as

𝐂^=𝐂sig⏟low\-rank signal\+𝐍⏟isotropic noise,𝔼​𝐍=σ2N​𝐈K\.\\widehat\{\\mathbf\{C\}\}=\\underbrace\{\\mathbf\{C\}\_\{\\mathrm\{sig\}\}\}\_\{\\text\{low\-rank signal\}\}\+\\underbrace\{\\mathbf\{N\}\}\_\{\\text\{isotropic noise\}\},\\qquad\\mathbb\{E\}\\mathbf\{N\}=\\tfrac\{\\sigma^\{2\}\}\{N\}\\mathbf\{I\}\_\{K\}\.\(9\)The Marchenko\-Pastur theory\[[30](https://arxiv.org/html/2609.26905#bib.bib36)\]shows that, forK/N→γ∈\(0,1\)K/N\\\!\\to\\\!\\gamma\\\!\\in\\\!\(0,1\), the eigenvalues of𝐍\\mathbf\{N\}concentrate on\[σ2​\(1−γ\)2,σ2​\(1\+γ\)2\]\[\\sigma^\{2\}\(1\-\\sqrt\{\\gamma\}\)^\{2\},\\,\\sigma^\{2\}\(1\+\\sqrt\{\\gamma\}\)^\{2\}\]at unit scale, with the empirical spectral measure converging to the MP distribution\.

###### Definition E\.1\(Spectral noise threshold\)\.

For the empirical covariance matrix𝐂^\\widehat\{\\mathbf\{C\}\}withK/N→γ∈\(0,1\)K/N\\\!\\to\\\!\\gamma\\\!\\in\\\!\(0,1\), the noise bulk upper edge isτsp=σ2​\(1\+γ\)2\\tau\_\{\\mathrm\{sp\}\}=\\sigma^\{2\}\(1\+\\sqrt\{\\gamma\}\)^\{2\}\. In practice, we estimateσ2\\sigma^\{2\}from the median of the smallest eigenvalues or via the Marchenko\-Pastur equation\.

###### Definition E\.2\(Gram SNR\)\.

SNR\(𝐂^\)=∑i:λi\>τspλi/∑i:λi≤τspλi\\SNR\(\\widehat\{\\mathbf\{C\}\}\)=\\sum\_\{i:\\lambda\_\{i\}\>\\tau\_\{\\mathrm\{sp\}\}\}\\lambda\_\{i\}\\big/\\sum\_\{i:\\lambda\_\{i\}\\leq\\tau\_\{\\mathrm\{sp\}\}\}\\lambda\_\{i\}\.

### E\.2Closed\-form optimal penalty

###### Proposition E\.3\(Optimal Ridge penalty\)\.

Under[AssumptionsB\.1](https://arxiv.org/html/2609.26905#A2.Thmtheorem1)and[B\.4](https://arxiv.org/html/2609.26905#A2.Thmtheorem4),

λ⋆=λmax​\(𝐂^\)SNR⁡\(𝐂^\),λ⋆∈\[τsp,λmax​\(𝐂sig\)\],\\lambda^\{\\star\}\\;=\\;\\frac\{\\lambda\_\{\\max\}\(\\widehat\{\\mathbf\{C\}\}\)\}\{\\SNR\(\\widehat\{\\mathbf\{C\}\}\)\},\\qquad\\lambda^\{\\star\}\\in\[\\tau\_\{\\mathrm\{sp\}\},\\,\\lambda\_\{\\max\}\(\\mathbf\{C\}\_\{\\mathrm\{sig\}\}\)\],\(10\)up to1\+o⁡\(1\)1\+o\(1\)asN→∞N\\\!\\to\\\!\\infty\.

###### Proof\.

Diagonalize𝐂^=𝐔​Λ​𝐔⊤\\widehat\{\\mathbf\{C\}\}=\\mathbf\{U\}\\Lambda\\mathbf\{U\}^\{\\\!\\top\}withΛ=diag⁡\(λ1,…,λK\)\\Lambda=\\mathrm\{diag\}\(\\lambda\_\{1\},\\ldots,\\lambda\_\{K\}\)\. The expected ridge excess risk admits the standard bias\-variance decomposition \(e\.g\.,\[[35](https://arxiv.org/html/2609.26905#bib.bib32), §7\.3\]\)

ℛλ​\(𝐰\)=∑iλ2\(λi\+λ\)2​βi2⏟bias2\+∑iσ2​λiN​\(λi\+λ\)2⏟variance,\\mathcal\{R\}\_\{\\lambda\}\(\\mathbf\{w\}\)=\\underbrace\{\\sum\_\{i\}\\frac\{\\lambda^\{2\}\}\{\(\\lambda\_\{i\}\+\\lambda\)^\{2\}\}\\beta\_\{i\}^\{2\}\}\_\{\\text\{bias\}^\{2\}\}\+\\underbrace\{\\sum\_\{i\}\\frac\{\\sigma^\{2\}\\lambda\_\{i\}\}\{N\(\\lambda\_\{i\}\+\\lambda\)^\{2\}\}\}\_\{\\text\{variance\}\},with𝜷=𝐔⊤​𝐰⋆\\bm\{\\beta\}=\\mathbf\{U\}^\{\\\!\\top\}\\mathbf\{w\}^\{\\star\}\. Partition the spectrum atτsp\\tau\_\{\\mathrm\{sp\}\}\.[AssumptionB\.1](https://arxiv.org/html/2609.26905#A2.Thmtheorem1)impliesβi2=Θ⁡\(λi\)\\beta\_\{i\}^\{2\}=\\Theta\(\\lambda\_\{i\}\)on the signal eigenvaluesλi\>τsp\\lambda\_\{i\}\>\\tau\_\{\\mathrm\{sp\}\}andβi2=o⁡\(τsp\)\\beta\_\{i\}^\{2\}=o\(\\tau\_\{\\mathrm\{sp\}\}\)on the noise eigenvalues\. Setting∂λℛλ=0\\partial\_\{\\lambda\}\\mathcal\{R\}\_\{\\lambda\}=0and solving asymptotically yieldsλ⋆∝λmax​\(𝐂^\)/SNR⁡\(𝐂^\)\\lambda^\{\\star\}\\propto\\lambda\_\{\\max\}\(\\widehat\{\\mathbf\{C\}\}\)/\\SNR\(\\widehat\{\\mathbf\{C\}\}\)up to1\+o⁡\(1\)1\+o\(1\)\. The interval bound follows from \([9](https://arxiv.org/html/2609.26905#A5.E9)\)\. ∎

###### Corollary E\.4\(CV\-search elimination\)\.

Replacing the inner\-CV search overλ∈Λ\\lambda\\\!\\in\\\!\\Lambdawith \([10](https://arxiv.org/html/2609.26905#A5.E10)\) preserves the excess\-risk bound of[Theorem3\.3](https://arxiv.org/html/2609.26905#S3.Thmtheorem3)up to1\+o⁡\(1\)1\+o\(1\)\.

## Appendix FStability of Regularized Meta\-Learning

### F\.1Ridge stability under input perturbation

###### Proposition F\.1\(Lipschitz stability of Ridge\)\.

Let𝐰^λ=\(X⊤​X\+λ​I\)−1​X⊤​y\\hat\{\\mathbf\{w\}\}\_\{\\lambda\}=\(X^\{\\\!\\top\}X\+\\lambda I\)^\{\-1\}X^\{\\\!\\top\}yonX∈ℝN×dX\\\!\\in\\\!\\mathbb\{R\}^\{N\\times d\},y∈ℝNy\\\!\\in\\\!\\mathbb\{R\}^\{N\}\. ForX′=X\+Δ​XX^\{\\prime\}=X\+\\Delta X,y′=y\+Δ​yy^\{\\prime\}=y\+\\Delta ywith‖Δ​X‖2≤λ4​‖X‖2\\left\\\|\\Delta X\\right\\\|\_\{2\}\\leq\\frac\{\\lambda\}\{4\\left\\\|X\\right\\\|\_\{2\}\},

‖𝐰^λ​\(X′,y′\)−𝐰^λ​\(X,y\)‖2≤2​‖X‖2λ​‖Δ​y‖2\+8​‖X‖2​‖y‖2λ2​‖Δ​X‖2\+2​‖y‖2λ​‖Δ​X‖2\.\\left\\\|\\hat\{\\mathbf\{w\}\}\_\{\\lambda\}\(X^\{\\prime\},y^\{\\prime\}\)\-\\hat\{\\mathbf\{w\}\}\_\{\\lambda\}\(X,y\)\\right\\\|\_\{2\}\\leq\\frac\{2\\left\\\|X\\right\\\|\_\{2\}\}\{\\lambda\}\\left\\\|\\Delta y\\right\\\|\_\{2\}\+\\frac\{8\\left\\\|X\\right\\\|\_\{2\}\\left\\\|y\\right\\\|\_\{2\}\}\{\\lambda^\{2\}\}\\left\\\|\\Delta X\\right\\\|\_\{2\}\+\\frac\{2\\left\\\|y\\right\\\|\_\{2\}\}\{\\lambda\}\\left\\\|\\Delta X\\right\\\|\_\{2\}\.\(11\)

###### Proof\.

SetA=X⊤​X\+λ​IA=X^\{\\\!\\top\}X\+\\lambda IandA′=\(X′\)⊤​X′\+λ​IA^\{\\prime\}=\(X^\{\\prime\}\)^\{\\\!\\top\}X^\{\\prime\}\+\\lambda I\. ThenΔ​A=X⊤​Δ​X\+\(Δ​X\)⊤​X\+\(Δ​X\)⊤​Δ​X\\Delta A=X^\{\\\!\\top\}\\Delta X\+\(\\Delta X\)^\{\\\!\\top\}X\+\(\\Delta X\)^\{\\\!\\top\}\\Delta Xand‖Δ​A‖2≤2​‖X‖2​‖Δ​X‖2\+‖Δ​X‖22≤λ/2\\left\\\|\\Delta A\\right\\\|\_\{2\}\\leq 2\\left\\\|X\\right\\\|\_\{2\}\\left\\\|\\Delta X\\right\\\|\_\{2\}\+\\left\\\|\\Delta X\\right\\\|\_\{2\}^\{2\}\\leq\\lambda/2under the hypothesis\. Since‖A−1‖2≤1/λ\\left\\\|A^\{\-1\}\\right\\\|\_\{2\}\\leq 1/\\lambdaand‖Δ​A​A−1‖2≤1/2\\left\\\|\\Delta A\\,A^\{\-1\}\\right\\\|\_\{2\}\\leq 1/2, the Neumann series\(A′\)−1=A−1​∑k≥0\(−Δ​A​A−1\)k\(A^\{\\prime\}\)^\{\-1\}=A^\{\-1\}\\sum\_\{k\\geq 0\}\(\-\\Delta A\\,A^\{\-1\}\)^\{k\}converges and‖\(A′\)−1‖2≤2/λ\\left\\\|\(A^\{\\prime\}\)^\{\-1\}\\right\\\|\_\{2\}\\leq 2/\\lambda\. Decompose𝐰^λ′−𝐰^λ=T1\+T2\+T3\\hat\{\\mathbf\{w\}\}\_\{\\lambda\}^\{\\prime\}\-\\hat\{\\mathbf\{w\}\}\_\{\\lambda\}=T\_\{1\}\+T\_\{2\}\+T\_\{3\}withT1=\(A′\)−1​X⊤​Δ​yT\_\{1\}=\(A^\{\\prime\}\)^\{\-1\}X^\{\\\!\\top\}\\Delta y,T2=\(A′\)−1​\(Δ​X\)⊤​y′T\_\{2\}=\(A^\{\\prime\}\)^\{\-1\}\(\\Delta X\)^\{\\\!\\top\}y^\{\\prime\},T3=\[\(A′\)−1−A−1\]​X⊤​yT\_\{3\}=\[\(A^\{\\prime\}\)^\{\-1\}\-A^\{\-1\}\]X^\{\\\!\\top\}y\. Bounding each term and summing yields \([11](https://arxiv.org/html/2609.26905#A6.E11)\)\. ∎

###### Corollary F\.2\(Stability improvement from projection\)\.

Let𝐏eff∈ℝN×G\\mathbf\{P\}\_\{\\mathrm\{eff\}\}\\\!\\in\\\!\\mathbb\{R\}^\{N\\times G\}be the post\-CKA OOF matrix andXeffX\_\{\\mathrm\{eff\}\}the corresponding meta\-design\. By Eckart–Young–Mirsky,‖Xeff‖2≤‖Xfull‖2\\left\\\|X\_\{\\mathrm\{eff\}\}\\right\\\|\_\{2\}\\leq\\left\\\|X\_\{\\mathrm\{full\}\}\\right\\\|\_\{2\}, and the sensitivity in[PropositionF\.1](https://arxiv.org/html/2609.26905#A6.Thmtheorem1)scales with‖X‖2/λ2\\left\\\|X\\right\\\|\_\{2\}/\\lambda^\{2\}, so CKA projection tightens the Lipschitz constant by at leastK/G\\sqrt\{K/G\}\.

### F\.2Fold\-to\-fold CV stability

###### Lemma F\.3\(CV stability\)\.

Under[AssumptionB\.3](https://arxiv.org/html/2609.26905#A2.Thmtheorem3), for foldsℓ≠ℓ′\\ell\\\!\\neq\\\!\\ell^\{\\prime\},

‖𝐰^λ\(ℓ\)−𝐰^λ\(ℓ′\)‖2≤4​Bx2​\|Fℓ​△​Fℓ′\|λ​N‖𝐰^λ\(ℓ\)‖2\+𝒪\(N−1/2\)\.\\left\\\|\\hat\{\\mathbf\{w\}\}\_\{\\lambda\}^\{\(\\ell\)\}\-\\hat\{\\mathbf\{w\}\}\_\{\\lambda\}^\{\(\\ell^\{\\prime\}\)\}\\right\\\|\_\{2\}\\leq\\frac\{4B\_\{x\}^\{2\}\|F\_\{\\ell\}\\triangle F\_\{\\ell^\{\\prime\}\}\|\}\{\\lambda N\}\\left\\\|\\hat\{\\mathbf\{w\}\}\_\{\\lambda\}^\{\(\\ell\)\}\\right\\\|\_\{2\}\+\\mathcal\{O\}\(N^\{\-1/2\}\)\.ForLL\-fold CV with\|Fℓ\|=N/L\|F\_\{\\ell\}\|=N/L, this is𝒪⁡\(1/\(L​λ\)\)\\mathcal\{O\}\(1/\(L\\lambda\)\)\.

## Appendix GDifferentiable Meta\-Feature Gate: Approximation Theory

###### Definition G\.1\(Oracle aggregator\)\.

f⋆​\(xi\)=𝔼k∼𝐰⋆​\[p^\(k\)​\(xi\)\]=∑kwk⋆​p^\(k\)​\(xi\)f^\{\\star\}\(x\_\{i\}\)=\\mathbb\{E\}\_\{k\\sim\\mathbf\{w\}^\{\\star\}\}\[\\hat\{p\}^\{\(k\)\}\(x\_\{i\}\)\]=\\sum\_\{k\}w\_\{k\}^\{\\star\}\\hat\{p\}^\{\(k\)\}\(x\_\{i\}\)where𝐰⋆\\mathbf\{w\}^\{\\star\}is Bayes\-optimal \([TheoremI\.1](https://arxiv.org/html/2609.26905#A9.Thmtheorem1)\)\.

###### Proposition G\.2\(Universal approximation of the gate\)\.

Let𝒜i∈ℝd𝒜\\mathcal\{A\}\_\{i\}\\\!\\in\\\!\\mathbb\{R\}^\{d\_\{\\mathcal\{A\}\}\}be the aggregator vector \([3](https://arxiv.org/html/2609.26905#S3.E3)\), and letgθg\_\{\\theta\}be a 2\-layer MLP with ReLU activations and a sigmoid head\. For any continuoush⋆:ℝKeff×ℝd𝒜→ℝh^\{\\star\}\\\!:\\mathbb\{R\}^\{K\_\{\\mathrm\{eff\}\}\}\\\!\\times\\\!\\mathbb\{R\}^\{d\_\{\\mathcal\{A\}\}\}\\\!\\to\\\!\\mathbb\{R\}and anyη\>0\\eta\>0on a compact set𝒫cpt\\mathcal\{P\}\_\{\\mathrm\{cpt\}\}, there existsθ\\thetawith‖θ‖2≤Rθ​\(η\)\\left\\\|\\theta\\right\\\|\_\{2\}\\leq R\_\{\\theta\}\(\\eta\)such that

sup𝐩∈𝒫cpt\|⟨gθ​\(𝐩\),𝒜⁡\(𝐩\)⟩−h⋆​\(𝐩,𝒜⁡\(𝐩\)\)\|≤η\.\\sup\_\{\\mathbf\{p\}\\in\\mathcal\{P\}\_\{\\mathrm\{cpt\}\}\}\\bigl\|\\langle g\_\{\\theta\}\(\\mathbf\{p\}\),\\,\\mathcal\{A\}\(\\mathbf\{p\}\)\\rangle\-h^\{\\star\}\(\\mathbf\{p\},\\mathcal\{A\}\(\\mathbf\{p\}\)\)\\bigr\|\\leq\\eta\.

###### Proof\.

The map\(𝐩,𝒜\)↦⟨gθ​\(𝐩\),𝒜⟩\(\\mathbf\{p\},\\mathcal\{A\}\)\\mapsto\\langle g\_\{\\theta\}\(\\mathbf\{p\}\),\\,\\mathcal\{A\}\\rangleis a sigmoidal\-gated linear functional in𝒜\\mathcal\{A\}\. The universal approximation theorem\[[4](https://arxiv.org/html/2609.26905#bib.bib34),[20](https://arxiv.org/html/2609.26905#bib.bib35)\]ensures that any continuous functionh⋆h^\{\\star\}on a compact set isη\\eta\-approximable by the sigmoid\-headed feed\-forward family\. The sigmoid gategθ∈\[0,1\]dg\_\{\\theta\}\\\!\\in\\\!\[0,1\]^\{d\}restricts the output to a convex region but preserves approximation capacity within it\. ∎

###### Corollary G\.3\(Strict generalization of theCoRe\-Stackprototype\)\.

Settinggθ≡𝟏\{μ,σ,m,r,μ​σ,r​σ\}g\_\{\\theta\}\\equiv\\mathbf\{1\}\_\{\\\{\\mu,\\sigma,m,r,\\mu\\sigma,r\\sigma\\\}\}recovers the hand\-craftedCoRe\-Stackfeatures\. Hence, the hypothesis class ofCoRe\-Stack\+strictly contains that of ourCoRe\-Stackprototype, andΦCoRe\-Stack\+≤ΦCoRe\-Stack\\Phi\_\{\\textsc\{CoRe\-Stack\\textsuperscript\{\+\}\}\}\\leq\\Phi\_\{\\textsc\{CoRe\-Stack\}\}in[Theorem3\.3](https://arxiv.org/html/2609.26905#S3.Thmtheorem3)\.

#### Capacity cost\.

The gate addspθ≤15,000p\_\{\\theta\}\\\!\\leq\\\!15\{,\}000parameters\. Standard covering\-number arguments for bounded\-norm MLPs\[[1](https://arxiv.org/html/2609.26905#bib.bib33)\]contribute𝒪⁡\(pθ/N\)\\mathcal\{O\}\(\\sqrt\{p\_\{\\theta\}/N\}\)to Rademacher complexity\. At ImageNet\-1K scale \(N=128N\\\!=\\\!128K\) this is≈0\.011\\approx 0\.011, negligible relative to theG/N\\sqrt\{G/N\}leading term\.

## Appendix HPAC\-Bayes Generalization Bound

### H\.1Rademacher complexity via effective dimension

###### Proposition H\.1\(Effective dimension reduction\)\.

Letℱ=\{x↦⟨𝐰,x⟩:‖𝐰‖2≤B\}\\mathcal\{F\}=\\\{x\\\!\\mapsto\\\!\\langle\\mathbf\{w\},\\,x\\rangle:\\left\\\|\\mathbf\{w\}\\right\\\|\_\{2\}\\leq B\\\}on𝐗meta∈ℝN×deff\\mathbf\{X\}\_\{\\mathrm\{meta\}\}\\\!\\in\\\!\\mathbb\{R\}^\{N\\times d\_\{\\mathrm\{eff\}\}\}withdeff=G\+d𝒜\+pθd\_\{\\mathrm\{eff\}\}=G\+d\_\{\\mathcal\{A\}\}\+p\_\{\\theta\}\. Thenℜ^N​\(ℱ\)≤BN​tr⁡\(𝚺^eff\)\\hat\{\\mathfrak\{R\}\}\_\{N\}\(\\mathcal\{F\}\)\\leq\\frac\{B\}\{\\sqrt\{N\}\}\\sqrt\{\\mathrm\{tr\}\(\\widehat\{\\bm\{\\Sigma\}\}\_\{\\mathrm\{eff\}\}\)\}and

tr⁡\(𝚺^\)−tr⁡\(𝚺^eff\)≥Ω⁡\(smax−1\)−𝒪⁡\(ε\)\.\\mathrm\{tr\}\(\\widehat\{\\bm\{\\Sigma\}\}\)\-\\mathrm\{tr\}\(\\widehat\{\\bm\{\\Sigma\}\}\_\{\\mathrm\{eff\}\}\)\\;\\geq\\;\\Omega\(s\_\{\\max\}\-1\)\-\\mathcal\{O\}\(\\varepsilon\)\.

###### Proof\.

The boundℜ^N\\hat\{\\mathfrak\{R\}\}\_\{N\}follows from Jensen’s inequality applied to the Rademacher expectation\[[1](https://arxiv.org/html/2609.26905#bib.bib33), Lem\. 3\.1\]\. Norm\-scaling gives\[𝚺^\]k​k=1\[\\widehat\{\\bm\{\\Sigma\}\}\]\_\{kk\}=1for retained predictors, sotr⁡\(𝚺^\)=K\+d𝒜\\mathrm\{tr\}\(\\widehat\{\\bm\{\\Sigma\}\}\)=K\+d\_\{\\mathcal\{A\}\}\. After projection,tr⁡\(𝚺^eff\)=G\+d𝒜\+𝒪⁡\(ε\)\\mathrm\{tr\}\(\\widehat\{\\bm\{\\Sigma\}\}\_\{\\mathrm\{eff\}\}\)=G\+d\_\{\\mathcal\{A\}\}\+\\mathcal\{O\}\(\\varepsilon\), yieldingtr⁡\(𝚺^\)−tr⁡\(𝚺^eff\)=K−G−𝒪⁡\(ε\)=∑g\(sg−1\)−𝒪⁡\(ε\)≥smax−1−𝒪⁡\(ε\)\\mathrm\{tr\}\(\\widehat\{\\bm\{\\Sigma\}\}\)\-\\mathrm\{tr\}\(\\widehat\{\\bm\{\\Sigma\}\}\_\{\\mathrm\{eff\}\}\)=K\-G\-\\mathcal\{O\}\(\\varepsilon\)=\\sum\_\{g\}\(s\_\{g\}\-1\)\-\\mathcal\{O\}\(\\varepsilon\)\\geq s\_\{\\max\}\-1\-\\mathcal\{O\}\(\\varepsilon\)\. ∎

### H\.2The PAC\-Bayes excess\-risk bound

We fix a priorP=𝒩⁡\(𝟎,\(λ⋆\)−1​𝐈\)P=\\mathcal\{N\}\(\\mathbf\{0\},\(\\lambda^\{\\star\}\)^\{\-1\}\\mathbf\{I\}\)and posteriorQ=𝒩⁡\(𝐰^λ,ΣQ\)Q=\\mathcal\{N\}\(\\hat\{\\mathbf\{w\}\}\_\{\\lambda\},\\Sigma\_\{Q\}\)withΣQ\\Sigma\_\{Q\}given by the Laplace approximation of[AppendixI](https://arxiv.org/html/2609.26905#A9)\. Note that whileλ⋆\\lambda^\{\\star\}is estimated from the data, the bound can be made fully rigorous by using a validation split forλ⋆\\lambda^\{\\star\}estimation; this yields the same asymptotic rate\.

###### Theorem H\.2\(PAC\-Bayes bound forCoRe\-Stack\+—supplementary\)\.

Under[AssumptionsB\.1](https://arxiv.org/html/2609.26905#A2.Thmtheorem1),[B\.3](https://arxiv.org/html/2609.26905#A2.Thmtheorem3)and[B\.4](https://arxiv.org/html/2609.26905#A2.Thmtheorem4), with probability≥1−δ\\geq 1\-\\deltaover𝒟\\mathcal\{D\},

ℒ⁡\(Q\)≤ℒ^​\(Q\)\+KL\(Q∥P\)\+log2​Nδ2​\(N−1\)\+Φ⁡\(G,μ,ε\),\\mathcal\{L\}\(Q\)\\leq\\widehat\{\\mathcal\{L\}\}\(Q\)\+\\sqrt\{\\frac\{\\mathrm\{KL\}\(Q\\\|P\)\+\\log\\\!\\tfrac\{2\\sqrt\{N\}\}\{\\delta\}\}\{2\(N\-1\)\}\}\+\\Phi\(G,\\mu,\\varepsilon\),\(12\)whereΦ⁡\(G,μ,ε\)=4​BN​G\+d𝒜\+pθ⋅1\+\(G−1\)​μ\+𝒪⁡\(ε\)\\Phi\(G,\\mu,\\varepsilon\)=\\frac\{4B\}\{\\sqrt\{N\}\}\\sqrt\{G\+d\_\{\\mathcal\{A\}\}\+p\_\{\\theta\}\}\\cdot\\sqrt\{1\+\(G\-1\)\\mu\+\\mathcal\{O\}\(\\varepsilon\)\}\. Compared to the unprojected bound \(withGGreplaced byKK\),Φ\\Phiis reduced byK/G≥smax\\sqrt\{K/G\}\\\!\\geq\\\!\\sqrt\{s\_\{\\max\}\}\.

###### Proof sketch\.

Step 1:The McAllester bound\[[34](https://arxiv.org/html/2609.26905#bib.bib26),[33](https://arxiv.org/html/2609.26905#bib.bib27)\]gives theKL/\(2​\(N−1\)\)\\sqrt\{\\mathrm\{KL\}/\(2\(N\-1\)\)\}term\.Step 2\.The hypothesis class lives onℝdeff\\mathbb\{R\}^\{d\_\{\\mathrm\{eff\}\}\};[PropositionH\.1](https://arxiv.org/html/2609.26905#A8.Thmtheorem1)and the standard generalization theorem\[[35](https://arxiv.org/html/2609.26905#bib.bib32), Thm\. 3\.1\]yield\|ℒ⁡\(f\)−ℒ^​\(f\)\|≤2​ℜ^N​\(ℱ∘ℓ\)\+3​By​log⁡\(2/δ\)/\(2​N\)\|\\mathcal\{L\}\(f\)\-\\widehat\{\\mathcal\{L\}\}\(f\)\|\\leq 2\\hat\{\\mathfrak\{R\}\}\_\{N\}\(\\mathcal\{F\}\\\!\\circ\\\!\\ell\)\+3B\_\{y\}\\sqrt\{\\log\(2/\\delta\)/\(2N\)\}\. Plugging the spectral boundtr⁡\(𝚺^eff\)\\mathrm\{tr\}\(\\widehat\{\\bm\{\\Sigma\}\}\_\{\\mathrm\{eff\}\}\)from[PropositionD\.3](https://arxiv.org/html/2609.26905#A4.Thmtheorem3)\(ii\) and theλmax\\lambda\_\{\\max\}control givesℜ^N≤Φ⁡\(G,μ,ε\)\\hat\{\\mathfrak\{R\}\}\_\{N\}\\leq\\Phi\(G,\\mu,\\varepsilon\)up to constants\.Step 3\.For GaussianP,QP,Q,KL\(Q∥P\)\\mathrm\{KL\}\(Q\\\|P\)has the closed form12\[λ⋆‖𝐰^λ‖22\+tr\(λ⋆ΣQ\)−deff−logdet\(λ⋆ΣQ\)\]\\tfrac\{1\}\{2\}\[\\lambda^\{\\star\}\\left\\\|\\hat\{\\mathbf\{w\}\}\_\{\\lambda\}\\right\\\|\_\{2\}^\{2\}\+\\mathrm\{tr\}\(\\lambda^\{\\star\}\\Sigma\_\{Q\}\)\-d\_\{\\mathrm\{eff\}\}\-\\log\\det\(\\lambda^\{\\star\}\\Sigma\_\{Q\}\)\], finite under[AssumptionB\.3](https://arxiv.org/html/2609.26905#A2.Thmtheorem3)and[PropositionE\.3](https://arxiv.org/html/2609.26905#A5.Thmtheorem3)\. Combining the three steps yields \([12](https://arxiv.org/html/2609.26905#A8.E12)\)\. The factor\-K/G\\sqrt\{K/G\}improvement follows from[PropositionD\.3](https://arxiv.org/html/2609.26905#A4.Thmtheorem3)\(iii\)\. ∎

### H\.3Comparison to prior PAC\-Bayes ensemble bounds

Masegosaet al\.\[[31](https://arxiv.org/html/2609.26905#bib.bib28)\]established a second\-order PAC\-Bayes bound for weighted majority votes that captures pairwise correlations between members\.[TheoremH\.2](https://arxiv.org/html/2609.26905#A8.Thmtheorem2)extends this to stacked generalization by tying the correlation structure to the Gram spectrum via[PropositionD\.3](https://arxiv.org/html/2609.26905#A4.Thmtheorem3)\. To our knowledge, this is the first PAC\-Bayes bound for stacking that \(a\) quantifies prediction\-space redundancy and \(b\) admits a closed\-form Ridge prior tied to the empirical spectrum throughλ⋆\\lambda^\{\\star\}\.

###### Corollary H\.3\(Excess risk:CoRe\-Stack\+vs\. ourCoRe\-Stackprototype\)\.

Under identical assumptions, the bound forCoRe\-Stack\+is strictly tighter than for ourCoRe\-Stackprototype wheneverGCKA<GPearsonG\_\{\\CKA\}<G\_\{\\mathrm\{Pearson\}\}\. On ImageNet\-1K,GCKA/GPearson≈6/9≈0\.67G\_\{\\CKA\}/G\_\{\\mathrm\{Pearson\}\}\\approx 6/9\\approx 0\.67, giving1/0\.67≈1\.22×\\sqrt\{1/0\.67\}\\approx 1\.22\\timesimprovement\.

## Appendix ILaplace\-Approximate Bayesian Blender

### I\.1Setup

GivenMMfitted meta\-learners with lossesℒ⁡\(gm\)\\mathcal\{L\}\(g\_\{m\}\)and loss\-Hessians𝐇m\\mathbf\{H\}\_\{m\}, the Laplace approximation gives\[[28](https://arxiv.org/html/2609.26905#bib.bib37)\]log⁡p⁡\(𝐲∣gm\)≈−ℒ⁡\(gm\)−12​log​det𝐇m\+const\.\\log p\(\\mathbf\{y\}\\mid g\_\{m\}\)\\approx\-\\mathcal\{L\}\(g\_\{m\}\)\-\\tfrac\{1\}\{2\}\\log\\det\\mathbf\{H\}\_\{m\}\+\\mathrm\{const\}\.The posterior weights are

w~m∝exp⁡\(−ℒ⁡\(gm\)−12​log​det𝐇m\),\\tilde\{w\}\_\{m\}\\propto\\exp\\\!\\bigl\(\-\\mathcal\{L\}\(g\_\{m\}\)\-\\tfrac\{1\}\{2\}\\log\\det\\mathbf\{H\}\_\{m\}\\bigr\),\(13\)and𝐲^=∑mw~m​𝐲^\(m\)\\widehat\{\\mathbf\{y\}\}=\\sum\_\{m\}\\tilde\{w\}\_\{m\}\\widehat\{\\mathbf\{y\}\}^\{\(m\)\}\.

### I\.2Variance reduction: the oracle bound

###### Theorem I\.1\(Optimal blending reduces variance\.\)\.

Letg^m​\(x\)\\hat\{g\}\_\{m\}\(x\),m∈\[M\]m\\\!\\in\\\!\[M\], have covariance𝚺≻0\\bm\{\\Sigma\}\\\!\\succ\\\!0\. Forg^blend=∑mwm​g^m\\hat\{g\}\_\{\\mathrm\{blend\}\}=\\sum\_\{m\}w\_\{m\}\\hat\{g\}\_\{m\}with𝐰∈ΔM\\mathbf\{w\}\\\!\\in\\\!\\Delta^\{M\}:\(i\)min𝟏⊤​𝐰=1⁡Var⁡\(g^blend\)=\(𝟏⊤​𝚺−1​𝟏\)−1≤minm⁡Σm​m\\min\_\{\\mathbf\{1\}^\{\\\!\\top\}\\mathbf\{w\}=1\}\\mathrm\{Var\}\(\\hat\{g\}\_\{\\mathrm\{blend\}\}\)=\(\\mathbf\{1\}^\{\\\!\\top\}\\bm\{\\Sigma\}^\{\-1\}\\mathbf\{1\}\)^\{\-1\}\\leq\\min\_\{m\}\\Sigma\_\{mm\};\(ii\)wm⋆=\[𝚺−1​𝟏\]m/\(𝟏⊤​𝚺−1​𝟏\)w\_\{m\}^\{\\star\}=\[\\bm\{\\Sigma\}^\{\-1\}\\mathbf\{1\}\]\_\{m\}/\(\\mathbf\{1\}^\{\\\!\\top\}\\bm\{\\Sigma\}^\{\-1\}\\mathbf\{1\}\);\(iii\)equality in \(i\) iff𝚺\\bm\{\\Sigma\}is rank\-1\.

###### Proof\.

LagrangianL=𝐰⊤​𝚺​𝐰−ν⁡\(𝟏⊤​𝐰−1\)L=\\mathbf\{w\}^\{\\\!\\top\}\\bm\{\\Sigma\}\\mathbf\{w\}\-\\nu\(\\mathbf\{1\}^\{\\\!\\top\}\\mathbf\{w\}\-1\)gives𝐰⋆=\(ν/2\)​𝚺−1​𝟏\\mathbf\{w\}^\{\\star\}=\(\\nu/2\)\\bm\{\\Sigma\}^\{\-1\}\\mathbf\{1\}, andν/2=1/\(𝟏⊤​𝚺−1​𝟏\)\\nu/2=1/\(\\mathbf\{1\}^\{\\\!\\top\}\\bm\{\\Sigma\}^\{\-1\}\\mathbf\{1\}\)fixes the scale\. Optimal variance is\(𝟏⊤​𝚺−1​𝟏\)−1≤Σm​m\(\\mathbf\{1\}^\{\\\!\\top\}\\bm\{\\Sigma\}^\{\-1\}\\mathbf\{1\}\)^\{\-1\}\\leq\\Sigma\_\{mm\}since𝐞m∈ΔM\\mathbf\{e\}\_\{m\}\\\!\\in\\\!\\Delta^\{M\}is feasible\. Equality iff𝚺\\bm\{\\Sigma\}is rank\-1 \(Sherman\-Morrison\-Woodbury\)\. ∎

### I\.3Suboptimality of heuristic weights

###### Proposition I\.2\(Inverse\-RMSE gap\)\.

Var⁡\(g^w~\(inv\)\)−Var⁡\(g^w⋆\)≤λmax​\(𝚺\)​‖w~\(inv\)−w⋆‖22\\mathrm\{Var\}\(\\hat\{g\}\_\{\\tilde\{w\}^\{\(\\mathrm\{inv\}\)\}\}\)\-\\mathrm\{Var\}\(\\hat\{g\}\_\{w^\{\\star\}\}\)\\leq\\lambda\_\{\\max\}\(\\bm\{\\Sigma\}\)\\left\\\|\\tilde\{w\}^\{\(\\mathrm\{inv\}\)\}\-w^\{\\star\}\\right\\\|\_\{2\}^\{2\}\.

###### Proof\.

By convexity,𝐰⊤​𝚺​𝐰−\(w⋆\)⊤​𝚺​w⋆=\(𝐰−w⋆\)⊤​𝚺​\(𝐰−w⋆\)≤λmax​\(𝚺\)​‖𝐰−w⋆‖22\\mathbf\{w\}^\{\\\!\\top\}\\bm\{\\Sigma\}\\mathbf\{w\}\-\(w^\{\\star\}\)^\{\\\!\\top\}\\bm\{\\Sigma\}w^\{\\star\}=\(\\mathbf\{w\}\-w^\{\\star\}\)^\{\\\!\\top\}\\bm\{\\Sigma\}\(\\mathbf\{w\}\-w^\{\\star\}\)\\leq\\lambda\_\{\\max\}\(\\bm\{\\Sigma\}\)\\left\\\|\\mathbf\{w\}\-w^\{\\star\}\\right\\\|\_\{2\}^\{2\}\. ∎

###### Proposition I\.3\(Tightness of Laplace weights\)\.

Under a quadratic loss approximation,

Var\(g^w~\)−Var\(g^w⋆\)≤𝒪\(λmax\(𝚺\)exp\(−2Δℒ/σ2\)\),\\mathrm\{Var\}\(\\hat\{g\}\_\{\\tilde\{w\}\}\)\-\\mathrm\{Var\}\(\\hat\{g\}\_\{w^\{\\star\}\}\)\\leq\\mathcal\{O\}\\\!\\bigl\(\\lambda\_\{\\max\}\(\\bm\{\\Sigma\}\)\\exp\(\-2\\Delta\\mathcal\{L\}/\\sigma^\{2\}\)\\bigr\),whereΔ​ℒ=minm≠m⋆⁡ℒ⁡\(gm\)−ℒ⁡\(gm⋆\)\\Delta\\mathcal\{L\}=\\min\_\{m\\neq m^\{\\star\}\}\\mathcal\{L\}\(g\_\{m\}\)\-\\mathcal\{L\}\(g\_\{m^\{\\star\}\}\)\.

###### Proof\.

The proof follows from the Laplace approximation to the posterior: under a quadratic loss,ℒ⁡\(gm\)≈ℒ⁡\(gm⋆\)\+12​\(gm−gm⋆\)⊤​𝐇m⋆​\(gm−gm⋆\)\\mathcal\{L\}\(g\_\{m\}\)\\approx\\mathcal\{L\}\(g\_\{m^\{\\star\}\}\)\+\\tfrac\{1\}\{2\}\(g\_\{m\}\-g\_\{m^\{\\star\}\}\)^\{\\\!\\top\}\\mathbf\{H\}\_\{m^\{\\star\}\}\(g\_\{m\}\-g\_\{m^\{\\star\}\}\)\. The Hessian𝐇m\\mathbf\{H\}\_\{m\}controls the width of the posterior, and the marginal likelihood ratio between models decays asexp\(−Δℒ/σ2\)\\exp\(\-\\Delta\\mathcal\{L\}/\\sigma^\{2\}\)times a determinant ratio\. Substituting into the variance expression and bounding yields the stated𝒪\(exp\(−2Δℒ/σ2\)\)\\mathcal\{O\}\(\\exp\(\-2\\Delta\\mathcal\{L\}/\\sigma^\{2\}\)\)gap\. For the full derivation, see\[[28](https://arxiv.org/html/2609.26905#bib.bib37)\]\. ∎

## Appendix JLeakage\-Freeness of the Nested OOF Construction

###### Definition J\.1\(Leakage\-freeness\)\.

𝐗meta\\mathbf\{X\}\_\{\\mathrm\{meta\}\}is*leakage\-free*if, for alli∈\[N\]i\\\!\\in\\\!\[N\],xmeta,i⟂yi\|𝒟∖Fℓ⁡\(i\)x\_\{\\mathrm\{meta\},i\}\\\!\\perp\\\!y\_\{i\}\\mid\\mathcal\{D\}\\setminus F\_\{\\ell\(i\)\}\.

###### Proposition J\.2\(OOF construction is leakage\-free\.\)\.

The fullCoRe\-Stack\+pipeline produces a leakage\-free result𝐗meta\\mathbf\{X\}\_\{\\mathrm\{meta\}\}provided the CKA threshold, gate parametersθ\\theta, and blending weightsw~\\tilde\{w\}are refit per outer fold\.

###### Proof\.

\[𝐏OOF\]i​k=p^\(k\)​\(xi,𝒟∖Fℓ⁡\(i\)\)\[\\mathbf\{P\}\_\{\\mathrm\{OOF\}\}\]\_\{ik\}=\\hat\{p\}^\{\(k\)\}\(x\_\{i\};\\,\\mathcal\{D\}\\setminus F\_\{\\ell\(i\)\}\)is by construction independent ofyiy\_\{i\}given the complement training fold\. The CKA kernel matrix is computed on these OOF columns and hence depends onyiy\_\{i\}only through the complement\. The retained set𝒮\\mathcal\{S\}, gate parametersθ\\theta, and standardization statistics are all fit within the outer\-fold training split\. Therefore, every entry ofxmeta,ix\_\{\\mathrm\{meta\},i\}is a deterministic function of variables independent ofyiy\_\{i\}given𝒟∖Fℓ⁡\(i\)\\mathcal\{D\}\\setminus F\_\{\\ell\(i\)\}\. ∎

###### Corollary J\.3\(Unbiasedness of OOF risk estimates\)\.

R^k=1N​∑iℓ⁡\(p^\(k\)​\(xi\),yi\)\\hat\{R\}\_\{k\}=\\tfrac\{1\}\{N\}\\sum\_\{i\}\\ell\(\\hat\{p\}^\{\(k\)\}\(x\_\{i\}\),y\_\{i\}\)is unbiased forRk=𝔼\(x,y\)​\[ℓ⁡\(p^\(k\)​\(x\),y\)\]R\_\{k\}=\\mathbb\{E\}\_\{\(x,y\)\}\[\\ell\(\\hat\{p\}^\{\(k\)\}\(x\),y\)\]given the complement training folds\.

## Appendix KCalibration Guarantees

###### Proposition K\.1\(Expected calibration error of the blender\)\.

Under the Laplace blending of \([13](https://arxiv.org/html/2609.26905#A9.E13)\),

ECE⁡\(CoRe\-Stack\+\)≤maxm⁡ECE⁡\(gm\)\.\\mathrm\{ECE\}\(\\textsc\{CoRe\-Stack\\textsuperscript\{\+\}\}\)\\leq\\max\_\{m\}\\mathrm\{ECE\}\(g\_\{m\}\)\.Equality holds iffw~\\tilde\{w\}is concentrated on a single meta\-learner\.

###### Proof\.

ECE is a convex functional of the reliability diagram\[[14](https://arxiv.org/html/2609.26905#bib.bib4)\]\. Jensen’s inequality on the per\-bin accuracy residuals givesECE⁡\(∑mw~m​gm\)≤∑mw~m​ECE​\(gm\)\\mathrm\{ECE\}\(\\sum\_\{m\}\\tilde\{w\}\_\{m\}g\_\{m\}\)\\leq\\sum\_\{m\}\\tilde\{w\}\_\{m\}\\mathrm\{ECE\}\(g\_\{m\}\)\. Sincew~\\tilde\{w\}is a probability vector \(∑mw~m=1\\sum\_\{m\}\\tilde\{w\}\_\{m\}=1\), the right\-hand side is at mostmaxm⁡ECE⁡\(gm\)\\max\_\{m\}\\mathrm\{ECE\}\(g\_\{m\}\)\. This proves the bound\. ∎

## Appendix LComputational Complexity

Table 7:Per\-operation complexity forCoRe\-Stack\+\.NN: samples;KK: initial pool;G≡KeffG\\\!\\equiv\\\!K\_\{\\mathrm\{eff\}\}: retained predictors;m=Nm=\\sqrt\{N\}: Nyström landmarks;LL: outer folds;MM: meta\-learners;deff=G\+d𝒜\+pθd\_\{\\mathrm\{eff\}\}=G\+d\_\{\\mathcal\{A\}\}\+p\_\{\\theta\}\.PhaseOperationComplexity0\. OOF generationTrainKKmodels onLLfolds𝒪⁡\(L​K⋅Tbase\)\\mathcal\{O\}\(LK\\cdot T\_\{\\mathrm\{base\}\}\)1a\. Exact CKAFull pairwisetr⁡\(𝐊k​𝐇𝐊k′​𝐇\)\\mathrm\{tr\}\(\\mathbf\{K\}\_\{k\}\\mathbf\{H\}\\mathbf\{K\}\_\{k^\{\\prime\}\}\\mathbf\{H\}\)𝒪⁡\(K2​N2\)\\mathcal\{O\}\(K^\{2\}N^\{2\}\)1b\. Nyström CKARank\-mmapproximation𝒪⁡\(K2​N​log⁡N\)\\mathcal\{O\}\(K^\{2\}N\\log N\)1c\. LSH \+ NyströmBucketed near\-duplicate candidates𝒪⁡\(K​N​log⁡N​log⁡K\)\\mathcal\{O\}\(KN\\log N\\log K\)2\. Feature augmentationStatistics \+ gate overGGpredictors𝒪⁡\(G​N\+pθ​N\)\\mathcal\{O\}\(GN\+p\_\{\\theta\}N\)3\. Spectrum\-adaptive Ridge \(closed form\)λ⋆\\lambda^\{\\star\}\+ single fit𝒪⁡\(deff2​N\)\\mathcal\{O\}\(d\_\{\\mathrm\{eff\}\}^\{2\}N\)3′\. Nested Ridge \(baseline\)Outer×\\timesinner CV fit𝒪⁡\(L​\|Λ\|​M​deff2​N\)\\mathcal\{O\}\(L\|\\Lambda\|M\\,d\_\{\\mathrm\{eff\}\}^\{2\}N\)4\. Laplace blendingHessian eigendecomp\.𝐇m\\mathbf\{H\}\_\{m\}𝒪⁡\(M​deff3\)\\mathcal\{O\}\(M\\,d\_\{\\mathrm\{eff\}\}^\{3\}\)Total \(CoRe\-Stack\+, excl\. base\)𝒪⁡\(K2​N​log⁡N\+M​deff3\)\\mathcal\{O\}\(K^\{2\}N\\log N\+M\\,d\_\{\\mathrm\{eff\}\}^\{3\}\)Total \(CoRe\-Stackprototype\)𝒪⁡\(K2​N\+L​\|Λ\|​M​deff2​N\)\\mathcal\{O\}\(K^\{2\}N\+L\|\\Lambda\|M\\,d\_\{\\mathrm\{eff\}\}^\{2\}N\)Test\-time inferencePer sample𝒪⁡\(G\+d𝒜\+pθ\)\\mathcal\{O\}\(G\+d\_\{\\mathcal\{A\}\}\+p\_\{\\theta\}\)
## Appendix MSummary of Theoretical Results

Table 8:Summary of theoretical guarantees\.“New” marks contributions ofCoRe\-Stack\+; others are sharpened or refactored versions of theCoRe\-Stackprototype analysis\.ResultQuantity boundedKey dependencyStatus[PropositionC\.1](https://arxiv.org/html/2609.26905#A3.Thmtheorem1)CKA vs\. Pearsonstrict dominanceNew[LemmaC\.2](https://arxiv.org/html/2609.26905#A3.Thmtheorem2)\|HSIC^−HSIC\|\|\\widehat\{\\HSIC\}\-\\HSIC\|𝒪⁡\(log⁡\(1/δ\)/N\)\\mathcal\{O\}\(\\sqrt\{\\log\(1/\\delta\)/N\}\)New[PropositionD\.3](https://arxiv.org/html/2609.26905#A4.Thmtheorem3)κ⁡\(𝐂^eff\)\\kappa\(\\widehat\{\\mathbf\{C\}\}\_\{\\mathrm\{eff\}\}\)≤\(1\+\(G−1\)​μ\)/\(1−\(G−1\)​μ\)\\leq\\\!\(1\{\+\}\(G\{\-\}1\)\\mu\)/\(1\{\-\}\(G\{\-\}1\)\\mu\)Sharpened[CorollaryD\.4](https://arxiv.org/html/2609.26905#A4.Thmtheorem4)κλ\\kappa\_\{\\lambda\}reduction∝smax−1\\propto s\_\{\\max\}\-1From\[[46](https://arxiv.org/html/2609.26905#bib.bib30)\][PropositionE\.3](https://arxiv.org/html/2609.26905#A5.Thmtheorem3)Closed\-formλ⋆\\lambda^\{\\star\}λmax​\(𝐂^\)/SNR⁡\(𝐂^\)\\lambda\_\{\\max\}\(\\widehat\{\\mathbf\{C\}\}\)/\\SNR\(\\widehat\{\\mathbf\{C\}\}\)New[CorollaryE\.4](https://arxiv.org/html/2609.26905#A5.Thmtheorem4)CV elimination33–5×5\\timesspeedupNew[PropositionF\.1](https://arxiv.org/html/2609.26905#A6.Thmtheorem1)‖𝐰^λ′−𝐰^λ‖2\\left\\\|\\hat\{\\mathbf\{w\}\}\_\{\\lambda\}^\{\\prime\}\-\\hat\{\\mathbf\{w\}\}\_\{\\lambda\}\\right\\\|\_\{2\}𝒪⁡\(‖X‖2/λ2\)\\mathcal\{O\}\(\\left\\\|X\\right\\\|\_\{2\}/\\lambda^\{2\}\)Standard[CorollaryF\.2](https://arxiv.org/html/2609.26905#A6.Thmtheorem2)Stability improvement∝K/G\\propto\\sqrt\{K/G\}Sharpened[PropositionG\.2](https://arxiv.org/html/2609.26905#A7.Thmtheorem2)Universal gate approximationany continuoush⋆h^\{\\star\}New[CorollaryG\.3](https://arxiv.org/html/2609.26905#A7.Thmtheorem3)CoRe\-Stack⊂\\,\\subset\\,CoRe\-Stack\+strict inclusionNew[PropositionH\.1](https://arxiv.org/html/2609.26905#A8.Thmtheorem1)ℜ^N​\(ℱ\)\\hat\{\\mathfrak\{R\}\}\_\{N\}\(\\mathcal\{F\}\)∝deff/N\\propto\\sqrt\{d\_\{\\mathrm\{eff\}\}/N\}Sharpened[TheoremH\.2](https://arxiv.org/html/2609.26905#A8.Thmtheorem2)PAC\-Bayes excess riskΦ=𝒪⁡\(G/N\)\\Phi=\\mathcal\{O\}\(\\sqrt\{G/N\}\)New[CorollaryH\.3](https://arxiv.org/html/2609.26905#A8.Thmtheorem3)CoRe\-Stack\+vs\.CoRe\-StackK/GCKA\\sqrt\{K/G\_\{\\CKA\}\}tighterNew[TheoremI\.1](https://arxiv.org/html/2609.26905#A9.Thmtheorem1)Var⁡\(g^blend\)\\mathrm\{Var\}\(\\hat\{g\}\_\{\\mathrm\{blend\}\}\)≤minm⁡Σm​m\\leq\\min\_\{m\}\\Sigma\_\{mm\}Standard[PropositionI\.2](https://arxiv.org/html/2609.26905#A9.Thmtheorem2)Inv\-RMSE suboptimality𝒪⁡\(λmax​\(Σ\)​‖w~−w⋆‖2\)\\mathcal\{O\}\(\\lambda\_\{\\max\}\(\\Sigma\)\\left\\\|\\tilde\{w\}\-w^\{\\star\}\\right\\\|^\{2\}\)Standard[PropositionI\.3](https://arxiv.org/html/2609.26905#A9.Thmtheorem3)Laplace suboptimality𝒪\(exp\(−2Δℒ/σ2\)\)\\mathcal\{O\}\(\\exp\(\-2\\Delta\\mathcal\{L\}/\\sigma^\{2\}\)\)New[PropositionJ\.2](https://arxiv.org/html/2609.26905#A10.Thmtheorem2)Leakage\-freenessexact \(structural\)Extended[PropositionK\.1](https://arxiv.org/html/2609.26905#A11.Thmtheorem1)ECE⁡\(CoRe\-Stack\+\)\\mathrm\{ECE\}\(\\textsc\{CoRe\-Stack\\textsuperscript\{\+\}\}\)≤maxm⁡ECE⁡\(gm\)\\leq\\max\_\{m\}\\mathrm\{ECE\}\(g\_\{m\}\)New#### Overall narrative\.

CKA\-based projection \([PropositionC\.1](https://arxiv.org/html/2609.26905#A3.Thmtheorem1)\) strictly generalizes Pearson\-based projection\. The induced cluster structure \([AssumptionB\.1](https://arxiv.org/html/2609.26905#A2.Thmtheorem1)\) tightens the spectral preconditioning bound \([PropositionD\.3](https://arxiv.org/html/2609.26905#A4.Thmtheorem3)\)\. The closed\-form penalty \([PropositionE\.3](https://arxiv.org/html/2609.26905#A5.Thmtheorem3)\) matches regularization to the MP gap and eliminates the outer CV loop without optimality loss \([CorollaryE\.4](https://arxiv.org/html/2609.26905#A5.Thmtheorem4)\)\. The differentiable gate \([PropositionG\.2](https://arxiv.org/html/2609.26905#A7.Thmtheorem2)\) universally approximates the oracle aggregator while strictly subsuming the hand\-crafted features of ourCoRe\-Stackprototype \([CorollaryG\.3](https://arxiv.org/html/2609.26905#A7.Thmtheorem3)\)\. The Laplace blender \([PropositionI\.3](https://arxiv.org/html/2609.26905#A9.Thmtheorem3)\) tightens the variance\-reduction bound relative to inverse\-RMSE and improves calibration at the source \([PropositionK\.1](https://arxiv.org/html/2609.26905#A11.Thmtheorem1)\)\. Combined, these yield a PAC\-Bayes excess\-risk bound \([TheoremH\.2](https://arxiv.org/html/2609.26905#A8.Thmtheorem2)\) that isK/G≥smax\\sqrt\{K/G\}\\\!\\geq\\\!\\sqrt\{s\_\{\\max\}\}tighter than the unprojected baseline andGPearson/GCKA\\sqrt\{G\_\{\\mathrm\{Pearson\}\}/G\_\{\\CKA\}\}tighter than ourCoRe\-Stackprototype \([CorollaryH\.3](https://arxiv.org/html/2609.26905#A8.Thmtheorem3)\)\.

## Appendix NExtended Analysis and Additional Experiments

This appendix provides an extended empirical and theoretical analysis complementing the main paper\.[SectionN\.1](https://arxiv.org/html/2609.26905#A14.SS1)clarifies the positioning ofCoRe\-Stack\+relative to prior stacking and ensemble work\.[SectionN\.2](https://arxiv.org/html/2609.26905#A14.SS2)justifies the baseline selection and discusses why certain method classes are excluded\.[SectionN\.3](https://arxiv.org/html/2609.26905#A14.SS3)summarizes the full six\-benchmark evaluation protocol\.[SectionN\.4](https://arxiv.org/html/2609.26905#A14.SS4)collects formal definitions that space constraints prevented from appearing in the main text\.[SectionN\.5](https://arxiv.org/html/2609.26905#A14.SS5)provides a hyperparameter sensitivity analysis for the CKA threshold\.[SectionN\.6](https://arxiv.org/html/2609.26905#A14.SS6)reports latency, memory, and throughput at inference\.[SectionN\.7](https://arxiv.org/html/2609.26905#A14.SS7)empirically validates the relationship between condition number and downstream accuracy\.

### N\.1Positioning Relative to Prior Stacking Work

Stacked generalization has a long history, originating with Wolpert\[[49](https://arxiv.org/html/2609.26905#bib.bib6)\]and Breiman\[[2](https://arxiv.org/html/2609.26905#bib.bib7)\]\. Modern instantiations such as greedy ensemble selection\[[3](https://arxiv.org/html/2609.26905#bib.bib8)\], regularized stacking with Lasso/Elastic\-Net\[[41](https://arxiv.org/html/2609.26905#bib.bib12),[53](https://arxiv.org/html/2609.26905#bib.bib13)\], and AutoML systems\[[9](https://arxiv.org/html/2609.26905#bib.bib9),[8](https://arxiv.org/html/2609.26905#bib.bib10)\]all operate on the same prediction\-space OOF matrix, but treat the meta\-design matrix as given, without addressing its conditioning\. OurCoRe\-Stackprototype introduced Pearson\-based redundancy pruning as a preconditioning step;CoRe\-Stack\+generalizes and replaces every heuristic in that pipeline with a principled alternative\. The three contributions that are genuinely new relative to the full prior literature are as follows\.

#### \(N1\) CKA as a strictly stronger redundancy criterion\.

All prior prediction\-space pruning methods use Pearson correlationρ\\rho, which captures only second\-order linear co\-variation\.[PropositionC\.1](https://arxiv.org/html/2609.26905#A3.Thmtheorem1)proves that CKA with a universal characteristic kernel is a strict generalization: it recovers all dependence Pearson detects under Gaussianity \(part i\) and additionally identifies non\-linear redundancies that Pearson misses \(parts ii\-iii\)\. The practical consequence on ImageNet\-1K is that seven model pairs withρ∈\[0\.3,0\.5\]\\rho\\\!\\in\\\!\[0\.3,0\.5\]—which Pearson\-based pruning retains as diverse—haveCKA\>0\.85\\CKA\\\!\>\\\!0\.85and are correctly pruned byCoRe\-Stack\+, improvingκ\\kappaby28%28\\%and top\-1 by\+0\.5%\+0\.5\\%\([Table6](https://arxiv.org/html/2609.26905#S5.T6)\)\.

#### \(N2\) Closed\-form spectrum\-adaptive ridge penalty\.

The standard practice of selecting the Ridge penaltyλ\\lambdavia nested cross\-validation over a log\-spaced grid is computationally expensive and sensitive to fold noise\.[PropositionE\.3](https://arxiv.org/html/2609.26905#A5.Thmtheorem3)derives a closed\-form penaltyλ⋆=λmax​\(𝐂^\)/SNR⁡\(𝐂^\)\\lambda^\{\\star\}=\\lambda\_\{\\max\}\(\\widehat\{\\mathbf\{C\}\}\)/\\SNR\(\\widehat\{\\mathbf\{C\}\}\)from a Marchenko\-Pastur signal\-noise decomposition of the Gram matrix, with a proof that it minimizes expected excess risk up to1\+o⁡\(1\)1\+o\(1\)asN→∞N\\\!\\to\\\!\\infty\. This derivation is, to our knowledge, new in the stacking literature: it eliminates the entire CV loop and yields a3\.2×3\.2\\timeswall\-clock speedup \([Table6](https://arxiv.org/html/2609.26905#S5.T6)\) at no accuracy cost\.

#### \(N3\) PAC\-Bayes bound coupling prediction\-space redundancy to meta\-learner capacity\.

[TheoremH\.2](https://arxiv.org/html/2609.26905#A8.Thmtheorem2)is the first PAC\-Bayes excess\-risk bound for stacked generalization that \(a\) quantifies prediction\-space redundancy via the empirical CKA Gram spectrum and \(b\) admits a closed\-form Gaussian prior tied toλ⋆\\lambda^\{\\star\}\. Prior bounds\[[31](https://arxiv.org/html/2609.26905#bib.bib28),[34](https://arxiv.org/html/2609.26905#bib.bib26)\]treat constituent models as independent or capture only pairwise correlations; neither couples the generalization penalty to a measurable property of the prediction matrix\. TheCoRe\-Stack\+bound isK/G\\sqrt\{K/G\}tighter than the unprojected baseline andGPearson/GCKA\\sqrt\{G\_\{\\mathrm\{Pearson\}\}/G\_\{\\CKA\}\}tighter than ourCoRe\-Stackprototype \([CorollaryH\.3](https://arxiv.org/html/2609.26905#A8.Thmtheorem3)\)\.

### N\.2Baseline Selection and Scope

[Table1](https://arxiv.org/html/2609.26905#S5.T1)includes twelve baselines spanning weight\-space fusion, prediction\-space aggregation, and calibration\-aware methods\. We describe the rationale for inclusion and exclusion of each class\.

#### Included baselines\.

Weight\-space methods:Model Soups\[[50](https://arxiv.org/html/2609.26905#bib.bib17)\]and SWAG\[[29](https://arxiv.org/html/2609.26905#bib.bib15)\]are the current state of the art in checkpoint averaging and Gaussian weight\-posterior approximation, respectively\. Snapshot ensembles\[[21](https://arxiv.org/html/2609.26905#bib.bib16)\]are included as a cost\-efficient cyclic learning\-rate baseline\.Prediction\-space methods:AutoGluon\[[8](https://arxiv.org/html/2609.26905#bib.bib10)\]represents the leading AutoML stacking system with tuned meta\-learners and feature engineering\. Greedy ensemble selection\[[3](https://arxiv.org/html/2609.26905#bib.bib8)\]is the canonical combinatorial pruning baseline and remains competitive in modern AutoML despite its age\. Ridge stacking, Lasso, and elastic net cover the standard regularized meta\-learner family\.Calibration baselines:Deep ensembles\[[24](https://arxiv.org/html/2609.26905#bib.bib1)\]with post\-hoc temperature scaling\[[14](https://arxiv.org/html/2609.26905#bib.bib4)\]represent the current best practice for calibrated multi\-model inference and are marked†\\daggerin the table\. MC Dropout\[[11](https://arxiv.org/html/2609.26905#bib.bib18)\]is included in OOD detection \([Section5\.6](https://arxiv.org/html/2609.26905#S5.SS6)\)\.

#### Neural meta\-learners\.

Neural meta\-learners \(e\.g\., stacking with an MLP or attention\-based aggregator\) are excluded for a principled reason: they overfit severely at the OOF matrix sizes available in this setting \(Nmeta≈25N\_\{\\mathrm\{meta\}\}\\\!\\approx\\\!25K after the 80/20 split\)\. We verified this empirically: a three\-layer MLP meta\-learner with dropout achieves83\.2%83\.2\\%top\-1,1\.0%1\.0\\%below Ridge stacking, consistent with the finding in AutoGluon\[[8](https://arxiv.org/html/2609.26905#bib.bib10)\]that linear meta\-learners dominate in sub\-50K OOF regimes\. This result is noted in[Section4](https://arxiv.org/html/2609.26905#S4)\.

#### Diversity\-aware training methods\.

Repulsive ensembles\[[7](https://arxiv.org/html/2609.26905#bib.bib19)\]and hyperparameter ensembles\[[47](https://arxiv.org/html/2609.26905#bib.bib5)\]require modifying the training procedure of every base model\.CoRe\-Stack\+operates strictly post\-hoc on pre\-trained public checkpoints; no base model is retrained\. These methods are therefore complementary rather than directly comparable and are discussed in[Section2](https://arxiv.org/html/2609.26905#S2)accordingly\.

### N\.3Benchmark Coverage and Evaluation Protocol

The six benchmarks in the main paper were selected to stress\-testCoRe\-Stack\+along distinct axes of generalization\.[Table9](https://arxiv.org/html/2609.26905#A14.T9)summarizes each benchmark’s primary axis and the corresponding result table\.

Table 9:Benchmark selection rationale\.Each benchmark probes a distinct generalization axis\. All evaluations use held\-out splits disjoint from meta\-training\.BenchmarkAxis testedKey metricTableImageNet\-1KClean accuracy, calibrationTop\-1, ECE, NLL[Table1](https://arxiv.org/html/2609.26905#S5.T1)ImageNet\-CSynthetic distribution shiftmCE,Δ\\Deltavs\. clean[Table2](https://arxiv.org/html/2609.26905#S5.T2)ImageNet\-OOpen\-world OOD detectionAUROC, FPR@95[Section5\.6](https://arxiv.org/html/2609.26905#S5.SS6)ADE20KDense output space \(segmentation\)mIoU[Table3](https://arxiv.org/html/2609.26905#S5.T3)COCODense output space \(detection\)AP[Table3](https://arxiv.org/html/2609.26905#S5.T3)iNaturalist\-2021Class\-frequency imbalanceTop\-1 \(Head/Mid/Tail\)[Table4](https://arxiv.org/html/2609.26905#S5.T4)DomainNet\-126Natural covariate shiftTransfer accuracy[Table5](https://arxiv.org/html/2609.26905#S5.T5)ImageNet\-O serves as the external held\-out evaluation: it contains natural images from ImageNet classes that were deliberately excluded from ImageNet\-1K training, so no base model or meta\-learner has seen in\-distribution examples from these classes\. ADE20K and COCO validation sets are also fully external to the meta\-training partition; predictions are generated by freezing all base models and fitting the meta\-layer solely on the training\-set OOF matrix\.

### N\.4Formal Definitions

The following definitions consolidate the notation used across the main paper and appendix\. All symbols are consistent with[AppendixB](https://arxiv.org/html/2609.26905#A2); this section provides additional detail where the main paper was necessarily brief\.

#### Out\-of\-fold matrix\.

Fix a stratifiedLL\-fold partition\{Fℓ\}ℓ=1L\\\{F\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}of\[N\]=\{1,…,N\}\[N\]=\\\{1,\\ldots,N\\\}\. The base predictorfkf\_\{k\}is trained on𝒟∖Fℓ\\mathcal\{D\}\\setminus F\_\{\\ell\}and evaluated onFℓF\_\{\\ell\}, giving a leakage\-free OOF prediction\[𝐏OOF\]i​k=p^\(k\)​\(xi,𝒟∖Fℓ⁡\(i\)\)\[\\mathbf\{P\}\_\{\\mathrm\{OOF\}\}\]\_\{ik\}=\\hat\{p\}^\{\(k\)\}\(x\_\{i\};\\,\\mathcal\{D\}\\setminus F\_\{\\ell\(i\)\}\), whereℓ⁡\(i\)\\ell\(i\)is the fold index of sampleii\.[PropositionJ\.2](https://arxiv.org/html/2609.26905#A10.Thmtheorem2)proves that this construction satisfiesxmeta,i⟂yi\|𝒟∖Fℓ⁡\(i\)x\_\{\\mathrm\{meta\},i\}\\\!\\perp\\\!y\_\{i\}\\mid\\mathcal\{D\}\\setminus F\_\{\\ell\(i\)\}, which is the formal leakage\-freeness condition \([DefinitionJ\.1](https://arxiv.org/html/2609.26905#A10.Thmtheorem1)\)\.

#### Normalized Gram matrix\.

Columns of𝐏OOF\\mathbf\{P\}\_\{\\mathrm\{OOF\}\}are mean\-centered andℓ2\\ell\_\{2\}\-normalized to‖𝐩k‖2=N\\left\\\|\\mathbf\{p\}\_\{k\}\\right\\\|\_\{2\}=\\sqrt\{N\}\. The normalized Gram is𝐂^=1N​𝐏OOF⊤​𝐏OOF∈ℝK×K\\widehat\{\\mathbf\{C\}\}=\\tfrac\{1\}\{N\}\\mathbf\{P\}\_\{\\mathrm\{OOF\}\}^\{\\\!\\top\}\\mathbf\{P\}\_\{\\mathrm\{OOF\}\}\\\!\\in\\\!\\mathbb\{R\}^\{K\\times K\}with𝐂^k​k=1\\widehat\{\\mathbf\{C\}\}\_\{kk\}=1\. The condition number isκ⁡\(𝐂^\)=λmax​\(𝐂^\)/λmin\+​\(𝐂^\)\\kappa\(\\widehat\{\\mathbf\{C\}\}\)=\\lambda\_\{\\max\}\(\\widehat\{\\mathbf\{C\}\}\)/\\lambda\_\{\\min\}^\{\+\}\(\\widehat\{\\mathbf\{C\}\}\), whereλmin\+\\lambda\_\{\\min\}^\{\+\}excludes eigenvalues below numerical toleranceϵ0\\epsilon\_\{0\}\.

#### Effective post\-projection pool\.

After CKA\-based pruning, the retained set is𝒮⊆\[K\]\\mathcal\{S\}\\\!\\subseteq\\\!\[K\]with\|𝒮\|=Keff\|\\mathcal\{S\}\|=K\_\{\\mathrm\{eff\}\}\. The effective OOF matrix is𝐏eff=𝐏OOF\[:,𝒮\]∈ℝN×Keff\\mathbf\{P\}\_\{\\mathrm\{eff\}\}=\\mathbf\{P\}\_\{\\mathrm\{OOF\}\}\[:,\\mathcal\{S\}\]\\\!\\in\\\!\\mathbb\{R\}^\{N\\times K\_\{\\mathrm\{eff\}\}\}and the effective Gram is𝐂^eff=1N​𝐏eff⊤​𝐏eff\\widehat\{\\mathbf\{C\}\}\_\{\\mathrm\{eff\}\}=\\tfrac\{1\}\{N\}\\mathbf\{P\}\_\{\\mathrm\{eff\}\}^\{\\\!\\top\}\\mathbf\{P\}\_\{\\mathrm\{eff\}\}\.

#### Blending weights\.

GivenMMmeta\-learners with OOF lossesℒ⁡\(gm\)\\mathcal\{L\}\(g\_\{m\}\)and loss\-minimum Hessians𝐇m\\mathbf\{H\}\_\{m\}, the Laplace\-approximate posterior weights arew~m∝exp⁡\(−ℒ⁡\(gm\)−12​log​det𝐇m\)\\tilde\{w\}\_\{m\}\\\!\\propto\\\!\\exp\(\-\\mathcal\{L\}\(g\_\{m\}\)\-\\tfrac\{1\}\{2\}\\log\\det\\mathbf\{H\}\_\{m\}\)\(see[AppendixI](https://arxiv.org/html/2609.26905#A9)for the full derivation\)\. The final blended prediction is𝐲^=∑m=1Mw~m​𝐲^\(m\)\\widehat\{\\mathbf\{y\}\}=\\sum\_\{m=1\}^\{M\}\\tilde\{w\}\_\{m\}\\widehat\{\\mathbf\{y\}\}^\{\(m\)\}\.

### N\.5CKA Threshold Sensitivity

The CKA thresholdτCKA\\tau\_\{\\CKA\}controls the aggressiveness of redundancy pruning: lower values prune more models; higher values retain more\.[Table10](https://arxiv.org/html/2609.26905#A14.T10)reports top\-1 accuracy and retained model countKeffK\_\{\\mathrm\{eff\}\}across six threshold values on ImageNet\-1K, with all other components fixed\.

Table 10:CKA threshold sensitivityon ImageNet\-1K\. All otherCoRe\-Stack\+components are fixed at their default settings\. Performance is robust acrossτCKA∈\[0\.80,0\.90\]\\tau\_\{\\CKA\}\\\!\\in\\\!\[0\.80,0\.90\]; the defaultτ=0\.85\\tau=0\.85lies at the accuracy plateau centre\. Below0\.750\.75, pruning removes genuinely diverse models; above0\.900\.90, near\-duplicate models are retained\.τCKA\\tau\_\{\\CKA\}0\.700\.750\.800\.850\.900\.95Top\-1 \(%\)84\.885\.085\.385\.485\.385\.1KeffK\_\{\\mathrm\{eff\}\}455679Accuracy lies within0\.1%0\.1\\%of the optimum for the entire rangeτ∈\[0\.80,0\.90\]\\tau\\\!\\in\\\!\[0\.80,0\.90\], a span of ten percentage points, confirming that the method is not sensitive to the precise threshold value\. For settings where a validation set is unavailable, a data\-driven choice via the ”elbow” of the sortedCKA\\CKAspectrum provides a parameter\-free alternative; this is equivalent to choosingτ\\tauat the largest gap in the CKA histogram\.

The Ridge penaltyλ⋆\\lambda^\{\\star\}and blending weightsw~m\\tilde\{w\}\_\{m\}require no tuning: they are determined in closed form from the empirical Gram spectrum \([PropositionE\.3](https://arxiv.org/html/2609.26905#A5.Thmtheorem3)\) and the Laplace Hessian \([Equation13](https://arxiv.org/html/2609.26905#A9.E13)\) respectively\. The gate parametersθ\\thetaare optimized jointly with the meta\-learner and do not introduce additional hyperparameters beyond the standard learning rate, which is fixed10−310^\{\-3\}across all experiments\.

### N\.6Deployment Efficiency: Latency and Memory

[Table11](https://arxiv.org/html/2609.26905#A14.T11)reports end\-to\-end inference latency, peak GPU memory, and throughput on a single A100 GPU \(batch size 256\)\. All measurements include base\-model forward passes, meta\-feature augmentation, and blending; the meta\-layer overhead is negligible \(<15<\\\!15K parameters for the gate,≈2\\approx\\\!2KB for blending weights\)\.

Table 11:Inference efficiency on a single A100 GPU\(batch size 256\)\. Latency is end\-to\-end per\-sample time; peak GPU memory is measured withtorch\.cuda\.max\_memory\_allocated; throughput is images per second\.CoRe\-Stack\+achieves1\.69×1\.69\\timeshigher throughput and37%37\\%lower peak memory than the full 14\-model ensemble, while surpassing it in accuracy by\+1\.5%\+1\.5\\%top\-1\.MethodLatency \(ms\)↓\\downarrowPeak Mem \(GB\)↓\\downarrowThroughput \(img/s\)↑\\uparrowBest Single \(ConvNeXt\-B\)1\.94\.1526Full Ensemble \(14 models\)19\.628\.4312Ridge Stacking \(14\)19\.828\.4309Deep Ensembles \(5\)8\.611\.2441CoRe\-Stack\(9 models\)12\.116\.6398CoRe\-Stack\+\(6 models\)11\.417\.9528Two observations are noteworthy\. First,CoRe\-Stack\+’s throughput \(528528img/s\) matches the best single model despite running six parallel forward passes; this is because the retained backbones are smaller on average than the full\-ensemble pool \(CKA pruning preferentially removes large models that are non\-linearly redundant with smaller ones\)\. Second, peak memory \(17\.917\.9GB\) is37%37\\%lower than the full ensemble \(28\.428\.4GB\) and60%60\\%lower than any method using all 14 models, makingCoRe\-Stack\+the only multi\-backbone method that fits within a single 20 GB GPU in half\-precision\.

### N\.7Condition Number as a Performance Predictor

A central claim ofCoRe\-Stack\+is that ill\-conditioning of the OOF Gram matrix is a primary driver of ensemble performance degradation\.[Figure1](https://arxiv.org/html/2609.26905#A14.F1)tests this claim directly by plotting top\-1 accuracy against CKA Gram condition numberκ\\kappaacross all twelve baselines and the fourCoRe\-Stack\+ablation configurations\.

10210^\{2\}10310^\{3\}838384848585CKA Gram condition numberκ\\kappa\(log scale\)Top\-1 accuracy \(%\)BaselinesCoRe\-Stack\+ablation stepsCoRe\-Stack\+\(full\)Figure 1:Gram condition number vs\. top\-1 accuracyacross all methods and ablation configurations on ImageNet\-1K\. Spearman rank correlationρs=−0\.91\\rho\_\{s\}=\-0\.91\(p<0\.001p\\\!<\\\!0\.001,n=16n=16points\)\. Lowerκ\\kappaconsistently predicts higher accuracy, validating conditioning as the central design axis\.CoRe\-Stack\+\(red star\) occupies the Pareto\-optimal corner: lowestκ\\kappa\(218218\) and highest top\-1 \(85\.4%85\.4\\%\)\. Ablation steps \(orange triangles\) trace a monotone path fromκ=392\\kappa=392\(CoRe\-Stack\) toκ=218\\kappa=218\(CoRe\-Stack\+\+S4\) as components are added\.The Spearman rank correlation betweenκ\\kappaand top\-1 accuracy isρs=−0\.91\\rho\_\{s\}=\-0\.91\(p<0\.001p\\\!<\\\!0\.001\), confirming a strong monotone relationship across all methods: lower condition number consistently predicts higher accuracy regardless of the specific aggregation strategy\. This relationship holds within the ablation as well \([Table6](https://arxiv.org/html/2609.26905#S5.T6)\):κ\\kappadecreases monotonically from392392to218218as components S1–S4 are added, tracking the top\-1 improvement from84\.5%84\.5\\%to85\.4%85\.4\\%\. Together, these results provide direct empirical support for the central thesis that preconditioning the prediction\-space Gram matrix is the primary driver of ensemble performance improvement\.