面向大语言模型可靠性检测的表征签名自适应迁移

# Adaptive Transfer of Representational Signatures for LLM Reliability Detection
Source: [https://arxiv.org/html/2610.33442](https://arxiv.org/html/2610.33442)
###### Abstract

Large language models \(LLMs\) are reported to be susceptible to reliability issues including hallucinations and reasoning errors\. Existing quality detectors can predict unreliable outputs based on internal representations with decent accuracies\. However, training these detectors often requires substantial domain\-specific labeled data, which limits their applicability to new domains\. We propose to address this challenge by transferring representational signatures across related domains\. More specifically, we build a statistical framework that first extracts low\-dimensional signatures for each domain, then uses cross\-fitting to learn composition weights of domain\-specific predictors\. The weights adapt to each source domain’s predictive relevance to the target domain\. Our asymptotic analysis of a least\-squares analogue characterizes the limiting prediction risk and aggregation weights in terms of domain discrepancies and representation bias\. Experiments across multiple hallucination datasets and open\-weight LLMs show that, with only 25 labeled questions in the new domain, adaptive transfer improves hallucination\-detection AUROC by 6\.8%\. For reasoning\-step prediction, it improves AUROC by 8\.4%\.

## 1Introduction

Large language models \(LLMs\) have demonstrated strong performance across a wide range of tasks, including language generation\([Brown et al\., 2020](https://arxiv.org/html/2610.33442#bib.bib1)\), mathematical reasoning\([Lewkowycz et al\., 2022](https://arxiv.org/html/2610.33442#bib.bib2)\), and code generation\([Chen et al\., 2021](https://arxiv.org/html/2610.33442#bib.bib20)\)\. However, strong performance on these tasks does not necessarily imply reliable behavior\. Hallucinations and misalignment can lead to incorrect decisions\([Hager et al\., 2024](https://arxiv.org/html/2610.33442#bib.bib10);[Hu et al\., 2026b](https://arxiv.org/html/2610.33442#bib.bib44)\)or unintended actions\([Zhang et al\., 2025a](https://arxiv.org/html/2610.33442#bib.bib15)\)\. In July 2026, for example, OpenAI reported that models undergoing internal cybersecurity evaluations with reduced safeguards bypassed containment controls and compromised Hugging Face’s production infrastructure\([OpenAI, 2026](https://arxiv.org/html/2610.33442#bib.bib21)\)\. These risks highlight the need to detect unreliable LLM behaviors\.

Recent studies have linked unreliable behavior in open\-weight LLMs to specific patterns in their internal representations\. For hallucination detection, for example, linear probes trained on hidden\-state activations from a single layer can distinguish correct answers from hallucinated ones with high accuracy\([O’Neill et al\., 2025](https://arxiv.org/html/2610.33442#bib.bib3);[Orgad et al\., 2025](https://arxiv.org/html/2610.33442#bib.bib39)\)\. Other approaches reduce the dimensionality of internal representations before classification\([Hu et al\., 2026a](https://arxiv.org/html/2610.33442#bib.bib37)\), or capture more complex patterns using nonlinear classifiers, such as multilayer perceptrons \(MLPs\)\([Azaria and Mitchell, 2023](https://arxiv.org/html/2610.33442#bib.bib6);[Zhou et al\., 2025](https://arxiv.org/html/2610.33442#bib.bib16)\)and vision transformers \(ViTs\)\([Bar\-Shalom et al\., 2025](https://arxiv.org/html/2610.33442#bib.bib19)\)\. Together, these methods suggest that internal representations provide informative signals for hallucination detection\.

Despite their success, many of these methods rely on labeled examples from a specific dataset or domain\([Shamsi, 2026](https://arxiv.org/html/2610.33442#bib.bib4)\)\. In practice, labeled examples can be costly to obtain in sufficient quantities for every new domain\. Existing work addresses this by reshaping internal states to make truthfulness signals more consistent across domains\([Zhang et al\., 2025b](https://arxiv.org/html/2610.33442#bib.bib41)\)or by constructing hallucination scores from uncertainty fluctuations that remain stable under domain shift\([Deng et al\., 2026](https://arxiv.org/html/2610.33442#bib.bib18)\)\. However, transfer performance can vary substantially across domains\. A detector trained on a single dataset may generalize better to some related domains while still performing poorly on others\([Alansari et al\., 2026](https://arxiv.org/html/2610.33442#bib.bib42)\)\. This raises a natural question:

Can we adaptively recompose reliability detectors trained on multiple related source domains to achieve accurate detection in a target domain with limited labeled data?

To answer this question, we develop an adaptive transfer method that learns a target\-domain detector using information from multiple source domains\. For each source domain, we train a classifier on a low\-dimensional principal component subspace that captures the dominant variation in the model’s internal representations from perturbed prompts\. We then linearly combine these source detectors with a detector trained on the available labeled target\-domain data\. Because the source detectors may differ in their predictive relevance to the target domain, we use cross\-fitting to learn composition weights that balance information transferred from the source domains with the information learned directly from the target domain\. We call the resulting method Adaptive Signature Transfer \(AdaST\)\.

To understand when source\-domain information benefits AdaST, we develop an asymptotic analysis of the adaptive transfer procedure in the regression setting\. The analysis characterizes how the learned composition weights converge to the population\-optimal counterparts, which are dependent on the discrepancy between source and target domains\. The theoretical analysis confirms that AdaST asymptotically performs better than or as well as the predictor that only relies on the target domain information\.

We use two tasks to validate the numerical performance of AdaST, including hallucination detection and mathematics reasoning correctness prediction\. AdaST achieves robust predictive performance across multiple domains on multiple open\-weight models\. Furthermore, we show that the learned detector vectors can be used as steering directions\. Our main contributions are summarized as:

1. 1\.We develop an adaptive cross\-domain transfer method, AdaST, for representational signatures of LLM behavior\. The method constructs principal\-component detectors in each domain and uses cross\-fitting to learn adaptive transfer weights\.
2. 2\.We characterize the limiting behavior of the adaptive transfer mechanism in the regression setting, which validates the effectiveness of adaptive transfer when the target domain is similar to the source domains\.
3. 3\.We demonstrate empirically that internal\-state detectors can benefit from cross\-domain transfer, with larger gains when fewer labeled examples are available in the target domain\.

### 1\.1Related Work

#### Representation\-based Reliability Detection\.

Existing representation\-based LLM reliability detection methods have explored a variety of designs, including linear probes applied directly to model activations\([O’Neill et al\., 2025](https://arxiv.org/html/2610.33442#bib.bib3);[Orgad et al\., 2025](https://arxiv.org/html/2610.33442#bib.bib39)\)and low\-dimensional subspace methods that extract compact behavioral representations\([Hu et al\., 2026a](https://arxiv.org/html/2610.33442#bib.bib37)\)\. More flexible neural classifiers have also been investigated, such as MLPs trained on hidden state activations\([Azaria and Mitchell, 2023](https://arxiv.org/html/2610.33442#bib.bib6);[Zhou et al\., 2025](https://arxiv.org/html/2610.33442#bib.bib16)\)and Vision Transformers that model the structured layer\-by\-token organization of activation tensors\([Bar\-Shalom et al\., 2025](https://arxiv.org/html/2610.33442#bib.bib19)\)\. These methods can achieve strong performance when sufficient labeled data are available, but their effectiveness can degrade under label scarcity\([Shamsi, 2026](https://arxiv.org/html/2610.33442#bib.bib4)\)\.

#### Cross\-Domain Reliability Detection\.

Recent work examines whether reliability signals learned from internal representations can directly transfer across domains\.[Orgad et al\. \(2025\)](https://arxiv.org/html/2610.33442#bib.bib39)report substantial cross\-dataset degradation, while[Alansari et al\. \(2026\)](https://arxiv.org/html/2610.33442#bib.bib42)find that transfer can succeed but varies with the training and test datasets\. Other work improves cross\-domain robustness by modifying internal representations or constructing more stable detection signals\([Zhang et al\., 2025b](https://arxiv.org/html/2610.33442#bib.bib41);[Hu et al\., 2026a](https://arxiv.org/html/2610.33442#bib.bib37);[Deng et al\., 2026](https://arxiv.org/html/2610.33442#bib.bib18)\)\. These methods train a detector on a single source domain and apply it unchanged to a new domain\. Our method instead studies how detectors learned from multiple source domains can be adaptively combined for a new domain with limited labeled data\.

#### Multi\-Source Transfer Learning\.

Multi\-source transfer learning studies how knowledge from several related source domains can be combined to improve prediction in a target domain\. Early studies formalized the multi\-source adaptation problem and developed principled ways to combine source\-specific predictors\([Mansour et al\., 2008](https://arxiv.org/html/2610.33442#bib.bib7);[Mansour et al\., 2009](https://arxiv.org/html/2610.33442#bib.bib8)\)\. Subsequent methods have explored a variety of strategies for source combination, including mixture\-of\-experts models with target\-dependent weights\([Guo et al\., 2018](https://arxiv.org/html/2610.33442#bib.bib5)\), representation alignment\([Peng et al\., 2019](https://arxiv.org/html/2610.33442#bib.bib9)\), and stacking or similarity\-based weighting\([Wu et al\., 2023](https://arxiv.org/html/2610.33442#bib.bib17)\)\. We build on these multi\-source transfer learning ideas to study adaptive transfer across representation\-based LLM reliability detectors\.

## 2Method

In this section, we present the proposed AdaST framework for cross\-domain binary behavioral prediction, as illustrated in Figure[1](https://arxiv.org/html/2610.33442#S2.F1)\. First, for each domain, we query the LLM with multiple semantically equivalent paraphrases of the same questions and extract the final\-token hidden states as representations of the generated responses\. We then construct question\-level contrast vectors by comparing hidden states associated with the two behavioral classes and use these contrasts to estimate a low\-dimensional behavioral subspace and train a domain\-specific detector\. Second, the learned detectors are adaptively combined using weights estimated from cross\-fitted predictions in the target domain\.

Figure 1:Overview of the proposed AdaST framework\.### 2\.1Paraphrase\-Based Behavioral Subspace

Let\[S\]=\{1,…,S\}\[S\]=\\\{1,\\ldots,S\\\}index the source domains andTTindex the target domain\. For each domain indexs∈\[S\]∪\{T\}s\\in\[S\]\\cup\\\{T\\\}, let𝒬s\\mathcal\{Q\}\_\{s\}denote the set of questions from the corresponding domain\. For example,𝒬1\\mathcal\{Q\}\_\{1\}may contain factual question\-answering prompts such as “What is the capital of France?”, while𝒬2\\mathcal\{Q\}\_\{2\}may contain conversational question\-answering prompts such as “Which city do you recommend visiting in France?” The semantic similarity across these domains allows us to transfer knowledge about LLM representations\.

Fix a domain indexs∈\[S\]∪\{T\}s\\in\[S\]\\cup\\\{T\\\}\. For each questionq∈𝒬sq\\in\\mathcal\{Q\}\_\{s\}, we retain the original wording ofqqand generateP−1P\-1semantically equivalent paraphrases, inspired by[Song et al\. \(2026\)](https://arxiv.org/html/2610.33442#bib.bib43)\. For example, the question “What is the capital of France?” can be paraphrased as “Name the capital of France\.” We then query the LLMRRtimes for each version of the questionqqto obtainRRresponses per version\. For each generated response, we extract the hidden state at the final generated token from a selected model layer and denote it by𝐡q​p​r∈ℝD\\mathbf\{h\}\_\{qpr\}\\in\\mathbb\{R\}^\{D\}, whereDDis the hidden\-state dimension,ppindexes the question version, andrrindexes the generated response\. Each response has a binary behavioral labelyq​p​r∈\{0,1\}y\_\{qpr\}\\in\\\{0,1\\\}, whereyq​p​r=1y\_\{qpr\}=1denotes the positive class andyq​p​r=0y\_\{qpr\}=0the negative class\. Let𝒬s±⊆𝒬s\\mathcal\{Q\}\_\{s\}^\{\\pm\}\\subseteq\\mathcal\{Q\}\_\{s\}denote the questions for which both behavioral classes are observed, and letms=\|𝒬s±\|m\_\{s\}=\|\\mathcal\{Q\}\_\{s\}^\{\\pm\}\|\.

Following prior work on contrastive representation learning\([Zou et al\., 2023](https://arxiv.org/html/2610.33442#bib.bib22);[Li et al\., 2023b](https://arxiv.org/html/2610.33442#bib.bib23);[Rimsky et al\., 2024](https://arxiv.org/html/2610.33442#bib.bib24)\), we represent the behavioral difference for each question by comparing the mean representations associated with its positive and negative responses\. Letℐq\+=\{\(p,r\):yq​p​r=1\}\\mathcal\{I\}\_\{q\}^\{\+\}=\\\{\(p,r\):y\_\{qpr\}=1\\\}andℐq−=\{\(p,r\):yq​p​r=0\}\\mathcal\{I\}\_\{q\}^\{\-\}=\\\{\(p,r\):y\_\{qpr\}=0\\\}denote the positive and negative classes for questionqq, respectively\. For eachq∈𝒬s±q\\in\\mathcal\{Q\}\_\{s\}^\{\\pm\}, we define the contrast vector:

𝜹q=1\|ℐq\+\|​∑\(p,r\)∈ℐq\+𝐡q​p​r−1\|ℐq−\|​∑\(p,r\)∈ℐq−𝐡q​p​r\.\\bm\{\\delta\}\_\{q\}=\\frac\{1\}\{\|\\mathcal\{I\}\_\{q\}^\{\+\}\|\}\\sum\_\{\(p,r\)\\in\\mathcal\{I\}\_\{q\}^\{\+\}\}\\mathbf\{h\}\_\{qpr\}\-\\frac\{1\}\{\|\\mathcal\{I\}\_\{q\}^\{\-\}\|\}\\sum\_\{\(p,r\)\\in\\mathcal\{I\}\_\{q\}^\{\-\}\}\\mathbf\{h\}\_\{qpr\}\.\(1\)
Here,𝜹q\\bm\{\\delta\}\_\{q\}quantifies the representation difference between the positive and negative behavioral classes for the same questionqq\. We stack the contrast vectors as rows of𝐂s\\mathbf\{C\}\_\{s\}, define their mean as𝜹¯s=1ms​∑q∈𝒬s±𝜹q\\bar\{\\bm\{\\delta\}\}\_\{s\}=\\frac\{1\}\{m\_\{s\}\}\\sum\_\{q\\in\\mathcal\{Q\}\_\{s\}^\{\\pm\}\}\\bm\{\\delta\}\_\{q\}, and center them across questions to obtain𝐂s−𝟏​𝜹¯s⊤\\mathbf\{C\}\_\{s\}\-\\mathbf\{1\}\\bar\{\\bm\{\\delta\}\}\_\{s\}^\{\\top\}\. Sincemsm\_\{s\}may be small relative to the hidden\-state dimensionDD, the problem becomes overparameterized\. Consequently, fitting a detector directly in the full residual space can be unstable and prone to overfitting\. We therefore perform principal component analysis on the centered contrast matrix to obtain a lower\-dimensional representation:

𝑪s−𝟏𝜹¯s⊤=𝑳s𝚲s𝑽s⊤,𝑩s=𝑽s,1:ks⊤∈ℝks×D,\{\\bm\{C\}\}\_\{s\}\-\\mathbf\{1\}\\bar\{\\bm\{\\delta\}\}\_\{s\}^\{\\top\}=\{\\bm\{L\}\}\_\{s\}\{\\bm\{\\Lambda\}\}\_\{s\}\{\\bm\{V\}\}\_\{s\}^\{\\top\},\\qquad\{\\bm\{B\}\}\_\{s\}=\{\\bm\{V\}\}\_\{s,1:k\_\{s\}\}^\{\\top\}\\in\\mathbb\{R\}^\{k\_\{s\}\\times D\},\(2\)We retainks=min⁡\{kmax,ms−1,D\}k\_\{s\}=\\min\\\{k\_\{\\max\},m\_\{s\}\-1,D\\\}principal directions, wherekmaxk\_\{\\max\}is selected using held\-out validation data, and the rows of𝑩s\{\\bm\{B\}\}\_\{s\}span the domain\-specific behavioral subspace\.

For a hidden state𝐡∈ℝD\\mathbf\{h\}\\in\\mathbb\{R\}^\{D\}, we first project it onto the behavioral subspace𝑩s\{\\bm\{B\}\}\_\{s\}\. Let𝝁s\\bm\{\\mu\}\_\{s\}and𝜸s\\bm\{\\gamma\}\_\{s\}denote the coordinate\-wise mean and standard deviation of𝑩s​𝐡\{\\bm\{B\}\}\_\{s\}\\mathbf\{h\}over the training responses in domainss\. For numerical stability, we standardize the projected representations coordinate\-wise and define

ϕs​\(𝐡\)=diag⁡\(𝜸s\)−1​\(𝑩s​𝐡−𝝁s\)\.\\bm\{\\phi\}\_\{s\}\(\\mathbf\{h\}\)=\\operatorname\{diag\}\(\\bm\{\\gamma\}\_\{s\}\)^\{\-1\}\(\{\\bm\{B\}\}\_\{s\}\\mathbf\{h\}\-\\bm\{\\mu\}\_\{s\}\)\.\(3\)
We then fit a logistic regression in this standardized subspace and denote its coefficient vector by𝜷s\\bm\{\\beta\}\_\{s\}\. The same procedure is applied independently to each source domain and the target domain\.

### 2\.2Adaptive Cross\-Domain Transfer

When labeled data in a target domain are scarce, a detector trained only on the target domain may generalize poorly to unseen examples\. Meanwhile, labeled datasets from related domains are often available\. When the underlying tasks are similar, the corresponding internal representations, here taken as residual\-stream activations, may share structure across domains\. We therefore supplement the target detector with detectors learned from the source domains\.

The domain\-specific detectors are learned in different low\-dimensional coordinate systems induced by their respective bases𝑩s\{\\bm\{B\}\}\_\{s\}\. Since these coordinate systems are not directly comparable, we map each domain\-specific coefficient vector𝜷s\\bm\{\\beta\}\_\{s\}back to the shared residual space\. Using the standardized projection in Equation[3](https://arxiv.org/html/2610.33442#S2.E3), the coefficient on𝐡\\mathbf\{h\}is

𝐯s=𝑩s⊤​diag⁡\(𝜸s\)−1​𝜷s\.\\mathbf\{v\}\_\{s\}=\{\\bm\{B\}\}\_\{s\}^\{\\top\}\\operatorname\{diag\}\(\\bm\{\\gamma\}\_\{s\}\)^\{\-1\}\\bm\{\\beta\}\_\{s\}\.\(4\)Since the detector scores are standardized over the labeled target responses before combination, the offset has no effect on transfer\. Only the direction of𝐯s\\mathbf\{v\}\_\{s\}matters after this standardization, so we normalize it as𝐯s←𝐯s/‖𝐯s‖2\\mathbf\{v\}\_\{s\}\\leftarrow\\mathbf\{v\}\_\{s\}/\\\|\\mathbf\{v\}\_\{s\}\\\|\_\{2\}and use𝐯s⊤​𝐡\\mathbf\{v\}\_\{s\}^\{\\top\}\\mathbf\{h\}to score target hidden states\.

The target detector is fitted using the labeled target data, so its predictions on the same examples are in\-sample\. Using these predictions to learn the combination weights can therefore favor the target detector over the source detectors\. To remove such an advantage, we use aKK\-fold cross\-fitting scheme\. Specifically, letnTn\_\{T\}denote the number of labeled target questions available for training\. These questions are partitioned intoKKequal folds\. For each held\-out fold, the target subspace and detector are refitted on the otherK−1K\-1folds, and the corresponding residual\-space direction𝐯T\\mathbf\{v\}\_\{T\}is recomputed, while the source detectors remain fixed\. For each held\-out foldkk, letStd−k⁡\(⋅\)\\operatorname\{Std\}\_\{\-k\}\(\\cdot\)denote standardization using the otherK−1K\-1target folds\. We applyStd−k\\operatorname\{Std\}\_\{\-k\}to the detector scores𝐯s⊤​𝐡\\mathbf\{v\}\_\{s\}^\{\\top\}\\mathbf\{h\}on the held\-out fold and concatenate the standardized scores over allKKfolds to form the cross\-fitted score vector𝝍s\\bm\{\\psi\}\_\{s\}\. These score vectors are concatenated as

𝚿=\[𝝍1⋯𝝍S𝝍T\]\.\\mathbf\{\\Psi\}=\\begin\{bmatrix\}\\bm\{\\psi\}\_\{1\}&\\cdots&\\bm\{\\psi\}\_\{S\}&\\bm\{\\psi\}\_\{T\}\\end\{bmatrix\}\.
Let𝐰=\(w1,…,wS,wT\)⊤\\mathbf\{w\}=\(w\_\{1\},\\ldots,w\_\{S\},w\_\{T\}\)^\{\\top\}denote the combination weights, wherew1,…,wSw\_\{1\},\\ldots,w\_\{S\}correspond to the source detectors andwTw\_\{T\}to the target detector, and letbbdenote the intercept\. Letiiindex the labeled target responses\(q,p,r\)\(q,p,r\)\. For each responseii, letyiy\_\{i\}be its binary label and𝚿i\\mathbf\{\\Psi\}\_\{i\}the corresponding row of𝚿\\mathbf\{\\Psi\}\. We estimate the combination weights and intercept once from the complete𝚿\\mathbf\{\\Psi\}by solving the constrained logistic regression problem:

\(b^,𝐰^\)=arg⁡min⁡∑ib∈ℝ,𝐰≥0𝟏⊤​𝐰=1⁡\[log⁡\(1\+exp⁡\(b\+𝚿i​𝐰\)\)−yi​\(b\+𝚿i​𝐰\)\]\.\(\\widehat\{b\},\\widehat\{\\mathbf\{w\}\}\)=\\arg\\min\_\{\\begin\{subarray\}\{c\}b\\in\\mathbb\{R\},\\;\\mathbf\{w\}\\geq 0\\\\ \\mathbf\{1\}^\{\\top\}\\mathbf\{w\}=1\\end\{subarray\}\}\\sum\_\{i\}\\left\[\\log\\\!\\left\(1\+\\exp\(b\+\\mathbf\{\\Psi\}\_\{i\}\\mathbf\{w\}\)\\right\)\-y\_\{i\}\(b\+\\mathbf\{\\Psi\}\_\{i\}\\mathbf\{w\}\)\\right\]\.\(5\)
Equation[5](https://arxiv.org/html/2610.33442#S2.E5)learns the transfer allocation weights from the cross\-fitted scores\. The learned weights determine how much each detector contributes to the target prediction, while the intercept adjusts the overall offset of the combined score\. For prediction, we train the target detector on all labeled target questions so that all available target supervision is used for prediction\. For the fixed source detectors and the trained target detector, we compute the score standardization statistics using all labeled target responses and denote the standardized score for a hidden state𝐡\\mathbf\{h\}byψs​\(𝐡\)\\psi\_\{s\}\(\\mathbf\{h\}\)\. For a new target response, we combine the standardized scores from the fixed source detectors and the trained target detector using the learned weights\. This defines the AdaST detector:

f^​\(𝐡\)=b^\+∑s∈\[S\]∪\{T\}w^s​ψs​\(𝐡\)\.\\widehat\{f\}\(\\mathbf\{h\}\)=\\widehat\{b\}\+\\sum\_\{s\\in\[S\]\\cup\\\{T\\\}\}\\widehat\{w\}\_\{s\}\\psi\_\{s\}\(\\mathbf\{h\}\)\.\(6\)
f^​\(𝐡\)\\widehat\{f\}\(\\mathbf\{h\}\)is a scalar logit for the new target response\. Applying the sigmoid function tof^​\(𝐡\)\\widehat\{f\}\(\\mathbf\{h\}\)gives the predicted probability that the response belongs to the positive class\. Larger values therefore indicate stronger evidence for the positive class, whereas smaller values indicate stronger evidence for the negative class\.

## 3Asymptotic Analysis

### 3\.1Risk Analysis

To characterize the geometry of transfer, we study a least\-squares analogue of the detector in Section[2](https://arxiv.org/html/2610.33442#S2), using fixed prediction subspaces and convex combinations of unstandardized linear predictions\. This formulation characterizes the limiting weights and prediction risk in terms of domain discrepancy and representation bias\.

For each domains∈\[S\]∪\{T\}s\\in\[S\]\\cup\\\{T\\\}withnsn\_\{s\}samples, consider the linear modelys=𝒙s⊤​𝜽s⋆\+εsy\_\{s\}=\{\\bm\{x\}\}\_\{s\}^\{\\top\}\{\\bm\{\\theta\}\}\_\{s\}^\{\\star\}\+\\varepsilon\_\{s\}, where𝒙s∼𝒩⁡\(0,𝚺s\)\{\\bm\{x\}\}\_\{s\}\\sim\\mathcal\{N\}\(0,\{\\bm\{\\Sigma\}\}\_\{s\}\)andεs\\varepsilon\_\{s\}is independent noise with mean zero and varianceσs2\\sigma\_\{s\}^\{2\}\. Let the orthonormal columns of𝑼s∈ℝD×k\{\\bm\{U\}\}\_\{s\}\\in\\mathbb\{R\}^\{D\\times k\}span a fixed prediction subspace, which corresponds to𝑩s⊤\{\\bm\{B\}\}\_\{s\}^\{\\top\}in Section[2](https://arxiv.org/html/2610.33442#S2)\. The population projected coefficient and the matrix of prediction discrepancies relative to the target are

𝜽s∘=𝑼s​\(𝑼s⊤​𝚺s​𝑼s\)−1​𝑼s⊤​𝚺s​𝜽s⋆,𝑸=\[𝜽1∘−𝜽T⋆,…,𝜽S∘−𝜽T⋆,𝒃T\],\{\\bm\{\\theta\}\}\_\{s\}^\{\\circ\}=\{\\bm\{U\}\}\_\{s\}\(\{\\bm\{U\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{s\}\{\\bm\{U\}\}\_\{s\}\)^\{\-1\}\{\\bm\{U\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{s\}\{\\bm\{\\theta\}\}\_\{s\}^\{\\star\},\\qquad\{\\bm\{Q\}\}=\\big\[\{\\bm\{\\theta\}\}\_\{1\}^\{\\circ\}\-\{\\bm\{\\theta\}\}\_\{T\}^\{\\star\},\\ldots,\{\\bm\{\\theta\}\}\_\{S\}^\{\\circ\}\-\{\\bm\{\\theta\}\}\_\{T\}^\{\\star\},\{\\bm\{b\}\}\_\{T\}\\big\],where𝒃T=𝜽T∘−𝜽T⋆\{\\bm\{b\}\}\_\{T\}=\{\\bm\{\\theta\}\}\_\{T\}^\{\\circ\}\-\{\\bm\{\\theta\}\}\_\{T\}^\{\\star\}is the target representation bias\. We work in the asymptotic limit of source and target sample sizes, while holdingSS,DDandkkfixed\. Assume that𝑯¯:=𝑸⊤​𝚺T​𝑸≻0\\overline\{\{\\bm\{H\}\}\}:=\{\\bm\{Q\}\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{Q\}\}\\succ 0\. We estimate weights by minimizing two\-fold target out\-of\-fold squared loss subject to𝒘≥0\{\\bm\{w\}\}\\geq 0and𝟏⊤​𝒘=1\\mathbf\{1\}^\{\\top\}\{\\bm\{w\}\}=1, then refit the target predictor on all target observations\. The last coordinate of the weight vector corresponds to the target domain, withwS\+1=wTw\_\{S\+1\}=w\_\{T\}\.𝜽^S\+1\\widehat\{\{\\bm\{\\theta\}\}\}\_\{S\+1\}denotes the target\-only coefficient refitted on all target observations\. Write𝜽^AdaST\\widehat\{\{\\bm\{\\theta\}\}\}\_\{\\text\{AdaST\}\}for the aggregate andℛT\\mathcal\{R\}\_\{T\}for its squared prediction risk on an independent target observation\. The full estimator and regularity assumptions are given in Appendix[D](https://arxiv.org/html/2610.33442#A4)\.

###### Theorem 1\(Asymptotic risk and weight convergence\)

Under the assumptions in Appendix[D\.1](https://arxiv.org/html/2610.33442#A4.SS1),

𝒘^\\displaystyle\\widehat\{\{\\bm\{w\}\}\}→a\.s\.𝒘⋆=arg⁡min𝒘≥0𝟏⊤​𝒘=1​𝒘⊤​𝑯¯​𝒘,\\displaystyle\\xrightarrow\{a\.s\.\}\{\\bm\{w\}\}^\{\\star\}=\\arg\\min\_\{\\begin\{subarray\}\{c\}\{\\bm\{w\}\}\\geq 0\\\\ \\mathbf\{1\}^\{\\top\}\{\\bm\{w\}\}=1\\end\{subarray\}\}\{\\bm\{w\}\}^\{\\top\}\\overline\{\{\\bm\{H\}\}\}\{\\bm\{w\}\},ℛT​\(𝜽^AdaST\)\\displaystyle\\mathcal\{R\}\_\{T\}\(\\widehat\{\{\\bm\{\\theta\}\}\}\_\{\\text\{AdaST\}\}\)→a\.s\.σT2\+\(𝒘⋆\)⊤​𝑯¯​𝒘⋆≤σT2\+H¯S\+1,S\+1\.\\displaystyle\\xrightarrow\{a\.s\.\}\\sigma\_\{T\}^\{2\}\+\(\{\\bm\{w\}\}^\{\\star\}\)^\{\\top\}\\overline\{\{\\bm\{H\}\}\}\{\\bm\{w\}\}^\{\\star\}\\leq\\sigma\_\{T\}^\{2\}\+\\overline\{H\}\_\{S\+1,S\+1\}\.The limiting weights are unique\. Equality with the target\-only risk holds exactly when𝐰⋆=𝐞S\+1\{\\bm\{w\}\}^\{\\star\}=\\mathbf\{e\}\_\{S\+1\}, the target\-only weight vector\.

Moreover, ifns≥c​nTn\_\{s\}\\geq cn\_\{T\}for alls∈\[S\]s\\in\[S\]and some fixedc\>1c\>1, then𝔼\|w^T−wT⋆\|=O\(nT−1/2\)\\mathbb\{E\}\|\\widehat\{w\}\_\{T\}\-w\_\{T\}^\{\\star\}\|=O\(n\_\{T\}^\{\-1/2\}\)\.

The proof and finite\-sample bounds are given in Appendices[D](https://arxiv.org/html/2610.33442#A4)and[E](https://arxiv.org/html/2610.33442#A5)\. The asymptotic result shows how complementary source predictors can reduce target representation bias, yielding a limiting risk no greater than that of the target\-only predictor\. Under the stated sample\-size conditions, the finite\-sample rate quantifies how quickly the learned target weight approaches its population\-optimal value and gives a vanishing bound on its expectation when sources alone are optimal\.

### 3\.2Synthetic Validation

We study nonnegative least\-squares aggregation with coefficient shifts𝚫s=𝜽s⋆−𝜽T⋆=τ​𝒅s\{\\bm\{\\Delta\}\}\_\{s\}=\{\\bm\{\\theta\}\}\_\{s\}^\{\\star\}\-\{\\bm\{\\theta\}\}\_\{T\}^\{\\star\}=\\tau\{\\bm\{d\}\}\_\{s\}, where𝒅s∈col⁡\(𝑼s\)\{\\bm\{d\}\}\_\{s\}\\in\\operatorname\{col\}\(\{\\bm\{U\}\}\_\{s\}\)and1S​∑s𝒅s⊤​𝚺T​𝒅s=1\\frac\{1\}\{S\}\\sum\_\{s\}\{\\bm\{d\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{d\}\}\_\{s\}=1\. The projected discrepancies satisfy𝚫~s​\(τ\)=𝜽s∘​\(τ\)−𝜽T⋆=𝒄s\+τ​𝒅s\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}\(\\tau\)=\{\\bm\{\\theta\}\}\_\{s\}^\{\\circ\}\(\\tau\)\-\{\\bm\{\\theta\}\}\_\{T\}^\{\\star\}=\{\\bm\{c\}\}\_\{s\}\+\\tau\{\\bm\{d\}\}\_\{s\}, with𝒄s=𝚫~s​\(0\)\{\\bm\{c\}\}\_\{s\}=\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}\(0\)\.τ\\tauquantifies how far the source coefficients drift from the target coefficient, with smaller values corresponding to source coefficients closer to the target coefficient, while larger values indicate greater source–target heterogeneity\. Even atτ=0\\tau=0, different prediction subspaces can produce different projected coefficients\. We considerτ∈\{0\.4,0\.5,0\.6,0\.75\}\\tau\\in\\\{0\.4,0\.5,0\.6,0\.75\\\}andnT∈\{100,200,400,800,1600,3200\}n\_\{T\}\\in\\\{100,200,400,800,1600,3200\\\}, with200200replications per setting\.

![Refer to caption](https://arxiv.org/html/2610.33442v1/figures/transfer_mass_panels_nonnegative.png)Figure 2:Synthetic validation of simplex\-constrained least\-squares aggregation\.A\.Target weightw^T\\widehat\{w\}\_\{T\}againstnTn\_\{T\}for several source driftsτ\\tau\.B\.Excess riskℛT−σT2\\mathcal\{R\}\_\{T\}\-\\sigma\_\{T\}^\{2\}againstnTn\_\{T\}atτ=0\.5\\tau=0\.5\. Solid lines summarize Monte Carlo means with95%95\\%confidence bands for the means\. Dashed lines mark the constrained population weights and risks from Theorem[1](https://arxiv.org/html/2610.33442#Thmtheorem1)\.Figure[2](https://arxiv.org/html/2610.33442#S3.F2)Ashows that for largerτ\\tau, the learned target weightw^T\\widehat\{w\}\_\{T\}approaches its limitwT⋆​\(τ\)w\_\{T\}^\{\\star\}\(\\tau\)as the sample sizes grow, and this limit increases withτ\\tau\. Intuitively, larger values ofτ\\tauimply that the source domains are less informative for the target, so more weight shifts to the target\. In this regime, the empirical target weight generally increases withnTn\_\{T\}, as additional target observations improve the target fit and reduce the need to borrow information from mismatched source domains\. Figure[2](https://arxiv.org/html/2610.33442#S3.F2)Bshows that the excess risksℛT−σT2\\mathcal\{R\}\_\{T\}\-\\sigma\_\{T\}^\{2\}of the transfer and target\-only estimators converge to their limits, with the transfer limit strictly below the target\-only one\.

## 4Experiments

In this section, we evaluate AdaST primarily on cross\-domain hallucination detection\. We further examine its generalization to reasoning\-step correctness\. We also analyze the learned representations and evaluate activation steering based on the resulting directions\.

### 4\.1Experimental Setup

#### Tasks and datasets\.

We consider several open\-weight language models, including OPT\-6\.7B\([Zhang et al\., 2022](https://arxiv.org/html/2610.33442#bib.bib25)\), Llama\-3\.1\-8B\([Grattafiori and others, 2024](https://arxiv.org/html/2610.33442#bib.bib26)\), Gemma\-7B\([Gemma Team and others, 2024](https://arxiv.org/html/2610.33442#bib.bib27)\), Mistral\-7B\([Jiang et al\., 2023](https://arxiv.org/html/2610.33442#bib.bib28)\), and DeepSeek\-LLM\-7B\([DeepSeek\-AI and others, 2024](https://arxiv.org/html/2610.33442#bib.bib29)\)\. Across these models, we evaluate two behavioral prediction tasks, hallucination detection and reasoning\-step correctness classification\. Hallucination detection is evaluated on Natural Questions\([Kwiatkowski et al\., 2019](https://arxiv.org/html/2610.33442#bib.bib30)\), TriviaQA\([Joshi et al\., 2017](https://arxiv.org/html/2610.33442#bib.bib31)\), CoQA\([Reddy et al\., 2019](https://arxiv.org/html/2610.33442#bib.bib32)\), HaluEval\-QA\([Li et al\., 2023a](https://arxiv.org/html/2610.33442#bib.bib33)\), SimpleQA\([Wei et al\., 2024](https://arxiv.org/html/2610.33442#bib.bib34)\), and HotpotQA\([Yang et al\., 2018](https://arxiv.org/html/2610.33442#bib.bib35)\)\. Reasoning\-step correctness is evaluated on PRM800K\([Lightman et al\., 2024](https://arxiv.org/html/2610.33442#bib.bib36)\), which contains questions from seven mathematical domains, including Algebra, Counting and Probability, Geometry, Intermediate Algebra, Number Theory, Prealgebra, and Precalculus\.

#### Evaluation metrics\.

We use AUROC as the primary discrimination metric, defined asAUROC=Pr⁡\(s⁡\(x\+\)\>s⁡\(x−\)\)\\mathrm\{AUROC\}=\\Pr\(s\(x^\{\+\}\)\>s\(x^\{\-\}\)\)\. Here,s⁡\(⋅\)s\(\\cdot\)denotes the detector score andx\+x^\{\+\}andx−x^\{\-\}denote positive and negative responses\. We also report Macro\-F1, defined asMacro​\-​F1=12​\(F​1positive\+F​1negative\)\\mathrm\{Macro\\text\{\-\}F1\}=\\frac\{1\}\{2\}\\left\(F1\_\{\\mathrm\{positive\}\}\+F1\_\{\\mathrm\{negative\}\}\\right\), whereF​1c=2​Pc​RcPc\+RcF1\_\{c\}=\\frac\{2P\_\{c\}R\_\{c\}\}\{P\_\{c\}\+R\_\{c\}\}for a classcc\. We report Macro\-F1 rather than the F1 score of either class because the response labels can be imbalanced\. Macro\-F1 gives equal weight to truthful and hallucinated responses and therefore avoids a high score driven primarily by the majority class\.

#### Baselines\.

We compare against five hallucination detection baselines\. HARP\([Hu et al\., 2026a](https://arxiv.org/html/2610.33442#bib.bib37)\)derives a reasoning subspace from the model’s unembedding matrix and performs detection on hidden states projected onto this subspace\. SinkProbe\([Binkowski et al\., 2026](https://arxiv.org/html/2610.33442#bib.bib38)\)detects hallucinations using statistics of attention\-sink patterns derived from attention maps\. HaloScope\([Du et al\., 2024](https://arxiv.org/html/2610.33442#bib.bib40)\)uses a latent subspace to separate unlabeled generations into candidate truthful and hallucinated sets, which are then used to train a binary truthfulness classifier\. PRISM\-SAPLMA\([Zhang et al\., 2025b](https://arxiv.org/html/2610.33442#bib.bib41)\)applies PRISM prompts before extracting hidden states and feeds the resulting representations to SAPLMA for cross\-domain detection\. CrossHallu\([Alansari et al\., 2026](https://arxiv.org/html/2610.33442#bib.bib42)\)trains an MLP detector on source\-domain features derived from hidden states, attention, and token probabilities, and studies its transfer to unseen target domains\.

#### Implementation details\.

For hallucination detection, we use six QA datasets as six domains\. Each dataset is treated in turn as the target domain, with the remaining five datasets serving as source domains\. Each question is augmented with two semantically equivalent paraphrases, giving three question versions in total\. We generate five responses per version and label each response by comparing it with all available reference answers using BLEURT\-20\([Sellam et al\., 2020](https://arxiv.org/html/2610.33442#bib.bib13);[Pu et al\., 2021](https://arxiv.org/html/2610.33442#bib.bib14)\)\. A response is labeled positive if its maximum BLEURT score exceeds0\.50\.5and negative otherwise\. The target\-label budgetnTn\_\{T\}counts original questions, with all paraphrases and generated responses from the same question kept together\.

For reasoning\-step correctness, we use PRM800K reasoning prefixes from the seven MATH subjects\. In our transfer setting, each subject is treated in turn as the target, with the remaining six serving as sources\. For each prefix, the evaluated model generates eight candidate next steps, which are scored by Qwen2\.5\-Math\-PRM\-7B\([Zhang et al\., 2025c](https://arxiv.org/html/2610.33442#bib.bib11)\)\. We retain the highest\-scoring candidate with score at least0\.80\.8as positive and the lowest\-scoring candidate with score at most0\.20\.2as negative, discarding prefixes for which both classes are unavailable\. The target\-label budgetnTn\_\{T\}counts retained reasoning prefixes\.

For both tasks, representations and domain\-specific detectors are constructed as described in Section[2](https://arxiv.org/html/2610.33442#S2), and transfer weights are estimated using three\-fold cross\-fitting\. Model\-specific layer selection are provided in Appendix[B\.1](https://arxiv.org/html/2610.33442#A2.SS1)\.

### 4\.2Main Results

Table 1:Performance of target\-only detectors and AdaST across different target\-label budgets, averaged over six datasets for hallucination detection and seven PRM800K MATH domains for reasoning\-step correctness\. Subscripts report standard errors\.Table[1](https://arxiv.org/html/2610.33442#S4.T1)compares the target\-only detector with AdaST across the two tasks and different target\-label budgets\. AdaST consistently improves both AUROC and Macro\-F1 across the two tasks, with larger gains when fewer target labels are available\. With limited target data, the target detector is estimated with greater uncertainty\. AdaST can compensate for this by assigning weight to source detectors that remain informative on held\-out target examples\. As more target labels become available, the target detector becomes more reliable and the additional benefit of source information decreases\. Results by model and target dataset are reported in Appendices[A\.1](https://arxiv.org/html/2610.33442#A1.SS1)and[A\.2](https://arxiv.org/html/2610.33442#A1.SS2)\.

![Refer to caption](https://arxiv.org/html/2610.33442v1/figures/fig_bar.png)Figure 3:Hallucination detection atnT=25n\_\{T\}=25, by target dataset\. Each bar is the mean over the five evaluated models and error bars are standard errors across models\.Figure[3](https://arxiv.org/html/2610.33442#S4.F3)compares AdaST with existing hallucination detectors atnT=25n\_\{T\}=25\. AdaST improves over the domain\-specific detector across all models and achieves the strongest average performance in both AUROC and Macro\-F1\. We also observe that some baselines remain competitive in AUROC but perform worse in Macro\-F1\. This gap shows that strong ranking performance does not always lead to balanced classification\. AdaST remains strong on both metrics\. Results by model are reported in Appendices[A\.3](https://arxiv.org/html/2610.33442#A1.SS3)\.

### 4\.3Further Analysis

#### Learned transfer path\.

Figure[4](https://arxiv.org/html/2610.33442#S4.F4)Ashows the learned expert allocation atnT=25n\_\{T\}=25\. The weights differ across target datasets, suggesting that different targets benefit from different source detectors rather than from a common source combination\. Figure[4](https://arxiv.org/html/2610.33442#S4.F4)Bshows how the overall allocation changes with target supervision\. With few target questions, most weights are placed on the source detectors\. AsnTn\_\{T\}grows, the target detector becomes more reliable, while the relative benefit of source information diminishes, leading AdaST to place more weight on the target detector\. Figure[4](https://arxiv.org/html/2610.33442#S4.F4)Cshows the same overall trend across model families\.

![Refer to caption](https://arxiv.org/html/2610.33442v1/weight.png)Figure 4:Learned AdaST combination weights\. \(a\) Detector weights atnT=25n\_\{T\}=25\. \(b\) Target weightwTw\_\{T\}and source weight1−wT1\-w\_\{T\}across target training sizes\. \(c\) Target weightwTw\_\{T\}across model families\.
#### Robustness to base detector choice\.

To test whether AdaST is robust to the choice of base detector, we use SinkProbe, the strongest single\-dataset baseline in Figure[3](https://arxiv.org/html/2610.33442#S4.F3)\. For each domain, a SinkProbe expert is a logistic classifier fitted to its attention\-sink features\. The resulting classifier score is used as the domain\-specific detector score in AdaST\. Table[2](https://arxiv.org/html/2610.33442#S4.T2)reports the results\.

Table 2:Robustness to base detector choice atnT=25n\_\{T\}=25\. AUROC \(%\)\.Table[2](https://arxiv.org/html/2610.33442#S4.T2)shows that AdaST improves SinkProbe across five models, with an average gain of 1\.61 AUROC points\. SinkProbe relies on attention\-sink features rather than the residual\-stream representations used by our default detector\. This suggests that the transfer gains are not specific to a particular detector construction\.

#### Effect of paraphrase\-aware estimation\.

Table[3](https://arxiv.org/html/2610.33442#S4.T3)examines the effect of paraphrase augmentation\. Within the evaluated range, adding paraphrased views improves AUROC for both the target\-only detector and AdaST\. The improvement suggests that paraphrases help produce a more stable domain\-specific behavioral representation\.

Table 3:Effect of paraphrase augmentation \(AUROC, %\)\.
#### From Detection to Intervention\.

We test whether the directions learned for detection can also steer generation\. Using the residual\-space directions in Equation[4](https://arxiv.org/html/2610.33442#S2.E4)and the transfer weights in Equation[5](https://arxiv.org/html/2610.33442#S2.E5), we define

𝐮mix=∑s∈\[S\]∪\{T\}w^s​𝐯s/κs‖∑s∈\[S\]∪\{T\}w^s​𝐯s/κs‖2,\\mathbf\{u\}\_\{\\mathrm\{mix\}\}=\\frac\{\\sum\_\{s\\in\[S\]\\cup\\\{T\\\}\}\\widehat\{w\}\_\{s\}\\,\\mathbf\{v\}\_\{s\}/\\kappa\_\{s\}\}\{\\left\\\|\\sum\_\{s\\in\[S\]\\cup\\\{T\\\}\}\\widehat\{w\}\_\{s\}\\,\\mathbf\{v\}\_\{s\}/\\kappa\_\{s\}\\right\\\|\_\{2\}\},whereκs\\kappa\_\{s\}is the target\-score normalization factor for detectorss\. During generation, we steer the residual state at the detection layer as𝐡t\(L\)←𝐡t\(L\)\+α​𝐮mix,\\mathbf\{h\}\_\{t\}^\{\(L\)\}\\leftarrow\\mathbf\{h\}\_\{t\}^\{\(L\)\}\+\\alpha\\mathbf\{u\}\_\{\\mathrm\{mix\}\},whereα\>0\\alpha\>0controls the steering strength\. The computation ofκs\\kappa\_\{s\}and the full activation\-steering implementation are detailed in Appendix[B\.2](https://arxiv.org/html/2610.33442#A2.SS2)\.

Table 4:Activation steering results on Llama\-3\.1\-8B, averaged over three seeds\.We use Phi\-4\([Abdin et al\., 2024](https://arxiv.org/html/2610.33442#bib.bib12)\)to assign semantic labels, from which we report semantic accuracy, hallucination and abstention\. Abstention is the fraction of responses that explicitly decline to answer or provide no substantive answer\. Steering improves semantic accuracy and reduces hallucination\. The abstention rate remains largely unchanged, suggesting that the improvement is not simply driven by the model avoiding difficult questions\.

## 5Conclusion

We study adaptive transfer of LLM reliability signals across domains with AdaST\. Experiments on hallucination detection and reasoning\-step correctness show that transfer is most beneficial when target labels are limited, with the learned weights gradually shifting toward the target detector as more supervision becomes available\. Our analysis characterizes this behavior in terms of source–target discrepancies and target representation bias\. Future research will include extending linear classifiers to nonlinear ones\.

## AI Use Statement

Generative AI tools were used to assist with language editing, figure presentation, and feedback on the manuscript\. All AI\-assisted content was reviewed and verified by the authors, who take full responsibility for the final content\.

## References

- Abdinet al\.\(2024\)M\. Abdin, J\. Aneja, H\. Behl, S\. Bubeck, R\. Eldan, S\. Gunasekar, M\. Harrison, R\. J\. Hewett, M\. Javaheripi, P\. Kauffmann,et al\.Phi\-4 technical report\.arXiv preprint arXiv:2412\.08905\.Cited by:[§4\.3](https://arxiv.org/html/2610.33442#S4.SS3.SSS0.Px4.p2.1)\.
- Alansariet al\.\(2026\)A\. Alansari, M\. Alkhorasani, and H\. LuqmanCrossHallu: do hallucination signals generalize across languages and domains in large language model’s internals?\.arXiv preprint arXiv:2607\.04029\.Cited by:[§1\.1](https://arxiv.org/html/2610.33442#S1.SS1.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.33442#S1.p3.1),[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px3.p1.1)\.
- Azaria and Mitchell \(2023\)A\. Azaria and T\. MitchellThe internal state of an llm knows when it’s lying\.InFindings of the Association for Computational Linguistics: EMNLP 2023,Singapore,pp\. 967–976\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.findings-emnlp.68),[Link](https://aclanthology.org/2023.findings-emnlp.68/)Cited by:[§1\.1](https://arxiv.org/html/2610.33442#S1.SS1.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.33442#S1.p2.1)\.
- Bar\-Shalomet al\.\(2025\)G\. Bar\-Shalom, F\. Frasca, Y\. Galron, Y\. Ziser, and H\. MaronBeyond token probes: hallucination detection via activation tensors with act\-vit\.InAdvances in Neural Information Processing Systems,Vol\.38,pp\. 95218–95248\.Cited by:[§1\.1](https://arxiv.org/html/2610.33442#S1.SS1.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.33442#S1.p2.1)\.
- Binkowskiet al\.\(2026\)J\. Binkowski, K\. Adamczewski, and T\. KajdanowiczAttention sinks as internal signals for hallucination detection in large language models\.InInternational Conference on Machine Learning,Cited by:[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px3.p1.1)\.
- Brownet al\.\(2020\)T\. B\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell,et al\.Language models are few\-shot learners\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 1877–1901\.Cited by:[§1](https://arxiv.org/html/2610.33442#S1.p1.1)\.
- Chenet al\.\(2021\)M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. d\. O\. Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman,et al\.Evaluating large language models trained on code\.arXiv preprint arXiv:2107\.03374\.Cited by:[§1](https://arxiv.org/html/2610.33442#S1.p1.1)\.
- DeepSeek\-AIet al\.\(2024\)DeepSeek\-AIet al\.DeepSeek LLM: scaling open\-source language models with longtermism\.arXiv preprint arXiv:2401\.02954\.Cited by:[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px1.p1.1)\.
- Denget al\.\(2026\)Y\. Deng, Z\. Fang, Y\. Li, and L\. ChenBeyond in\-domain detection: spikescore for cross\-domain hallucination detection\.External Links:2601\.19245,[Link](https://arxiv.org/abs/2601.19245)Cited by:[§1\.1](https://arxiv.org/html/2610.33442#S1.SS1.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.33442#S1.p3.1)\.
- Duet al\.\(2024\)X\. Du, C\. Xiao, and Y\. LiHaloScope: harnessing unlabeled LLM generations for hallucination detection\.InAdvances in Neural Information Processing Systems,Cited by:[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px3.p1.1)\.
- Gemma Teamet al\.\(2024\)Gemma Teamet al\.Gemma: open models based on gemini research and technology\.arXiv preprint arXiv:2403\.08295\.Cited by:[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px1.p1.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafioriet al\.The Llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px1.p1.1)\.
- Guoet al\.\(2018\)J\. Guo, D\. Shah, and R\. BarzilayMulti\-source domain adaptation with mixture of experts\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,pp\. 4694–4703\.External Links:[Document](https://dx.doi.org/10.18653/v1/D18-1498)Cited by:[§1\.1](https://arxiv.org/html/2610.33442#S1.SS1.SSS0.Px3.p1.1)\.
- Hageret al\.\(2024\)P\. Hager, F\. Jungmann, R\. Holland, K\. Bhagat, I\. Hubrecht, M\. Knauer, J\. Vielhauer, M\. Makowski, R\. Braren, G\. Kaissis, and D\. RueckertEvaluation and mitigation of the limitations of large language models in clinical decision\-making\.Nature Medicine30,pp\. 2613–2622\.External Links:[Document](https://dx.doi.org/10.1038/s41591-024-03097-1)Cited by:[§1](https://arxiv.org/html/2610.33442#S1.p1.1)\.
- Huet al\.\(2026a\)J\. Hu, G\. Tu, S\. Cheng, J\. Li, J\. Wang, R\. Chen, Z\. Zhou, and D\. ShanHARP: hallucination detection via reasoning subspace projection\.InInternational Conference on Learning Representations,Cited by:[§1\.1](https://arxiv.org/html/2610.33442#S1.SS1.SSS0.Px1.p1.1),[§1\.1](https://arxiv.org/html/2610.33442#S1.SS1.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.33442#S1.p2.1),[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px3.p1.1)\.
- Huet al\.\(2026b\)Y\. Hu, X\. Tong, H\. Bai, X\. Ding, S\. M\. Bharadwaj, S\. Cao, R\. Nowak, and J\. ZhangUnderstanding why language models hallucinate: testing reasoning against priors\.arXiv preprint arXiv:2607\.00447\.Cited by:[§1](https://arxiv.org/html/2610.33442#S1.p1.1)\.
- Jianget al\.\(2023\)A\. Q\. Jiang, A\. Sablayrolles, A\. Mensch, C\. Bamford, D\. S\. Chaplot, D\. de las Casas, F\. Bressand, G\. Lengyel, G\. Lample, L\. Saulnier, L\. Renard Lavaud, M\. Lachaux, P\. Stock, T\. Le Scao, T\. Lavril, T\. Wang, T\. Lacroix, and W\. El SayedMistral 7b\.arXiv preprint arXiv:2310\.06825\.Cited by:[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px1.p1.1)\.
- Joshiet al\.\(2017\)M\. Joshi, E\. Choi, D\. Weld, and L\. ZettlemoyerTriviaQA: a large scale distantly supervised challenge dataset for reading comprehension\.InProceedings of the 55th Annual Meeting of the Association for Computational Linguistics,pp\. 1601–1611\.Cited by:[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px1.p1.1)\.
- Kwiatkowskiet al\.\(2019\)T\. Kwiatkowski, J\. Palomaki, O\. Redfield, M\. Collins, A\. Parikh, C\. Alberti, D\. Epstein, I\. Polosukhin, J\. Devlin, K\. Lee, K\. Toutanova, L\. Jones, M\. Kelcey, M\. Chang, A\. M\. Dai, J\. Uszkoreit, Q\. Le, and S\. PetrovNatural questions: a benchmark for question answering research\.Transactions of the Association for Computational Linguistics7,pp\. 452–466\.Cited by:[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px1.p1.1)\.
- Lewkowyczet al\.\(2022\)A\. Lewkowycz, A\. Andreassen, D\. Dohan, E\. Dyer, H\. Michalewski, V\. Ramasesh, A\. Slone, C\. Anil, I\. Schlag, T\. Gutman\-Solo,et al\.Solving quantitative reasoning problems with language models\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 3843–3857\.Cited by:[§1](https://arxiv.org/html/2610.33442#S1.p1.1)\.
- Liet al\.\(2023a\)J\. Li, X\. Cheng, X\. Zhao, J\. Nie, and J\. WenHaluEval: a large\-scale hallucination evaluation benchmark for large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 6449–6464\.Cited by:[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px1.p1.1)\.
- Liet al\.\(2023b\)K\. Li, O\. Patel, F\. Viégas, H\. Pfister, and M\. WattenbergInference\-time intervention: eliciting truthful answers from a language model\.InAdvances in Neural Information Processing Systems,Vol\.36\.Cited by:[§2\.1](https://arxiv.org/html/2610.33442#S2.SS1.p3.1)\.
- Lightmanet al\.\(2024\)H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. CobbeLet’s verify step by step\.InInternational Conference on Learning Representations,Cited by:[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px1.p1.1)\.
- Mansouret al\.\(2008\)Y\. Mansour, M\. Mohri, and A\. RostamizadehDomain adaptation with multiple sources\.InAdvances in Neural Information Processing Systems,Vol\.21,pp\. 1041–1048\.Cited by:[§1\.1](https://arxiv.org/html/2610.33442#S1.SS1.SSS0.Px3.p1.1)\.
- Mansouret al\.\(2009\)Y\. Mansour, M\. Mohri, and A\. RostamizadehMultiple source adaptation and the rényi divergence\.InProceedings of the Twenty\-Fifth Conference on Uncertainty in Artificial Intelligence,pp\. 367–374\.Cited by:[§1\.1](https://arxiv.org/html/2610.33442#S1.SS1.SSS0.Px3.p1.1)\.
- OpenAI \(2026\)OpenAIThe hugging face incident and the road ahead\.Note:OpenAIExternal Links:[Link](https://openai.com/index/hugging-face-incident-and-the-road-ahead/)Cited by:[§1](https://arxiv.org/html/2610.33442#S1.p1.1)\.
- Orgadet al\.\(2025\)H\. Orgad, M\. Toker, Z\. Gekhman, R\. Reichart, I\. Szpektor, H\. Kotek, and Y\. BelinkovLLMs know more than they show: on the intrinsic representation of LLM hallucinations\.InInternational Conference on Learning Representations,Cited by:[§1\.1](https://arxiv.org/html/2610.33442#S1.SS1.SSS0.Px1.p1.1),[§1\.1](https://arxiv.org/html/2610.33442#S1.SS1.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.33442#S1.p2.1)\.
- O’Neillet al\.\(2025\)C\. O’Neill, S\. Chalnev, C\. C\. Zhao, M\. Kirkby, and M\. JayasekaraA single direction of truth: an observer model’s linear residual probe exposes and steers contextual hallucinations\.arXiv preprint arXiv:2507\.23221\.Cited by:[§1\.1](https://arxiv.org/html/2610.33442#S1.SS1.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.33442#S1.p2.1)\.
- Penget al\.\(2019\)X\. Peng, Q\. Bai, X\. Xia, Z\. Huang, K\. Saenko, and B\. WangMoment matching for multi\-source domain adaptation\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 1406–1415\.External Links:[Document](https://dx.doi.org/10.1109/ICCV.2019.00149)Cited by:[§1\.1](https://arxiv.org/html/2610.33442#S1.SS1.SSS0.Px3.p1.1)\.
- Puet al\.\(2021\)A\. Pu, H\. W\. Chung, A\. P\. Parikh, S\. Gehrmann, and T\. SellamLearning compact metrics for mt\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,pp\. 751–762\.External Links:[Document](https://dx.doi.org/10.18653/v1/2021.emnlp-main.58)Cited by:[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px4.p1.1)\.
- Reddyet al\.\(2019\)S\. Reddy, D\. Chen, and C\. D\. ManningCoQA: a conversational question answering challenge\.Transactions of the Association for Computational Linguistics7,pp\. 249–266\.Cited by:[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px1.p1.1)\.
- Rimskyet al\.\(2024\)N\. Rimsky, N\. Gabrieli, J\. Schulz, M\. Tong, E\. Hubinger, and A\. TurnerSteering llama 2 via contrastive activation addition\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 15504–15522\.Cited by:[§2\.1](https://arxiv.org/html/2610.33442#S2.SS1.p3.1)\.
- Sellamet al\.\(2020\)T\. Sellam, D\. Das, and A\. P\. ParikhBLEURT: learning robust metrics for text generation\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,pp\. 7881–7892\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.704)Cited by:[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px4.p1.1)\.
- Shamsi \(2026\)N\. I\. ShamsiDensity ridge selective prediction for llm and vlm hallucination detection under calibration label scarcity\.External Links:2606\.10198,[Link](https://arxiv.org/abs/2606.10198)Cited by:[§1\.1](https://arxiv.org/html/2610.33442#S1.SS1.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.33442#S1.p3.1)\.
- Songet al\.\(2026\)H\. Song, R\. Ji, N\. Shi, F\. Lai, and R\. Al KontarInv\-entropy: a fully probabilistic framework for uncertainty quantification in language models\.Advances in Neural Information Processing Systems38,pp\. 107889–107922\.Cited by:[§2\.1](https://arxiv.org/html/2610.33442#S2.SS1.p2.1)\.
- Weiet al\.\(2024\)J\. Wei, N\. Karina, H\. W\. Chung, Y\. J\. Jiao, S\. Papay, A\. Glaese, J\. Schulman, and W\. FedusMeasuring short\-form factuality in large language models\.arXiv preprint arXiv:2411\.04368\.Cited by:[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px1.p1.1)\.
- Wuet al\.\(2023\)Y\. Wu, G\. Parmigiani, and B\. RenMulti\-source domain adaptation for regression\.arXiv preprint arXiv:2312\.05460\.Cited by:[§1\.1](https://arxiv.org/html/2610.33442#S1.SS1.SSS0.Px3.p1.1)\.
- Yanget al\.\(2018\)Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. Cohen, R\. Salakhutdinov, and C\. D\. ManningHotpotQA: a dataset for diverse, explainable multi\-hop question answering\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,pp\. 2369–2380\.Cited by:[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2025a\)B\. Zhang, Y\. Tan, Y\. Shen, A\. Salem, M\. Backes, S\. Zannettou, and Y\. ZhangBreaking agents: compromising autonomous LLM agents through malfunction amplification\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,Suzhou, China,pp\. 34964–34976\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1771),[Link](https://aclanthology.org/2025.emnlp-main.1771/)Cited by:[§1](https://arxiv.org/html/2610.33442#S1.p1.1)\.
- Zhanget al\.\(2025b\)F\. Zhang, P\. Yu, B\. Yi, B\. Zhang, T\. Li, and Z\. LiuPrompt\-guided internal states for hallucination detection of large language models\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics,pp\. 21806–21818\.Cited by:[§1\.1](https://arxiv.org/html/2610.33442#S1.SS1.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.33442#S1.p3.1),[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px3.p1.1)\.
- Zhanget al\.\(2022\)S\. Zhang, S\. Roller, N\. Goyal, M\. Artetxe, M\. Chen, S\. Chen, C\. Dewan, M\. Diab, X\. Li, X\. V\. Lin, T\. Mihaylov, M\. Ott, S\. Shleifer, K\. Shuster, D\. Simig, P\. S\. Koura, A\. Sridhar, T\. Wang, and L\. ZettlemoyerOPT: open pre\-trained transformer language models\.arXiv preprint arXiv:2205\.01068\.Cited by:[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2025c\)Z\. Zhang, C\. Zheng, Y\. Wu, B\. Zhang, R\. Lin, B\. Yu, D\. Liu, J\. Zhou, and J\. LinThe lessons of developing process reward models in mathematical reasoning\.arXiv preprint arXiv:2501\.07301\.Cited by:[§4\.1](https://arxiv.org/html/2610.33442#S4.SS1.SSS0.Px4.p2.1)\.
- Zhouet al\.\(2025\)X\. Zhou, M\. Zhang, Z\. Lee, W\. Ye, and S\. ZhangHaDeMiF: hallucination detection and mitigation in large language models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=VwOYxPScxB)Cited by:[§1\.1](https://arxiv.org/html/2610.33442#S1.SS1.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.33442#S1.p2.1)\.
- Zouet al\.\(2023\)A\. Zou, L\. Phan, S\. Chen, J\. Campbell, P\. Guo, R\. Ren, A\. Pan, X\. Yin, M\. Mazeika, A\. Dombrowski, S\. Goel, N\. Li, M\. J\. Byun, Z\. Wang, A\. Mallen, S\. Basart, S\. Koyejo, D\. Song, M\. Fredrikson, J\. Z\. Kolter, and D\. HendrycksRepresentation engineering: a top\-down approach to ai transparency\.arXiv preprint arXiv:2310\.01405\.Cited by:[§2\.1](https://arxiv.org/html/2610.33442#S2.SS1.p3.1)\.

## Appendix AAdditional Results

### A\.1Comparisons Across Models

Table[5](https://arxiv.org/html/2610.33442#A1.T5)breaks thenT=25n\_\{T\}=25column of Table[1](https://arxiv.org/html/2610.33442#S4.T1)down by model\. The gain from the source domains is positive for every model on both tasks\.

Table 5:Per\-model breakdown of Table[1](https://arxiv.org/html/2610.33442#S4.T1)atnT=25n\_\{T\}=25\.
### A\.2Comparisons Across Dataset

Tables[6](https://arxiv.org/html/2610.33442#A1.T6)and[7](https://arxiv.org/html/2610.33442#A1.T7)break Table[1](https://arxiv.org/html/2610.33442#S4.T1)down by target dataset\. The gain from the source domains is positive on every target dataset of both tasks atnT=25n\_\{T\}=25\.

Table 6:AdaST for hallucination detection against the same detector without source data, by dataset and target label budgetnTn\_\{T\}\.Table 7:AdaST for Reasoning\-step correctness against the same detector without source data, by MATH domain and target label budgetnTn\_\{T\}\. Prealgebra has fewer than100100labeled training questions, so thenT=100n\_\{T\}=100row omits it and averages over the remaining six domains\.
### A\.3Comparisons Across Baseline

Table[8](https://arxiv.org/html/2610.33442#A1.T8)gives the per\-model, per\-dataset breakdown behind Figure[3](https://arxiv.org/html/2610.33442#S4.F3)\.

Table 8:Matched\-budget hallucination detection atnT=25n\_\{T\}=25, by model and target dataset\. Subscripts are bootstrap standard errors

## Appendix BAdditional Implementation Details

### B\.1Layer Selection

For each model and target domainTT, we select the representation layer using labeled source data only\. For each source domains∈\[S\]s\\in\[S\], we divide its questions \(or reasoning prefixes\) into a70%70\\%fitting partition and a30%30\\%validation partition\. At each candidate layerjj, we estimate a mean\-difference direction between the positive and negative classes using the fitting partition and evaluate its AUROC on the corresponding validation partition\. Here, the positive class corresponds to truthful responses for hallucination detection and correct next steps for reasoning\-step correctness\. We select

Lm,T=arg⁡maxj​1S​∑s∈\[S\]AUROCm,s,jval,L\_\{m,T\}=\\arg\\max\_\{j\}\\frac\{1\}\{S\}\\sum\_\{s\\in\[S\]\}\\operatorname\{AUROC\}^\{\\mathrm\{val\}\}\_\{m,s,j\},wheremmindexes the evaluated language model\. The selected layer is then fixed for all target\-label budgets, cross\-fitting folds, and experimental repetitions for that model–target pair\.

For hallucination detection, the five remaining QA datasets serve as source domains when one dataset is held out as the target\. For reasoning\-step correctness, the six remaining MATH subjects serve as source domains\. Neither labeled target data nor the target test set is used for layer selection, preserving the target\-label budget and avoiding target\-test leakage\. After selectingLm,TL\_\{m,T\}, the source detectors are refitted using all available labeled data from their respective source domains\.

### B\.2Intervention Direction and Activation Steering

For detectors∈\[S\]∪\{T\}s\\in\[S\]\\cup\\\{T\\\}, recall that its raw residual\-space score is𝐯s⊤​𝐡\\mathbf\{v\}\_\{s\}^\{\\top\}\\mathbf\{h\}\. We denote it bygs​\(𝐡\)g\_\{s\}\(\\mathbf\{h\}\)\.

To place the scores of different detectors on a common scale, we standardize them using the labeled target responses\. Specifically, we define the mean raw score of detectorsson the target set as

g¯s=1nT​P​R​∑q∈𝒬T∑p=1P∑r=1R𝐯s⊤​𝐡q​p​r,\\bar\{g\}\_\{s\}=\\frac\{1\}\{n\_\{T\}PR\}\\sum\_\{q\\in\\mathcal\{Q\}\_\{T\}\}\\sum\_\{p=1\}^\{P\}\\sum\_\{r=1\}^\{R\}\\mathbf\{v\}\_\{s\}^\{\\top\}\\mathbf\{h\}\_\{qpr\},and define

κs=1nT​P​R−1​∑q∈𝒬T∑p=1P∑r=1R\(𝐯s⊤​𝐡q​p​r−g¯s\)2\.\\kappa\_\{s\}=\\sqrt\{\\frac\{1\}\{n\_\{T\}PR\-1\}\\sum\_\{q\\in\\mathcal\{Q\}\_\{T\}\}\\sum\_\{p=1\}^\{P\}\\sum\_\{r=1\}^\{R\}\\left\(\\mathbf\{v\}\_\{s\}^\{\\top\}\\mathbf\{h\}\_\{qpr\}\-\\bar\{g\}\_\{s\}\\right\)^\{2\}\}\.Thus,κs\\kappa\_\{s\}is the empirical standard deviation of detectorss’s raw scores on the labeled target responses\. The standardized detector score is therefore

ψs​\(𝐡\)=𝐯s⊤​𝐡−g¯sκs\.\\psi\_\{s\}\(\\mathbf\{h\}\)=\\frac\{\\mathbf\{v\}\_\{s\}^\{\\top\}\\mathbf\{h\}\-\\bar\{g\}\_\{s\}\}\{\\kappa\_\{s\}\}\.Sinceg¯s\\bar\{g\}\_\{s\}is constant with respect to𝐡\\mathbf\{h\}, the direction in residual space along which the standardized detector score increases is

∇𝐡ψs​\(𝐡\)=𝐯sκs\.\\nabla\_\{\\mathbf\{h\}\}\\psi\_\{s\}\(\\mathbf\{h\}\)=\\frac\{\\mathbf\{v\}\_\{s\}\}\{\\kappa\_\{s\}\}\.
The final AdaST detector combines the standardized detector scores as

f^​\(𝐡\)=b^\+∑s∈\[S\]∪\{T\}w^s​ψs​\(𝐡\)\.\\widehat\{f\}\(\\mathbf\{h\}\)=\\widehat\{b\}\+\\sum\_\{s\\in\[S\]\\cup\\\{T\\\}\}\\widehat\{w\}\_\{s\}\\psi\_\{s\}\(\\mathbf\{h\}\)\.Its corresponding residual\-space direction is therefore

∇𝐡f^​\(𝐡\)=∑s∈\[S\]∪\{T\}w^s​𝐯sκs\.\\nabla\_\{\\mathbf\{h\}\}\\widehat\{f\}\(\\mathbf\{h\}\)=\\sum\_\{s\\in\[S\]\\cup\\\{T\\\}\}\\widehat\{w\}\_\{s\}\\frac\{\\mathbf\{v\}\_\{s\}\}\{\\kappa\_\{s\}\}\.We normalize this vector to unit norm to obtain the steering direction𝐮mix\\mathbf\{u\}\_\{\\mathrm\{mix\}\}used in the intervention experiment\. We next describe how the resulting direction is applied during generation\. We implement activation steering using a forward hook on transformer blockLL\. During the initial prompt pass, the hook addsα​𝐮mix\\alpha\\mathbf\{u\}\_\{\\mathrm\{mix\}\}only to the block output at the final prompt position\. During cached autoregressive decoding, the same update is applied to the current token position at every decoding step:

𝐡t\(L\)←𝐡t\(L\)\+α​𝐮mix\.\\mathbf\{h\}\_\{t\}^\{\(L\)\}\\leftarrow\\mathbf\{h\}\_\{t\}^\{\(L\)\}\+\\alpha\\mathbf\{u\}\_\{\\mathrm\{mix\}\}\.The modified state is then passed to blockL\+1L\+1to predict the next token\. The steering direction𝐮mix\\mathbf\{u\}\_\{\\mathrm\{mix\}\}and strengthα\\alphaare fixed throughout the generation of a response\. No earlier prompt positions or other transformer layers are modified\.

For a unit steering direction𝐮mix\\mathbf\{u\}\_\{\\mathrm\{mix\}\}, we first compute its activation scale

Amix=maxi∈𝒱⁡\|⟨𝐡i,−1\(L\),𝐮mix⟩\|,A\_\{\\mathrm\{mix\}\}=\\max\_\{i\\in\\mathcal\{V\}\}\\left\|\\left\\langle\\mathbf\{h\}\_\{i,\-1\}^\{\(L\)\},\\mathbf\{u\}\_\{\\mathrm\{mix\}\}\\right\\rangle\\right\|,where𝒱\\mathcal\{V\}denotes the steering validation set and𝐡i,−1\(L\)\\mathbf\{h\}\_\{i,\-1\}^\{\(L\)\}is the residual state of the final prompt token at layerLL\. We evaluate strengths of the formα=c​Amix\\alpha=cA\_\{\\mathrm\{mix\}\}forc∈\{0,0\.1,0\.3,1,3,10,20,40\}c\\in\\\{0,0\.1,0\.3,1,3,10,20,40\\\}and select the value that maximizes mean BLEURT on the validation responses\. Ties are resolved in favor of the smaller magnitude\. The selectedα\\alphais then frozen and evaluated on 500 disjoint test questions\.

## Appendix CAdditional Lemmas

###### Lemma 1\(Sandwiched sample second moments\.\)

Letx1,…,xn∈ℝDx\_\{1\},\\dots,x\_\{n\}\\in\\mathbb\{R\}^\{D\}be i\.i\.d\. with𝔼​‖xℓ‖22<∞\\mathbb\{E\}\\\|x\_\{\\ell\}\\\|\_\{2\}^\{2\}<\\inftyand𝔼⁡\[xℓ​xℓ⊤\]=Σ\\mathbb\{E\}\[x\_\{\\ell\}x\_\{\\ell\}^\{\\top\}\]=\\Sigma, and letΣ^=1n​∑ℓ=1nxℓ​xℓ⊤\\widehat\{\\Sigma\}=\\frac\{1\}\{n\}\\sum\_\{\\ell=1\}^\{n\}x\_\{\\ell\}x\_\{\\ell\}^\{\\top\}\. LetA∈ℝD×mAA\\in\\mathbb\{R\}^\{D\\times m\_\{A\}\}andB∈ℝD×mBB\\in\\mathbb\{R\}^\{D\\times m\_\{B\}\}be fixed matrices whose numbers of columns do not depend onnn\. WithDD,AA, andBBfixed, asn→∞n\\to\\infty,

A⊤​Σ^​B→a\.s\.A⊤​Σ​B\.A^\{\\top\}\\widehat\{\\Sigma\}B\\xrightarrow\{\\mathrm\{a\.s\.\}\}A^\{\\top\}\\Sigma B\.If in additionA⊤​Σ​AA^\{\\top\}\\Sigma Ais positive definite, then almost surelyA⊤​Σ^​AA^\{\\top\}\\widehat\{\\Sigma\}Ais invertible for all largennand\(A⊤​Σ^​A\)−1→\(A⊤​Σ​A\)−1\(A^\{\\top\}\\widehat\{\\Sigma\}A\)^\{\-1\}\\to\(A^\{\\top\}\\Sigma A\)^\{\-1\}\.

#### Proof\.

Fix an entry\(a,b\)\(a,b\)witha≤mAa\\leq m\_\{A\}andb≤mBb\\leq m\_\{B\}, and letea,ebe\_\{a\},e\_\{b\}denote standard basis vectors ofℝmA\\mathbb\{R\}^\{m\_\{A\}\}andℝmB\\mathbb\{R\}^\{m\_\{B\}\}\. The entry is a sample mean,

\(A⊤​Σ^​B\)a​b=1n​∑ℓ=1nYℓ,Yℓ=\(ea⊤​A⊤​xℓ\)​\(eb⊤​B⊤​xℓ\)\.\\big\(A^\{\\top\}\\widehat\{\\Sigma\}B\\big\)\_\{ab\}=\\frac\{1\}\{n\}\\sum\_\{\\ell=1\}^\{n\}Y\_\{\\ell\},\\qquad Y\_\{\\ell\}=\\big\(e\_\{a\}^\{\\top\}A^\{\\top\}x\_\{\\ell\}\\big\)\\big\(e\_\{b\}^\{\\top\}B^\{\\top\}x\_\{\\ell\}\\big\)\.The variablesYℓY\_\{\\ell\}are i\.i\.d\. acrossℓ\\ell, being a fixed function of the i\.i\.d\. vectorsxℓx\_\{\\ell\}\. They are integrable by the Cauchy Schwarz inequality,

𝔼​\|Yℓ\|≤\(𝔼​\(ea⊤​A⊤​xℓ\)2\)1/2​\(𝔼​\(eb⊤​B⊤​xℓ\)2\)1/2=\(ea⊤​A⊤​Σ​A​ea\)1/2​\(eb⊤​B⊤​Σ​B​eb\)1/2<∞\.\\mathbb\{E\}\|Y\_\{\\ell\}\|\\leq\\big\(\\mathbb\{E\}\(e\_\{a\}^\{\\top\}A^\{\\top\}x\_\{\\ell\}\)^\{2\}\\big\)^\{1/2\}\\big\(\\mathbb\{E\}\(e\_\{b\}^\{\\top\}B^\{\\top\}x\_\{\\ell\}\)^\{2\}\\big\)^\{1/2\}=\\big\(e\_\{a\}^\{\\top\}A^\{\\top\}\\Sigma Ae\_\{a\}\\big\)^\{1/2\}\\big\(e\_\{b\}^\{\\top\}B^\{\\top\}\\Sigma Be\_\{b\}\\big\)^\{1/2\}<\\infty\.Their mean is𝔼​Yℓ=\(A⊤​Σ​B\)a​b\\mathbb\{E\}Y\_\{\\ell\}=\(A^\{\\top\}\\Sigma B\)\_\{ab\}\. Kolmogorov’s strong law of large numbers gives1n​∑ℓYℓ→\(A⊤​Σ​B\)a​b\\frac\{1\}\{n\}\\sum\_\{\\ell\}Y\_\{\\ell\}\\to\(A^\{\\top\}\\Sigma B\)\_\{ab\}almost surely\. This holds on an eventΩa​b\\Omega\_\{ab\}withℙ⁡\(Ωa​b\)=1\\mathbb\{P\}\(\\Omega\_\{ab\}\)=1\. There aremA​mBm\_\{A\}m\_\{B\}entries, a finite number that does not depend onnn, soℙ⁡\(⋂a,bΩa​b\)=1\\mathbb\{P\}\(\\bigcap\_\{a,b\}\\Omega\_\{ab\}\)=1\. On this event every entry converges\. Entrywise convergence of matrices of fixed size is convergence in every matrix norm, which proves the first claim\.

For the second claim, work on the same probability one event withA=BA=B\. By Weyl’s inequality,

λmin​\(A⊤​Σ^​A\)≥λmin​\(A⊤​Σ​A\)−‖A⊤​Σ^​A−A⊤​Σ​A‖op\.\\lambda\_\{\\min\}\(A^\{\\top\}\\widehat\{\\Sigma\}A\)\\geq\\lambda\_\{\\min\}\(A^\{\\top\}\\Sigma A\)\-\\\|A^\{\\top\}\\widehat\{\\Sigma\}A\-A^\{\\top\}\\Sigma A\\\|\_\{\\mathrm\{op\}\}\.The right side converges toλmin​\(A⊤​Σ​A\)\>0\\lambda\_\{\\min\}\(A^\{\\top\}\\Sigma A\)\>0, soA⊤​Σ^​AA^\{\\top\}\\widehat\{\\Sigma\}Ais invertible for all largenn\. Continuity of matrix inversion gives\(A⊤​Σ^​A\)−1→\(A⊤​Σ​A\)−1\(A^\{\\top\}\\widehat\{\\Sigma\}A\)^\{\-1\}\\to\(A^\{\\top\}\\Sigma A\)^\{\-1\}\.

###### Lemma 2\(Projected noise sample means\.\)

Let\(xℓ,εℓ\)\(x\_\{\\ell\},\\varepsilon\_\{\\ell\}\),ℓ=1,…,n\\ell=1,\\dots,n, be i\.i\.d\. pairs with𝔼⁡\[εℓ∣xℓ\]=0\\mathbb\{E\}\[\\varepsilon\_\{\\ell\}\\mid x\_\{\\ell\}\]=0,𝔼​εℓ2<∞\\mathbb\{E\}\\varepsilon\_\{\\ell\}^\{2\}<\\infty, and𝔼​‖xℓ‖22<∞\\mathbb\{E\}\\\|x\_\{\\ell\}\\\|\_\{2\}^\{2\}<\\infty\. WriteΣ=𝔼⁡\[xℓ​xℓ⊤\]\\Sigma=\\mathbb\{E\}\[x\_\{\\ell\}x\_\{\\ell\}^\{\\top\}\],X=\[x1,…,xn\]∈ℝD×nX=\[x\_\{1\},\\dots,x\_\{n\}\]\\in\\mathbb\{R\}^\{D\\times n\}, andε=\(ε1,…,εn\)⊤\\varepsilon=\(\\varepsilon\_\{1\},\\dots,\\varepsilon\_\{n\}\)^\{\\top\}\. LetA∈ℝD×mAA\\in\\mathbb\{R\}^\{D\\times m\_\{A\}\}be a fixed matrix whose number of columns does not depend onnn\. WithDDandAAfixed, asn→∞n\\to\\infty,

1n​A⊤​X​ε=1n​∑ℓ=1n\(A⊤​xℓ\)​εℓ→a\.s\.0\.\\frac\{1\}\{n\}A^\{\\top\}X\\varepsilon=\\frac\{1\}\{n\}\\sum\_\{\\ell=1\}^\{n\}\(A^\{\\top\}x\_\{\\ell\}\)\\varepsilon\_\{\\ell\}\\xrightarrow\{\\mathrm\{a\.s\.\}\}0\.

#### Proof\.

Fix a coordinatea≤mAa\\leq m\_\{A\}and leteae\_\{a\}be theaath standard basis vector ofℝmA\\mathbb\{R\}^\{m\_\{A\}\}\. The coordinate is the sample mean1n​∑ℓYℓ\\frac\{1\}\{n\}\\sum\_\{\\ell\}Y\_\{\\ell\}withYℓ=\(ea⊤​A⊤​xℓ\)​εℓY\_\{\\ell\}=\(e\_\{a\}^\{\\top\}A^\{\\top\}x\_\{\\ell\}\)\\varepsilon\_\{\\ell\}\. These variables are i\.i\.d\. acrossℓ\\elland integrable by the Cauchy Schwarz inequality,

𝔼​\|Yℓ\|≤\(𝔼​\(ea⊤​A⊤​xℓ\)2\)1/2​\(𝔼​εℓ2\)1/2=\(ea⊤​A⊤​Σ​A​ea\)1/2​\(𝔼​εℓ2\)1/2<∞\.\\mathbb\{E\}\|Y\_\{\\ell\}\|\\leq\\big\(\\mathbb\{E\}\(e\_\{a\}^\{\\top\}A^\{\\top\}x\_\{\\ell\}\)^\{2\}\\big\)^\{1/2\}\\big\(\\mathbb\{E\}\\varepsilon\_\{\\ell\}^\{2\}\\big\)^\{1/2\}=\\big\(e\_\{a\}^\{\\top\}A^\{\\top\}\\Sigma Ae\_\{a\}\\big\)^\{1/2\}\\big\(\\mathbb\{E\}\\varepsilon\_\{\\ell\}^\{2\}\\big\)^\{1/2\}<\\infty\.Their mean is zero by the tower property,

𝔼​Yℓ=𝔼⁡\[\(ea⊤​A⊤​xℓ\)​𝔼​\[εℓ∣xℓ\]\]=0\.\\mathbb\{E\}Y\_\{\\ell\}=\\mathbb\{E\}\\Big\[\(e\_\{a\}^\{\\top\}A^\{\\top\}x\_\{\\ell\}\)\\mathbb\{E\}\[\\varepsilon\_\{\\ell\}\\mid x\_\{\\ell\}\]\\Big\]=0\.Kolmogorov’s strong law of large numbers gives1n​∑ℓYℓ→0\\frac\{1\}\{n\}\\sum\_\{\\ell\}Y\_\{\\ell\}\\to 0almost surely\. Since there are finitely many coordinates, the entire vector converges to zero almost surely\.

## Appendix DProof of Theorem[1](https://arxiv.org/html/2610.33442#Thmtheorem1)

### D\.1Model and Assumptions

We use the Gaussian linear model and fixed prediction subspaces defined in the main text\. Observations are i\.i\.d\. within each domain, and datasets from different domains are independent\. Noise is independent of features, with mean zero and fixed finite variance\. The target test observation is independent of all training data\.

All source and target sample sizes tend to infinity, whileSS,DD,kk, the prediction subspaces, and the population distributions remain fixed\. Assume that the projected covariances𝑼s⊤​𝚺s​𝑼s\{\\bm\{U\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{s\}\{\\bm\{U\}\}\_\{s\}are positive definite in every domain and that𝑯¯=𝑸⊤​𝚺T​𝑸≻0\\overline\{\{\\bm\{H\}\}\}=\{\\bm\{Q\}\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{Q\}\}\\succ 0\. The target fold assignments are fixed across the asymptotic sequence and independent of the observations, withnT,f/nT→ρf∈\(0,1\)n\_\{T,f\}/n\_\{T\}\\to\\rho\_\{f\}\\in\(0,1\)forf=1,2f=1,2\.

### D\.2Two\-Fold Out\-of\-Fold Weight Estimation

Write𝑿s=\[𝒙s,1,…,𝒙s,ns\]\{\\bm\{X\}\}\_\{s\}=\[\{\\bm\{x\}\}\_\{s,1\},\\ldots,\{\\bm\{x\}\}\_\{s,n\_\{s\}\}\],𝒚s=\(ys,1,…,ys,ns\)⊤\{\\bm\{y\}\}\_\{s\}=\(y\_\{s,1\},\\ldots,y\_\{s,n\_\{s\}\}\)^\{\\top\}, and𝜺s=\(εs,1,…,εs,ns\)⊤\{\\bm\{\\varepsilon\}\}\_\{s\}=\(\\varepsilon\_\{s,1\},\\ldots,\\varepsilon\_\{s,n\_\{s\}\}\)^\{\\top\}\. For each source domain, fit the projected linear regression

𝜷^s=arg⁡min𝜷∈ℝk​1ns​‖𝒚s−𝑿s⊤​𝑼s​𝜷‖22,𝜽^s=𝑼s​𝜷^s\.\\widehat\{\\bm\{\\beta\}\}\_\{s\}=\\arg\\min\_\{\{\\bm\{\\beta\}\}\\in\\mathbb\{R\}^\{k\}\}\\frac\{1\}\{n\_\{s\}\}\\\|\{\\bm\{y\}\}\_\{s\}\-\{\\bm\{X\}\}\_\{s\}^\{\\top\}\{\\bm\{U\}\}\_\{s\}\{\\bm\{\\beta\}\}\\\|\_\{2\}^\{2\},\\qquad\\widehat\{\\bm\{\\theta\}\}\_\{s\}=\{\\bm\{U\}\}\_\{s\}\\widehat\{\\bm\{\\beta\}\}\_\{s\}\.Split the target observations into two disjoint folds with sample sizesnT,1\+nT,2=nTn\_\{T,1\}\+n\_\{T,2\}=n\_\{T\}\. Using only foldf∈\{1,2\}f\\in\\\{1,2\\\}, fit the projected linear regression in the target subspace

𝜷^T,f=arg⁡min𝜷∈ℝk​1nT,f​‖𝒚T,f−𝑿T,f⊤​𝑼T​𝜷‖22,𝜽^T,f=𝑼T​𝜷^T,f\.\\widehat\{\\bm\{\\beta\}\}\_\{T,f\}=\\arg\\min\_\{\{\\bm\{\\beta\}\}\\in\\mathbb\{R\}^\{k\}\}\\frac\{1\}\{n\_\{T,f\}\}\\\|\{\\bm\{y\}\}\_\{T,f\}\-\{\\bm\{X\}\}\_\{T,f\}^\{\\top\}\{\\bm\{U\}\}\_\{T\}\{\\bm\{\\beta\}\}\\\|\_\{2\}^\{2\},\\qquad\\widehat\{\\bm\{\\theta\}\}\_\{T,f\}=\{\\bm\{U\}\}\_\{T\}\\widehat\{\\bm\{\\beta\}\}\_\{T,f\}\.The aggregation weights𝒘=\(w1,…,wS,wS\+1\)⊤\{\\bm\{w\}\}=\(w\_\{1\},\\ldots,w\_\{S\},w\_\{S\+1\}\)^\{\\top\}are estimated by

𝒘^=argmin𝒘≥0𝟏⊤​𝒘=11nT\[\\displaystyle\\widehat\{\\bm\{w\}\}=\\arg\\min\_\{\\begin\{subarray\}\{c\}\{\\bm\{w\}\}\\geq 0\\\\ \\mathbf\{1\}^\{\\top\}\{\\bm\{w\}\}=1\\end\{subarray\}\}\\frac\{1\}\{n\_\{T\}\}\\Bigg\[‖𝒚T,1−𝑿T,1⊤​\(∑s=1Sws​𝜽^s\+wS\+1​𝜽^T,2\)‖22\\displaystyle\\left\\\|\{\\bm\{y\}\}\_\{T,1\}\-\{\\bm\{X\}\}\_\{T,1\}^\{\\top\}\\left\(\\sum\_\{s=1\}^\{S\}w\_\{s\}\\widehat\{\\bm\{\\theta\}\}\_\{s\}\+w\_\{S\+1\}\\widehat\{\\bm\{\\theta\}\}\_\{T,2\}\\right\)\\right\\\|\_\{2\}^\{2\}\+‖𝒚T,2−𝑿T,2⊤\(∑s=1Sws𝜽^s\+wS\+1𝜽^T,1\)‖22\]\.\\displaystyle\+\\left\\\|\{\\bm\{y\}\}\_\{T,2\}\-\{\\bm\{X\}\}\_\{T,2\}^\{\\top\}\\left\(\\sum\_\{s=1\}^\{S\}w\_\{s\}\\widehat\{\\bm\{\\theta\}\}\_\{s\}\+w\_\{S\+1\}\\widehat\{\\bm\{\\theta\}\}\_\{T,1\}\\right\)\\right\\\|\_\{2\}^\{2\}\\Bigg\]\.Any measurable minimizer can be used if the empirical optimum is not unique\.

### D\.3Risk Calculation

The projected linear regression using the full target dataset is

𝜷^T=arg⁡min𝜷∈ℝk​1nT​‖𝒚T−𝑿T⊤​𝑼T​𝜷‖22,𝜽^S\+1=𝑼T​𝜷^T\.\\widehat\{\\bm\{\\beta\}\}\_\{T\}=\\arg\\min\_\{\{\\bm\{\\beta\}\}\\in\\mathbb\{R\}^\{k\}\}\\frac\{1\}\{n\_\{T\}\}\\\|\{\\bm\{y\}\}\_\{T\}\-\{\\bm\{X\}\}\_\{T\}^\{\\top\}\{\\bm\{U\}\}\_\{T\}\{\\bm\{\\beta\}\}\\\|\_\{2\}^\{2\},\\qquad\\widehat\{\\bm\{\\theta\}\}\_\{S\+1\}=\{\\bm\{U\}\}\_\{T\}\\widehat\{\\bm\{\\beta\}\}\_\{T\}\.Hence, the transferred estimator is

𝜽^AdaST=∑s=1Sw^s​𝜽^s\+w^S\+1​𝜽^S\+1,\\widehat\{\{\\bm\{\\theta\}\}\}\_\{\\text\{AdaST\}\}=\\sum\_\{s=1\}^\{S\}\\widehat\{w\}\_\{s\}\\widehat\{\\bm\{\\theta\}\}\_\{s\}\+\\widehat\{w\}\_\{S\+1\}\\widehat\{\\bm\{\\theta\}\}\_\{S\+1\},\(7\)with the last coordinate of the weight vector corresponds to the target domain andw^S\+1=w^T\\widehat\{w\}\_\{S\+1\}=\\widehat\{w\}\_\{T\}\. For an independent target observation\(𝒙0,y0\)\(\{\\bm\{x\}\}\_\{0\},y\_\{0\}\), the noise has mean zero and is independent of the features, so

ℛT​\(𝜽^AdaST\)=𝔼⁡\[\(y0−𝒙0⊤​𝜽^AdaST\)2∣𝜽^AdaST\]=σT2\+\(𝜽^AdaST−𝜽T⋆\)⊤​𝚺T​\(𝜽^AdaST−𝜽T⋆\)\.\\mathcal\{R\}\_\{T\}\(\\widehat\{\{\\bm\{\\theta\}\}\}\_\{\\text\{AdaST\}\}\)=\\mathbb\{E\}\\big\[\(y\_\{0\}\-\{\\bm\{x\}\}\_\{0\}^\{\\top\}\\widehat\{\{\\bm\{\\theta\}\}\}\_\{\\text\{AdaST\}\}\)^\{2\}\\mid\\widehat\{\{\\bm\{\\theta\}\}\}\_\{\\text\{AdaST\}\}\\big\]=\\sigma\_\{T\}^\{2\}\+\(\\widehat\{\{\\bm\{\\theta\}\}\}\_\{\\text\{AdaST\}\}\-\{\\bm\{\\theta\}\}\_\{T\}^\{\\star\}\)^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\(\\widehat\{\{\\bm\{\\theta\}\}\}\_\{\\text\{AdaST\}\}\-\{\\bm\{\\theta\}\}\_\{T\}^\{\\star\}\)\.Using Equation[7](https://arxiv.org/html/2610.33442#A4.E7)and the constraint𝟏⊤​𝒘^=1\\mathbf\{1\}^\{\\top\}\\widehat\{\\bm\{w\}\}=1,

𝜽^AdaST−𝜽T⋆\\displaystyle\\widehat\{\{\\bm\{\\theta\}\}\}\_\{\\text\{AdaST\}\}\-\{\\bm\{\\theta\}\}\_\{T\}^\{\\star\}=∑s=1Sw^s​𝜽^s\+w^S\+1​𝜽^S\+1−\(∑s=1Sw^s\+w^S\+1\)​𝜽T⋆\\displaystyle=\\sum\_\{s=1\}^\{S\}\\widehat\{w\}\_\{s\}\\widehat\{\\bm\{\\theta\}\}\_\{s\}\+\\widehat\{w\}\_\{S\+1\}\\widehat\{\\bm\{\\theta\}\}\_\{S\+1\}\-\\left\(\\sum\_\{s=1\}^\{S\}\\widehat\{w\}\_\{s\}\+\\widehat\{w\}\_\{S\+1\}\\right\)\{\\bm\{\\theta\}\}\_\{T\}^\{\\star\}=∑s=1Sw^s​\(𝜽^s−𝜽T⋆\)\+w^S\+1​\(𝜽^S\+1−𝜽T⋆\)\.\\displaystyle=\\sum\_\{s=1\}^\{S\}\\widehat\{w\}\_\{s\}\(\\widehat\{\\bm\{\\theta\}\}\_\{s\}\-\{\\bm\{\\theta\}\}\_\{T\}^\{\\star\}\)\+\\widehat\{w\}\_\{S\+1\}\(\\widehat\{\\bm\{\\theta\}\}\_\{S\+1\}\-\{\\bm\{\\theta\}\}\_\{T\}^\{\\star\}\)\.Here,𝜽^S\+1\\widehat\{\\bm\{\\theta\}\}\_\{S\+1\}is the target\-only coefficient\. For each source task, the source–target shift, representation bias, and estimation error are

𝚫s=𝜽s⋆−𝜽T⋆,𝒃s=𝜽s∘−𝜽s⋆,𝝃s=𝜽^s−𝜽s∘\.\{\\bm\{\\Delta\}\}\_\{s\}=\{\\bm\{\\theta\}\}\_\{s\}^\{\\star\}\-\{\\bm\{\\theta\}\}\_\{T\}^\{\\star\},\\qquad\{\\bm\{b\}\}\_\{s\}=\{\\bm\{\\theta\}\}\_\{s\}^\{\\circ\}\-\{\\bm\{\\theta\}\}\_\{s\}^\{\\star\},\\qquad\{\\bm\{\\xi\}\}\_\{s\}=\\widehat\{\\bm\{\\theta\}\}\_\{s\}\-\{\\bm\{\\theta\}\}\_\{s\}^\{\\circ\}\.The effective discrepancy between the projected source coefficient and the target coefficient is therefore

𝚫~s=𝜽s∘−𝜽T⋆=𝚫s\+𝒃s,𝜽^s−𝜽T⋆=𝚫~s\+𝝃s\.\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}=\{\\bm\{\\theta\}\}\_\{s\}^\{\\circ\}\-\{\\bm\{\\theta\}\}\_\{T\}^\{\\star\}=\{\\bm\{\\Delta\}\}\_\{s\}\+\{\\bm\{b\}\}\_\{s\},\\qquad\\widehat\{\\bm\{\\theta\}\}\_\{s\}\-\{\\bm\{\\theta\}\}\_\{T\}^\{\\star\}=\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}\+\{\\bm\{\\xi\}\}\_\{s\}\.For the target task, the representation bias and estimation error are

𝒃T=𝜽T∘−𝜽T⋆,𝝃S\+1=𝜽^S\+1−𝜽T∘\.\{\\bm\{b\}\}\_\{T\}=\{\\bm\{\\theta\}\}\_\{T\}^\{\\circ\}\-\{\\bm\{\\theta\}\}\_\{T\}^\{\\star\},\\qquad\{\\bm\{\\xi\}\}\_\{S\+1\}=\\widehat\{\\bm\{\\theta\}\}\_\{S\+1\}\-\{\\bm\{\\theta\}\}\_\{T\}^\{\\circ\}\.Hence,

𝜽^AdaST−𝜽T⋆=∑s=1Sw^s​\(𝚫~s\+𝝃s\)\+w^S\+1​\(𝒃T\+𝝃S\+1\),\\widehat\{\{\\bm\{\\theta\}\}\}\_\{\\text\{AdaST\}\}\-\{\\bm\{\\theta\}\}\_\{T\}^\{\\star\}=\\sum\_\{s=1\}^\{S\}\\widehat\{w\}\_\{s\}\(\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}\+\{\\bm\{\\xi\}\}\_\{s\}\)\+\\widehat\{w\}\_\{S\+1\}\(\{\\bm\{b\}\}\_\{T\}\+\{\\bm\{\\xi\}\}\_\{S\+1\}\),and expanding the quadratic risk gives

ℛT​\(𝜽^AdaST\)\\displaystyle\\mathcal\{R\}\_\{T\}\(\\widehat\{\{\\bm\{\\theta\}\}\}\_\{\\text\{AdaST\}\}\)=σT2\+∑s=1S∑s′=1Sw^s​w^s′​\[𝚫~s⊤​𝚺T​𝚫~s′\+𝚫~s⊤​𝚺T​𝝃s′\+𝝃s⊤​𝚺T​𝚫~s′\+𝝃s⊤​𝚺T​𝝃s′\]\\displaystyle=\\sigma\_\{T\}^\{2\}\+\\sum\_\{s=1\}^\{S\}\\sum\_\{s^\{\\prime\}=1\}^\{S\}\\widehat\{w\}\_\{s\}\\widehat\{w\}\_\{s^\{\\prime\}\}\\Big\[\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\\widetilde\{\\bm\{\\Delta\}\}\_\{s^\{\\prime\}\}\+\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{\\xi\}\}\_\{s^\{\\prime\}\}\+\{\\bm\{\\xi\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\\widetilde\{\\bm\{\\Delta\}\}\_\{s^\{\\prime\}\}\+\{\\bm\{\\xi\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{\\xi\}\}\_\{s^\{\\prime\}\}\\Big\]\+2∑s=1Sw^sw^S\+1\[𝚫~s⊤𝚺T𝒃T\+𝚫~s⊤𝚺T𝝃S\+1\+𝝃s⊤𝚺T𝒃T\+𝝃s⊤𝚺T𝝃S\+1\]\\displaystyle\\quad\+2\\sum\_\{s=1\}^\{S\}\\widehat\{w\}\_\{s\}\\widehat\{w\}\_\{S\+1\}\\Big\[\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{b\}\}\_\{T\}\+\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{\\xi\}\}\_\{S\+1\}\+\{\\bm\{\\xi\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{b\}\}\_\{T\}\+\{\\bm\{\\xi\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{\\xi\}\}\_\{S\+1\}\\Big\]\+w^S\+12​\[𝒃T⊤​𝚺T​𝒃T\+𝒃T⊤​𝚺T​𝝃S\+1\+𝝃S\+1⊤​𝚺T​𝒃T\+𝝃S\+1⊤​𝚺T​𝝃S\+1\]\.\\displaystyle\\quad\+\\widehat\{w\}\_\{S\+1\}^\{2\}\\Big\[\{\\bm\{b\}\}\_\{T\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{b\}\}\_\{T\}\+\{\\bm\{b\}\}\_\{T\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{\\xi\}\}\_\{S\+1\}\+\{\\bm\{\\xi\}\}\_\{S\+1\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{b\}\}\_\{T\}\+\{\\bm\{\\xi\}\}\_\{S\+1\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{\\xi\}\}\_\{S\+1\}\\Big\]\.
We first show that the estimation errors vanish\. Let𝚺^s=𝑿s​𝑿s⊤/ns\\widehat\{\\bm\{\\Sigma\}\}\_\{s\}=\{\\bm\{X\}\}\_\{s\}\{\\bm\{X\}\}\_\{s\}^\{\\top\}/n\_\{s\}\. Substituting𝒚s=𝑿s⊤​𝜽s⋆\+𝜺s\{\\bm\{y\}\}\_\{s\}=\{\\bm\{X\}\}\_\{s\}^\{\\top\}\{\\bm\{\\theta\}\}\_\{s\}^\{\\star\}\+\{\\bm\{\\varepsilon\}\}\_\{s\}into the projected least\-squares normal equations gives

𝜷^s\\displaystyle\\widehat\{\\bm\{\\beta\}\}\_\{s\}=\(𝑼s⊤​𝚺^s​𝑼s\)−1​𝑼s⊤​𝚺^s​𝜽s⋆\\displaystyle=\(\{\\bm\{U\}\}\_\{s\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{s\}\{\\bm\{U\}\}\_\{s\}\)^\{\-1\}\{\\bm\{U\}\}\_\{s\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{s\}\{\\bm\{\\theta\}\}\_\{s\}^\{\\star\}\+\(𝑼s⊤​𝚺^s​𝑼s\)−1​1ns​𝑼s⊤​𝑿s​𝜺s\.\\displaystyle\\quad\+\(\{\\bm\{U\}\}\_\{s\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{s\}\{\\bm\{U\}\}\_\{s\}\)^\{\-1\}\\frac\{1\}\{n\_\{s\}\}\{\\bm\{U\}\}\_\{s\}^\{\\top\}\{\\bm\{X\}\}\_\{s\}\{\\bm\{\\varepsilon\}\}\_\{s\}\.By Lemma[1](https://arxiv.org/html/2610.33442#Thmlemma1),

𝑼s⊤​𝚺^s​𝑼s→a\.s\.𝑼s⊤​𝚺s​𝑼s,𝑼s⊤​𝚺^s​𝜽s⋆→a\.s\.𝑼s⊤​𝚺s​𝜽s⋆\.\{\\bm\{U\}\}\_\{s\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{s\}\{\\bm\{U\}\}\_\{s\}\\xrightarrow\{a\.s\.\}\{\\bm\{U\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{s\}\{\\bm\{U\}\}\_\{s\},\\qquad\{\\bm\{U\}\}\_\{s\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{s\}\{\\bm\{\\theta\}\}\_\{s\}^\{\\star\}\\xrightarrow\{a\.s\.\}\{\\bm\{U\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{s\}\{\\bm\{\\theta\}\}\_\{s\}^\{\\star\}\.The first limit is positive definite, so the sample matrix is almost surely invertible for all sufficiently large sample sizes and its inverse converges to\(𝑼s⊤​𝚺s​𝑼s\)−1\(\{\\bm\{U\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{s\}\{\\bm\{U\}\}\_\{s\}\)^\{\-1\}\. By Lemma[2](https://arxiv.org/html/2610.33442#Thmlemma2),

1ns​𝑼s⊤​𝑿s​𝜺s→a\.s\.0\.\\frac\{1\}\{n\_\{s\}\}\{\\bm\{U\}\}\_\{s\}^\{\\top\}\{\\bm\{X\}\}\_\{s\}\{\\bm\{\\varepsilon\}\}\_\{s\}\\xrightarrow\{a\.s\.\}0\.Therefore,

𝜷^s→a\.s\.\(𝑼s⊤​𝚺s​𝑼s\)−1​𝑼s⊤​𝚺s​𝜽s⋆=𝜷s∘,\\widehat\{\\bm\{\\beta\}\}\_\{s\}\\xrightarrow\{a\.s\.\}\(\{\\bm\{U\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{s\}\{\\bm\{U\}\}\_\{s\}\)^\{\-1\}\{\\bm\{U\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{s\}\{\\bm\{\\theta\}\}\_\{s\}^\{\\star\}=\{\\bm\{\\beta\}\}\_\{s\}^\{\\circ\},and hence

𝝃s=𝜽^s−𝜽s∘=𝑼s​\(𝜷^s−𝜷s∘\)→a\.s\.0\.\{\\bm\{\\xi\}\}\_\{s\}=\\widehat\{\\bm\{\\theta\}\}\_\{s\}\-\{\\bm\{\\theta\}\}\_\{s\}^\{\\circ\}=\{\\bm\{U\}\}\_\{s\}\(\\widehat\{\\bm\{\\beta\}\}\_\{s\}\-\{\\bm\{\\beta\}\}\_\{s\}^\{\\circ\}\)\\xrightarrow\{a\.s\.\}0\.The same argument withs=Ts=Tdefines𝜷T∘\{\\bm\{\\beta\}\}\_\{T\}^\{\\circ\}and applies to the two target\-fold estimators and the full\-sample target estimator\. Defining𝝃T,f=𝜽^T,f−𝜽T∘\{\\bm\{\\xi\}\}\_\{T,f\}=\\widehat\{\\bm\{\\theta\}\}\_\{T,f\}\-\{\\bm\{\\theta\}\}\_\{T\}^\{\\circ\}forf=1,2f=1,2, we obtain

𝝃T,1→a\.s\.0,𝝃T,2→a\.s\.0,𝝃S\+1→a\.s\.0\.\{\\bm\{\\xi\}\}\_\{T,1\}\\xrightarrow\{a\.s\.\}0,\\qquad\{\\bm\{\\xi\}\}\_\{T,2\}\\xrightarrow\{a\.s\.\}0,\\qquad\{\\bm\{\\xi\}\}\_\{S\+1\}\\xrightarrow\{a\.s\.\}0\.

### D\.4Asymptotic Out\-of\-Fold Weights

For foldf∈\{1,2\}f\\in\\\{1,2\\\}, substituting the target model and using𝟏⊤​𝒘=1\\mathbf\{1\}^\{\\top\}\{\\bm\{w\}\}=1, the residual in the out\-of\-fold objective becomes

𝜺T,f−𝑿T,f⊤​\(∑s=1Sws​\(𝚫~s\+𝝃s\)\+wS\+1​\(𝒃T\+𝝃T,3−f\)\)\.\{\\bm\{\\varepsilon\}\}\_\{T,f\}\-\{\\bm\{X\}\}\_\{T,f\}^\{\\top\}\\left\(\\sum\_\{s=1\}^\{S\}w\_\{s\}\(\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}\+\{\\bm\{\\xi\}\}\_\{s\}\)\+w\_\{S\+1\}\(\{\\bm\{b\}\}\_\{T\}\+\{\\bm\{\\xi\}\}\_\{T,3\-f\}\)\\right\)\.Define

𝑬^f=\[𝚫~1\+𝝃1⋯𝚫~S\+𝝃S𝒃T\+𝝃T,3−f\]\.\\widehat\{\\bm\{E\}\}\_\{f\}=\\begin\{bmatrix\}\\widetilde\{\\bm\{\\Delta\}\}\_\{1\}\+\{\\bm\{\\xi\}\}\_\{1\}&\\cdots&\\widetilde\{\\bm\{\\Delta\}\}\_\{S\}\+\{\\bm\{\\xi\}\}\_\{S\}&\{\\bm\{b\}\}\_\{T\}\+\{\\bm\{\\xi\}\}\_\{T,3\-f\}\\end\{bmatrix\}\.Expanding the squared residuals, the out\-of\-fold objective is

L⁡\(𝒘\)=1nT​‖𝜺T‖22−2​𝒘⊤​𝒈^\+𝒘⊤​𝑯^​𝒘,L\(\{\\bm\{w\}\}\)=\\frac\{1\}\{n\_\{T\}\}\\\|\{\\bm\{\\varepsilon\}\}\_\{T\}\\\|\_\{2\}^\{2\}\-2\{\\bm\{w\}\}^\{\\top\}\\widehat\{\\bm\{g\}\}\+\{\\bm\{w\}\}^\{\\top\}\\widehat\{\\bm\{H\}\}\{\\bm\{w\}\},where

𝒈^=1nT​∑f=12𝑬^f⊤​𝑿T,f​𝜺T,f,𝑯^=1nT​∑f=12𝑬^f⊤​𝑿T,f​𝑿T,f⊤​𝑬^f\.\\widehat\{\\bm\{g\}\}=\\frac\{1\}\{n\_\{T\}\}\\sum\_\{f=1\}^\{2\}\\widehat\{\\bm\{E\}\}\_\{f\}^\{\\top\}\{\\bm\{X\}\}\_\{T,f\}\{\\bm\{\\varepsilon\}\}\_\{T,f\},\\qquad\\widehat\{\\bm\{H\}\}=\\frac\{1\}\{n\_\{T\}\}\\sum\_\{f=1\}^\{2\}\\widehat\{\\bm\{E\}\}\_\{f\}^\{\\top\}\{\\bm\{X\}\}\_\{T,f\}\{\\bm\{X\}\}\_\{T,f\}^\{\\top\}\\widehat\{\\bm\{E\}\}\_\{f\}\.Fors≤Ss\\leq S, the source predictor is the same on both target folds, so

g^s\\displaystyle\\widehat\{g\}\_\{s\}=1nT​\(𝚫~s\+𝝃s\)⊤​𝑿T​𝜺T\\displaystyle=\\frac\{1\}\{n\_\{T\}\}\(\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}\+\{\\bm\{\\xi\}\}\_\{s\}\)^\{\\top\}\{\\bm\{X\}\}\_\{T\}\{\\bm\{\\varepsilon\}\}\_\{T\}=1nT​𝚫~s⊤​𝑿T​𝜺T\+\(𝜷^s−𝜷s∘\)⊤​1nT​𝑼s⊤​𝑿T​𝜺T→a\.s\.0\.\\displaystyle=\\frac\{1\}\{n\_\{T\}\}\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}^\{\\top\}\{\\bm\{X\}\}\_\{T\}\{\\bm\{\\varepsilon\}\}\_\{T\}\+\(\\widehat\{\\bm\{\\beta\}\}\_\{s\}\-\{\\bm\{\\beta\}\}\_\{s\}^\{\\circ\}\)^\{\\top\}\\frac\{1\}\{n\_\{T\}\}\{\\bm\{U\}\}\_\{s\}^\{\\top\}\{\\bm\{X\}\}\_\{T\}\{\\bm\{\\varepsilon\}\}\_\{T\}\\xrightarrow\{a\.s\.\}0\.Here Lemma[2](https://arxiv.org/html/2610.33442#Thmlemma2)applies withA=𝚫~sA=\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}andA=𝑼sA=\{\\bm\{U\}\}\_\{s\}, and𝜷^s−𝜷s∘→a\.s\.0\\widehat\{\\bm\{\\beta\}\}\_\{s\}\-\{\\bm\{\\beta\}\}\_\{s\}^\{\\circ\}\\xrightarrow\{a\.s\.\}0\. For the target coordinate, the same argument gives

g^S\+1\\displaystyle\\widehat\{g\}\_\{S\+1\}=1nT​∑f=12\(𝒃T\+𝝃T,3−f\)⊤​𝑿T,f​𝜺T,f\\displaystyle=\\frac\{1\}\{n\_\{T\}\}\\sum\_\{f=1\}^\{2\}\(\{\\bm\{b\}\}\_\{T\}\+\{\\bm\{\\xi\}\}\_\{T,3\-f\}\)^\{\\top\}\{\\bm\{X\}\}\_\{T,f\}\{\\bm\{\\varepsilon\}\}\_\{T,f\}=∑f=12nT,fnT​\[1nT,f​𝒃T⊤​𝑿T,f​𝜺T,f\+\(𝜷^T,3−f−𝜷T∘\)⊤​1nT,f​𝑼T⊤​𝑿T,f​𝜺T,f\]→a\.s\.0\.\\displaystyle=\\sum\_\{f=1\}^\{2\}\\frac\{n\_\{T,f\}\}\{n\_\{T\}\}\\left\[\\frac\{1\}\{n\_\{T,f\}\}\{\\bm\{b\}\}\_\{T\}^\{\\top\}\{\\bm\{X\}\}\_\{T,f\}\{\\bm\{\\varepsilon\}\}\_\{T,f\}\+\(\\widehat\{\\bm\{\\beta\}\}\_\{T,3\-f\}\-\{\\bm\{\\beta\}\}\_\{T\}^\{\\circ\}\)^\{\\top\}\\frac\{1\}\{n\_\{T,f\}\}\{\\bm\{U\}\}\_\{T\}^\{\\top\}\{\\bm\{X\}\}\_\{T,f\}\{\\bm\{\\varepsilon\}\}\_\{T,f\}\\right\]\\xrightarrow\{a\.s\.\}0\.Thus𝒈^→a\.s\.0\\widehat\{\\bm\{g\}\}\\xrightarrow\{a\.s\.\}0\.

We next consider𝑯^\\widehat\{\\bm\{H\}\}\. Fors,s′≤Ss,s^\{\\prime\}\\leq S,

H^s​s′\\displaystyle\\widehat\{H\}\_\{ss^\{\\prime\}\}=\(𝚫~s\+𝝃s\)⊤​𝚺^T​\(𝚫~s′\+𝝃s′\)\\displaystyle=\(\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}\+\{\\bm\{\\xi\}\}\_\{s\}\)^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T\}\(\\widetilde\{\\bm\{\\Delta\}\}\_\{s^\{\\prime\}\}\+\{\\bm\{\\xi\}\}\_\{s^\{\\prime\}\}\)=𝚫~s⊤​𝚺^T​𝚫~s′\+𝚫~s⊤​𝚺^T​𝝃s′\+𝝃s⊤​𝚺^T​𝚫~s′\+𝝃s⊤​𝚺^T​𝝃s′,\\displaystyle=\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T\}\\widetilde\{\\bm\{\\Delta\}\}\_\{s^\{\\prime\}\}\+\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{\\xi\}\}\_\{s^\{\\prime\}\}\+\{\\bm\{\\xi\}\}\_\{s\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T\}\\widetilde\{\\bm\{\\Delta\}\}\_\{s^\{\\prime\}\}\+\{\\bm\{\\xi\}\}\_\{s\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{\\xi\}\}\_\{s^\{\\prime\}\},where𝚺^T=𝑿T​𝑿T⊤/nT\\widehat\{\\bm\{\\Sigma\}\}\_\{T\}=\{\\bm\{X\}\}\_\{T\}\{\\bm\{X\}\}\_\{T\}^\{\\top\}/n\_\{T\}\. Applying Lemma[1](https://arxiv.org/html/2610.33442#Thmlemma1)to coordinate vectors gives𝚺^T→a\.s\.𝚺T\\widehat\{\\bm\{\\Sigma\}\}\_\{T\}\\xrightarrow\{a\.s\.\}\{\\bm\{\\Sigma\}\}\_\{T\}in operator norm becauseDDis fixed\. Each term involving an estimation error vanishes\. For example,

\|𝚫~s⊤​𝚺^T​𝝃s′\|≤‖𝚫~s‖2​‖𝚺^T‖op​‖𝝃s′‖2→a\.s\.0,\|𝝃s⊤​𝚺^T​𝝃s′\|≤‖𝝃s‖2​‖𝚺^T‖op​‖𝝃s′‖2→a\.s\.0\.\|\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{\\xi\}\}\_\{s^\{\\prime\}\}\|\\leq\\\|\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}\\\|\_\{2\}\\\|\\widehat\{\\bm\{\\Sigma\}\}\_\{T\}\\\|\_\{\\mathrm\{op\}\}\\\|\{\\bm\{\\xi\}\}\_\{s^\{\\prime\}\}\\\|\_\{2\}\\xrightarrow\{a\.s\.\}0,\\qquad\|\{\\bm\{\\xi\}\}\_\{s\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{\\xi\}\}\_\{s^\{\\prime\}\}\|\\leq\\\|\{\\bm\{\\xi\}\}\_\{s\}\\\|\_\{2\}\\\|\\widehat\{\\bm\{\\Sigma\}\}\_\{T\}\\\|\_\{\\mathrm\{op\}\}\\\|\{\\bm\{\\xi\}\}\_\{s^\{\\prime\}\}\\\|\_\{2\}\\xrightarrow\{a\.s\.\}0\.Consequently,

H^s​s′→a\.s\.𝚫~s⊤​𝚺T​𝚫~s′\.\\widehat\{H\}\_\{ss^\{\\prime\}\}\\xrightarrow\{a\.s\.\}\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\\widetilde\{\\bm\{\\Delta\}\}\_\{s^\{\\prime\}\}\.Fors≤Ss\\leq S,

H^s,S\+1=∑f=12nT,fnT​\(𝚫~s\+𝝃s\)⊤​𝚺^T,f​\(𝒃T\+𝝃T,3−f\),\\widehat\{H\}\_\{s,S\+1\}=\\sum\_\{f=1\}^\{2\}\\frac\{n\_\{T,f\}\}\{n\_\{T\}\}\(\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}\+\{\\bm\{\\xi\}\}\_\{s\}\)^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T,f\}\(\{\\bm\{b\}\}\_\{T\}\+\{\\bm\{\\xi\}\}\_\{T,3\-f\}\),where𝚺^T,f=𝑿T,f​𝑿T,f⊤/nT,f\\widehat\{\\bm\{\\Sigma\}\}\_\{T,f\}=\{\\bm\{X\}\}\_\{T,f\}\{\\bm\{X\}\}\_\{T,f\}^\{\\top\}/n\_\{T,f\}\. For each fold, expanding this product gives

\(𝚫~s\+𝝃s\)⊤​𝚺^T,f​\(𝒃T\+𝝃T,3−f\)\\displaystyle\(\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}\+\{\\bm\{\\xi\}\}\_\{s\}\)^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T,f\}\(\{\\bm\{b\}\}\_\{T\}\+\{\\bm\{\\xi\}\}\_\{T,3\-f\}\)=𝚫~s⊤​𝚺^T,f​𝒃T\+𝚫~s⊤​𝚺^T,f​𝝃T,3−f\+𝝃s⊤​𝚺^T,f​𝒃T\+𝝃s⊤​𝚺^T,f​𝝃T,3−f\\displaystyle\\quad=\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T,f\}\{\\bm\{b\}\}\_\{T\}\+\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T,f\}\{\\bm\{\\xi\}\}\_\{T,3\-f\}\+\{\\bm\{\\xi\}\}\_\{s\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T,f\}\{\\bm\{b\}\}\_\{T\}\+\{\\bm\{\\xi\}\}\_\{s\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T,f\}\{\\bm\{\\xi\}\}\_\{T,3\-f\}→a\.s\.𝚫~s⊤​𝚺T​𝒃T\.\\displaystyle\\quad\\xrightarrow\{a\.s\.\}\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{b\}\}\_\{T\}\.The error terms vanish by the same norm bounds, since𝚺^T,f→a\.s\.𝚺T\\widehat\{\\bm\{\\Sigma\}\}\_\{T,f\}\\xrightarrow\{a\.s\.\}\{\\bm\{\\Sigma\}\}\_\{T\}and𝝃T,3−f→a\.s\.0\{\\bm\{\\xi\}\}\_\{T,3\-f\}\\xrightarrow\{a\.s\.\}0\. Using∑fnT,f/nT=1\\sum\_\{f\}n\_\{T,f\}/n\_\{T\}=1gives

H^s,S\+1→a\.s\.𝚫~s⊤​𝚺T​𝒃T\.\\widehat\{H\}\_\{s,S\+1\}\\xrightarrow\{a\.s\.\}\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{b\}\}\_\{T\}\.Finally,

H^S\+1,S\+1\\displaystyle\\widehat\{H\}\_\{S\+1,S\+1\}=∑f=12nT,fnT​\(𝒃T\+𝝃T,3−f\)⊤​𝚺^T,f​\(𝒃T\+𝝃T,3−f\)\\displaystyle=\\sum\_\{f=1\}^\{2\}\\frac\{n\_\{T,f\}\}\{n\_\{T\}\}\(\{\\bm\{b\}\}\_\{T\}\+\{\\bm\{\\xi\}\}\_\{T,3\-f\}\)^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T,f\}\(\{\\bm\{b\}\}\_\{T\}\+\{\\bm\{\\xi\}\}\_\{T,3\-f\}\)=∑f=12nT,fnT​\[𝒃T⊤​𝚺^T,f​𝒃T\+2​𝒃T⊤​𝚺^T,f​𝝃T,3−f\+𝝃T,3−f⊤​𝚺^T,f​𝝃T,3−f\]\\displaystyle=\\sum\_\{f=1\}^\{2\}\\frac\{n\_\{T,f\}\}\{n\_\{T\}\}\\Big\[\{\\bm\{b\}\}\_\{T\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T,f\}\{\\bm\{b\}\}\_\{T\}\+2\{\\bm\{b\}\}\_\{T\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T,f\}\{\\bm\{\\xi\}\}\_\{T,3\-f\}\+\{\\bm\{\\xi\}\}\_\{T,3\-f\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T,f\}\{\\bm\{\\xi\}\}\_\{T,3\-f\}\\Big\]→a\.s\.𝒃T⊤​𝚺T​𝒃T\.\\displaystyle\\xrightarrow\{a\.s\.\}\{\\bm\{b\}\}\_\{T\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{b\}\}\_\{T\}\.These entrywise limits give

𝑯^→a\.s\.\[𝚫~1⋯𝚫~S𝒃T\]⊤​𝚺T​\[𝚫~1⋯𝚫~S𝒃T\]=𝑸⊤​𝚺T​𝑸=𝑯¯\.\\widehat\{\\bm\{H\}\}\\xrightarrow\{a\.s\.\}\\begin\{bmatrix\}\\widetilde\{\\bm\{\\Delta\}\}\_\{1\}&\\cdots&\\widetilde\{\\bm\{\\Delta\}\}\_\{S\}&\{\\bm\{b\}\}\_\{T\}\\end\{bmatrix\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\\begin\{bmatrix\}\\widetilde\{\\bm\{\\Delta\}\}\_\{1\}&\\cdots&\\widetilde\{\\bm\{\\Delta\}\}\_\{S\}&\{\\bm\{b\}\}\_\{T\}\\end\{bmatrix\}=\{\\bm\{Q\}\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{Q\}\}=\\overline\{\{\\bm\{H\}\}\}\.
The limiting weights minimize𝒘⊤​𝑯¯​𝒘\{\\bm\{w\}\}^\{\\top\}\\overline\{\{\\bm\{H\}\}\}\{\\bm\{w\}\}subject to𝒘≥0\{\\bm\{w\}\}\\geq 0and𝟏⊤​𝒘=1\\mathbf\{1\}^\{\\top\}\{\\bm\{w\}\}=1\. The feasible set is compact and the objective is strictly convex because𝑯¯≻0\\overline\{\{\\bm\{H\}\}\}\\succ 0, so its minimizer𝒘⋆\{\\bm\{w\}\}^\{\\star\}exists and is unique\. Every feasible vector satisfies‖𝒘‖2≤1\\\|\{\\bm\{w\}\}\\\|\_\{2\}\\leq 1, giving the uniform bound

sup𝒘≥0𝟏⊤​𝒘=1\|L⁡\(𝒘\)−1nT​‖𝜺T‖22−𝒘⊤​𝑯¯​𝒘\|≤2​‖𝒈^‖2\+‖𝑯^−𝑯¯‖op→a\.s\.0\.\\sup\_\{\\begin\{subarray\}\{c\}\{\\bm\{w\}\}\\geq 0\\\\ \\mathbf\{1\}^\{\\top\}\{\\bm\{w\}\}=1\\end\{subarray\}\}\\left\|L\(\{\\bm\{w\}\}\)\-\\frac\{1\}\{n\_\{T\}\}\\\|\{\\bm\{\\varepsilon\}\}\_\{T\}\\\|\_\{2\}^\{2\}\-\{\\bm\{w\}\}^\{\\top\}\\overline\{\{\\bm\{H\}\}\}\{\\bm\{w\}\}\\right\|\\leq 2\\\|\\widehat\{\\bm\{g\}\}\\\|\_\{2\}\+\\\|\\widehat\{\\bm\{H\}\}\-\\overline\{\{\\bm\{H\}\}\}\\\|\_\{\\mathrm\{op\}\}\\xrightarrow\{a\.s\.\}0\.To show that the empirical minimizer converges, take any feasible𝒘\{\\bm\{w\}\}\. The segment from𝒘⋆\{\\bm\{w\}\}^\{\\star\}to𝒘\{\\bm\{w\}\}is feasible, and optimality at𝒘⋆\{\\bm\{w\}\}^\{\\star\}makes the directional derivative nonnegative\. Thus\(𝒘−𝒘⋆\)⊤​𝑯¯​𝒘⋆≥0\(\{\\bm\{w\}\}\-\{\\bm\{w\}\}^\{\\star\}\)^\{\\top\}\\overline\{\{\\bm\{H\}\}\}\{\\bm\{w\}\}^\{\\star\}\\geq 0, and

𝒘⊤​𝑯¯​𝒘−\(𝒘⋆\)⊤​𝑯¯​𝒘⋆\\displaystyle\{\\bm\{w\}\}^\{\\top\}\\overline\{\{\\bm\{H\}\}\}\{\\bm\{w\}\}\-\(\{\\bm\{w\}\}^\{\\star\}\)^\{\\top\}\\overline\{\{\\bm\{H\}\}\}\{\\bm\{w\}\}^\{\\star\}=2​\(𝒘−𝒘⋆\)⊤​𝑯¯​𝒘⋆\+\(𝒘−𝒘⋆\)⊤​𝑯¯​\(𝒘−𝒘⋆\)\\displaystyle=2\(\{\\bm\{w\}\}\-\{\\bm\{w\}\}^\{\\star\}\)^\{\\top\}\\overline\{\{\\bm\{H\}\}\}\{\\bm\{w\}\}^\{\\star\}\+\(\{\\bm\{w\}\}\-\{\\bm\{w\}\}^\{\\star\}\)^\{\\top\}\\overline\{\{\\bm\{H\}\}\}\(\{\\bm\{w\}\}\-\{\\bm\{w\}\}^\{\\star\}\)≥λmin​\(𝑯¯\)​‖𝒘−𝒘⋆‖22\.\\displaystyle\\geq\\lambda\_\{\\min\}\(\\overline\{\{\\bm\{H\}\}\}\)\\\|\{\\bm\{w\}\}\-\{\\bm\{w\}\}^\{\\star\}\\\|\_\{2\}^\{2\}\.SinceL⁡\(𝒘^\)≤L⁡\(𝒘⋆\)L\(\\widehat\{\\bm\{w\}\}\)\\leq L\(\{\\bm\{w\}\}^\{\\star\}\), applying the uniform bound to both weights gives

λmin​\(𝑯¯\)​‖𝒘^−𝒘⋆‖22\\displaystyle\\lambda\_\{\\min\}\(\\overline\{\{\\bm\{H\}\}\}\)\\\|\\widehat\{\\bm\{w\}\}\-\{\\bm\{w\}\}^\{\\star\}\\\|\_\{2\}^\{2\}≤𝒘^⊤​𝑯¯​𝒘^−\(𝒘⋆\)⊤​𝑯¯​𝒘⋆\\displaystyle\\leq\\widehat\{\\bm\{w\}\}^\{\\top\}\\overline\{\{\\bm\{H\}\}\}\\widehat\{\\bm\{w\}\}\-\(\{\\bm\{w\}\}^\{\\star\}\)^\{\\top\}\\overline\{\{\\bm\{H\}\}\}\{\\bm\{w\}\}^\{\\star\}≤4​‖𝒈^‖2\+2​‖𝑯^−𝑯¯‖op→a\.s\.0\.\\displaystyle\\leq 4\\\|\\widehat\{\\bm\{g\}\}\\\|\_\{2\}\+2\\\|\\widehat\{\\bm\{H\}\}\-\\overline\{\{\\bm\{H\}\}\}\\\|\_\{\\mathrm\{op\}\}\\xrightarrow\{a\.s\.\}0\.Becauseλmin​\(𝑯¯\)\>0\\lambda\_\{\\min\}\(\\overline\{\{\\bm\{H\}\}\}\)\>0, this proves

𝒘^→a\.s\.𝒘⋆=arg⁡min𝒘≥0𝟏⊤​𝒘=1​𝒘⊤​𝑯¯​𝒘\.\\widehat\{\\bm\{w\}\}\\xrightarrow\{a\.s\.\}\{\\bm\{w\}\}^\{\\star\}=\\arg\\min\_\{\\begin\{subarray\}\{c\}\{\\bm\{w\}\}\\geq 0\\\\ \\mathbf\{1\}^\{\\top\}\{\\bm\{w\}\}=1\\end\{subarray\}\}\{\\bm\{w\}\}^\{\\top\}\\overline\{\{\\bm\{H\}\}\}\{\\bm\{w\}\}\.
Returning to the risk expansion, every weight lies in\[0,1\]\[0,1\]\. Terms involving estimation errors therefore vanish\. For example,

\|w^s​w^s′​𝚫~s⊤​𝚺T​𝝃s′\|≤‖𝚫~s‖2​‖𝚺T‖op​‖𝝃s′‖2→a\.s\.0,\|\\widehat\{w\}\_\{s\}\\widehat\{w\}\_\{s^\{\\prime\}\}\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{\\xi\}\}\_\{s^\{\\prime\}\}\|\\leq\\\|\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}\\\|\_\{2\}\\\|\{\\bm\{\\Sigma\}\}\_\{T\}\\\|\_\{\\mathrm\{op\}\}\\\|\{\\bm\{\\xi\}\}\_\{s^\{\\prime\}\}\\\|\_\{2\}\\xrightarrow\{a\.s\.\}0,and terms involving two errors are bounded by‖𝚺T‖op\\\|\{\\bm\{\\Sigma\}\}\_\{T\}\\\|\_\{\\mathrm\{op\}\}times the product of their norms\. Since there are finitely many terms, substituting the weight limits yields

ℛT​\(𝜽^AdaST\)\\displaystyle\\mathcal\{R\}\_\{T\}\(\\widehat\{\{\\bm\{\\theta\}\}\}\_\{\\text\{AdaST\}\}\)→a\.s\.σT2\+∑s=1S∑s′=1Sws⋆​ws′⋆​𝚫~s⊤​𝚺T​𝚫~s′\\displaystyle\\xrightarrow\{a\.s\.\}\\sigma\_\{T\}^\{2\}\+\\sum\_\{s=1\}^\{S\}\\sum\_\{s^\{\\prime\}=1\}^\{S\}w\_\{s\}^\{\\star\}w\_\{s^\{\\prime\}\}^\{\\star\}\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\\widetilde\{\\bm\{\\Delta\}\}\_\{s^\{\\prime\}\}\+2∑s=1Sws⋆wS\+1⋆𝚫~s⊤𝚺T𝒃T\+\(wS\+1⋆\)2𝒃T⊤𝚺T𝒃T\\displaystyle\\quad\+2\\sum\_\{s=1\}^\{S\}w\_\{s\}^\{\\star\}w\_\{S\+1\}^\{\\star\}\\widetilde\{\\bm\{\\Delta\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{b\}\}\_\{T\}\+\(w\_\{S\+1\}^\{\\star\}\)^\{2\}\{\\bm\{b\}\}\_\{T\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{b\}\}\_\{T\}=σT2\+\(𝒘⋆\)⊤​𝑯¯​𝒘⋆\.\\displaystyle=\\sigma\_\{T\}^\{2\}\+\(\{\\bm\{w\}\}^\{\\star\}\)^\{\\top\}\\overline\{\{\\bm\{H\}\}\}\{\\bm\{w\}\}^\{\\star\}\.The target\-only estimator has limiting riskσT2\+𝒃T⊤​𝚺T​𝒃T=σT2\+H¯S\+1,S\+1\\sigma\_\{T\}^\{2\}\+\{\\bm\{b\}\}\_\{T\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{T\}\{\\bm\{b\}\}\_\{T\}=\\sigma\_\{T\}^\{2\}\+\\overline\{H\}\_\{S\+1,S\+1\}\. Since𝐞S\+1\\mathbf\{e\}\_\{S\+1\}is feasible, optimality gives

\(𝒘⋆\)⊤​𝑯¯​𝒘⋆≤𝐞S\+1⊤​𝑯¯​𝐞S\+1=H¯S\+1,S\+1\.\(\{\\bm\{w\}\}^\{\\star\}\)^\{\\top\}\\overline\{\{\\bm\{H\}\}\}\{\\bm\{w\}\}^\{\\star\}\\leq\\mathbf\{e\}\_\{S\+1\}^\{\\top\}\\overline\{\{\\bm\{H\}\}\}\\mathbf\{e\}\_\{S\+1\}=\\overline\{H\}\_\{S\+1,S\+1\}\.Equality holds exactly when𝒘⋆=𝐞S\+1\{\\bm\{w\}\}^\{\\star\}=\\mathbf\{e\}\_\{S\+1\}, since the limiting minimizer is unique\. This completes the proof\.

## Appendix EFinite\-Sample Decay of the Target Weight

We quantify thefinite\-sampleerror under the Gaussian\-design and fixed\-subspace assumptions of Theorem[1](https://arxiv.org/html/2610.33442#Thmtheorem1)\. Writew^T=w^S\+1\\widehat\{w\}\_\{T\}=\\widehat\{w\}\_\{S\+1\},λ=λmin​\(𝑯¯\)\>0\\lambda=\\lambda\_\{\\min\}\(\\overline\{\{\\bm\{H\}\}\}\)\>0, and

rn2=∑s=1S1ns−k−1\+∑f=121nT,f−k−1\.r\_\{n\}^\{2\}=\\sum\_\{s=1\}^\{S\}\\frac\{1\}\{n\_\{s\}\-k\-1\}\+\\sum\_\{f=1\}^\{2\}\\frac\{1\}\{n\_\{T,f\}\-k\-1\}\.Assume all source and target\-fold sample sizes exceedk\+1k\+1\. Below,CCdenotes a constant depending only on the fixed population parameters and dimensions\.

For source domainss, let𝝃s=𝜽^s−𝜽s∘\{\\bm\{\\xi\}\}\_\{s\}=\\widehat\{\\bm\{\\theta\}\}\_\{s\}\-\{\\bm\{\\theta\}\}\_\{s\}^\{\\circ\}\. The projected residual has varianceνs2=σs2\+𝒃s⊤​𝚺s​𝒃s\\nu\_\{s\}^\{2\}=\\sigma\_\{s\}^\{2\}\+\{\\bm\{b\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{s\}\{\\bm\{b\}\}\_\{s\}and is independent of the projected covariates under Gaussian design\. The Gaussian inverse\-Gram moment gives

𝔼​‖𝝃s‖22=νs2ns−k−1​tr⁡\[\(𝑼s⊤​𝚺s​𝑼s\)−1\]\.\\mathbb\{E\}\\\|\{\\bm\{\\xi\}\}\_\{s\}\\\|\_\{2\}^\{2\}=\\frac\{\\nu\_\{s\}^\{2\}\}\{n\_\{s\}\-k\-1\}\\operatorname\{tr\}\\\!\\left\[\(\{\\bm\{U\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{s\}\{\\bm\{U\}\}\_\{s\}\)^\{\-1\}\\right\]\.\(8\)The same identity applies to target\-fold estimation errors, withs=Ts=Tand sample sizenT,fn\_\{T,f\}\.

Next, use the out\-of\-fold matrices𝑯^\\widehat\{\\bm\{H\}\},𝒈^\\widehat\{\\bm\{g\}\}, and𝑬^f\\widehat\{\\bm\{E\}\}\_\{f\}defined in Appendix[D](https://arxiv.org/html/2610.33442#A4)\. Set

pf=nT,f/nT,𝚵f=𝑬^f−𝑸,𝚺^T,f=𝑿T,f​𝑿T,f⊤nT,f\.p\_\{f\}=n\_\{T,f\}/n\_\{T\},\\qquad\\bm\{\\Xi\}\_\{f\}=\\widehat\{\\bm\{E\}\}\_\{f\}\-\{\\bm\{Q\}\},\\qquad\\widehat\{\\bm\{\\Sigma\}\}\_\{T,f\}=\\frac\{\{\\bm\{X\}\}\_\{T,f\}\{\\bm\{X\}\}\_\{T,f\}^\{\\top\}\}\{n\_\{T,f\}\}\.Since𝚵f\\bm\{\\Xi\}\_\{f\}contains source fitting errors and the target fitting error from fold3−f3\-f, it is independent of the observations in foldff\. Expanding𝔼​‖𝚵f‖F2=∑s=1S𝔼​‖𝝃s‖22\+𝔼​‖𝝃T,3−f‖22\\mathbb\{E\}\\\|\\bm\{\\Xi\}\_\{f\}\\\|\_\{\\mathrm\{F\}\}^\{2\}=\\sum\_\{s=1\}^\{S\}\\mathbb\{E\}\\\|\{\\bm\{\\xi\}\}\_\{s\}\\\|^\{2\}\_\{2\}\+\\mathbb\{E\}\\\|\{\\bm\{\\xi\}\}\_\{T,3\-f\}\\\|^\{2\}\_\{2\}and with Equation[8](https://arxiv.org/html/2610.33442#A5.E8)gives𝔼​‖𝚵f‖F2≤C​rn2\\mathbb\{E\}\\\|\\bm\{\\Xi\}\_\{f\}\\\|\_\{\\mathrm\{F\}\}^\{2\}\\leq Cr\_\{n\}^\{2\}, whereC=maxs⁡νs2​tr⁡\[\(𝑼s⊤​𝚺s​𝑼s\)−1\]C=\\max\_\{s\}\\nu\_\{s\}^\{2\}\\operatorname\{tr\}\\\!\\left\[\(\{\\bm\{U\}\}\_\{s\}^\{\\top\}\{\\bm\{\\Sigma\}\}\_\{s\}\{\\bm\{U\}\}\_\{s\}\)^\{\-1\}\\right\]\. Expanding the quadratic gives

𝑯^−𝑯¯\\displaystyle\\widehat\{\\bm\{H\}\}\-\\overline\{\{\\bm\{H\}\}\}=𝑸⊤​\(𝚺^T−𝚺T\)​𝑸\\displaystyle=\{\\bm\{Q\}\}^\{\\top\}\(\\widehat\{\\bm\{\\Sigma\}\}\_\{T\}\-\{\\bm\{\\Sigma\}\}\_\{T\}\)\{\\bm\{Q\}\}\+∑f=12pf\[𝑸⊤𝚺^T,f𝚵f\+𝚵f⊤𝚺^T,f𝑸\+𝚵f⊤𝚺^T,f𝚵f\]\.\\displaystyle\\quad\+\\sum\_\{f=1\}^\{2\}p\_\{f\}\\left\[\{\\bm\{Q\}\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T,f\}\\bm\{\\Xi\}\_\{f\}\+\\bm\{\\Xi\}\_\{f\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T,f\}\{\\bm\{Q\}\}\+\\bm\{\\Xi\}\_\{f\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T,f\}\\bm\{\\Xi\}\_\{f\}\\right\]\.Since the sample covariance𝚺^T\\widehat\{\{\\bm\{\\Sigma\}\}\}\_\{T\}concentrates around its population counterpart𝚺T\{\\bm\{\\Sigma\}\}\_\{T\}, Gaussian covariance moment bounds yield𝔼‖𝑸⊤\(𝚺^T−𝚺T\)𝑸‖≤CnT−1/2\\mathbb\{E\}\\left\\\|\{\\bm\{Q\}\}^\{\\top\}\(\\widehat\{\{\\bm\{\\Sigma\}\}\}\_\{T\}\-\{\\bm\{\\Sigma\}\}\_\{T\}\)\{\\bm\{Q\}\}\\right\\\|\\leq Cn\_\{T\}^\{\-1/2\}\. Since𝔼​‖𝚺^T,f‖op≤tr⁡\(𝚺T\)\\mathbb\{E\}\\\|\\widehat\{\\bm\{\\Sigma\}\}\_\{T,f\}\\\|\_\{\\mathrm\{op\}\}\\leq\\operatorname\{tr\}\(\{\\bm\{\\Sigma\}\}\_\{T\}\), then𝔼​‖𝑸⊤​𝚺^T,f​𝚵f‖op≤𝔼​‖𝑸‖op​tr⁡\(𝚺T\)​𝔼​‖𝚵f‖F2=𝒪⁡\(rn\)\\mathbb\{E\}\\\|\{\\bm\{Q\}\}^\{\\top\}\\widehat\{\\bm\{\\Sigma\}\}\_\{T,f\}\\bm\{\\Xi\}\_\{f\}\\\|\_\{\\mathrm\{op\}\}\\leq\\mathbb\{E\}\\\|\{\\bm\{Q\}\}\\\|\_\{\\mathrm\{op\}\}\\operatorname\{tr\}\(\{\\bm\{\\Sigma\}\}\_\{T\}\)\\sqrt\{\\mathbb\{E\}\\\|\\bm\{\\Xi\}\_\{f\}\\\|\_\{\\mathrm\{F\}\}^\{2\}\}=\\mathcal\{O\}\(r\_\{n\}\)\. Similarly, the quadratic term is bounded by𝒪⁡\(rn2\)\\mathcal\{O\}\(r\_\{n\}^\{2\}\)\.

For the variance term,

𝒈^=∑f=12pf​\(𝑸\+𝚵f\)⊤​𝑿T,f​𝜺T,fnT,f,𝔼​‖𝑿T,f​𝜺T,fnT,f‖22=σT2​tr⁡\(𝚺T\)nT,f\.\\widehat\{\\bm\{g\}\}=\\sum\_\{f=1\}^\{2\}p\_\{f\}\(\{\\bm\{Q\}\}\+\\bm\{\\Xi\}\_\{f\}\)^\{\\top\}\\frac\{\{\\bm\{X\}\}\_\{T,f\}\{\\bm\{\\varepsilon\}\}\_\{T,f\}\}\{n\_\{T,f\}\},\\qquad\\mathbb\{E\}\\\|\\frac\{\{\\bm\{X\}\}\_\{T,f\}\{\\bm\{\\varepsilon\}\}\_\{T,f\}\}\{n\_\{T,f\}\}\\\|\_\{2\}^\{2\}=\\frac\{\\sigma\_\{T\}^\{2\}\\operatorname\{tr\}\(\{\\bm\{\\Sigma\}\}\_\{T\}\)\}\{n\_\{T,f\}\}\.Similar calculations as for𝑯^−𝑯¯\\widehat\{\\bm\{H\}\}\-\\overline\{\{\\bm\{H\}\}\}give𝔼​‖𝒈^‖2≤C⁡\(1\+rn\)​∑fpf/nT,f\\mathbb\{E\}\\\|\\widehat\{\\bm\{g\}\}\\\|\_\{2\}\\leq C\(1\+r\_\{n\}\)\\sum\_\{f\}p\_\{f\}/\\sqrt\{n\_\{T,f\}\}\. Together,

𝔼⁡\[‖𝑯^−𝑯¯‖op\+‖𝒈^‖2\]≤C⁡\(rn\+rn2\)\.\\mathbb\{E\}\\left\[\\\|\\widehat\{\\bm\{H\}\}\-\\overline\{\{\\bm\{H\}\}\}\\\|\_\{\\mathrm\{op\}\}\+\\\|\\widehat\{\\bm\{g\}\}\\\|\_\{2\}\\right\]\\leq C\(r\_\{n\}\+r\_\{n\}^\{2\}\)\.\(9\)
Finally, let

𝒅=𝒘^−𝒘⋆\.\{\\bm\{d\}\}=\\widehat\{\{\\bm\{w\}\}\}\-\{\\bm\{w\}\}^\{\\star\}\.Since𝒘^\\widehat\{\{\\bm\{w\}\}\}minimizes the empirical quadratic objective over the simplex and𝒘⋆\{\\bm\{w\}\}^\{\\star\}is feasible, the first\-order optimality condition gives

\(𝒘⋆−𝒘^\)⊤​\(𝑯^​𝒘^−𝒈^\)≥0,\(\{\\bm\{w\}\}^\{\\star\}\-\\widehat\{\{\\bm\{w\}\}\}\)^\{\\top\}\\left\(\\widehat\{\{\\bm\{H\}\}\}\\widehat\{\{\\bm\{w\}\}\}\-\\widehat\{\{\\bm\{g\}\}\}\\right\)\\geq 0,or equivalently,

𝒅⊤​\(𝑯^​𝒘^−𝒈^\)≤0\.\{\\bm\{d\}\}^\{\\top\}\\left\(\\widehat\{\{\\bm\{H\}\}\}\\widehat\{\{\\bm\{w\}\}\}\-\\widehat\{\{\\bm\{g\}\}\}\\right\)\\leq 0\.Similarly, since𝒘⋆\{\\bm\{w\}\}^\{\\star\}minimizes the population quadratic objective over the simplex and𝒘^\\widehat\{\{\\bm\{w\}\}\}is feasible,

\(𝒘^−𝒘⋆\)⊤​𝑯¯​𝒘⋆≥0,\(\\widehat\{\{\\bm\{w\}\}\}\-\{\\bm\{w\}\}^\{\\star\}\)^\{\\top\}\\overline\{\{\\bm\{H\}\}\}\{\\bm\{w\}\}^\{\\star\}\\geq 0,so that

𝒅⊤​𝑯¯​𝒘⋆≥0\.\{\\bm\{d\}\}^\{\\top\}\\overline\{\{\\bm\{H\}\}\}\{\\bm\{w\}\}^\{\\star\}\\geq 0\.
Sinceλ=λmin​\(𝑯¯\)\\lambda=\\lambda\_\{\\min\}\(\\overline\{\{\\bm\{H\}\}\}\), we have

λ​‖𝒅‖22\\displaystyle\\lambda\\\|\{\\bm\{d\}\}\\\|\_\{2\}^\{2\}≤𝒅⊤​𝑯¯​𝒅≤𝒅⊤​𝑯¯​𝒘^\\displaystyle\\leq\{\\bm\{d\}\}^\{\\top\}\\overline\{\{\\bm\{H\}\}\}\{\\bm\{d\}\}\\leq\{\\bm\{d\}\}^\{\\top\}\\overline\{\{\\bm\{H\}\}\}\\widehat\{\{\\bm\{w\}\}\}=𝒅⊤​\[\(𝑯¯−𝑯^\)​𝒘^\+𝑯^​𝒘^\]\\displaystyle=\{\\bm\{d\}\}^\{\\top\}\\left\[\(\\overline\{\{\\bm\{H\}\}\}\-\\widehat\{\{\\bm\{H\}\}\}\)\\widehat\{\{\\bm\{w\}\}\}\+\\widehat\{\{\\bm\{H\}\}\}\\widehat\{\{\\bm\{w\}\}\}\\right\]=𝒅⊤​\[\(𝑯¯−𝑯^\)​𝒘^\+𝒈^\]\+𝒅⊤​\(𝑯^​𝒘^−𝒈^\)\\displaystyle=\{\\bm\{d\}\}^\{\\top\}\\left\[\(\\overline\{\{\\bm\{H\}\}\}\-\\widehat\{\{\\bm\{H\}\}\}\)\\widehat\{\{\\bm\{w\}\}\}\+\\widehat\{\{\\bm\{g\}\}\}\\right\]\+\{\\bm\{d\}\}^\{\\top\}\\left\(\\widehat\{\{\\bm\{H\}\}\}\\widehat\{\{\\bm\{w\}\}\}\-\\widehat\{\{\\bm\{g\}\}\}\\right\)≤𝒅⊤​\[\(𝑯¯−𝑯^\)​𝒘^\+𝒈^\]\\displaystyle\\leq\{\\bm\{d\}\}^\{\\top\}\\left\[\(\\overline\{\{\\bm\{H\}\}\}\-\\widehat\{\{\\bm\{H\}\}\}\)\\widehat\{\{\\bm\{w\}\}\}\+\\widehat\{\{\\bm\{g\}\}\}\\right\]≤\(‖𝑯^−𝑯¯‖op\+‖𝒈^‖2\)​‖𝒅‖2\.\\displaystyle\\leq\\left\(\\\|\\widehat\{\{\\bm\{H\}\}\}\-\\overline\{\{\\bm\{H\}\}\}\\\|\_\{\\mathrm\{op\}\}\+\\\|\\widehat\{\{\\bm\{g\}\}\}\\\|\_\{2\}\\right\)\\\|\{\\bm\{d\}\}\\\|\_\{2\}\.
Therefore‖𝒘^−𝒘⋆‖2≤\(‖𝑯^−𝑯¯‖op\+‖𝒈^‖2\)/λ\\\|\\widehat\{\\bm\{w\}\}\-\{\\bm\{w\}\}^\{\\star\}\\\|\_\{2\}\\leq\(\\\|\\widehat\{\\bm\{H\}\}\-\\overline\{\{\\bm\{H\}\}\}\\\|\_\{\\mathrm\{op\}\}\+\\\|\\widehat\{\\bm\{g\}\}\\\|\_\{2\}\)/\\lambda\. Taking expectations gives

𝔼​‖𝒘^−𝒘⋆‖2≤Cλ​\(rn\+rn2\)\.\\mathbb\{E\}\\\|\\widehat\{\\bm\{w\}\}\-\{\\bm\{w\}\}^\{\\star\}\\\|\_\{2\}\\leq\\frac\{C\}\{\\lambda\}\(r\_\{n\}\+r\_\{n\}^\{2\}\)\.\(10\)In any setting whose population solution haswT⋆=0w\_\{T\}^\{\\star\}=0, simplex nonnegativity then gives

0≤𝔼​w^T≤Cλ​\(rn\+rn2\)\.0\\leq\\mathbb\{E\}\\widehat\{w\}\_\{T\}\\leq\\frac\{C\}\{\\lambda\}\(r\_\{n\}\+r\_\{n\}^\{2\}\)\.\(11\)In the empirically\-interested regime where source samples are more abundant than target samples, withns≥c​nTn\_\{s\}\\geq cn\_\{T\}for a fixedc≥1c\\geq 1andpf→ρf∈\(0,1\)p\_\{f\}\\to\\rho\_\{f\}\\in\(0,1\)forf=1,2f=1,2, we havern=O\(nT−1/2\)r\_\{n\}=O\(n\_\{T\}^\{\-1/2\}\)\. Consequently, ifwT⋆=0w\_\{T\}^\{\\star\}=0,

𝔼w^T=O\(nT−1/2\)\.\\mathbb\{E\}\\widehat\{w\}\_\{T\}=O\(n\_\{T\}^\{\-1/2\}\)\.This establishes a vanishing upper bound on the expected target weight\. More generally, the same argument gives𝔼\|w^T−wT⋆\|=O\(nT−1/2\)\\mathbb\{E\}\|\\widehat\{w\}\_\{T\}\-w\_\{T\}^\{\\star\}\|=O\(n\_\{T\}^\{\-1/2\}\), including whenwT⋆\>0w\_\{T\}^\{\\star\}\>0\.