FCPRAG: Fusion-Controller Parametric Retrieval-Augmented Generation for Stable Multi-Passage LoRA Injection

# FCPRAG: Fusion-Controller Parametric Retrieval-Augmented Generation for Stable Multi-Passage LoRA Injection
Source: [https://arxiv.org/html/2608.21750](https://arxiv.org/html/2608.21750)
Jindong LiAffiliation:The Hong Kong University of Science and Technology \(Guangzhou\)YI DINGAffiliation:The Hong Kong University of Science and Technology \(Guangzhou\)Nie XiaojianAffiliation:The Hong Kong University of Science and Technology \(Guangzhou\)Rong FuAffiliation:University of MacauShuangyong SongAffiliation:Institute of Artificial Intelligence \(TeleAI\), China Telecomjzhu997@connect\.hkust\-gz\.edu\.cn menglinyang@hkust\-gz\.edu\.cnHaowei HeAffiliation:Institute of Artificial Intelligence \(TeleAI\), China Telecomjzhu997@connect\.hkust\-gz\.edu\.cn menglinyang@hkust\-gz\.edu\.cnMenglin YangAffiliation:The Hong Kong University of Science and Technology \(Guangzhou\)

###### Abstract

Parametric retrieval\-augmented generation \(PRAG\) injects retrieved evidence into a large language model \(LLM\) through passage\-specific LoRA adapters, reducing reliance on long in\-context prompts\. When multiple passages are retrieved for the same query, however,*evidence\-level fusion*becomes a bottleneck: equal\-weight merging can amplify weak or conflicting evidence, and translating retrieval signals into fusion weights often requires fragile global tuning\. We proposeFCPRAG, a fusion\-controlled parametric RAG framework that adds a lightweight controller for retrieval\-conditioned, sample\-level adapter fusion\. The controller predicts per\-passage fusion scores together with sample\-level calibration signals, including a mixing gate and an adaptive temperature, enabling fusion that stays selective under informative retrieval signals and conservative under uncertainty\.FCPRAGis trained with merge\-aware supervision derived from each adapter’s marginal contribution within a multi\-adapter merge, using training data only\. We further show that a single dataset\-level temperature is suboptimal under heteroscedastic retrieval uncertainty, motivating sample\-level adaptation\. Experiments on HotpotQA, 2WikiMultiHopQA, PopQA, and ComplexWebQuestions \(CWQ\) across three LLM backbones show thatFCPRAGconsistently improves F1 over standard RAG and parametric RAG baselines, with gains of up to 4\.65% on 2WikiMultiHopQA and 7\.55% on CWQ, while also reducing tuning cost and improving robustness under retrieval perturbations\.

22footnotetext:Corresponding author\.## 1Introduction

Retrieval\-augmented generation \(RAG\) improves the factuality of large language models \(LLMs\) by retrieving external evidence and conditioning generation on the retrieved content[Lewis et al\. 2020](https://arxiv.org/html/2608.21750#bib.bib21);[Karpukhin et al\. 2020](https://arxiv.org/html/2608.21750#bib.bib17);[Izacard and Grave 2021](https://arxiv.org/html/2608.21750#bib.bib12)\. However, classical in\-context RAG increases prompt length with the number and size of retrieved passages, leading to higher inference cost and potential degradation under long or conflicting contexts\. Recent RAG research therefore explores stronger retrieval and generation control to reduce redundancy and error propagation[Jiang et al\. 2023](https://arxiv.org/html/2608.21750#bib.bib15);[Asai et al\. 2024](https://arxiv.org/html/2608.21750#bib.bib2), but the fundamental trade\-off remains: multi\-evidence reasoning often benefits from more context, while longer contexts are expensive and fragile\.

Parametric RAG \(PRAG\)[Su et al\. 2025a](https://arxiv.org/html/2608.21750#bib.bib33)offers an alternative: instead of appending passages to the prompt, it*parameterizes*each passage into lightweight LoRA weights[Hu et al\. 2022](https://arxiv.org/html/2608.21750#bib.bib10)during an offline stage, and loads these passage\-specific parameters at test time to inject knowledge into the model\. This design reduces online context length and admits an optional combine setting that also appends retrieved passages to the input prompt \(Section[3\.1](https://arxiv.org/html/2608.21750#S3.SS1)\)\. Recent PRAG\-family methods further reduce cost by using translators or routing to reuse a small set of latent experts[Tan et al\. 2025](https://arxiv.org/html/2608.21750#bib.bib36);[Su et al\. 2025b](https://arxiv.org/html/2608.21750#bib.bib34)\.

While these methods successfully reduce inference cost by moving evidence from the prompt into model parameters, they introduce a new challenge:*multi\-evidence fusion*\. For many knowledge\-intensive queries, retrieval returns multiple relevant passages \(often partially relevant, redundant, or mutually inconsistent\)\. In PRAG\-family systems, this yields multiple passage\-specific LoRA experts that must be merged before generation\. Uniform fusion \(equal weights\) can inject noise from weak passages, while hand\-tuned fusion heuristics \(e\.g\., globally set softmax temperature or blending factors\) often fail to generalize across queries, tasks, and model scales\. This sensitivity affects both efficiency and robustness: in PRAG\-style parametric injection, the fusion weights directly control which evidence is written into the model’s effective parameters at test time, so even small calibration mismatches can over\- or under\-inject passage updates \(Figure[2](https://arxiv.org/html/2608.21750#S5.F2)\(a\)\)\. As a result, PRAG\-family pipelines frequently rely on expensive grid search over global temperature and blending coefficients to reach strong performance\.

To understand why this grid search is difficult to eliminate, we identify three properties that make this fusion bottleneck particularly challenging in parametric RAG\.First, evidence utility is highly sample\-dependent\.Even under a fixed retriever, the same query can surface a mix of strongly supportive, weakly relevant, redundant, or distracting passages, and the*marginal contribution*of each passage\-specific adapter \(estimated by removing one adapter from the merge and measuring the F1 drop\) can vary substantially across instances and subtasks\. Uniform fusion therefore tends to either dilute a dominant adapter \(when a single passage is decisive\) or over\-inject weak adapters \(when retrieval is noisy\)\. For example, in bridge\-style multi\-hop QA, one retrieved passage may contain the bridge entity needed to answer the question, while others are distractors\.Second, parametric injection amplifies fusion mistakes\.Unlike in\-context conditioning, where the model may ignore or down\-weight unhelpful tokens at generation time, parametric RAG fuses evidence through weight\-space updates: an over\-weighted passage adapter directly alters the model’s effective parameters for the current query, making conflicts and noise harder to “wash out” via prompting\.Third, mapping scores to weights is itself a calibration problem\.Even if a scoring model ranks passages reasonably well, the optimal sharpness of the resulting distribution \(temperature\) and the degree of deviation from uniform fusion \(blending\) can change across backbones, tasks, and retrieval quality, which is why grid search is often needed to stabilize performance\. This motivates learning*per\-sample*calibration so that fusion can be selective when confident and conservative when uncertain, while avoiding costly per\-model tuning \(Figures[2](https://arxiv.org/html/2608.21750#S5.F2)and[3](https://arxiv.org/html/2608.21750#S5.F3)\)\.

Table 1:Capability comparison across RAG, PRAG\-family systems, and FCPRAG\.MethodParametricevidence injectionFusion inparameter spaceSample\-levelcalibrated fusionUniform\-priorfallbackRAG[Lewis et al\. 2020](https://arxiv.org/html/2608.21750#bib.bib21)××××PRAG[Su et al\. 2025a](https://arxiv.org/html/2608.21750#bib.bib33)✓✓××DyPRAG[Tan et al\. 2025](https://arxiv.org/html/2608.21750#bib.bib36)✓✓××Poly\-PRAG[Su et al\. 2025b](https://arxiv.org/html/2608.21750#bib.bib34)✓✓××FCPRAG\(Ours\)✓✓✓✓

At a high level, this fusion problem resembles model and adapter merging[Wortsman et al\. 2022](https://arxiv.org/html/2608.21750#bib.bib38);[Ilharco et al\. 2022](https://arxiv.org/html/2608.21750#bib.bib11);[Yadav et al\. 2023](https://arxiv.org/html/2608.21750#bib.bib39);[Pfeiffer et al\. 2021](https://arxiv.org/html/2608.21750#bib.bib27);[Ainsworth et al\. 2022](https://arxiv.org/html/2608.21750#bib.bib1);[Jin et al\. 2022](https://arxiv.org/html/2608.21750#bib.bib16);[Stoica et al\. 2023](https://arxiv.org/html/2608.21750#bib.bib32);[Izmailov et al\. 2018](https://arxiv.org/html/2608.21750#bib.bib14);[Matena and Raffel 2022](https://arxiv.org/html/2608.21750#bib.bib25)\. However, PRAG\-family fusion is fundamentally*retrieval\-conditioned*: the adapters to be merged are tied to passages retrieved for a specific query, and the appropriate confidence in a peaked vs\. conservative weighting can change from sample to sample\. This motivates treating fusion as a learned, calibrated decision rather than a fixed global heuristic, analogous in spirit to classical calibration of predictive distributions[Guo et al\. 2017](https://arxiv.org/html/2608.21750#bib.bib5);[Platt et al\. 1999](https://arxiv.org/html/2608.21750#bib.bib28);[Niculescu\-Mizil and Caruana 2005](https://arxiv.org/html/2608.21750#bib.bib26);[Kuleshov et al\. 2018](https://arxiv.org/html/2608.21750#bib.bib19)\.

To solve the above challenges, we proposeFCPRAGwhich introduces a learned*Fusion Controller*into the PRAG inference pipeline\. The controller predicts sample\-level fusion weights for the retrieved passages and jointly learns per\-sample calibration signals to stabilize the mapping from raw fusion scores to final merge weights\. Importantly,FCPRAGis designed as a plug\-in: it does not change the underlying PRAG offline parameterization, retrieval, or generation, and can replace the merge step in PRAG/DyPRAG with minimal engineering\. We empirically validate that this learned calibration can replace manual hyperparameter tuning \(Figure[3](https://arxiv.org/html/2608.21750#S5.F3)\)\.

![Refer to caption](https://arxiv.org/html/2608.21750v1/figures/framework.png)Figure 1:Overview ofFCPRAG\. Given a query, retrieve multiple passages and load their passage\-specific LoRA adapters \(offline\-trained\)\. A Fusion Controller predicts per\-passage weights and per\-sample calibration, then a weighted adapter merger \(WAM\) merges the adapters for parametric injection \(optionally combined with in\-context passages\)\.Why fusion control is the right interface\.A key shift in parametric RAG is that evidence is injected through*parameters*rather than*tokens*\. When retrieval returnsNNpassages, PRAG\-family systems produceNNpassage\-LoRA adapters that must be fused into a single injected update\. This fusion step becomes the main place where evidence relevance, redundancy, and conflict must be resolved\. Our design therefore targets the fusion interface directly: we learn \(i\) evidence\-level weights and \(ii\) per\-sample calibration, so the system can be both selective when confident and conservative when uncertain\. To summarize, in this study, we made the following contributions\.

- •FCPRAGarchitecture\.We introduce a learned Fusion Controller for PRAG\-family inference that performs retrieval\-conditioned, sample\-level LoRA fusion and calibration, adding<1%<1\\%trainable parameters relative to the backbone LLM\.
- •Theoretical motivation\.We prove \(Proposition[1](https://arxiv.org/html/2608.21750#Thmproposition1)\) that under heteroscedastic retrieval uncertainty, any single dataset\-level fusion temperature incurs strictly positive calibration regret, formally motivating sample\-level\(g,τ\)\(g,\\tau\)adaptation\.
- •Merge\-aware supervision under no\-leak\.We propose a training\-signal construction that supervises fusion weights using each passage adapter’s marginal contribution within a multi\-LoRA merge, while enforcing a no\-leak train/eval separation\.
- •Empirical gains and key ablations\.On HotpotQA[Yang et al\. 2018](https://arxiv.org/html/2608.21750#bib.bib40)and 2WikiMultiHopQA[Ho et al\. 2020](https://arxiv.org/html/2608.21750#bib.bib8),FCPRAGimproves F1 across three LLM backbones \(Llama\-3\.2\-1B[Touvron et al\. 2023](https://arxiv.org/html/2608.21750#bib.bib37), Qwen2\.5\-1\.5B[Qwen et al\. 2025](https://arxiv.org/html/2608.21750#bib.bib29), Llama\-3\-8B[Touvron et al\. 2023](https://arxiv.org/html/2608.21750#bib.bib37)\) and achieves the best Avg\. Rank in Table[2](https://arxiv.org/html/2608.21750#S3.T2)\. A random controller ablation is substantially worse, validating the learned fusion mechanism\.

## 2Related Work

Unlike in\-context RAG, which injects retrieved evidence by concatenating passages to the prompt, parametric RAG injects retrieved knowledge as parameter updates[Lewis et al\. 2020](https://arxiv.org/html/2608.21750#bib.bib21);[Karpukhin et al\. 2020](https://arxiv.org/html/2608.21750#bib.bib17);[Izacard and Grave 2021](https://arxiv.org/html/2608.21750#bib.bib12);[Guu et al\. 2020](https://arxiv.org/html/2608.21750#bib.bib6);[Borgeaud et al\. 2022](https://arxiv.org/html/2608.21750#bib.bib3);[Izacard et al\. 2022](https://arxiv.org/html/2608.21750#bib.bib13);[Su et al\. 2025a](https://arxiv.org/html/2608.21750#bib.bib33)\. This avoids long\-context overhead, but under multi\-passage retrieval it introduces a different online bottleneck: multiple evidence\-conditioned updates must be combined before generation\.

PRAG[Su et al\. 2025a](https://arxiv.org/html/2608.21750#bib.bib33)converts each retrieved passage into passage\-specific LoRA weights via offline fine\-tuning[Hu et al\. 2022](https://arxiv.org/html/2608.21750#bib.bib10);[Houlsby et al\. 2019](https://arxiv.org/html/2608.21750#bib.bib9);[Li and Liang 2021](https://arxiv.org/html/2608.21750#bib.bib22);[Han et al\. 2024](https://arxiv.org/html/2608.21750#bib.bib7)\. DyPRAG[Tan et al\. 2025](https://arxiv.org/html/2608.21750#bib.bib36)reduces per\-passage training and storage by dynamically generating parametric knowledge at inference time, while Poly\-PRAG[Su et al\. 2025b](https://arxiv.org/html/2608.21750#bib.bib34)routes queries to a small set of latent LoRA experts\. Despite these differences in offline parameterization, PRAG\-family systems share the same online fusion problem: when several passages are retrieved, the model must merge multiple passage\-conditioned parametric updates, often under noisy or partially conflicting evidence\.

Existing fusion methods mainly operate either in token/prediction space, such as Fusion\-in\-Decoder and RePlug[Izacard and Grave 2021](https://arxiv.org/html/2608.21750#bib.bib12);[Shi et al\. 2024](https://arxiv.org/html/2608.21750#bib.bib31), or in task\-level parameter composition, such as model soups, task arithmetic, TIES\-Merging, and AdapterFusion[Wortsman et al\. 2022](https://arxiv.org/html/2608.21750#bib.bib38);[Ilharco et al\. 2022](https://arxiv.org/html/2608.21750#bib.bib11);[Yadav et al\. 2023](https://arxiv.org/html/2608.21750#bib.bib39);[Pfeiffer et al\. 2021](https://arxiv.org/html/2608.21750#bib.bib27)\. Our setting is different: we study*retrieval\-conditioned evidence\-level fusion in parameter space*, where useful merge weights vary across queries and over\-weighting weak evidence can inject spurious knowledge\.FCPRAGaddresses this gap by learning both per\-query fusion scores and calibration that controls how confidently the model departs from uniform fusion\. Extended background is provided in Appendix[B\.2](https://arxiv.org/html/2608.21750#A2.SS2)\.

## 3Methodology

### 3\.1Problem Formulation and PRAG Baseline

We consider knowledge\-intensive QA where, given a queryqq, a retriever returns a set of passages𝒫=\{p1,…,pN\}\\mathcal\{P\}=\\\{p\_\{1\},\\dots,p\_\{N\}\\\}\. PRAG[Su et al\. 2025a](https://arxiv.org/html/2608.21750#bib.bib33)parameterizes each passagepip\_\{i\}into a lightweight passage\-LoRA adapterΔi\\Delta\_\{i\}by offline fine\-tuning on synthetic QA\-style supervision derived frompip\_\{i\}\. At test time, PRAG loads the offline\-trained adapters\{Δi\}i=1N\\\{\\Delta\_\{i\}\\\}\_\{i=1\}^\{N\}corresponding to the retrieved passages𝒫\\mathcal\{P\}and merges them into a single adapterΔ⁡\(𝒫\)\\Delta\(\\mathcal\{P\}\), which is injected into the LLM to generate an answer\. We denote this parametric\-only inference asParametric\-only\(Param\.\), and an optional variantCombinewhere the same retrieved passages are also provided to the LLM in\-context in addition to parametric injection \(following the “PRAG \+ in\-context” combination described in[Su et al\. 2025a](https://arxiv.org/html/2608.21750#bib.bib33)\)\.

#### Data augmentation \(training\-side\)\.

Following the PRAG\-family pipeline, we construct synthetic supervision for each passagepip\_\{i\}using an instruction\-tuned LLM: \(i\) a rewritten variant ofpip\_\{i\}and \(ii\) a small set of QA pairs whose answers are supported bypip\_\{i\}\. These augmentation artifacts define a passage\-specific training setDiD\_\{i\}\.

#### Passage\-LoRA training \(offline parameterization\)\.

We then fine\-tune a passage\-specific LoRA adapterΔi\\Delta\_\{i\}on QA\-style prompts built fromDiD\_\{i\}, optimizing the standard causal language modeling loss while keeping the backbone LLM frozen\. To encourage parametric encoding rather than relying only on in\-context evidence, training mixes*passage\-conditioned*examples \(the prompt includespip\_\{i\}or its rewrite\) and*question\-only*examples \(the prompt omits the passage but still supervises the answer\)\. Appendix[B\.8](https://arxiv.org/html/2608.21750#A2.SS8)lists the training\-side prompt templates used for augmentation and passage\-LoRA fine\-tuning\.

### 3\.2Evidence\-level Multi\-LoRA Fusion Setting

In the multi\-passage setting, each passage yields a passage\-LoRA adapterΔi\\Delta\_\{i\}and the system must fuse\{Δi\}i=1N\\\{\\Delta\_\{i\}\\\}\_\{i=1\}^\{N\}before generation\. Uniform fusion \(equal weights\) treats all passages as equally useful, which can be suboptimal when passages have different relevance or when some passages introduce conflicting knowledge\. We aim to learn a*fusion policy*that assigns weightswi​\(q,𝒫\)w\_\{i\}\(q,\\mathcal\{P\}\)per query so that the merged adapter emphasizes helpful evidence and suppresses noise\.

### 3\.3FCPRAG: Fusion Controller with Learned Calibration

Figure[1](https://arxiv.org/html/2608.21750#S1.F1)shows the high\-level pipeline\.FCPRAGaugments PRAG\-family inference with a lightweight Fusion Controllerfθf\_\{\\theta\}\. Given a query and its retrieved passages, the controller predicts retrieval\-conditioned fusion scores for the corresponding passage\-LoRA adapters, and outputs per\-sample calibration signals to stabilize the mapping from scores to weights\. We denote the downstream weighted merge block as a*Weighted Adapter Merger*\(WAM\), which applies controller weights to retrieved adapters and outputs one merged adapter for injection\. Inference follows a simple sequence: encode the query and retrieved passages, predict fusion scores and sample\-wise calibration, map them to merge weights, merge the retrieved adapters, and generate in either Parametric\-only or Combine mode\. Appendix[B\.3](https://arxiv.org/html/2608.21750#A2.SS3)gives pseudocode\.

#### Inputs\.

For controller inputs, we construct interaction features from dense query and passage embeddings produced by a sentence embedding model\. Let𝐞q∈ℝd\\mathbf\{e\}\_\{q\}\\in\\mathbb\{R\}^\{d\}be the query embedding and𝐄p=\[𝐞p1,…,𝐞pN\]∈ℝN×d\\mathbf\{E\}\_\{p\}=\[\\mathbf\{e\}\_\{p\_\{1\}\},\\dots,\\mathbf\{e\}\_\{p\_\{N\}\}\]\\in\\mathbb\{R\}^\{N\\times d\}be passage embeddings \(normalized\)\. We form interaction features for each passage:

𝐱i=\[𝐞q;𝐞pi;𝐞q⊙𝐞pi;\|𝐞q−𝐞pi\|\]\.\\mathbf\{x\}\_\{i\}=\\left\[\\mathbf\{e\}\_\{q\};\\ \\mathbf\{e\}\_\{p\_\{i\}\};\\ \\mathbf\{e\}\_\{q\}\\odot\\mathbf\{e\}\_\{p\_\{i\}\};\\ \|\\mathbf\{e\}\_\{q\}\-\\mathbf\{e\}\_\{p\_\{i\}\}\|\\right\]\.\(1\)

#### Outputs and weight mapping\.

The controller predicts a scalar scoresi∈\(0,1\)s\_\{i\}\\in\(0,1\)for each passage via an MLP, and a calibration head predicts: \(i\) a gateg∈\(0,1\)g\\in\(0,1\)and \(ii\) a temperatureτ∈\[τmin,τmax\]\\tau\\in\[\\tau\_\{\\min\},\\tau\_\{\\max\}\]\. We map scores to a probability distribution and blend with a uniform prior:

πi\\displaystyle\\pi\_\{i\}=softmax⁡\(siτ\),\\displaystyle=\\mathrm\{softmax\}\\left\(\\frac\{s\_\{i\}\}\{\\tau\}\\right\),\(2\)wi\\displaystyle w\_\{i\}=g⋅πi\+\(1−g\)⋅1N\.\\displaystyle=g\\cdot\\pi\_\{i\}\+\(1\-g\)\\cdot\\frac\{1\}\{N\}\.\(3\)Intuitively,τ\\taucontrols how sharply scores are converted into weights, whileggcontrols how far the controller deviates from uniform fusion\. This mirrors the theoretical role of calibration below: heteroscedastic retrieval uncertainty motivates sample\-level sharpness, and the uniform\-prior interpolation in Eq\.[3](https://arxiv.org/html/2608.21750#S3.E3)implements shrinkage toward conservative fusion when score signals are unreliable\. The theory supports this calibration choice rather than claiming optimality of the full controller architecture\.

#### Weighted Adapter Merger \(WAM\)\.

Given retrieved adapters\{Δi\}i=1N\\\{\\Delta\_\{i\}\\\}\_\{i=1\}^\{N\}and controller outputs\{wi\}i=1N\\\{w\_\{i\}\\\}\_\{i=1\}^\{N\}from Eq\.[3](https://arxiv.org/html/2608.21750#S3.E3), we first apply merge\-time normalization:

w~i=N​wi∑j=1Nwj\+ϵ\.\\tilde\{w\}\_\{i\}=\\frac\{N\\,w\_\{i\}\}\{\\sum\_\{j=1\}^\{N\}w\_\{j\}\+\\epsilon\}\.\(4\)WAM then applies the normalized weights as scaling factors and composes the scaled adapters into a single merged adapter:

Δ⋆=ℳWAM​\(\{w~i⋅Δi\}i=1N\)\.\\Delta^\{\\star\}=\\mathcal\{M\}\_\{\\mathrm\{WAM\}\}\\\!\\left\(\\\{\\tilde\{w\}\_\{i\}\\cdot\\Delta\_\{i\}\\\}\_\{i=1\}^\{N\}\\right\)\.\(5\)Finally, we injectΔ⋆\\Delta^\{\\star\}into the frozen backbone model and denote the adapted model byℳΔ⋆\\mathcal\{M\}\_\{\\Delta^\{\\star\}\}:

ℳΔ⋆\\displaystyle\\mathcal\{M\}\_\{\\Delta^\{\\star\}\}←Inject​\(ℳ,Δ⋆\),\\displaystyle\\leftarrow\\textsc\{Inject\}\(\\mathcal\{M\},\\Delta^\{\\star\}\),\(6\)a^\\displaystyle\\hat\{a\}←Generate​\(ℳΔ⋆,prompt\)\.\\displaystyle\\leftarrow\\textsc\{Generate\}\(\\mathcal\{M\}\_\{\\Delta^\{\\star\}\},\\text\{prompt\}\)\.

### 3\.4Merge\-aware Training Signal and No\-leak Protocol

Training the Fusion Controller requires supervision that reflects how each passage\-LoRA adapter contributes under*multi\-LoRA fusion*\(as opposed to isolated single\-adapter behavior\)\. We therefore construct*merge\-aware*marginal contribution labels using training\-side data only: for each training instance\(q,𝒫,a\)\(q,\\mathcal\{P\},a\), we evaluate the system with allNNadapters merged under uniform fusion to obtainF1allF\_\{1\}^\{\\text\{all\}\}, then remove theii\-th adapter and re\-evaluate to obtainF1∖iF\_\{1\}^\{\\setminus i\}; the difference measures the marginal effect of that adapter under fusion\. We define

δi=F1all−F1∖i,\\delta\_\{i\}=F\_\{1\}^\{\\text\{all\}\}\-F\_\{1\}^\{\\setminus i\},\(7\)where we useδi\\delta\_\{i\}to denote a scalar marginal contribution \(to avoid confusion with LoRA deltasΔi\\Delta\_\{i\}\)\. We convert\{δi\}\\\{\\delta\_\{i\}\\\}into a soft target distribution

yi=softmax⁡\(δiTlabel\)\.y\_\{i\}=\\mathrm\{softmax\}\\left\(\\frac\{\\delta\_\{i\}\}\{T\_\{\\text\{label\}\}\}\\right\)\.\(8\)We then train the controller to match this target distribution via KL divergence \(below\); when contributions are flat \(no preference\), the target becomes uniform and we down\-weight these samples to prevent learning a trivial uniform mapping\. A schematic illustration and label\-source analysis are deferred to Appendix[B\.5](https://arxiv.org/html/2608.21750#A2.SS5)\. Label construction and controller training use training data only, while evaluation is performed on a held\-out split\.

#### Objective\.

Let𝐰θ​\(q,𝒫\)\\mathbf\{w\}\_\{\\theta\}\(q,\\mathcal\{P\}\)be the controller’s predicted distribution from Eq\.[3](https://arxiv.org/html/2608.21750#S3.E3)\. For samplebb, we define

ℓKL\(b\)\\displaystyle\\ell\_\{\\text\{KL\}\}^\{\(b\)\}=∑i=1Nyi\(b\)​log⁡yi\(b\)wi\(b\)\+ϵ,\\displaystyle=\\sum\_\{i=1\}^\{N\}y\_\{i\}^\{\(b\)\}\\log\\frac\{y\_\{i\}^\{\(b\)\}\}\{w\_\{i\}^\{\(b\)\}\+\\epsilon\},\(9\)and optimize the weighted batch objective

ℒKL\\displaystyle\\mathcal\{L\}\_\{\\text\{KL\}\}=∑bω\(b\)​ℓKL\(b\)∑bω\(b\)\+ϵ,\\displaystyle=\\frac\{\\sum\_\{b\}\\omega^\{\(b\)\}\\ell\_\{\\text\{KL\}\}^\{\(b\)\}\}\{\\sum\_\{b\}\\omega^\{\(b\)\}\+\\epsilon\},\(10\)whereω\(b\)\\omega^\{\(b\)\}is the sample weight\. When the deltas are flat \(no preference\),𝐲\\mathbf\{y\}becomes uniform; we down\-weight these samples to prevent the model from learning a trivial uniform mapping\.

Table 2:Main results \(F1, %\) on 2WikiMultiHopQA, HotpotQA, PopQA, and ComplexWebQuestions \(CWQ\)\. Suffix “\-Combine” denotes results under the Combine setting\. All baseline methods \(Vanilla, Standard RAG, PRAG, DyPRAG\) were manually reproduced under the same experimental setup for fair comparison\.Avg\. Rankis the mean rank \(lower is better\) over the 8 non\-Avg columns within each backbone block\. Best results are highlighted inboldand second\-best results areunderlined\.Base LLMMethod2WikiMultiHopQAHotpotQAPopQACWQAvg\.RankCompareBridgeInferenceComposeAvg\.BridgeCompareLlama\-3\.2\-1BVanilla36\.4834\.2313\.805\.6022\.539\.8537\.8414\.5628\.528\.00Standard RAG41\.2239\.3115\.577\.2225\.8311\.4239\.3917\.1531\.786\.38PRAG46\.8344\.3217\.168\.5529\.2212\.9941\.8419\.1632\.093\.62PRAG\-Combine41\.0341\.2619\.378\.2127\.4721\.3542\.7332\.9431\.824\.31DyPRAG41\.5337\.9818\.678\.2126\.6010\.8442\.9118\.6931\.925\.31DyPRAG\-Combine44\.8741\.4219\.319\.4228\.7611\.3545\.6721\.1433\.873\.25FCPRAG\(Our\)44\.2942\.1420\.215\.9728\.1514\.5647\.4917\.2834\.933\.12FCPRAG\-Combine44\.1141\.4721\.6710\.6529\.4823\.0545\.6038\.3334\.172\.00Qwen2\.5\-1\.5BVanilla39\.5337\.2615\.206\.4024\.6013\.4343\.2316\.5831\.256\.62Standard RAG40\.8933\.1218\.7911\.1025\.9716\.3243\.9619\.7331\.504\.88PRAG43\.8535\.4120\.1712\.7728\.0517\.3546\.2218\.9032\.683\.62PRAG\-Combine40\.6237\.9518\.037\.5826\.0418\.7139\.3323\.0121\.485\.12DyPRAG43\.9136\.5417\.989\.1226\.8911\.6943\.1819\.9231\.545\.25DyPRAG\-Combine42\.2340\.8318\.569\.7527\.8412\.1346\.4132\.0832\.293\.50FCPRAG\(Our\)45\.5942\.8917\.8113\.1829\.8718\.5851\.5421\.3035\.272\.38FCPRAG\-Combine38\.9743\.7319\.047\.7827\.3819\.9840\.8020\.7720\.024\.62Llama\-3\-8BVanilla50\.8848\.6021\.4912\.6833\.4137\.8061\.0036\.3939\.887\.25Standard RAG55\.4653\.6820\.9411\.8035\.4741\.2563\.4337\.9741\.296\.50PRAG58\.1156\.2921\.3412\.7337\.1243\.4966\.0339\.0643\.085\.00PRAG\-Combine62\.4758\.2329\.9418\.9242\.3975\.1882\.8342\.2942\.492\.38DyPRAG55\.3252\.8720\.4411\.9235\.1443\.7662\.9235\.8741\.147\.00DyPRAG\-Combine59\.8156\.9427\.6518\.8240\.8171\.6781\.1239\.2142\.963\.38FCPRAG\(Our\)67\.0756\.5325\.8816\.1641\.4143\.9570\.6441\.3350\.633\.12FCPRAG\-Combine67\.0462\.3932\.8625\.8847\.0476\.1581\.8743\.7948\.731\.38

### 3\.5Plug\-and\-play Integration

FCPRAGis designed to be a drop\-in fusion module:PRAGuses uniform weights in the merge;FCPRAGreplaces those weights with𝐰θ\\mathbf\{w\}\_\{\\theta\}\.DyPRAGadditionally uses translator\-generated deltas;FCPRAGcan be applied by replacing mean pooling/averaging in the aggregation stage with a weighted sum driven by𝐰θ\\mathbf\{w\}\_\{\\theta\}\.Poly\-PRAGperforms routing over latent experts; we discuss how a calibrated fusion controller could complement routing decisions in future work, but do not claim full plug\-and\-play integration\.

### 3\.6Efficiency and Overhead

The Fusion Controller is a small MLP operating on retriever embeddings\. Compared to PRAG\-family inference, the additional overhead is a single embedding forward pass for the query and passages plus lightweight MLP computation\. Controller training is fast and does not require backpropagating through the LLM; label construction is the main offline cost but can be parallelized and reused across controller variants\.

### 3\.7Theoretical Motivation

We provide a formal justification for sample\-level fusion calibration\. The full proof is in Appendix[A](https://arxiv.org/html/2608.21750#A1)\.

###### Proposition 1\(Limitation of Dataset\-level Fusion Calibration\)\.

Under heteroscedastic retrieval uncertainty, different queries induce different optimal fusion sharpness levels\. Consequently, no single dataset\-level temperature can be optimal for all queries, and any fixed global temperature incurs positive expected calibration regret under the risk\-aware objective analyzed in Appendix[A](https://arxiv.org/html/2608.21750#A1):

Reg⁡\(T\)≜𝔼t,𝐬​\[ℛt​\(𝐰⁡\(Tt⋆,𝐬\)\)−ℛt​\(𝐰⁡\(T,𝐬\)\)\],\\mathrm\{Reg\}\(T\)\\;\\triangleq\\;\\mathbb\{E\}\_\{t,\\,\\mathbf\{s\}\}\\\!\\Big\[\\mathcal\{R\}\_\{t\}\\\!\\bigl\(\\mathbf\{w\}\(T\_\{t\}^\{\\star\};\\,\\mathbf\{s\}\)\\bigr\)\-\\mathcal\{R\}\_\{t\}\\\!\\bigl\(\\mathbf\{w\}\(T;\\,\\mathbf\{s\}\)\\bigr\)\\Big\],\(11\)In particular, even the best dataset\-level temperature

Tglobal⋆∈arg⁡maxT\>0​𝔼t,𝐬​\[ℛt​\(𝐰⁡\(T,𝐬\)\)\]T\_\{\\mathrm\{global\}\}^\{\\star\}\\;\\in\\;\\arg\\max\_\{T\>0\}\\;\\mathbb\{E\}\_\{t,\\,\\mathbf\{s\}\}\\\!\\Big\[\\mathcal\{R\}\_\{t\}\\\!\\bigl\(\\mathbf\{w\}\(T;\\,\\mathbf\{s\}\)\\bigr\)\\Big\]\(12\)satisfiesReg⁡\(Tglobal⋆\)\>0\\mathrm\{Reg\}\(T\_\{\\mathrm\{global\}\}^\{\\star\}\)\>0\. The full assumptions and proof are deferred to Appendix[A](https://arxiv.org/html/2608.21750#A1)\.

Intuition\.Softmax temperature controls how sharply evidence is fused in parameter space\. Under low retrieval uncertainty, sharp fusion is beneficial; under high uncertainty, conservative \(near\-uniform\) fusion is safer\. When uncertainty varies across queries, a single dataset\-level temperature cannot satisfy both regimes\. This directly motivates the calibration head in Eq\.[3](https://arxiv.org/html/2608.21750#S3.E3):τ\\tauadapts fusion sharpness, whileggallows the learned distribution to shrink back toward the uniform prior under unreliable retrieval signals\. Appendix[A\.3](https://arxiv.org/html/2608.21750#A1.SS3)derives the corresponding oracle\-weight form\.

## 4Experimental Setup

We evaluate on HotpotQA[Yang et al\. 2018](https://arxiv.org/html/2608.21750#bib.bib40), 2WikiMultiHopQA[Ho et al\. 2020](https://arxiv.org/html/2608.21750#bib.bib8), PopQA[Mallen et al\. 2023](https://arxiv.org/html/2608.21750#bib.bib24), and ComplexWebQuestions \(CWQ\)[Talmor and Berant 2018](https://arxiv.org/html/2608.21750#bib.bib35), using three instruction\-tuned backbones: Llama\-3\.2\-1B, Qwen2\.5\-1\.5B, and Llama\-3\-8B[Touvron et al\. 2023](https://arxiv.org/html/2608.21750#bib.bib37);[Qwen et al\. 2025](https://arxiv.org/html/2608.21750#bib.bib29)\. Following PRAG\-family evaluation practice[Su et al\. 2025a](https://arxiv.org/html/2608.21750#bib.bib33);[Tan et al\. 2025](https://arxiv.org/html/2608.21750#bib.bib36), we useF1\(%\) as the primary metric\. For HotpotQA and 2WikiMultiHopQA, we report subtask\-level F1; for 2Wiki we also report macro average over its four subtasks\.

We compare Vanilla, Standard RAG[Lewis et al\. 2020](https://arxiv.org/html/2608.21750#bib.bib21);[Karpukhin et al\. 2020](https://arxiv.org/html/2608.21750#bib.bib17);[Izacard and Grave 2021](https://arxiv.org/html/2608.21750#bib.bib12), PRAG[Su et al\. 2025a](https://arxiv.org/html/2608.21750#bib.bib33), DyPRAG[Tan et al\. 2025](https://arxiv.org/html/2608.21750#bib.bib36), andFCPRAG\. All methods use the same retriever and the same top\-KKpassages per query; only the knowledge injection and fusion mechanism differ\. For parametric methods \(PRAG, DyPRAG, andFCPRAG\), we report both Parametric\-only and Combine settings \(Section[3\.1](https://arxiv.org/html/2608.21750#S3.SS1)\)\. A randomly initialized controller is evaluated only as an ablation \(Table[3](https://arxiv.org/html/2608.21750#S5.T3)\)\. Full implementation details are deferred to Appendix[B\.4](https://arxiv.org/html/2608.21750#A2.SS4)\. Runtime and implementation overhead are discussed in Appendix[B\.4](https://arxiv.org/html/2608.21750#A2.SS4)\.

## 5Experiments

Table 3:Ablation on 2WikiMultiHopQA \(F1 scores, Qwen2\.5\-1\.5B and Llama\-3\-8B\)\. We compare uniform fusion \(PRAG\), a random \(untrained\) fusion controller, and our learned fusion controller \(FCPRAG\) under Parametric\-only and Combine settings\.Base LLMMethodCmp\.Br\.Inf\.Compst\.Avg\.Qwen2\.5\-1\.5BPRAG43\.8535\.4120\.1712\.7728\.05PRAG\-Combine40\.6237\.9518\.037\.5826\.04Random Ctrl\.44\.3936\.0516\.107\.9526\.12Random Ctrl\.\-Combine38\.9841\.8718\.327\.3226\.62FCPRAG\(Ours\)45\.5942\.8917\.8113\.1829\.87FCPRAG\-Combine38\.9743\.7319\.047\.7827\.38Llama\-3\-8BPRAG58\.1156\.2921\.3412\.7337\.12PRAG\-Combine62\.4758\.2329\.9418\.9242\.39Random Ctrl\.60\.1854\.1222\.0512\.1137\.12Random Ctrl\.\-Combine64\.5859\.0128\.4717\.1842\.31FCPRAG\(Ours\)67\.0756\.5325\.8816\.1641\.41FCPRAG\-Combine67\.0462\.3932\.8625\.8847\.04

### 5\.1Main Results

Table[2](https://arxiv.org/html/2608.21750#S3.T2)reports results on 2WikiMultiHopQA, HotpotQA, PopQA, and ComplexWebQuestions \(CWQ\)\. Across backbones and benchmarks,FCPRAGgenerally improves over PRAG’s uniform merge, with gains most pronounced on multi\-evidence settings where retrieved passages vary in usefulness or contain distractors\. These results support our thesis that retrieval\-conditioned evidence fusion is a first\-class bottleneck in parametric RAG, and that learning both weights and calibration provides a practical upgrade over globally tuned heuristics\. A varying\-KKcheck is reported in Appendix[B\.6](https://arxiv.org/html/2608.21750#A2.SS6): performance is not monotonic in the Parametric\-only setting, butFCPRAG\-Combine improves over PRAG\-Combine atK=4K=4andK=5K=5\.

### 5\.2Ablation: Random Fusion Controller

To verify that gains come from*learned*fusion rather than architectural side effects, we replace the Fusion Controller with a random \(untrained\) one while keeping the same parametric\-fusion pipeline\. Table[3](https://arxiv.org/html/2608.21750#S5.T3)\(Qwen2\.5\-1\.5B and Llama\-3\-8B\) shows that the random controller underperformsFCPRAGacross subtasks under both Parametric\-only and Combine settings\. This indicates that improvements are not explained by adding an extra module or changing the merge operator; they come from learning retrieval\-conditioned fusion weights \(and calibration\) that map evidence to meaningful fusion decisions\.

#### Merge\-aware supervision\.

The controller also depends on how fusion labels are constructed\. Holding the scoring projector fixed on HotpotQA \(Qwen2\.5\-1\.5B, Parametric\-only\), replacing isolated single\-adapter labels with our merge\-aware labels improves macro F1 from 32\.47 to 33\.91, with the largest gain on comparison questions \(\+2\.63\)\. This links the leave\-one\-out construction in Section[3\.4](https://arxiv.org/html/2608.21750#S3.SS4)to downstream fusion quality; Appendix[B\.5](https://arxiv.org/html/2608.21750#A2.SS5)reports the full label\-source ablation, attribution diagnostics, and offline cost\.

### 5\.3Analysis: Where Does Fusion Control Help?

The four 2Wiki subtasks stress different reasoning patterns and evidence usage\. In bridge/comparison questions, a single passage can dominate the answer, and uniform fusion may over\-inject irrelevant passage\-specific updates\. In compositional/inference questions, multiple pieces of evidence may be required, and the system must balance contributions without amplifying conflicts\.FCPRAGaddresses both regimes by \(i\) learning to emphasize dominant experts when confident \(highggand lowτ\\tau\) and \(ii\) falling back toward uniform fusion when signals are ambiguous \(lowgg\), improving robustness\.

#### Calibration replaces grid search\.

Figures[2](https://arxiv.org/html/2608.21750#S5.F2)–[3](https://arxiv.org/html/2608.21750#S5.F3)show that learned calibration can replace per\-model global tuning\. The score\-only global baseline is sensitive to blend/temperature choices, whereasFCPRAGpredicts sample\-wise\(g,τ\)\(g,\\tau\)that avoid degenerate pure\-uniform or overly sharp regimes and remain in a stable high\-performing basin\. This is consistent with Proposition[1](https://arxiv.org/html/2608.21750#Thmproposition1): fixed dataset\-level calibration is brittle under heteroscedastic retrieval uncertainty\. Although the learned mean\(g,τ\)\(g,\\tau\)does not exactly match the single best global\(α,T\)\(\\alpha,T\)cell, it stays in the same high\-performance region, yielding strong F1 without exhaustive tuning\. On 2Wiki in the*Combine*setting, learned calibration achieves comparable performance to grid\-selected settings with substantially lower tuning cost, without changing the retriever, merge operator, or offline passage parameterization\.

![Refer to caption](https://arxiv.org/html/2608.21750v1/fig-calibration-ab.png)Figure 2:Calibration diagnostics on HotpotQA\.\(a\)F1 over global blendα\\alphaand temperatureTTfor a score\-only fusion baseline\.\(b\)Learned sample\-wise calibration outputs\(g,τ\)\(g,\\tau\)fromFCPRAG\.Figure 3:Tuning cost vs\. performance on 2WikiMultiHopQA \(Combine\): each point is macro\-average F1 over the four 2Wiki subtasks\. Circles denote grid\-selected global calibration; stars denoteFCPRAG’s learned calibration\.

### 5\.4Robustness to Retrieval Perturbations

Real retrieval systems can return irrelevant or redundant passages, which is especially harmful when evidence is injected through adapter weights\. We test retrieval noise atK=3K=3with two one\-passage perturbations:replace\-onesubstitutes an irrelevant passage, andrepeat\-oneduplicates a retrieved passage\. Figure[4](https://arxiv.org/html/2608.21750#S5.F4)shows thatFCPRAGretains more relative performance than PRAG under both perturbations, consistent with conservative fusion when retrieval signals are unreliable\. Because each panel is normalized by the method’s own unperturbed score, the figure emphasizes relative performance retention rather than absolute score differences\. This matches the role of adaptive calibration: noisier retrieved sets should shift fusion toward less aggressive injection instead of applying a fixed global temperature\.

Figure 4:Retrieval perturbation robustness on 2WikiMultiHopQA \(Llama\-3\-8B, Parametric\-only;K=3K=3\)\. We reportreplace\-oneandrepeat\-oneperturbations, with each panel normalized by the method’s own unperturbed score \(none=1=1\)\.

## 6Conclusion

We presentedFCPRAG, a learned fusion controller for stable, retrieval\-conditioned multi\-passage LoRA fusion in parametric retrieval augmented generation\. By predicting per\-passage weights and per\-sample calibration,FCPRAGimproves robustness and reduces the need for expensive global calibration\. Across four datasets, 2WikiMultiHopQA, HotpotQA, PopQA, and ComplexWebQuestions\(CWQ\) and three LLM backbones,FCPRAGimproves F1 scores over the baseline methods in most settings, while a random controller performs significantly worse\. These results suggest that sample\-level calibration is a practical mechanism for adapting evidence fusion to query\-dependent retrieval uncertainty, rather than relying on a single global setting across tasks and backbones\. Because the method operates at the fusion stage, it can strengthen multi\-passage parametric RAG with minimal changes to the underlying retrieval, offline parameterization, and generation pipeline\. These results suggest that learned fusion control is a promising and practical direction for parametric RAG systems under multi\-passage retrieval and parametric fusion\.

## Limitations

Offline label cost:merge\-aware supervision requires evaluating one all\-merge andKKleave\-one\-out merges per training sample \(e\.g\., four forward passes forK=3K=3\), increasing offline computation cost and making largerKKmore expensive\.Embedding dependence:the controller operates on query/passage embeddings; if embedding similarity is misaligned with actual answer utility, fusion predictions may degrade\.Scaling to larger evidence sets:we focus on a small, fixed number of passages per query; extending to largerKKmay require sparse or hierarchical fusion policies\.

## Ethical Considerations

This work studies a retrieval\-augmented QA system using public benchmarks and does not involve new human\-subject data collection\. The main ethical risk is that parametric injection can make retrieved errors or biased evidence more persistent in the model’s effective parameters during inference\.FCPRAGmitigates this risk by learning to down\-weight weak or conflicting evidence, but it does not guarantee factual correctness or fairness\. Deployments should therefore pair the method with source filtering, answer attribution, and task\-specific safety evaluation before use in high\-stakes settings\.

## References

- Ainsworth et al\. \(2022\)Samuel K Ainsworth, Jonathan Hayase, and Siddhartha Srinivasa\. 2022\.Git re\-basin: Merging models modulo permutation symmetries\.*arXiv preprint arXiv:2209\.04836*\.
- Asai et al\. \(2024\)Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi\. 2024\.Self\-rag: Learning to retrieve, generate, and critique through self\-reflection\.
- Borgeaud et al\. \(2022\)Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean\-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, and 1 others\. 2022\.Improving language models by retrieving from trillions of tokens\.In*International conference on machine learning*, pages 2206–2240\. PMLR\.
- Fedus et al\. \(2022\)William Fedus, Barret Zoph, and Noam Shazeer\. 2022\.Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity\.*Journal of Machine Learning Research*, 23\(120\):1–39\.
- Guo et al\. \(2017\)Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger\. 2017\.On calibration of modern neural networks\.In*International conference on machine learning*, pages 1321–1330\. PMLR\.
- Guu et al\. \(2020\)Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang\. 2020\.Retrieval augmented language model pre\-training\.In*International conference on machine learning*, pages 3929–3938\. PMLR\.
- Han et al\. \(2024\)Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang\. 2024\.Parameter\-efficient fine\-tuning for large models: A comprehensive survey\.*arXiv preprint arXiv:2403\.14608*\.
- Ho et al\. \(2020\)Xanh Ho, Anh\-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa\. 2020\.Constructing a multi\-hop qa dataset for comprehensive evaluation of reasoning steps\.*arXiv preprint arXiv:2011\.01060*\.
- Houlsby et al\. \(2019\)Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly\. 2019\.Parameter\-efficient transfer learning for nlp\.In*International conference on machine learning*, pages 2790–2799\. PMLR\.
- Hu et al\. \(2022\)Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen\-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others\. 2022\.Lora: Low\-rank adaptation of large language models\.*ICLR*, 1\(2\):3\.
- Ilharco et al\. \(2022\)Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi\. 2022\.Editing models with task arithmetic\.*arXiv preprint arXiv:2212\.04089*\.
- Izacard and Grave \(2021\)Gautier Izacard and Edouard Grave\. 2021\.Leveraging passage retrieval with generative models for open domain question answering\.In*Proceedings of the 16th conference of the european chapter of the association for computational linguistics: main volume*, pages 874–880\.
- Izacard et al\. \(2022\)Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi\-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave\. 2022\.Few\-shot learning with retrieval augmented language models\.*arXiv preprint arXiv:2208\.03299*, 1\(2\):4\.
- Izmailov et al\. \(2018\)Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson\. 2018\.Averaging weights leads to wider optima and better generalization\.*arXiv preprint arXiv:1803\.05407*\.
- Jiang et al\. \(2023\)Zhengbao Jiang, Frank F Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi\-Yu, Yiming Yang, Jamie Callan, and Graham Neubig\. 2023\.Active retrieval augmented generation\.In*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pages 7969–7992\.
- Jin et al\. \(2022\)Xisen Jin, Xiang Ren, Daniel Preotiuc\-Pietro, and Pengxiang Cheng\. 2022\.Dataless knowledge fusion by merging weights of language models\.*arXiv preprint arXiv:2212\.09849*\.
- Karpukhin et al\. \(2020\)Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick SH Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen\-tau Yih\. 2020\.Dense passage retrieval for open\-domain question answering\.In*EMNLP \(1\)*, pages 6769–6781\.
- Khandelwal et al\. \(2019\)Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis\. 2019\.Generalization through memorization: Nearest neighbor language models\.*arXiv preprint arXiv:1911\.00172*\.
- Kuleshov et al\. \(2018\)Volodymyr Kuleshov, Nathan Fenner, and Stefano Ermon\. 2018\.Accurate uncertainties for deep learning using calibrated regression\.In*International conference on machine learning*, pages 2796–2804\. PMLR\.
- Lepikhin et al\. \(2020\)D Lepikhin, H Lee, Y Xu, D Chen, O Firat, Y Huang, M Krikun, N Shazeer, and Z Gshard\. 2020\.Scaling giant models with conditional computation and automatic sharding\.*arXiv preprint arXiv:2006\.16668*\.
- Lewis et al\. \(2020\)Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen\-tau Yih, Tim Rocktäschel, and 1 others\. 2020\.Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.*Advances in neural information processing systems*, 33:9459–9474\.
- Li and Liang \(2021\)Xiang Lisa Li and Percy Liang\. 2021\.Prefix\-tuning: Optimizing continuous prompts for generation\.*arXiv preprint arXiv:2101\.00190*\.
- Lu et al\. \(2024\)Yi Lu, Jing Nathan Yan, Songlin Yang, Justin T Chiu, Siyu Ren, Fei Yuan, Wenting Zhao, Zhiyong Wu, and Alexander M Rush\. 2024\.A controlled study on long context extension and generalization in llms\.*arXiv preprint arXiv:2409\.12181*\.
- Mallen et al\. \(2023\)Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi\. 2023\.When not to trust language models: Investigating effectiveness of parametric and non\-parametric memories\.In*Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 9802–9822\.
- Matena and Raffel \(2022\)Michael S Matena and Colin A Raffel\. 2022\.Merging models with fisher\-weighted averaging\.*Advances in Neural Information Processing Systems*, 35:17703–17716\.
- Niculescu\-Mizil and Caruana \(2005\)Alexandru Niculescu\-Mizil and Rich Caruana\. 2005\.Predicting good probabilities with supervised learning\.In*Proceedings of the 22nd international conference on Machine learning*, pages 625–632\.
- Pfeiffer et al\. \(2021\)Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych\. 2021\.Adapterfusion: Non\-destructive task composition for transfer learning\.In*Proceedings of the 16th conference of the European chapter of the association for computational linguistics: main volume*, pages 487–503\.
- Platt et al\. \(1999\)John Platt and 1 others\. 1999\.Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods\.*Advances in large margin classifiers*, 10\(3\):61–74\.
- Qwen et al\. \(2025\)Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others\. 2025\.[Qwen2\.5 technical report](https://arxiv.org/abs/2412.15115)\.*Preprint*, arXiv:2412\.15115\.
- Shazeer et al\. \(2017\)Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean\. 2017\.Outrageously large neural networks: The sparsely\-gated mixture\-of\-experts layer\.*arXiv preprint arXiv:1701\.06538*\.
- Shi et al\. \(2024\)Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Richard James, Mike Lewis, Luke Zettlemoyer, and Wen\-tau Yih\. 2024\.Replug: Retrieval\-augmented black\-box language models\.In*Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\)*, pages 8371–8384\.
- Stoica et al\. \(2023\)George Stoica, Daniel Bolya, Jakob Bjorner, Pratik Ramesh, Taylor Hearn, and Judy Hoffman\. 2023\.Zipit\! merging models from different tasks without training\.*arXiv preprint arXiv:2305\.03053*\.
- Su et al\. \(2025a\)Weihang Su, Yichen Tang, Qingyao Ai, Junxi Yan, Changyue Wang, Hongning Wang, Ziyi Ye, Yujia Zhou, and Yiqun Liu\. 2025a\.Parametric retrieval augmented generation\.In*Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval*, pages 1240–1250\.
- Su et al\. \(2025b\)Zhan Su, Fengran Mo, and Jian\-yun Nie\. 2025b\.Parametric retrieval\-augmented generation using latent routing of lora adapters\.*arXiv preprint arXiv:2511\.17044*\.
- Talmor and Berant \(2018\)Alon Talmor and Jonathan Berant\. 2018\.The web as a knowledge\-base for answering complex questions\.*arXiv preprint arXiv:1803\.06643*\.
- Tan et al\. \(2025\)Yuqiao Tan, Shizhu He, Huanxuan Liao, Jun Zhao, and Kang Liu\. 2025\.Dynamic parametric retrieval augmented generation for test\-time knowledge enhancement\.*arXiv preprint arXiv:2503\.23895*\.
- Touvron et al\. \(2023\)Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie\-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others\. 2023\.Llama: Open and efficient foundation language models\.*arXiv preprint arXiv:2302\.13971*\.
- Wortsman et al\. \(2022\)Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo\-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and 1 others\. 2022\.Model soups: averaging weights of multiple fine\-tuned models improves accuracy without increasing inference time\.In*International conference on machine learning*, pages 23965–23998\. PMLR\.
- Yadav et al\. \(2023\)Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal\. 2023\.Ties\-merging: Resolving interference when merging models\.*Advances in Neural Information Processing Systems*, 36:7093–7115\.
- Yang et al\. \(2018\)Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning\. 2018\.Hotpotqa: A dataset for diverse, explainable multi\-hop question answering\.In*Proceedings of the 2018 conference on empirical methods in natural language processing*, pages 2369–2380\.
- Zhu et al\. \(2024\)Andrew Zhu, Alyssa Hwang, Liam Dugan, and Chris Callison\-Burch\. 2024\.Fanoutqa: A multi\-hop, multi\-document question answering benchmark for large language models\.*arXiv preprint arXiv:2402\.14116*\.

## Appendix AProof of Proposition[1](https://arxiv.org/html/2608.21750#Thmproposition1)

For a given queryqq, retrieval returns passages𝒫=\{p1,…,pN\}\\mathcal\{P\}=\\\{p\_\{1\},\\dots,p\_\{N\}\\\}, each associated with a passage\-specific LoRA adapterΔi\\Delta\_\{i\}\. A scoring model assigns per\-passage scores𝐬=\(s1,…,sN\)\\mathbf\{s\}=\(s\_\{1\},\\dots,s\_\{N\}\), which may vary across queries\. We assume scores are non\-degenerate, i\.e\.,Pr\[∃i≠j:si≠sj\]\>0\\Pr\[\\exists\\,i\\neq j:s\_\{i\}\\neq s\_\{j\}\]\>0; otherwise𝐰⁡\(T,𝐬\)\\mathbf\{w\}\(T;\\,\\mathbf\{s\}\)is uniform for allTTand temperature has no effect\. We allow the score distribution𝐬\|t\\mathbf\{s\}\\mid tto differ across query types\. The proof proceeds by expanding the risk\-aware objective, showing that the optimal temperature depends monotonically on the noise variance, and concluding that heteroscedastic types force distinct optima that no single temperature can satisfy\. Fusion produces a merged update

Δ⁡\(T,𝐬\)=∑i=1Nwi​\(T,𝐬\)​Δi,\\Delta\(T;\\,\\mathbf\{s\}\)=\\sum\_\{i=1\}^\{N\}w\_\{i\}\(T;\\,\\mathbf\{s\}\)\\,\\Delta\_\{i\},\(13\)where weights are given by a softmax with temperatureT\>0T\>0:

wi​\(T,𝐬\)=exp⁡\(si/T\)∑j=1Nexp⁡\(sj/T\)\.w\_\{i\}\(T;\\,\\mathbf\{s\}\)=\\frac\{\\exp\(s\_\{i\}/T\)\}\{\\sum\_\{j=1\}^\{N\}\\exp\(s\_\{j\}/T\)\}\.\(14\)Note that even under a*global*\(dataset\-level\) temperatureTT, the weights𝐰⁡\(T,𝐬\)\\mathbf\{w\}\(T;\\,\\mathbf\{s\}\)are sample\-dependent through the scores𝐬\\mathbf\{s\}\. We show that allowing the temperature to also adapt per sample can strictly reduce the expected calibration regret under heteroscedastic uncertainty\.

We model the effective utility of adapteriion a given sample as

Ui=μi\+εi,U\_\{i\}=\\mu\_\{i\}\+\\varepsilon\_\{i\},\(15\)whereμi\\mu\_\{i\}is the mean contribution \(independent of query typett\) andεi\\varepsilon\_\{i\}is zero\-mean noise induced by retrieval uncertainty\.

#### Assumptions\.

1. \(A1\)Independence\.εi\\varepsilon\_\{i\}are independent across passages, withVar⁡\(εi∣type​t\)=σt2\\mathrm\{Var\}\(\\varepsilon\_\{i\}\\mid\\text\{type \}t\)=\\sigma\_\{t\}^\{2\}\.
2. \(A2\)Non\-degenerate heteroscedastic mixture\.There exist at least two query typesA,BA,BwithσA2≠σB2\\sigma\_\{A\}^\{2\}\\neq\\sigma\_\{B\}^\{2\}, each occurring with positive probability \(0<π<10<\\pi<1, whereπ=Pr\[t=A\]\\pi=\\Pr\[t\\\!=\\\!A\]\)\.
3. \(A3\)Score–utility alignment \(local\)\.For each typett, the expected score–utility interaction𝔼𝐬\|t​\[∑i∂wi​\(T,𝐬\)∂T​μi\]\\mathbb\{E\}\_\{\\mathbf\{s\}\\mid t\}\\\!\\left\[\\sum\_\{i\}\\frac\{\\partial w\_\{i\}\(T;\\,\\mathbf\{s\}\)\}\{\\partial T\}\\,\\mu\_\{i\}\\right\]is strictly decreasing inTTon an interval containing bothTA⋆T\_\{A\}^\{\\star\}andTB⋆T\_\{B\}^\{\\star\}; intuitively, increasingTTshifts mass from higher\-utility to lower\-utility passages on average\.
4. \(A4\)Interior optimum\. 1. \(a\)For each typett, the expected objective𝔼𝐬\|t​\[ℛt​\(𝐰⁡\(T,𝐬\)\)\]\\mathbb\{E\}\_\{\\mathbf\{s\}\\mid t\}\[\\mathcal\{R\}\_\{t\}\(\\mathbf\{w\}\(T;\\,\\mathbf\{s\}\)\)\]admits a unique maximizerTt⋆∈\(0,∞\)T\_\{t\}^\{\\star\}\\in\(0,\\infty\)\. 2. \(b\)The derivative term𝔼𝐬\|t​\[∑iwi​∂wi∂T\]\\mathbb\{E\}\_\{\\mathbf\{s\}\\mid t\}\\\!\\left\[\\sum\_\{i\}w\_\{i\}\\frac\{\\partial w\_\{i\}\}\{\\partial T\}\\right\]is non\-zero and does not change sign in a neighborhood ofTt⋆T\_\{t\}^\{\\star\}\.

#### Remark on \(A1\)\.

The independence assumption simplifies the variance term; allowing correlations adds cross\-terms∑i≠jwi​wj​Cov​\(εi,εj\)\\sum\_\{i\\neq j\}w\_\{i\}w\_\{j\}\\,\\mathrm\{Cov\}\(\\varepsilon\_\{i\},\\varepsilon\_\{j\}\)but the variance remains type\-dependent throughσt2\\sigma\_\{t\}^\{2\}, so the monotonicity argument in §[A\.1](https://arxiv.org/html/2608.21750#A1.SS1)still applies and Proposition[1](https://arxiv.org/html/2608.21750#Thmproposition1)holds\. In particular, positive correlations among passage noises can further motivate conservative \(near\-uniform\) fusion under high uncertainty\.

We adopt a mean–variance objective as a tractable proxy that captures the key tension between evidence sharpness and noise amplification; the empirical sections validate the conclusion under actual QA metrics\. We define the risk\-aware objective for fusion as

ℛt​\(𝐰\)=𝔼⁡\[∑iwi​Ui\]−λ​Var​\(∑iwi​Ui\),λ\>0\.\\mathcal\{R\}\_\{t\}\(\\mathbf\{w\}\)=\\mathbb\{E\}\\\!\\left\[\\sum\_\{i\}w\_\{i\}\\,U\_\{i\}\\right\]\-\\lambda\\,\\mathrm\{Var\}\\\!\\left\(\\sum\_\{i\}w\_\{i\}\\,U\_\{i\}\\right\),\\quad\\lambda\>0\.\(16\)
The per\-type optimal temperature maximizes the expected risk\-aware objective:

Tt⋆∈arg⁡maxT\>0​𝔼𝐬\|t​\[ℛt​\(𝐰⁡\(T,𝐬\)\)\],T\_\{t\}^\{\\star\}\\;\\in\\;\\arg\\max\_\{T\>0\}\\;\\mathbb\{E\}\_\{\\mathbf\{s\}\\mid t\}\\\!\\Big\[\\mathcal\{R\}\_\{t\}\\\!\\bigl\(\\mathbf\{w\}\(T;\\,\\mathbf\{s\}\)\\bigr\)\\Big\],\(17\)which is unique by assumption \(A4a\)\. Note thatTt⋆T\_\{t\}^\{\\star\}is a type\-level constant \(not a function of𝐬\\mathbf\{s\}\)\.

By linearity of expectation,

𝔼⁡\[∑iwi​Ui\]=∑iwi​μi\.\\mathbb\{E\}\\\!\\left\[\\sum\_\{i\}w\_\{i\}\\,U\_\{i\}\\right\]=\\sum\_\{i\}w\_\{i\}\\,\\mu\_\{i\}\.\(18\)By independence \(A1\),

Var⁡\(∑iwi​Ui\)=σt2​∑iwi2\.\\mathrm\{Var\}\\\!\\left\(\\sum\_\{i\}w\_\{i\}\\,U\_\{i\}\\right\)=\\sigma\_\{t\}^\{2\}\\sum\_\{i\}w\_\{i\}^\{2\}\.\(19\)Thus,

ℛt​\(𝐰\)=∑iwi​μi−λ​σt2​∑iwi2\.\\mathcal\{R\}\_\{t\}\(\\mathbf\{w\}\)=\\sum\_\{i\}w\_\{i\}\\,\\mu\_\{i\}\-\\lambda\\,\\sigma\_\{t\}^\{2\}\\sum\_\{i\}w\_\{i\}^\{2\}\.\(20\)

### A\.1Effect of Temperature and Retrieval Uncertainty

The softmax temperatureTTcontrols the concentration of𝐰⁡\(T,𝐬\)\\mathbf\{w\}\(T;\\,\\mathbf\{s\}\):

- •AsT↓0T\\downarrow 0,𝐰⁡\(T,𝐬\)\\mathbf\{w\}\(T;\\,\\mathbf\{s\}\)becomes increasingly peaked \(approaching one\-hot\), and∑iwi2\\sum\_\{i\}w\_\{i\}^\{2\}increases\.
- •AsT↑∞T\\uparrow\\infty,𝐰⁡\(T,𝐬\)\\mathbf\{w\}\(T;\\,\\mathbf\{s\}\)approaches the uniform distribution, and∑iwi2\\sum\_\{i\}w\_\{i\}^\{2\}decreases to1/N1/N\.

This reveals the core trade\-off: decreasingTTincreases both the alignment with high\-μi\\mu\_\{i\}passages \(higher expected mean\) and the variance of the fused signal \(larger∑iwi2\\sum\_\{i\}w\_\{i\}^\{2\}\)\. The variance penalty inℛt\\mathcal\{R\}\_\{t\}is scaled byσt2\\sigma\_\{t\}^\{2\}, so:

- •Whenσt2\\sigma\_\{t\}^\{2\}is small, the penalty is weak and the optimal temperatureTt⋆T\_\{t\}^\{\\star\}is small \(sharp fusion\)\.
- •Whenσt2\\sigma\_\{t\}^\{2\}is large, the penalty dominates andTt⋆T\_\{t\}^\{\\star\}shifts upward \(flatter fusion\)\.

More precisely, differentiating the expected objective \(Eq\. \([20](https://arxiv.org/html/2608.21750#A1.E20)\), with𝐰=𝐰⁡\(T,𝐬\)\\mathbf\{w\}=\\mathbf\{w\}\(T;\\,\\mathbf\{s\}\)\) with respect toTTand taking the expectation over𝐬\|t\\mathbf\{s\}\\mid t\(justified by dominated convergence, since softmax derivatives are bounded\) yields the first\-order condition:

𝔼𝐬\|t​\[∑i∂wi∂T​μi\]=2​λ​σt2​𝔼𝐬\|t​\[∑iwi​∂wi∂T\]\.\\mathbb\{E\}\_\{\\mathbf\{s\}\\mid t\}\\\!\\left\[\\sum\_\{i\}\\frac\{\\partial w\_\{i\}\}\{\\partial T\}\\,\\mu\_\{i\}\\right\]\\;=\\;2\\lambda\\,\\sigma\_\{t\}^\{2\}\\;\\mathbb\{E\}\_\{\\mathbf\{s\}\\mid t\}\\\!\\left\[\\sum\_\{i\}w\_\{i\}\\frac\{\\partial w\_\{i\}\}\{\\partial T\}\\right\]\.\(21\)By the score–utility alignment assumption \(A3\), the left\-hand side is strictly decreasing inTTnear the optimum\. The right\-hand side scales linearly withσt2\\sigma\_\{t\}^\{2\}, and the derivative term is non\-zero and of constant sign nearTt⋆T\_\{t\}^\{\\star\}by \(A4b\)\. By the implicit function theorem applied to \([21](https://arxiv.org/html/2608.21750#A1.E21)\),Tt⋆T\_\{t\}^\{\\star\}is a smooth function ofσt2\\sigma\_\{t\}^\{2\}\. Whenσt2\\sigma\_\{t\}^\{2\}increases, the right\-hand side of \([21](https://arxiv.org/html/2608.21750#A1.E21)\) increases at the oldTt⋆T\_\{t\}^\{\\star\}, violating the first\-order condition; since the left\-hand side is decreasing inTT, the balance point must shift to a largerTT\. Combined with uniqueness of the interior maximizer \(A4a\), the balance pointTt⋆T\_\{t\}^\{\\star\}is strictly increasing inσt2\\sigma\_\{t\}^\{2\}\.

### A\.2Impossibility of a Single Dataset\-level Temperature

By \(A2\), there exist two typesAAandBBwithσA2≠σB2\\sigma\_\{A\}^\{2\}\\neq\\sigma\_\{B\}^\{2\}, each with positive probability\. By the strict monotonicity established in §[A\.1](https://arxiv.org/html/2608.21750#A1.SS1), the type\-level optimal temperatures \(Eq\.[17](https://arxiv.org/html/2608.21750#A1.E17)\) satisfy

TA⋆≠TB⋆\.T\_\{A\}^\{\\star\}\\neq T\_\{B\}^\{\\star\}\.\(22\)
Any dataset\-level heuristic fixes a single temperatureTTfor all samples\. SinceTA⋆≠TB⋆T\_\{A\}^\{\\star\}\\neq T\_\{B\}^\{\\star\}and both types occur with positive probability \(A2\), any fixedTTdiffers from at least one type optimum\. By uniqueness of the interior maximizer \(A4a\), using anyT≠Tt⋆T\\neq T\_\{t\}^\{\\star\}on typettyields a strict loss; that is, at least one of the following holds:

𝔼𝐬\|A​\[ℛA​\(𝐰⁡\(T,𝐬\)\)\]<𝔼𝐬\|A​\[ℛA​\(𝐰⁡\(TA⋆,𝐬\)\)\]\\displaystyle\\mathbb\{E\}\_\{\\mathbf\{s\}\\mid A\}\\\!\\Big\[\\mathcal\{R\}\_\{A\}\\\!\\bigl\(\\mathbf\{w\}\(T;\\,\\mathbf\{s\}\)\\bigr\)\\Big\]<\\mathbb\{E\}\_\{\\mathbf\{s\}\\mid A\}\\\!\\Big\[\\mathcal\{R\}\_\{A\}\\\!\\bigl\(\\mathbf\{w\}\(T\_\{A\}^\{\\star\};\\,\\mathbf\{s\}\)\\bigr\)\\Big\]𝔼𝐬\|B​\[ℛB​\(𝐰⁡\(T,𝐬\)\)\]<𝔼𝐬\|B​\[ℛB​\(𝐰⁡\(TB⋆,𝐬\)\)\]\\displaystyle\\mathbb\{E\}\_\{\\mathbf\{s\}\\mid B\}\\\!\\Big\[\\mathcal\{R\}\_\{B\}\\\!\\bigl\(\\mathbf\{w\}\(T;\\,\\mathbf\{s\}\)\\bigr\)\\Big\]<\\mathbb\{E\}\_\{\\mathbf\{s\}\\mid B\}\\\!\\Big\[\\mathcal\{R\}\_\{B\}\\\!\\bigl\(\\mathbf\{w\}\(T\_\{B\}^\{\\star\};\\,\\mathbf\{s\}\)\\bigr\)\\Big\]\(23\)\(or both\), so the expected calibration regret \([11](https://arxiv.org/html/2608.21750#S3.E11)\) is strictly positive\. In particular, this holds forTglobal⋆T\_\{\\mathrm\{global\}\}^\{\\star\}\(Eq\.[12](https://arxiv.org/html/2608.21750#S3.E12)\), soReg⁡\(Tglobal⋆\)\>0\\mathrm\{Reg\}\(T\_\{\\mathrm\{global\}\}^\{\\star\}\)\>0\. This completes the proof that sample\-level temperature adaptation is strictly beneficial under heteroscedastic retrieval uncertainty\. ∎

### A\.3Oracle Weights and Connection to FCPRAG

To make the architectural connection explicit, we derive the oracle weights in theN=2N=2case, relaxing the softmax parameterization to optimize overwwdirectly\. The resulting oracle provides a lower bound on what softmax\-parameterized fusion can achieve; the first\-order expansion below shows that FCPRAG approximates this oracle in the near\-uniform regime\. Letw1=ww\_\{1\}=wandw2=1−ww\_\{2\}=1\-w, withΔ​μ=μ1−μ2\>0\\Delta\\mu=\\mu\_\{1\}\-\\mu\_\{2\}\>0\. The objective \([20](https://arxiv.org/html/2608.21750#A1.E20)\) becomes

ℛt​\(w\)=w​μ1\+\(1−w\)​μ2−λ​σt2​\[w2\+\(1−w\)2\]\.\\mathcal\{R\}\_\{t\}\(w\)=w\\,\\mu\_\{1\}\+\(1\-w\)\\,\\mu\_\{2\}\-\\lambda\\,\\sigma\_\{t\}^\{2\}\\bigl\[w^\{2\}\+\(1\-w\)^\{2\}\\bigr\]\.\(24\)Taking the derivative and setting it to zero:

Δ​μ−2​λ​σt2​\(2​w−1\)=0⟹wt⋆=12\+Δ​μ4​λ​σt2\.\\Delta\\mu\-2\\lambda\\,\\sigma\_\{t\}^\{2\}\\,\(2w\-1\)=0\\quad\\Longrightarrow\\quad w\_\{t\}^\{\\star\}=\\frac\{1\}\{2\}\+\\frac\{\\Delta\\mu\}\{4\\lambda\\,\\sigma\_\{t\}^\{2\}\}\.\(25\)This oracle weight has the form*uniform baseline\+\+utility\-proportional deviation that shrinks with noise*, which directly motivates the\(g,τ\)\(g,\\tau\)gating mechanism:ggcontrols the deviation magnitude andτ\\taucontrols the sharpness, together playing the role ofΔ​μ/\(4​λ​σt2\)\\Delta\\mu/\(4\\lambda\\,\\sigma\_\{t\}^\{2\}\)\. We can match this to FCPRAG’s gated fusion \(Eq\.[3](https://arxiv.org/html/2608.21750#S3.E3)\)\. ForN=2N=2, FCPRAG produces

wiFC=g⋅softmax⁡\(si/τ\)\+\(1−g\)⋅12\.w\_\{i\}^\{\\mathrm\{FC\}\}=g\\cdot\\mathrm\{softmax\}\(s\_\{i\}/\\tau\)\+\(1\-g\)\\cdot\\tfrac\{1\}\{2\}\.\(26\)Under a first\-order expansion around the near\-uniform regime \(\|s1−s2\|/τ≪1\|s\_\{1\}\-s\_\{2\}\|/\\tau\\ll 1; used here for interpretability, though the learned controller is not restricted to this regime\):

softmax⁡\(s1/τ\)≈12\+s1−s24​τ,\\mathrm\{softmax\}\(s\_\{1\}/\\tau\)\\;\\approx\\;\\frac\{1\}\{2\}\+\\frac\{s\_\{1\}\-s\_\{2\}\}\{4\\tau\},\(27\)so

w1FC≈12\+g⁡\(s1−s2\)4​τ\.w\_\{1\}^\{\\mathrm\{FC\}\}\\;\\approx\\;\\frac\{1\}\{2\}\\;\+\\;\\frac\{g\\,\(s\_\{1\}\-s\_\{2\}\)\}\{4\\tau\}\.\(28\)Comparing with the oracle \([25](https://arxiv.org/html/2608.21750#A1.E25)\): under score–utility alignment,s1−s2s\_\{1\}\-s\_\{2\}is positively correlated withΔ​μ\\Delta\\muin expectation, so the oracle suggests settingg/τg/\\tauto decrease withσt2\\sigma\_\{t\}^\{2\}\. Note thatλ\\lambdais a free parameter in the theoretical model with no direct architectural counterpart; the theory therefore provides qualitative guidance \(the*direction*of adaptation\) rather than quantitative calibration, while the controller learns the appropriate scale from data\. In particular, the controller should increaseg/τg/\\tau\(sharper, more confident fusion\) whenσt2\\sigma\_\{t\}^\{2\}is small and decrease it \(conservative, near\-uniform fusion\) whenσt2\\sigma\_\{t\}^\{2\}is large, which is precisely the adaptive behavior that FCPRAG learns from data\.

#### Remark \(observability\)\.

In practice, the Fusion Controller does not observeσt2\\sigma\_\{t\}^\{2\}directly; it infers signals correlated with retrieval uncertainty from the query–passage interaction features \(Eq\.[1](https://arxiv.org/html/2608.21750#S3.E1)\)\. Proposition[1](https://arxiv.org/html/2608.21750#Thmproposition1)establishes that such adaptation is*beneficial in principle*; whether the learned controller captures sufficient signal is an empirical question validated by our experiments\. The case studies in Appendix[C](https://arxiv.org/html/2608.21750#A3)show that the learnedggandτ\\tauvalues vary meaningfully across queries with different evidence quality, suggesting the controller captures relevant uncertainty signals\.

### A\.4Discussion

This result formally motivates predicting fusion calibration \(e\.g\., temperature and a gate interpolating with a uniform prior\) at the sample level\. Such a mechanism can adapt to heteroscedastic retrieval uncertainty and avoid the inherent limitations of dataset\-level fusion heuristics\. We note several simplifications in the theoretical model: the mean–variance objective is a tractable proxy for downstream QA metrics, the two\-type mixture is a minimal construction \(the argument extends to any finite number of types with distinct noise variances\), and theN=2N=2oracle relaxes the softmax constraint\. Despite these simplifications, the qualitative prediction aligns with our empirical findings: FCPRAG’s gains over PRAG are most pronounced on subtasks with heterogeneous evidence quality \(Table[2](https://arxiv.org/html/2608.21750#S3.T2), Figure[4](https://arxiv.org/html/2608.21750#S5.F4)\)\. This is exactly the regime where the calibration regret of a global policy is largest: the retrieved set mixes strongly and weakly relevant passages, amplifying the cost of a fixed temperature\.

## Appendix BAdditional Details

### B\.1Use of Large Language Models

Large language models were used for language polishing and proofreading to improve the clarity and readability of this manuscript\.

### B\.2Extended Related Work

#### In\-context RAG and document aggregation\.

Classical RAG pipelines inject retrieved evidence by concatenating passages with the query or by aggregating document information inside the decoder[Lewis et al\. 2020](https://arxiv.org/html/2608.21750#bib.bib21);[Karpukhin et al\. 2020](https://arxiv.org/html/2608.21750#bib.bib17);[Izacard and Grave 2021](https://arxiv.org/html/2608.21750#bib.bib12);[Guu et al\. 2020](https://arxiv.org/html/2608.21750#bib.bib6);[Borgeaud et al\. 2022](https://arxiv.org/html/2608.21750#bib.bib3);[Izacard et al\. 2022](https://arxiv.org/html/2608.21750#bib.bib13);[Khandelwal et al\. 2019](https://arxiv.org/html/2608.21750#bib.bib18)\. This design has been highly effective, but it also makes performance sensitive to context length, retrieval noise, and evidence conflicts, especially in multi\-hop reasoning settings where useful and distracting passages may coexist[Jiang et al\. 2023](https://arxiv.org/html/2608.21750#bib.bib15);[Asai et al\. 2024](https://arxiv.org/html/2608.21750#bib.bib2);[Yang et al\. 2018](https://arxiv.org/html/2608.21750#bib.bib40);[Ho et al\. 2020](https://arxiv.org/html/2608.21750#bib.bib8);[Zhu et al\. 2024](https://arxiv.org/html/2608.21750#bib.bib41);[Lu et al\. 2024](https://arxiv.org/html/2608.21750#bib.bib23)\. Methods such as RePlug further improve how retrieved documents are used without updating the backbone LLM[Shi et al\. 2024](https://arxiv.org/html/2608.21750#bib.bib31)\. Our setting differs because the retrieved evidence is fused in*parameter space*rather than prompt space: each passage induces a candidate parametric update, and the core question becomes how strongly each update should influence the final merged adapter\.

#### Expert routing and mixture\-of\-experts\.

Learning to select or weight components conditioned on an input has a long history in mixture\-of\-experts models\. Sparsely\-gated MoE layers[Shazeer et al\. 2017](https://arxiv.org/html/2608.21750#bib.bib30)and efficient variants such as Switch Transformers[Fedus et al\. 2022](https://arxiv.org/html/2608.21750#bib.bib4)use routing to activate a subset of experts per input \(or token\) to improve capacity and efficiency; large\-scale conditional computation systems such as GShard further highlight the benefits of input\-dependent expert selection[Lepikhin et al\. 2020](https://arxiv.org/html/2608.21750#bib.bib20)\. While our Fusion Controller is not an MoE layer inside the backbone LLM, it plays an analogous role at the*evidence level*: it produces retrieval\-conditioned weights over passage\-specific parametric updates\. This difference in granularity matters: evidence\-level fusion must decide not only which updates are helpful, but also how*confidently*to deviate from uniform fusion under retrieval noise and evidence conflicts\.

#### Calibration and re\-weighting\.

Calibration methods such as Platt scaling and temperature scaling[Platt et al\. 1999](https://arxiv.org/html/2608.21750#bib.bib28);[Guo et al\. 2017](https://arxiv.org/html/2608.21750#bib.bib5);[Niculescu\-Mizil and Caruana 2005](https://arxiv.org/html/2608.21750#bib.bib26);[Kuleshov et al\. 2018](https://arxiv.org/html/2608.21750#bib.bib19)adjust predictive distributions to better reflect uncertainty\. In PRAG\-family fusion, mapping controller scores to merge weights often requires careful temperature and blending choices\.FCPRAGintegrates calibration into the fusion controller by learning a temperature and a mixing gate that blends learned weights with a uniform prior, yielding stable fusion without expensive global tuning\. This perspective aligns calibrated fusion with classical calibration: the goal is not only to rank evidence, but to control how*confidently*the system deviates from uniform fusion when score signals are unreliable\. More broadly, reducing tuning and selection overhead is a recurring theme in RAG systems: hyperparameters that work well for one backbone or dataset can shift under changes in retrievers, context length, or evidence noise\. Our diagnostic analysis therefore treats calibration as an explicit design target rather than an afterthought\.

### B\.3Inference Pseudocode

Algorithm 1Fusion\-controlled parametric RAG inference \(FCPRAG\)\.1:Query

qq; retrieved passages

𝒫=\{pi\}i=1N\\mathcal\{P\}=\\\{p\_\{i\}\\\}\_\{i=1\}^\{N\}; passage adapters

\{Δi\}i=1N\\\{\\Delta\_\{i\}\\\}\_\{i=1\}^\{N\}; Fusion Controller

fθf\_\{\\theta\}; frozen backbone LLM

ℳ\\mathcal\{M\}; weighted adapter merger

ℳWAM​\(⋅\)\\mathcal\{M\}\_\{\\mathrm\{WAM\}\}\(\\cdot\)\.

2:Generated answer

a^\\hat\{a\}\.

3:Compute query embedding

𝐞q\\mathbf\{e\}\_\{q\}and passage embeddings

\{𝐞pi\}\\\{\\mathbf\{e\}\_\{p\_\{i\}\}\\\}using the sentence encoder\.

4:for

i←1i\\leftarrow 1to

NNdo

5:Build interaction feature

𝐱i=\[𝐞q;𝐞pi;𝐞q⊙𝐞pi;\|𝐞q−𝐞pi\|\]\\mathbf\{x\}\_\{i\}=\[\\mathbf\{e\}\_\{q\};\\ \\mathbf\{e\}\_\{p\_\{i\}\};\\ \\mathbf\{e\}\_\{q\}\\odot\\mathbf\{e\}\_\{p\_\{i\}\};\\ \|\\mathbf\{e\}\_\{q\}\-\\mathbf\{e\}\_\{p\_\{i\}\}\|\]\.

6:endfor

7:Predict fusion scores

\{si\}\\\{s\_\{i\}\\\}and per\-sample calibration

\(g,τ\)\(g,\\tau\)using

fθf\_\{\\theta\}\.

8:

π←softmax⁡\(\{si/τ\}i=1N\)\\pi\\leftarrow\\mathrm\{softmax\}\(\\\{s\_\{i\}/\\tau\\\}\_\{i=1\}^\{N\}\)\.

9:for

i←1i\\leftarrow 1to

NNdo

10:

wi←g⋅πi\+\(1−g\)⋅1Nw\_\{i\}\\leftarrow g\\cdot\\pi\_\{i\}\+\(1\-g\)\\cdot\\frac\{1\}\{N\}\.

11:endfor

12:for

i←1i\\leftarrow 1to

NNdo

13:

w~i←N​wi∑j=1Nwj\+ϵ\\tilde\{w\}\_\{i\}\\leftarrow\\frac\{N\\,w\_\{i\}\}\{\\sum\_\{j=1\}^\{N\}w\_\{j\}\+\\epsilon\}\.

14:endfor

15:

Δ⋆←ℳWAM​\(\{Δi\},\{w~i\}\)\\Delta^\{\\star\}\\leftarrow\\mathcal\{M\}\_\{\\mathrm\{WAM\}\}\(\\\{\\Delta\_\{i\}\\\},\\\{\\tilde\{w\}\_\{i\}\\\}\)
16:Inject

Δ⋆\\Delta^\{\\star\}into

ℳ\\mathcal\{M\}to obtain adapted model

ℳΔ⋆\\mathcal\{M\}\_\{\\Delta^\{\\star\}\}\.

17:ifCombinemodethen

18:Build the in\-context prompt with

qqand passages

𝒫\\mathcal\{P\}\.

19:else

20:Build the prompt with

qqonly \(Parametric\-only\)\.

21:endif

22:

a^←Generate​\(ℳΔ⋆,prompt\)\\hat\{a\}\\leftarrow\\textsc\{Generate\}\(\\mathcal\{M\}\_\{\\Delta^\{\\star\}\},\\text\{prompt\}\)\.

23:return

a^\\hat\{a\}\.

### B\.4Reproducibility Details

All experiments were conducted on 2× NVIDIA H100 GPUs\. We evaluate three instruction\-tuned backbones: Llama\-3\.2\-1B, Qwen2\.5\-1\.5B, and Llama\-3\-8B[Touvron et al\. 2023](https://arxiv.org/html/2608.21750#bib.bib37);[Qwen et al\. 2025](https://arxiv.org/html/2608.21750#bib.bib29)\. All compared methods share the same retriever and use top\-K=3K=3passages per query, matching the default setting in PRAG\-family evaluations\. We usebge\-base\-en\-v1\.5from SentenceTransformers for query and passage embeddings\. The Fusion Controller consists of two independent networks: a 3\-layer scoring network \(hidden dimensions: 2048, 1024\) that predicts per\-passage fusion scores, and a 2\-layer calibration network \(hidden dimension: 256\) that predicts per\-sample calibration parameters\(g,τ\)\(g,\\tau\)\. Both networks use ReLU activations and dropout rate 0\.1\. Training uses AdamW optimizer with learning rate10−410^\{\-4\}, weight decay 0\.01, batch size 32, and 10 epochs\. Following PRAG[Su et al\. 2025a](https://arxiv.org/html/2608.21750#bib.bib33), we use LoRA rankr=2r=2, scaling factorα=32\\alpha=32, and dropout=0=0\. For the multi\-hop benchmarks, we follow a fixed\-subset evaluation protocol per subtask and report subtask\-level F1 together with the 2Wiki macro average\. For retrieval perturbation analysis, we corrupt one of the three retrieved passages viareplace\-oneorrepeat\-one, and measure relative performance retention in the Parametric\-only setting\.

#### Runtime and overhead\.

The Fusion Controller operates on fixed\-size query and passage embeddings and adds only a small MLP forward pass at inference time\. Unlike global fusion calibration via grid search,FCPRAGlearns calibration parameters directly from data without backpropagating through the backbone LLM\.

### B\.5Merge\-aware Label Construction and Analysis

Figure[5](https://arxiv.org/html/2608.21750#A2.F5)gives a schematic view of the leave\-one\-out label construction process described in Section[3\.4](https://arxiv.org/html/2608.21750#S3.SS4)\.

![[Uncaptioned image]](https://arxiv.org/html/2608.21750v1/figures/merge_label_generation.png)

Figure 5:Merge\-aware label construction\. For each sample, computeF1F\_\{1\}with all adapters merged and with each adapter removed; marginal contributions are softmax\-normalized into a soft label distributionyythat supervises fusion\-weight prediction\. This procedure uses training data only\.
To test whether merge\-aware labels provide useful supervision beyond isolated adapter utility, we compare two label sources while keeping the scoring projector, backbone, and Parametric\-only evaluation setting fixed\. Table[4](https://arxiv.org/html/2608.21750#A2.T4)shows that merge\-aware labels improve macro F1 over single\-adapter labels on HotpotQA\.

Table 4:Label\-source ablation on HotpotQA \(Qwen2\.5\-1\.5B, Parametric\-only\)\.Label sourceBridgeComparisonMacroSingle\-adapter labels17\.7147\.2432\.47Merge\-aware labels17\.9549\.8733\.91Δ\\Delta\+0\.24\+2\.63\+1\.43

We also compare isolated single\-adapter utility rankings with the merge\-aware leave\-one\-out rankings\. Table[5](https://arxiv.org/html/2608.21750#A2.T5)indicates that isolated adapter behavior is a weak proxy for the full merged ranking, even when the top contributor often agrees\. ForNNretrieved adapters, label construction requires one all\-merge evaluation andNNleave\-one\-out evaluations per training sample, i\.e\.,N\+1N\+1merged evaluations; this cost is offline, uses training data only, and the resulting labels can be reused across controller variants\.

Table 5:Agreement between isolated single\-adapter utility and merge\-aware leave\-one\-out labels on HotpotQA \(Qwen2\.5\-1\.5B\)\.MetricBridgeComparisonOverallAvg\. Spearmanρ\\rho0\.00980\.01570\.0127Top\-1 match \(%\)76\.0058\.5067\.25

### B\.6Varying Number of Retrieved Passages

Table[6](https://arxiv.org/html/2608.21750#A2.T6)reports a sensitivity check on 2WikiMultiHopQA with Qwen2\.5\-1\.5B when the number of retrieved passages varies\. We include this analysis to test behavior beyond the defaultK=3K=3setting\. Parametric\-only performance is not monotonic asKKincreases, reflecting the difficulty of fusing more passage\-specific updates without also injecting more noise\. In the Combine setting, however,FCPRAGimproves over PRAG\-Combine atK=4K=4andK=5K=5\.

Table 6:Varying\-KKresults on 2WikiMultiHopQA with Qwen2\.5\-1\.5B \(F1, %\)\.MethodK=3K=3K=4K=4K=5K=5Standard RAG25\.9728\.5828\.82PRAG28\.0527\.0527\.01PRAG\-Combine26\.0431\.1233\.90FCPRAG29\.8726\.9728\.21FCPRAG\-Combine27\.3833\.6335\.08

### B\.7Dataset Overview

Table 7\. Dataset overview and subtask structure\.

DatasetDomain / FocusSubtasks2WikiMultiHopQAWikipedia\-basedmulti\-hop QABridge\-comparison; Comparison;Compositional; InferenceHotpotQAWikipedia\-basedmulti\-hop QABridge; ComparisonPopQAEntity\-centricopen\-domain QASingle taskCWQWeb\-basedcomplex QASingle task

### B\.8Training\-side Prompts

This appendix lists the key prompt templates used to construct synthetic supervision and train passage\-LoRA adapters offline\. Placeholders such as\{passage\}and\{question\}indicate runtime fields\.

#### Passage rewrite \(augmentation\)\.

Training prompt: passage rewriteRewrite the following passage\. While keeping the entities, proper nouns, and key details such as names, locations, and terminology intact, create a new version of the text that expresses the same ideas in a different way\. Make sure the revised passage is distinct from the original one, but preserves the core meaning and relevant information\. passage

#### QA\-pair generation from a passage \(augmentation\)\.

Training prompt: passage\-to\-QA generationI will provide a passage of text, and you need to generate three different questions based on the content of this passage\. Each question should be answerable using the information provided in the passage\. Additionally, please provide an appropriate answer for each question derived from the passage\. You need to generate the question and answer in the following format: \[ "question": "What is the capital of France?", "answer": "Paris", "full\_answer": "The capital of France is Paris\." \] This list should have at least three elements\. You only need to output this list in the above format\. Passage: passage

#### Passage\-LoRA fine\-tuning prompts \(offline parameterization\)\.

Given QA pairs derived from passagepip\_\{i\}, we fine\-tune a passage\-specific LoRA adapterΔi\\Delta\_\{i\}using a mix of passage\-conditioned and question\-only prompts:

Training prompt: passage\-conditioned and question\-only fine\-tuning\(A\) Passage\-conditioned: You should answer the question by referring to the knowledge provided below and integrating your own knowledge\. Passage 1: passageQuestion: question\(B\) Question\-only \(no passage in context\): You should answer the question by referring to the knowledge provided below and integrating your own knowledge\.Question: question

## Appendix CCase Studies

This section provides one real case per dataset from our main evaluation setting\. Each case reports the question, retrieved passages, rendered prompt structure, model output, and fusion signals\.

2WikiMultiHopQA \(Llama\-3\-8B\): real case \(combined inference\)Question\. Which film came out earlier, Morecambe Church Lads’ Brigade At Drill or Little Funny Guy?Retrieved passages \(top\-KK\)\. Passage 1: \.\.\.Church Lads’ and Church Girls’ Brigade \.\.\. origins in 1891 \.\.\. Passage 2: \.\.\.Church Lads’ and Church Girls’ Brigade \.\.\. historical notes \.\.\. Passage 3: \.\.\.Boys’ Life Brigade \.\.\. related youth\-brigade timeline \.\.\.Rendered prompt \(combined setting\)\. You should reference the knowledge provided below and combine it with your own knowledge to answer the question\. Please follow the format of the example I provided above\. Here are some examples about how to answer the questions\. Question: When did the director of film Hypocrite \(Film\) die? Answer: The film Hypocrite was directed by Miguel Morayta\. Miguel Morayta died on 19 June 2013\. So the answer is 19 June 2013\. Here are some reference\. Passage 1: \.\.\.Church Lads’ and Church Girls’ Brigade\.\.\. Passage 2: \.\.\.historical notes on Church Lads’ Brigade\.\.\. Passage 3: \.\.\.Boys’ Life Brigade and related organizations\.\.\. Let’s think step by step\. Answer the questions in the same format as above\. Question: Which film came out earlier, Morecambe Church Lads’ Brigade At Drill or Little Funny Guy?Model output and evaluation\. Generated text: \.\.\.Morecambe Church Lads’ Brigade At Drill \(1908\) vs Little Funny Guy \(1912\)\.\.\. Eval answer: Morecambe Church Lads’ Brigade At Drill Gold answer: Morecambe Church Lads’ Brigade at Drill EM/F1: 1 / 1\.0Fusion signals \(projector\-enabled run\)\. scores: \[0\.5088, 0\.5350, 0\.5331\] g: 0\.8357, tau: 0\.0988 weights: \[0\.8644, 1\.0767, 1\.0588\]

HotpotQA \(Qwen2\.5\-1\.5B\): real case \(combined inference\)Question\. Who has a wider scope of profession, José Echegaray or Graham Swift?Retrieved passages \(top\-KK\)\. Passage 1: María Catalina Irigoyen Echegaray \.\.\. Spanish Roman Catholic professed religious \.\.\. Passage 2: \.\.\.Beatification record of María Catalina Irigoyen Echegaray \.\.\. Passage 3: \.\.\.José Echegaray taught mathematics/hydraulics \.\.\. also served in government \.\.\.Rendered prompt \(combined setting\)\. You should reference the knowledge provided below and combine it with your own knowledge to answer the question\. Please follow the format of the example I provided above\. Here are some examples about how to answer the questions\. Question: Jeremy Theobald and Christopher Nolan share what profession? Answer: Jeremy Theobald is an actor and producer\. Christopher Nolan is a director, producer, and screenwriter\. Therefore, they both share the profession of being a producer\. So the answer is producer\. Here are some reference\. Passage 1: \.\.\.María Catalina Irigoyen Echegaray\.\.\. Passage 2: \.\.\.beatification context\.\.\. Passage 3: \.\.\.José Echegaray’s teaching and political roles\.\.\. Let’s think step by step\. Answer the questions in the same format as above\. Question: Who has a wider scope of profession, José Echegaray or Graham Swift?Model output and evaluation\. Generated text: \.\.\.José Echegaray was an engineer/politician/teacher, while Graham Swift is a novelist\.\.\. Eval answer: José Echegaray Gold answer: José Echegaray EM/F1: 1 / 1\.0Fusion signals \(projector\-enabled run\)\. scores: \[0\.4886, 0\.4649, 0\.4761\] g: 0\.8178, tau: 0\.0817 weights: \[1\.1235, 0\.8864, 0\.9901\]

PopQA \(Llama\-3\-8B\): real case \(combined inference\)Question\. What is Brigitte Bardot’s occupation?Retrieved passages \(top\-KK\)\. Passage 1: \.\.\.Fondation Brigitte Bardot \.\.\. animal protection foundation \.\.\. Passage 2: \.\.\.Brigitte Bardot Foundation \.\.\. created by French actress Brigitte Bardot \.\.\. Passage 3: \.\.\.Brigitte Bardot \.\.\. French actress, singer, dancer, and fashion model \.\.\.Rendered prompt \(combined setting\)\. You should reference the knowledge provided below and combine it with your own knowledge to answer the question\. Please follow the format of the example I provided above\. Here are some examples about how to answer the questions\. Question: When did the director of film Hypocrite \(Film\) die? Answer: The film Hypocrite was directed by Miguel Morayta\. Miguel Morayta died on 19 June 2013\. So the answer is 19 June 2013\. Here are some reference\. Passage 1: \.\.\.Fondation Brigitte Bardot\.\.\. Passage 2: \.\.\.French actress Brigitte Bardot\.\.\. Passage 3: \.\.\.Brigitte Bardot biography\.\.\. Let’s think step by step\. Answer the questions in the same format as above\. Question: What is Brigitte Bardot’s occupation?Model output and evaluation\. Generated text: \.\.\.Brigitte Bardot is a French actress, singer, dancer, and fashion model\.\.\. Eval answer: actress Gold answer: \[actor, actress, \.\.\.\] EM/F1: 1 / 1\.0Fusion signals \(projector\-enabled run\)\. scores: \[0\.5344, 0\.4883, 0\.5225\] g: 0\.7373, tau: 0\.1661 weights: \[1\.0854, 0\.8860, 1\.0286\]

ComplexWebQuestions \(Llama\-3\-8B\): real case \(combined inference\)Question\. What countries border the country whose capital is Sucre?Retrieved passages \(top\-KK\)\. Passage 1: Sucre \.\.\. constitutional capital of Bolivia\.\.\. Passage 2: What a Country\! \.\.\. \(distractor passage\) Passage 3: Capital Country \.\.\. New South Wales tourism region\.\.\. \(distractor\)Rendered prompt \(combined setting\)\. You should reference the knowledge provided below and combine it with your own knowledge to answer the question\. Please follow the format of the example I provided above\. Here are some examples about how to answer the questions\. Question: When did the director of film Hypocrite \(Film\) die? Answer: The film Hypocrite was directed by Miguel Morayta\. Miguel Morayta died on 19 June 2013\. So the answer is 19 June 2013\. Here are some reference\. Passage 1: Sucre\.\.\. constitutional capital of Bolivia\.\.\. Passage 2: What a Country\!\.\.\. Passage 3: Capital Country\.\.\. Let’s think step by step\. Answer the questions in the same format as above\. Question: What countries border the country whose capital is Sucre?Model output and evaluation\. Generated text: \.\.\.countries bordering Bolivia are Brazil, Argentina, Paraguay, and Chile\. Eval answer: Brazil Gold answer: \[Brazil, Argentina, Paraguay, Chile, \.\.\.\] EM/F1: 1 / 1\.0Fusion signals \(projector\-enabled run\)\. scores: \[0\.5126, 0\.5649, 0\.5041\] g: 0\.7724, tau: 0\.1105 weights: \[0\.8840, 1\.2808, 0\.8352\]