Large Language Models in Resolving Contextual Knowledge Conflicts

# Large Language Models in Resolving Contextual Knowledge Conflicts
Source: [https://arxiv.org/html/2609.03148](https://arxiv.org/html/2609.03148)
Xinye YangZhenyang LiuAffiliation:Northwestern University, Evanston, ILEmail:[yuanyuan\.lei@ufl\.edu](mailto:[email protected])Ruisi LiAffiliation:New York University, New York, NYYuanyuan LeiAffiliation:Computer & Information Science and Engineering, University of Florida, Gainesville, FL

###### Abstract

Most prior works focused on conflicts between an LLM’s internal parametric knowledge and externally provided context\. In contrast, we investigate how LLMs handle conflicts that arise within contextual knowledge itself\. We introduce a taxonomy of six types of contextual conflicts \(misinformation, inferential, temporal, granularity, perspective, and ambiguity\) and contribute a comprehensive datasetContextConflictfor this setting\. The dataset contains 5,781 samples, covers both reasoning and summarization tasks, and includes both explicit contradictions and implicit conflicts that require multi\-step reasoning\. Experiments on seven LLMs show that current models still fall short in resolving contextual knowledge conflicts\. We further provide mechanistic interpretability insights into how LLMs process such conflicts, revealing their latent awareness of conflicts and the representational geometry underlying conflict processing\. In addition, our analysis uncovers a consistent model bias towards earlier evidence, and this positional preference serves as a key obstacle to effective conflict resolution\. Motivated by these findings, we further propose a simple training\-free, label\-free steering method that steers activations to encourage a more comprehensive incorporation of evidences for better conflict resolution\. On our dataset, the method consistently improves accuracy on reasoning tasks and generates higher\-quality, more balanced summaries for summarization tasks\.111The link for dataset and code is:[https://github\.com/lei\-nlp\-lab/context\_conflict\_emnlp\_2026](https://github.com/lei-nlp-lab/context_conflict_emnlp_2026)\.222ContextConflictdataset is also released on Hugging Face:[https://huggingface\.co/datasets/AsherYang/ContextConflict](https://huggingface.co/datasets/AsherYang/ContextConflict)

## 1Introduction

Large language models \(LLMs\) deployed in retrieval\-augmented or multi\-document settings routinely encounter conflicting information from multiple sources\. Figure[1](https://arxiv.org/html/2609.03148#S1.F1)illustrates a representative case: when asked about social media identity\-verification policies, an LLM must synthesize opposing viewpoints that prioritize different values \(safety versus privacy\)\. The reliability of downstream applications therefore depends on how well models handle such conflicts\. When conflict resolution fails, outputs may become hallucinated, factually incorrect, or one\-sided[Shi et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib9);[Chen et al\. \(2022\)](https://arxiv.org/html/2609.03148#bib.bib4), with the failure mode extending to news aggregation, medical decision support, misinformation detection, and legal reasoning in high\-stakes settings\. Characterizing how LLMs process multi\-source conflicts is therefore a prerequisite for diagnosing and correcting these failures[Longpre et al\. \(2021\)](https://arxiv.org/html/2609.03148#bib.bib1);[Chen et al\. \(2022\)](https://arxiv.org/html/2609.03148#bib.bib4)\.

![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/knowledge_conflict.png)Figure 1:Example of contextual knowledge conflict: two contexts provide divergent perspectives\.Prior datasets on contextual knowledge conflicts exhibit four interrelated limitations\. First, they rely on template\-based synthetic construction, most commonly entity replacement, which fails to capture real\-world complexity[Su et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib10);[Longpre et al\. \(2021\)](https://arxiv.org/html/2609.03148#bib.bib1)\. Second, they focus on explicit factual contradictions and overlook implicit conflicts that require multi\-step reasoning[Su et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib10);[Lazaridou et al\. \(2021\)](https://arxiv.org/html/2609.03148#bib.bib5);[Du et al\. \(2022\)](https://arxiv.org/html/2609.03148#bib.bib6)\. Third, they offer limited domain breadth and conflict\-type coverage\. Fourth, they can suffer from class imbalance, which makes category\-level analysis statistically unreliable[Xu et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib7);[Xie et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib13)\.

To address the above gaps, we contributeContextConflict, a comprehensive multi\-domain dataset of contextual knowledge conflicts comprising 5,781 samples\. We define a taxonomy of six conflict types, includingmisinformation,inferential,temporal,granularity,perspective,ambiguityconflicts\. The first three correspond toreasoningtasks, where LLMs are required to perform reasoning and identify the correct answer from multiple conflicting evidences\. The latter three correspond tosummarizationtasks, where LLMs are expected to produce balanced summaries that preserve divergent perspectives\. Within each conflict type, we further distinguishexplicitconflicts that are identifiable by direct surface\-level comparison, andimplicitconflicts that require multi\-step cross\-evidence reasoning to resolve\. An evaluation of seven LLMs covering both closed\-source and open\-source models on this dataset show that modern LLMs still fall short in resolving contextual knowledge conflicts\.

We also provide mechanistic interpretability insights into how LLMs process contextual conflicts\. Specifically, we investigate the latent awareness and representational geometry of each conflict type using concept activation vectors[Kim et al\. \(2018\)](https://arxiv.org/html/2609.03148#bib.bib11)and spectral energy decomposition[Eckart and Young \(1936\)](https://arxiv.org/html/2609.03148#bib.bib37), respectively\. We also examine evidence attributions at both representation and output levels: at the representation level, we compare the activation geometry induced by single evidence versus combined evidence; at the output level, we quantify evidence contributions using Shapley\-based attribution scores[Lundberg and Lee \(2017\)](https://arxiv.org/html/2609.03148#bib.bib38)\. Our analysis reveals strong awareness of contextual conflicts internalized in models, with different conflict types emerging at different layer depths and represented as distinct geometry in model’s latent space\. We also uncover a consistent model bias favoring earlier\-positioned evidences, suggesting that this positional preference is a key obstacle to comprehensive evidence integration\.

To address this issue, we then design a training\-free, label\-free steering method that mitigates positional preference and encourages more comprehensive consideration of conflicting evidences\. Specifically, we construct a steering direction by computing the centroid of LLMs activations when processing each piece of evidence individually\. During inference, we nudge the model’s activations along this steering direction, guiding it to attend more evenly across evidence positions and thereby integrate evidence more comprehensively\. The results show that our simple method effectively improves LLMs’ ability to resolve knowledge conflicts, yielding more balanced summaries in summarization tasks and higher accuracy on reasoning tasks\.

Our main contributions are summarized below\.

- •We contribute a comprehensive multi\-domain dataset covering six types of contextual knowledge conflicts, spanning reasoning and summarization tasks across diverse domains
- •We present a mechanistic interpretability analysis of contextual conflict processing, revealing how each conflict are detected, geometrically represented, and processed within LLMs
- •We design a training\-free, label\-free steering method that encourages more comprehensive integration of conflicting evidences, yielding better performance in conflict resolution

## 2ContextConflict: Contextual Knowledge Conflict Dataset

### 2\.1Overview

We contribute a multi\-domain contextual knowledge conflict dataset to address the limitations of existing datasets\. We define a taxonomy of six conflict types and each conflict type probes a distinct capability, includinginferentialreasoning \(multi\-step reasoning over conflicting premises\),misinformationrobustness \(resistance to plausible disinformation\),temporalreasoning \(tracking claim validity over time\),granularityalignment \(reconciling information at different abstraction levels\),perspectiveintegration \(balancing diverse viewpoints\), andambiguityresolution \(disambiguating co\-referential entities across documents\)\. These six conflict types can be organized into two task families:Reasoningtasks \(inferential,misinformation,temporal\) require models to identify the correct answer under conflicting evidence, andSummarizationtasks \(ambiguity,perspective,granularity\) require models to produce balanced summaries that preserve divergent viewpoints\. Within each family, conflicts are further divided intoexplicitandimplicitcases\. Explicit cases are detectable by direct surface\-level comparison and require at most one inferential step\. Implicit cases emerge through multi\-step cross\-evidence reasoning and require at least two inferential steps\. We provide the per\-type implicit proportions in Table[1](https://arxiv.org/html/2609.03148#S2.T1)\.

The desired model output differs by task family\. In Reasoning tasks, each instance has a verifiable ground\-truth answer, so the model is expected to identify the correct conclusion under the conflicting evidence, and Accuracy serve as evaluation metric\. In Summarization tasks, each instance admit multiple valid responses:perspectiveandambiguityoften involve diverging political opinions or social questions, whilegranularityallows compatible answers at different specificity levels\. The model is expected to comprehensively integrate the divergent viewpoints and generate a balanced summary: forperspectiveconflicts, it should present and contrast all stances without privileging any; forambiguityconflicts, it should identify the underlying name collision and cover each referenced entity; forgranularityconflicts, it should integrate the different levels of specificity and state their compatibility\. The Shapley\-based Balance score is employed as evaluation metric to measure evenness of evidence use\. More evaluation details are in[3\.1](https://arxiv.org/html/2609.03148#S3.SS1)\.

Table 1:ContextConflictstatistics: size, average number of evidence pieces, implicit conflict proportion, and base sources for each conflict type\.
### 2\.2Dataset Composition

We construct the dataset from ten base datasets spanning diverse domains; Appendix Table[5](https://arxiv.org/html/2609.03148#A2.T5)lists each source with its domain, license, and per\-conflict\-type usage\. The dataset consists of 1,734 semi\-synthetic instances \(29\.99%\) and 4,047 preserved\-original instances manually categorized by conflict type \(70\.01%\), for a total of 5,781\.

Unlike prior synthetic conflict datasets that build conflicts via entity replacement[Su et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib10);[Longpre et al\. \(2021\)](https://arxiv.org/html/2609.03148#bib.bib1), which keeps the surrounding context unchanged, swaps only the entity, and produces shallow lexical contradictions, our semi\-synthetic instances use GPT\-5[Singh et al\. \(2025\)](https://arxiv.org/html/2609.03148#bib.bib3)only as an auxiliary generator for conflicting evidence or timestamps; gold labels are inherited from the source dataset or human re\-annotated, never produced by GPT\-5\. This preserves discourse\-level coherence in the conflicting evidence while keeping label integrity independent of the generator\.

Quality assurance is risk\-proportional, scaling with how much each construction step can corrupt labels\. Subsets whose perturbations may flip the gold label receive full re\-annotation by two independent annotators; subsets whose augmentations rarely change the label are audited on a 30% sample; preserved\-original subsets are spot\-checked at 10%\. Per\-source provenance counts are released in the metadata; Appendix[B\.1](https://arxiv.org/html/2609.03148#A2.SS1)reports the full GPT\-5 prompts and validation rules, and Appendix[B\.3](https://arxiv.org/html/2609.03148#A2.SS3)provides representative samples\.

##### Granularity and Inferential Conflicts\.

NEJM\-MedQA[Savage et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib25)combines U\.S\. medical\-licensing exam questions with real clinical cases from the*New England Journal of Medicine*\. Each instance provides clinical evidence \(symptoms, lab results, patient history\) and several candidate reasoning chains\. Each chain is generated under a distinct diagnostic\-reasoning prompt strategy from the source dataset and carries a binary gold\-correctness flag adjudicated by clinicians\. We select cases where two chains disagree on the final diagnosis\. When both diagnoses are gold\-correct but at different abstraction levels \(e\.g\., a broad syndrome label versus a specific pathogen it subsumes\), we label agranularityconflict: the two answers are compatible and differ only in diagnostic specificity\. When only one diagnosis is gold\-correct and the other reaches a wrong conclusion through a flawed intermediate step \(e\.g\., a misapplied clinical heuristic\), we label aninferentialconflict: the disagreement is at the reasoning\-process level\. The original question and reasoning chains are preserved as evidence\.

##### Inferential Conflicts\.

FOLIO[Han et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib26)and ENTAILMENTBANK[Dalvi et al\. \(2021\)](https://arxiv.org/html/2609.03148#bib.bib24)are logical inference datasets\. Each instance gives a set of premises and a hypothesis, labeled by whether the premises entail, contradict, or are neutral to the hypothesis\. We add one or two extra statements that perturb the original reasoning chain\. Because the added evidence can change label validity, every resulting instance is re\-annotated by two independent annotators, with disagreements resolved by discussion \(100% double\-annotation coverage; annotation interface in Appendix Figure[8](https://arxiv.org/html/2609.03148#A2.F8)\)\.

##### Misinformation Conflicts\.

SciFact[Wadden et al\. \(2020\)](https://arxiv.org/html/2609.03148#bib.bib29)is a fact\-verification dataset\. Each instance pairs a claim with evidence sentences from research abstracts, labeled as supporting or refuting the claim\. We generate conflicting evidence in a matching writing style, including experimental\-style citations, to buildmisinformationconflicts\. Because this augmentation rarely changes gold labels, we audit a 30% sample for stylistic and argumentative consistency\.

##### Temporal Conflicts\.

ConflictBank\-temporal[Su et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib10)instance contains evidence statements with an implicit chronological order and a time\-sensitive question\. We attach an explicit timestamp to each statement, without modifying the original evidence, so that temporal relationships become unambiguous\. Because timestamps can change gold labels, each instance is verified by two independent annotators with discussion\-based adjudication\.

##### Curation of Remaining Datasets\.

For datasets used without synthetic generation \(AmbigDocs, ROAST\-ABSA, AllSides, Perspectrum, CONFLICTS\), we randomly sample 10% of each source for quality review and conflict\-category validation\.

## 3Evaluation

### 3\.1Evaluation Metrics

We evaluate model performance using complementary metrics with task\-specific emphasis\.

Accuracy\.Forreasoning tasks\(inferential,misinformation, andtemporalconflicts\), we measure accuracy as the proportion of model outputs that match the gold\-standard labels\.

Evidence Balance\.Forsummarization tasks, all evidence sources are equally valid despite their conflicting content, so we quantify how evenly a response integrates them with a Shapley\-based attribution framework\. We assign each evidence piece a Shapley\-value contribution to the response’s likelihood and then measure how unequally these contributions are distributed\. Givennnevidence pieces, index setN=\{1,…,n\}N=\\\{1,\\ldots,n\\\}, and model responseRR, the marginal contribution of pieceiiis

ϕi=∑S⊆N∖\{i\}1n​\(n−1\|S\|\)​\(v⁡\(S∪\{i\}\)−v⁡\(S\)\),\\phi\_\{i\}=\\sum\_\{S\\subseteq N\\setminus\\\{i\\\}\}\\frac\{1\}\{n\\binom\{n\-1\}\{\|S\|\}\}\\bigl\(v\(S\\cup\\\{i\\\}\)\-v\(S\)\\bigr\),\(1\)wherev⁡\(S\)=1\|R\|​∑t=1\|R\|log⁡pscorer​\(rt∣r<t,ℰS\)v\(S\)=\\frac\{1\}\{\|R\|\}\\sum\_\{t=1\}^\{\|R\|\}\\log p\_\{\\text\{scorer\}\}\(r\_\{t\}\\mid r\_\{<t\},\\mathcal\{E\}\_\{S\}\)is the length\-normalized log\-likelihood ofRRunder a frozen external scorer \(default: Llama\-3\.2\-1B; scorer\-size sensitivity is examined in Section[3\.2](https://arxiv.org/html/2609.03148#S3.SS2)\)\. We clip negative contributions, which arise when the response contradicts a piece, and normalize the non\-negative mass into a share distributionpi=max⁡\(0,ϕi\)/∑jmax⁡\(0,ϕj\)p\_\{i\}=\\max\(0,\\phi\_\{i\}\)/\\sum\_\{j\}\\max\(0,\\phi\_\{j\}\)\. We then report Balance as the normalized Gini coefficient of𝐩\\mathbf\{p\}:

Balance\(R\)=nn−1⋅1n∑i=1n\(2i−n−1\)p\[i\],\\text\{Balance\}\(R\)=\\frac\{n\}\{n\-1\}\\cdot\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\(2i\-n\-1\)\\,p\_\{\[i\]\},\(2\)wherep\[1\]≤⋯≤p\[n\]p\_\{\[1\]\}\\leq\\cdots\\leq p\_\{\[n\]\}are sorted contributions\. Lower scores indicate more balanced integration \(0 = perfect equality; 1 = maximum inequality\)\.

Faithfulness\.333Computed using RAGAS[Es et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib36):[https://github\.com/vibrantlabsai/ragas](https://github.com/vibrantlabsai/ragas)To detect hallucinated content that goes beyond the given evidence in conflict scenarios, we report Faithfulness, applicable to all conflict types\. It decomposes a response into atomic claims and reports the proportion judged supported by the given evidence\. Faithfulness measures grounding rather than factual correctness: a factually accurate response still scores low if some claims are not grounded in the provided evidence\.

### 3\.2Validating the Balance Metric

Unlike established metrics such as Accuracy and Faithfulness, Evidence Balance relies on our Shapley\-based attribution framework\. We confirm its reliability with three independent checks: a human and LLM agreement study, a causal intervention on the same human\-verified subset, and a scorer\-size robustness check\.

Human Evaluation and LLM\-as\-a\-Judge\.To assess agreement between our automatic metric and human judgment of evidence balance, we collect annotations from two independent humans and a GPT\-5 LLM\-as\-a\-Judge on 108 samples spanning three models and three summarization tasks\. Our metric reachesκ=0\.50\\kappa=0\.50–0\.520\.52against the human annotators \(Table[2](https://arxiv.org/html/2609.03148#S3.T2)\), comparable to inter\-human agreement \(κ=0\.52\\kappa=0\.52\) and to the LLM\-as\-a\-Judge \(κ=0\.51\\kappa=0\.51\)\. Annotation setup, interface, and per\-model Balance appear in Appendix[C\.2](https://arxiv.org/html/2609.03148#A3.SS2)\.

Causal Intervention\.To probe whether Shapley rankings reflect actual evidence dependence, we run a causal\-intervention check on the human\-annotated subset above, restricted to the two open\-weight models where each model’s own log\-probabilities are accessible \(Llama\-3\.1\-8B\-Instruct and GPT\-OSS\-20B\), giving 72 already human\-verified samples\. We remove the highest\- and lowest\-contributing evidence piece and measure the change in each model’s own log\-likelihood\. As Table[3](https://arxiv.org/html/2609.03148#S3.T3)shows, removing the highest\-contributing evidence causes substantially larger drops than removing the lowest, indicating that attribution rankings align with actual evidence dependence\.

Scorer\-Size Robustness\.To rule out artifacts of the 1B default scorer, we rerun the Shapley computation with two alternative scorers on the three summarization tasks: one larger from the same family \(Llama\-3\.1\-8B\) and one comparable in size from a different family \(Gemma\-2B\)\. As shown in Appendix Table[6](https://arxiv.org/html/2609.03148#A3.T6), absolute Balance values shift but the ranking of evaluated models is essentially unchanged, confirming that our results do not depend on the default scorer\.

Table 2:Inter\-annotator agreement \(Cohen’s Kappa\)\.Table 3:Causal intervention on the 72 human\-verified samples\.Δhigh=ℓfull−ℓhigh\\Delta\_\{\\text\{high\}\}\{=\}\\ell\_\{\\text\{full\}\}\{\-\}\\ell\_\{\\text\{high\}\},Δlow=ℓfull−ℓlow\\Delta\_\{\\text\{low\}\}\{=\}\\ell\_\{\\text\{full\}\}\{\-\}\\ell\_\{\\text\{low\}\}, using each model’s own log\-likelihood\.
### 3\.3Evaluation Results

Table 4:Performance across all conflict tasks \(%\)\. Acc▲\\blacktriangleand Fth▲\\blacktriangle\(higher is better\) are reported for reasoning; Bal▼\\blacktriangledown\(lower is better\) and Fth▲\\blacktrianglefor summarization\. All\-Tokens and First\-Generated are our two injection schedules ofu\(l\)u^\{\(l\)\}\(§[5](https://arxiv.org/html/2609.03148#S5)\); CAS is the Context\-Aware Steering alternative direction we compare against \(§[5](https://arxiv.org/html/2609.03148#S5)\)\. Best and second\-best within each model block arebold/underlined\.Task Difficulty and Model Scaling\.Table[4](https://arxiv.org/html/2609.03148#S3.T4)reports performance across seven base models and six conflict categories\. On reasoning tasks, we observe a difficulty hierarchy: Temporal reaches the highest accuracy \(up to 68\.0%\), followed by Misinformation \(up to 61\.5%\) and Inferential \(up to 44\.3%\), indicating increasing difficulty\. Performance scales with model capacity \(Inferential: 24\.8% on llama\-3\.1\-8b\-instruct vs\. 44\.3% on claude\-4\.5\-sonnet\), yet Inferential stays below 50% even for the strongest model, suggesting multi\-step reasoning under conflict is a shared SOTA bottleneck\.

Pervasive Position Bias\.Our Shapley\-based attribution reveals strong evidence\-position bias in summarization tasks\. As shown in Appendix Figure[23](https://arxiv.org/html/2609.03148#A4.F23)for Llama\-3\.1\-8B\-Instruct, Ambiguity shows the strongest bias, with the first evidence contributing 69\.0%, while Perspective and Granularity are less extreme but still imbalanced\. Even top\-performing models remain far from perfectly balanced integration\. Detailed per\-model evidence distributions are reported in Appendix[D\.5](https://arxiv.org/html/2609.03148#A4.SS5)\. Balance is also not monotonic in capacity \(GPT\-OSS\-120B 31\.9 vs\. 20B 39\.8\), suggesting that scale alone cannot eliminate position bias and motivating the correction in Section[5](https://arxiv.org/html/2609.03148#S5)\.

Faithfulness Patterns\.Faithfulness is higher on summarization than on reasoning and is not tightly coupled with Balance \(Table[4](https://arxiv.org/html/2609.03148#S3.T4)\): GPT\-OSS\-20B on Perspective reaches 92\.5% Faithfulness while its Balance stays at 34\.5, showing that a model can be simultaneously well\-grounded in the provided evidence and positionally biased, so the bias operates primarily at the evidence\-selection stage\.

## 4Analysis

We conduct three complementary analyses to understand how LLMs internally process conflicting knowledge\. \(i\)Conflict awarenessmeasures the initial detection point via the linear separability of hidden states\. \(ii\)Representational geometryexamines structural segregation via spectral energy analysis\. \(iii\)Position biastraces how evidence order skews latent representations and final outputs via Shapley attribution\. Together, these lenses provide a layered map of how conflict is encoded, organized, and resolved\.

Our mechanistic analyses focus on Llama\-3\.1\-8B\-Instruct\. To ensure our findings reflect fundamental cognitive mechanisms rather than architectural quirks, we replicate key trends on GPT\-OSS\-20B[OpenAI et al\. \(2025\)](https://arxiv.org/html/2609.03148#bib.bib2), which differs in both lineage and scale\. Appendix[D\.4](https://arxiv.org/html/2609.03148#A4.SS4)reports the layer\-wise AUC,Δ​ER\\Delta\\text\{ER\}curves, and projections for this model\. The consistent conflict\-type ordering and emergence patterns across both architectures confirm that these conflict\-processing mechanisms are robust, inherent behaviors of modern LLMs\.

### 4\.1Conflict Awareness via Concept Vectors

![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/auc_comparison_llama8b.png)Figure 2:Layer\-wise AUC for conflict vs\. consistent sample classification\. Higher AUC indicates stronger conflict awareness\.To understand how LLMs process confliction, we must first verify if they internally register it\. Our core premise is that if a model possesses latent conflict awareness, its hidden states for conflicting inputs should be linearly separable from consistent ones\. We quantify this by constructing paired datasets for each conflict typett, consisting of a conflict versionxconf\(t\)x\_\{\\text\{conf\}\}^\{\(t\)\}and a consistent versionxcons\(t\)x\_\{\\text\{cons\}\}^\{\(t\)\}\(details in Appendix[D\.1](https://arxiv.org/html/2609.03148#A4.SS1)\)\. At each layerll, we train a linear logistic\-regression probe onhl​\(x\)h\_\{l\}\(x\)and measure the classification performance via AUC:

AUCl\(t\)=AUC​\(\{\(hl​\(xconf\(t\)\),1\)\},\{\(hl​\(xcons\(t\)\),0\)\}\)\\text\{AUC\}\_\{l\}^\{\(t\)\}=\\text\{AUC\}\\big\(\\\{\(h\_\{l\}\(x\_\{\\text\{conf\}\}^\{\(t\)\}\),1\)\\\},\\\{\(h\_\{l\}\(x\_\{\\text\{cons\}\}^\{\(t\)\}\),0\)\\\}\\big\)\(3\)This approach allows us to map the precise trajectory of conflict awareness\. We reveal not only if the model detects a contradiction, but exactly where and how it emerges within the architecture\.

Our findings show that models exhibit robust, layer\-wise conflict awareness, with signal emergence tied to semantic complexity\. Figure[2](https://arxiv.org/html/2609.03148#S4.F2)illustrates that most conflict types reach high separability \(AUC \> 0\.85\) in mid\-to\-late layers, yet their developmental paths differ\.Temporalandambiguityconflicts saturate earliest, as they rely on explicit markers handled by early syntactic processing[Tenney et al\. \(2019\)](https://arxiv.org/html/2609.03148#bib.bib33)\. Conversely,inferentialandmisinformationconflicts emerge later\. These require multi\-step reasoning or world knowledge, which rely on abstract semantic representations from deeper layers\. Other types show non\-monotonic patterns:granularityfollows a U\-shape, whileperspectiveconflicts peak in middle layers before declining during viewpoint reconciliation\. These trends demonstrate that conflict awareness is not a single trigger\. It is a dynamic, heterogeneous process that aligns with the model’s progressive semantic refinement, as confirmed across multiple models in Appendix[D\.2](https://arxiv.org/html/2609.03148#A4.SS2)\.

![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/bias_simple_prompt_all_conflicts.png)Figure 3:Layer\-wise directional projection\.c\(l\)c^\{\(l\)\}consistently aligns with specific evidence directions, deviating from uniform\.
### 4\.2Spectral Energy Analysis Reveals Conflict Dimensionality

Spectral energy analysis allows us to look beyondwhethera model detects a conflict, and instead uncoverhowthat information is geometrically organized in the latent space\. Our core motivation is to determine the rank structure of hidden states: do conflict representations concentrate along a few dominant directions, or do they disperse across many dimensions? Answering this is vital, as it dictates the optimal subspace for targeted steering interventions\. To capture this geometry, we analyze the rank structure via the spectral energy of the activations\.

For each conflict type, we extract hidden states at the last non\-padding token, forming matrices𝐇conf\(l\),𝐇cons\(l\)∈ℝn×d\\mathbf\{H\}\_\{\\text\{conf\}\}^\{\(l\)\},\\mathbf\{H\}\_\{\\text\{cons\}\}^\{\(l\)\}\\in\\mathbb\{R\}^\{n\\times d\}\. We center each matrix as𝐇~=𝐇−𝐇¯\\tilde\{\\mathbf\{H\}\}=\\mathbf\{H\}\-\\bar\{\\mathbf\{H\}\}and calculate the energy ratio \(ER\) and its delta:

ER=∑i=1kσi2‖𝐇~‖F2,Δ​ERl\(t\)=ERl,conf\(t\)−ERl,cons\(t\)\\text\{ER\}=\\frac\{\\sum\_\{i=1\}^\{k\}\\sigma\_\{i\}^\{2\}\}\{\\\|\\tilde\{\\mathbf\{H\}\}\\\|\_\{F\}^\{2\}\},\\;\\Delta\\text\{ER\}\_\{l\}^\{\(t\)\}=\\text\{ER\}\_\{l,\\text\{conf\}\}^\{\(t\)\}\-\\text\{ER\}\_\{l,\\text\{cons\}\}^\{\(t\)\}\(4\)Here,σ1,…,σk\\sigma\_\{1\},\\ldots,\\sigma\_\{k\}represent the top\-kksingular values \(withk=10k\{=\}10\)\. A positiveΔ​ER\\Delta\\text\{ER\}signifies low\-rank compression in dominant directions, while a negativeΔ​ER\\Delta\\text\{ER\}indicates dispersion across the broader dimensional space\. Further implementation details are provided in Appendix[D\.3](https://arxiv.org/html/2609.03148#A4.SS3)\.

![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/delta_er.png)Figure 4:Delta energy ratio across layers\.Δ​ER\>0\\Delta\\text\{ER\}\>0indicates concentrated representations;Δ​ER<0\\Delta\\text\{ER\}<0indicates dispersed ones\.The resulting geometric patterns in Figure[4](https://arxiv.org/html/2609.03148#S4.F4)demonstrate that models handle different conflict types through distinct dimensional strategies\.Temporalconflicts compress into low\-rank representations \(Δ​ER\>0\\Delta\\text\{ER\}\>0\), effectively reducing the conflict to the singular dimension of event sequencing\. Conversely,granularityandambiguityconflicts disperse across dimensions \(Δ​ER<0\\Delta\\text\{ER\}<0\); the former spans multiple levels of specificity, while the latter activates parallel lexical interpretations\.Inferentialandmisinformationconflicts hover near zero, suggesting that reasoning processes reweight existing features rather than reorganizing the latent geometry\. Finally,perspectiveconflicts exhibit strong layer\-dependence, reflecting the gradual emergence of stance as an abstract property\. These distinctions indicate that the effective rank change is not merely a byproduct of conflict presence, but a diagnostic signal of how each conflict type is internally structured, represented, and resolved\. Collectively, these trends are qualitatively consistent across models in Appendix[D\.4](https://arxiv.org/html/2609.03148#A4.SS4)and reveal that conflict resolution is not a one\-size\-fits\-all geometric process, but is instead dynamically tailored to the nature of the contradiction\.

### 4\.3Evidence Position Bias in Internal Representations

We investigate "position bias" in LLMs during conflict resolution\. Unlike standard retrieval tasks where bias often stems from irrelevant noise, our setting involves multiple legitimate but contradictory evidence pieces\. The observed pattern therefore reveals an*implicit trust allocation*: the model’s priority under direct competition\.

Standard mitigations like reordering or positional adjustments are ineffective here, as they assume a single correct position or noise\. Our empirical tests in Appendix[D\.8](https://arxiv.org/html/2609.03148#A4.SS8)confirm that random reordering fails to override the dominance of Evidence 1\. As shown in Section[3\.3](https://arxiv.org/html/2609.03148#S3.SS3), models disproportionately rely on the first evidence regardless of content\. We analyze this systematic bias through two lenses: activation\-space geometry and output\-level Shapley attribution\.

![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/method.png)Figure 5:Three\-stage activation steering method: \(1\) collect single\-evidence and combined\-evidence activations from calibration samples, \(2\) compute layer\-wise steering directions toward neutral integration, \(3\) apply corrections during generation to mitigate positional bias\.Representation\-Level Analysis\.To test whether the bias originates at the representation level rather than at decoding, we compare the model’s combined\-evidence activation against an unbiased reference built from single\-evidence activations\. For a sample withKKpieces, we build one combined\-evidence prompt andKKsingle\-evidence prompts\. At each layerll, we extract the final\-token residual\-stream activations:c\(l\)c^\{\(l\)\}from the combined prompt, andai\(l\)a\_\{i\}^\{\(l\)\}from theii\-th single\-evidence prompt\. If the model integrates all evidence neutrally,c\(l\)c^\{\(l\)\}should align with theneutral centerμ\(l\)=1K​∑ai\(l\)\\mu^\{\(l\)\}=\\frac\{1\}\{K\}\\sum a\_\{i\}^\{\(l\)\}\. We quantify this alignment using the projection:

bi\(l\)=\(c\(l\)−μ\(l\)\)⋅\(ai\(l\)−μ\(l\)\)\(∥c\(l\)−μ\(l\)∥\+ϵb\)​\(∥ai\(l\)−μ\(l\)∥\+ϵb\)b\_\{i\}^\{\(l\)\}=\\frac\{\(c^\{\(l\)\}\-\\mu^\{\(l\)\}\)\\cdot\(a\_\{i\}^\{\(l\)\}\-\\mu^\{\(l\)\}\)\}\{\\bigl\(\\lVert c^\{\(l\)\}\-\\mu^\{\(l\)\}\\rVert\+\\epsilon\_\{b\}\\bigr\)\\bigl\(\\lVert a\_\{i\}^\{\(l\)\}\-\\mu^\{\(l\)\}\\rVert\+\\epsilon\_\{b\}\\bigr\)\}\(5\)As shown in Figure[3](https://arxiv.org/html/2609.03148#S4.F3), this geometric bias persists across all layers\. The combined representation consistently shifts toward the first evidence\. This suggests the bias is a fundamental representation\-level effect rather than a late\-stage decoding artifact\.

Output\-Level Corroboration\.Our Shapley\-based attribution framework further confirms this first\-evidence dominance\. As seen in Appendix Figure[23](https://arxiv.org/html/2609.03148#A4.F23), Evidence 1 disproportionately drives model outputs: accounting for 69\.0% of contributions in Ambiguity, 38\.4% in Granularity, and 35\.0% in Perspective\. These values significantly exceed uniform baselines\. Across all seven tested models, ranging from 8B to 120B parameters, Evidence 1 dominance remains a consistent, systematic artifact\. This reinforces the finding that models possess an inherent architectural preference for early evidence\.

## 5Activation Steering for Effective Conflict Resolution

We propose a training\-free, label\-free activation steering method to mitigate position bias\. Our previous analysis \(Section[4](https://arxiv.org/html/2609.03148#S4)\) demonstrates that position bias is encoded as a geometric asymmetry in the residual stream\. This suggests that the intervention must operate directly within the hidden\-state space rather than at the input or decoding stages\. If evidence were integrated uniformly, the combined\-prompt activationc\(l\)c^\{\(l\)\}would align with a position\-agnostic neutral centerμ\(l\)\\mu^\{\(l\)\}\. Since the observed gapc\(l\)−μ\(l\)c^\{\(l\)\}\-\\mu^\{\(l\)\}consistently points toward the first evidence, we steer the representation by translatingc\(l\)c^\{\(l\)\}back toward the neutral center\. As illustrated in Figure[5](https://arxiv.org/html/2609.03148#S4.F5), our method comprises three stages: activation collection, direction computation, and additive injection\.

Activation Collection and Direction Computation\.We partition the data into calibration \(20%\) and test sets \(80%\)\. For each calibration samplenn, we extract the final\-token residual\-stream activations at layerll:\{an,i\(l\)\}\\\{a\_\{n,i\}^\{\(l\)\}\\\}fromKKsingle\-evidence prompts andcn\(l\)c\_\{n\}^\{\(l\)\}from the combined prompt\. We calculate the neutral centerμn\(l\)=1K​∑i=1Kan,i\(l\)\\mu\_\{n\}^\{\(l\)\}=\\frac\{1\}\{K\}\\sum\_\{i=1\}^\{K\}a\_\{n,i\}^\{\(l\)\}and average these values across the calibration set intoc¯\(l\)\\bar\{c\}^\{\(l\)\}andμ¯\(l\)\\bar\{\\mu\}^\{\(l\)\}\. We define our steering direction as:

u\(l\)=μ¯\(l\)−c¯\(l\)max⁡\(‖μ¯\(l\)−c¯\(l\)‖,ϵu\)u^\{\(l\)\}=\\frac\{\\bar\{\\mu\}^\{\(l\)\}\-\\bar\{c\}^\{\(l\)\}\}\{\\max\\\!\\left\(\\\|\\bar\{\\mu\}^\{\(l\)\}\-\\bar\{c\}^\{\(l\)\}\\\|,\\,\\epsilon\_\{u\}\\right\)\}\(6\)This fixed directionu\(l\)u^\{\(l\)\}is then applied to all test instances\.

Steered Inference\.At inference time, we apply the precomputed direction to nudge the residual stream toward the neutral center:ht\(l\)←ht\(l\)\+α⋅u\(l\)h\_\{t\}^\{\(l\)\}\\leftarrow h\_\{t\}^\{\(l\)\}\+\\alpha\\cdot u^\{\(l\)\}withα=1\.0\\alpha=1\.0\. We evaluate two injection schedules:First\-Generated\(first step only\) andAll\-Tokens\(every step\)\. As shown in Table[4](https://arxiv.org/html/2609.03148#S3.T4), both models achieve significant performance gains under All\-Tokens\. Llama\-3\.1\-8B\-Instruct shows improved summarization balance and substantial accuracy gains in reasoning \(e\.g\., \+19\.2 points on Temporal\)\. These improvements provide causal and targeted validation of our claim that the observed bias is rooted in the internal representation\.

Analysis and Comparisons\.To verify that the gains come from our contrastive design rather than generic context steering, we compareu\(l\)u^\{\(l\)\}withContext\-Aware Steering \(CAS\)\. CAS amplifies the model’s overall reliance on context, whereasu\(l\)u^\{\(l\)\}targets the evidence\-integration axis identified in our analysis\. Our direction wins five of six model×\\timesreasoning comparisons and remains robust on summarization balance, showing that mechanistically grounded representation\-level rebalancing is more effective for tasks requiring selective cross\-evidence integration\.

The one exception is GPT\-OSS\-20B Temporal, where CAS outperforms ouru\(l\)u^\{\(l\)\}\(Accuracy 65\.5 vs\. 64\.1\)\. We attribute this to task\-mechanism alignment: temporal reasoning requires using every timestamped piece uniformly, so the task itself rewards CAS’s blanket amplification of context reliance and offers little headroom for the selective rebalancing our direction is designed for\. The boundary clarifies when each direction is preferred: CAS suits tasks demanding uniform integration; ouru\(l\)u^\{\(l\)\}suits tasks requiring selective cross\-evidence integration, which covers most reasoning conflicts\.

The All\-Tokens schedule yields the largest accuracy gains; the minor decrease in reasoning faithfulness reflects a shift toward multi\-evidence reasoning that exceeds verbatim grounding\.

## 6Conclusion

We release a 5,781\-sample multi\-domain dataset of contextual knowledge conflicts spanning six conflict types across reasoning and summarization tasks\. Our mechanistic analysis shows that LLMs often detect conflicts internally, yet still allocate disproportionate trust to earlier evidence, producing representation\-level positional bias\. Based on this finding, we propose a training\-free, label\-free activation steering method that mitigates this bias and improves evidence integration under conflict\.

## Limitations and Future Work

We acknowledge several limitations\. First, our method requires white\-box access to residual\-stream activations, so it applies only to open\-weight models and adds inference\-time overhead\. Appendix[D\.5](https://arxiv.org/html/2609.03148#A4.SS5)shows the same position bias in closed\-source systems, so the phenomenon itself is not limited to white\-box settings\. Second, compute constraints restrict our steering experiments to GPT and LLaMA models at the 8B and 20B scale; scaling to 70B or 120B is left for future work\. Third, our dataset uses synthetic samples for controlled manipulation; evaluating on noisy, real\-world retrieval with heterogeneous evidence remains open\. Fourth, our Shapley\-based attribution assumes each evidence piece contributes equally in expectation, which may not hold when evidence quality varies in real\-world pipelines\. A weighted Shapley formulation with source\-reliability priors, used alongside Faithfulness, is a promising extension\.

## Ethical considerations

Our dataset contains synthetically generated misinformation designed to test model robustness against false information\. The dataset does not include personally identifying information or offensive content\. To mitigate potential risks of misuse, we clearly label each evidence piece as factually correct or incorrect in the dataset metadata\. We emphasize that the misinformation samples are constructed solely for research purposes to evaluate conflict handling capabilities and should not be used to train models for generating misleading content\. We will include explicit usage guidelines with the dataset release to prevent misuse\. All base datasets used in constructing this dataset are publicly available and published through established academic channels, governed by open\-access licenses that permit research reuse\.

## Acknowledgments

We thank the University of Florida Research Computing HiPerGator for providing computational resources and UF NaviGator for providing access to LLM APIs\. We also acknowledge Delta at the National Center for Supercomputing Applications through allocation CIS251209 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support \(ACCESS\) program, which is supported by U\.S\. National Science Foundation grants \#2138259, \#2138286, \#2138307, \#2137603, and \#2138296\.

## References

- Anet al\.\(2024\)S\. An, Z\. Ma, Z\. Lin, N\. Zheng, J\. Lou, and W\. ChenMake your llm fully utilize the context\.InAdvances in Neural Information Processing Systems,A\. Globerson, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. Tomczak, and C\. Zhang \(Eds\.\),Vol\.37,pp\. 62160–62188\.External Links:[Document](https://dx.doi.org/10.52202/079017-1986),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/file/71c3451f6cd6a4f82bb822db25cea4fd-Paper-Conference.pdf)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px5.p1.1)\.
- Cattanet al\.\(2025\)A\. Cattan, A\. Jacovi, O\. Ram, J\. Herzig, R\. Aharoni, S\. Goldshtein, E\. Ofek, I\. Szpektor, and A\. CaciularuDRAGged into conflicts: detecting and addressing conflicting sources in search\-augmented llms\.External Links:2506\.08500,[Link](https://arxiv.org/abs/2506.08500)Cited by:[Table 5](https://arxiv.org/html/2609.03148#A2.T5.2.20.1.1.1)\.
- Cheboluet al\.\(2024\)S\. U\. S\. Chebolu, F\. Dernoncourt, N\. Lipka, and T\. SolorioROAST: review\-level opinion aspect sentiment target joint detection for absa\.External Links:2405\.20274,[Link](https://arxiv.org/abs/2405.20274)Cited by:[Table 5](https://arxiv.org/html/2609.03148#A2.T5.2.14.1.1.1)\.
- Chenet al\.\(2022\)H\. Chen, M\. Zhang, and E\. ChoiRich knowledge sources bring complex knowledge conflicts: recalibrating models to reflect conflicting evidence\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,Y\. Goldberg, Z\. Kozareva, and Y\. Zhang \(Eds\.\),Abu Dhabi, United Arab Emirates,pp\. 2292–2307\.External Links:[Link](https://aclanthology.org/2022.emnlp-main.146/),[Document](https://dx.doi.org/10.18653/v1/2022.emnlp-main.146)Cited by:[§1](https://arxiv.org/html/2609.03148#S1.p1.1)\.
- Chenet al\.\(2025\)J\. Chen, B\. Bi, W\. Zhang, J\. Sui, X\. Zhu, Y\. Wang, L\. Mei, and S\. LiuRethinking all evidence: enhancing trustworthy retrieval\-augmented generation via conflict\-driven summarization\.External Links:2507\.01281,[Link](https://arxiv.org/abs/2507.01281)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px2.p1.1)\.
- Chenet al\.\(2019\)S\. Chen, D\. Khashabi, W\. Yin, C\. Callison\-Burch, and D\. RothSeeing things from a different angle:discovering diverse perspectives about claims\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),J\. Burstein, C\. Doran, and T\. Solorio \(Eds\.\),Minneapolis, Minnesota,pp\. 542–557\.External Links:[Link](https://aclanthology.org/N19-1053/),[Document](https://dx.doi.org/10.18653/v1/N19-1053)Cited by:[Table 5](https://arxiv.org/html/2609.03148#A2.T5.2.12.1.1.1)\.
- Choiet al\.\(2025\)E\. Choi, J\. Park, H\. Lee, and J\. LeeConflict\-aware soft prompting for retrieval\-augmented generation\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 26969–26983\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.1371/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1371),ISBN 979\-8\-89176\-332\-6Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px2.p1.1)\.
- Dalviet al\.\(2021\)B\. Dalvi, P\. Jansen, O\. Tafjord, Z\. Xie, H\. Smith, L\. Pipatanangkura, and P\. ClarkExplaining answers with entailment trees\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,M\. Moens, X\. Huang, L\. Specia, and S\. W\. Yih \(Eds\.\),Online and Punta Cana, Dominican Republic,pp\. 7358–7370\.External Links:[Link](https://aclanthology.org/2021.emnlp-main.585/),[Document](https://dx.doi.org/10.18653/v1/2021.emnlp-main.585)Cited by:[Table 5](https://arxiv.org/html/2609.03148#A2.T5.2.5.1.1.1),[§2\.2](https://arxiv.org/html/2609.03148#S2.SS2.SSS0.Px2.p1.1)\.
- Duet al\.\(2022\)Y\. Du, A\. Bosselut, and C\. D\. ManningSynthetic disinformation attacks on automated fact verification systems\.Proceedings of the AAAI Conference on Artificial Intelligence36\(10\),pp\. 10581–10589\.External Links:[Link](https://ojs.aaai.org/index.php/AAAI/article/view/21302),[Document](https://dx.doi.org/10.1609/aaai.v36i10.21302)Cited by:[§1](https://arxiv.org/html/2609.03148#S1.p2.1)\.
- Eckart and Young \(1936\)C\. Eckart and G\. YoungThe approximation of one matrix by another of lower rank\.Psychometrika1\(3\),pp\. 211–218\.External Links:ISSN 1860\-0980,[Document](https://dx.doi.org/10.1007/BF02288367),[Link](https://doi.org/10.1007/BF02288367)Cited by:[§1](https://arxiv.org/html/2609.03148#S1.p4.1)\.
- Eset al\.\(2024\)S\. Es, J\. James, L\. Espinosa Anke, and S\. SchockaertRAGAs: automated evaluation of retrieval augmented generation\.InProceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations,N\. Aletras and O\. De Clercq \(Eds\.\),St\. Julians, Malta,pp\. 150–158\.External Links:[Link](https://aclanthology.org/2024.eacl-demo.16/),[Document](https://dx.doi.org/10.18653/v1/2024.eacl-demo.16)Cited by:[footnote 3](https://arxiv.org/html/2609.03148#footnote3)\.
- Hanet al\.\(2024\)S\. Han, H\. Schoelkopf, Y\. Zhao, Z\. Qi, M\. Riddell, W\. Zhou, J\. Coady, D\. Peng, Y\. Qiao, L\. Benson, L\. Sun, A\. Wardle\-Solano, H\. Szabó, E\. Zubova, M\. Burtell, J\. Fan, Y\. Liu, B\. Wong, M\. Sailor, A\. Ni, L\. Nan, J\. Kasai, T\. Yu, R\. Zhang, A\. Fabbri, W\. M\. Kryscinski, S\. Yavuz, Y\. Liu, X\. V\. Lin, S\. Joty, Y\. Zhou, C\. Xiong, R\. Ying, A\. Cohan, and D\. RadevFOLIO: natural language reasoning with first\-order logic\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 22017–22031\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.1229/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.1229)Cited by:[Table 5](https://arxiv.org/html/2609.03148#A2.T5.2.9.1.1.1),[§2\.2](https://arxiv.org/html/2609.03148#S2.SS2.SSS0.Px2.p1.1)\.
- Hubenet al\.\(2024\)R\. Huben, H\. Cunningham, L\. R\. Smith, A\. Ewart, and L\. SharkeySparse autoencoders find highly interpretable features in language models\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=F76bwRSLeK)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px4.p1.1)\.
- Javadiet al\.\(2026\)S\. Javadi, S\. Mirabi, M\. Gangar, and B\. OfoghiContradictions in context: challenges for retrieval\-augmented generation in healthcare\.InAdvances in Information Retrieval: 48th European Conference on Information Retrieval, ECIR 2026, Delft, The Netherlands, March 29 – April 2, 2026, Proceedings, Part I,Berlin, Heidelberg,pp\. 34–48\.External Links:ISBN 978\-3\-032\-21288\-7,[Link](https://doi.org/10.1007/978-3-032-21289-4_3),[Document](https://dx.doi.org/10.1007/978-3-032-21289-4%5F3)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px2.p1.1)\.
- Jinet al\.\(2025\)J\. Jin, Y\. Song, W\. Luo, and H\. WangFrom bias to benefit: place good documents in good positions\.External Links:[Link](https://openreview.net/forum?id=XNar6WUIit)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px5.p1.1)\.
- Kimet al\.\(2018\)B\. Kim, M\. Wattenberg, J\. Gilmer, C\. Cai, J\. Wexler, F\. Viegas, and R\. sayresInterpretability beyond feature attribution: quantitative testing with concept activation vectors \(TCAV\)\.InProceedings of the 35th International Conference on Machine Learning,J\. Dy and A\. Krause \(Eds\.\),Proceedings of Machine Learning Research, Vol\.80,pp\. 2668–2677\.External Links:[Link](https://proceedings.mlr.press/v80/kim18d.html)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2609.03148#S1.p4.1)\.
- Lazaridouet al\.\(2021\)A\. Lazaridou, A\. Kuncoro, E\. Gribovskaya, D\. Agrawal, A\. Liška, T\. Terzi, M\. Gimenez, C\. d\. M\. d’Autume, T\. Kocisky, S\. Ruder, D\. Yogatama, K\. Cao, S\. Young, and P\. BlunsomMind the gap: assessing temporal generalization in neural language models\.InProceedings of the 35th International Conference on Neural Information Processing Systems,NIPS ’21,Red Hook, NY, USA\.External Links:ISBN 9781713845393Cited by:[§1](https://arxiv.org/html/2609.03148#S1.p2.1)\.
- Leeet al\.\(2024\)Y\. Lee, X\. Ye, and E\. ChoiAmbigDocs: reasoning across documents on different entities under the same name\.InFirst Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=mkYCfO822n)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px1.p1.1),[Table 5](https://arxiv.org/html/2609.03148#A2.T5.2.3.1.1.1)\.
- Liet al\.\(2025\)G\. Li, Y\. Chen, and H\. TongTaming knowledge conflicts in language models\.InForty\-second International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=0cEZyhHEks)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px2.p1.1)\.
- Liuet al\.\(2024\)N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. LiangLost in the middle: how language models use long contexts\.Transactions of the Association for Computational Linguistics12,pp\. 157–173\.External Links:[Link](https://aclanthology.org/2024.tacl-1.9/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00638)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px5.p1.1)\.
- Longpreet al\.\(2021\)S\. Longpre, K\. Perisetla, A\. Chen, N\. Ramesh, C\. DuBois, and S\. SinghEntity\-based knowledge conflicts in question answering\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,M\. Moens, X\. Huang, L\. Specia, and S\. W\. Yih \(Eds\.\),Online and Punta Cana, Dominican Republic,pp\. 7052–7063\.External Links:[Link](https://aclanthology.org/2021.emnlp-main.565/),[Document](https://dx.doi.org/10.18653/v1/2021.emnlp-main.565)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.03148#S1.p1.1),[§1](https://arxiv.org/html/2609.03148#S1.p2.1),[§2\.2](https://arxiv.org/html/2609.03148#S2.SS2.p2.1)\.
- Lundberg and Lee \(2017\)S\. M\. Lundberg and S\. LeeA unified approach to interpreting model predictions\.InAdvances in Neural Information Processing Systems,I\. Guyon, U\. V\. Luxburg, S\. Bengio, H\. Wallach, R\. Fergus, S\. Vishwanathan, and R\. Garnett \(Eds\.\),Vol\.30,pp\.\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2017/file/8a20a8621978632d76c43dfd28b67767-Paper.pdf)Cited by:[§1](https://arxiv.org/html/2609.03148#S1.p4.1)\.
- OpenAIet al\.\(2025\)OpenAI, :, S\. Agarwal, L\. Ahmad, J\. Ai, S\. Altman, A\. Applebaum, E\. Arbus, R\. K\. Arora, Y\. Bai, B\. Baker, H\. Bao, B\. Barak, A\. Bennett, T\. Bertao, N\. Brett, E\. Brevdo, G\. Brockman, S\. Bubeck, C\. Chang, K\. Chen, M\. Chen, E\. Cheung, A\. Clark, D\. Cook, M\. Dukhan, C\. Dvorak, K\. Fives, V\. Fomenko, T\. Garipov, K\. Georgiev, M\. Glaese, T\. Gogineni, A\. Goucher, L\. Gross, K\. G\. Guzman, J\. Hallman, J\. Hehir, J\. Heidecke, A\. Helyar, H\. Hu, R\. Huet, J\. Huh, S\. Jain, Z\. Johnson, C\. Koch, I\. Kofman, D\. Kundel, J\. Kwon, V\. Kyrylov, E\. Y\. Le, G\. Leclerc, J\. P\. Lennon, S\. Lessans, M\. Lezcano\-Casado, Y\. Li, Z\. Li, J\. Lin, J\. Liss, Lily, Liu, J\. Liu, K\. Lu, C\. Lu, Z\. Martinovic, L\. McCallum, J\. McGrath, S\. McKinney, A\. McLaughlin, S\. Mei, S\. Mostovoy, T\. Mu, G\. Myles, A\. Neitz, A\. Nichol, J\. Pachocki, A\. Paino, D\. Palmie, A\. Pantuliano, G\. Parascandolo, J\. Park, L\. Pathak, C\. Paz, L\. Peran, D\. Pimenov, M\. Pokrass, E\. Proehl, H\. Qiu, G\. Raila, F\. Raso, H\. Ren, K\. Richardson, D\. Robinson, B\. Rotsted, H\. Salman, S\. Sanjeev, M\. Schwarzer, D\. Sculley, H\. Sikchi, K\. Simon, K\. Singhal, Y\. Song, D\. Stuckey, Z\. Sun, P\. Tillet, S\. Toizer, F\. Tsimpourlas, N\. Vyas, E\. Wallace, X\. Wang, M\. Wang, O\. Watkins, K\. Weil, A\. Wendling, K\. Whinnery, C\. Whitney, H\. Wong, L\. Yang, Y\. Yang, M\. Yasunaga, K\. Ying, W\. Zaremba, W\. Zhan, C\. Zhang, B\. Zhang, E\. Zhang, and S\. ZhaoGpt\-oss\-120b & gpt\-oss\-20b model card\.External Links:2508\.10925,[Link](https://arxiv.org/abs/2508.10925)Cited by:[§4](https://arxiv.org/html/2609.03148#S4.p2.1)\.
- Patelet al\.\(2025\)S\. Patel, M\. Zhou, and G\. FantiMaxShapley: towards incentive\-compatible generative search with fair context attribution\.External Links:2512\.05958,[Link](https://arxiv.org/abs/2512.05958)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px3.p1.1)\.
- Raghuet al\.\(2017\)M\. Raghu, J\. Gilmer, J\. Yosinski, and J\. Sohl\-DicksteinSVCCA: singular vector canonical correlation analysis for deep learning dynamics and interpretability\.InAdvances in Neural Information Processing Systems,I\. Guyon, U\. V\. Luxburg, S\. Bengio, H\. Wallach, R\. Fergus, S\. Vishwanathan, and R\. Garnett \(Eds\.\),Vol\.30,pp\.\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2017/file/dc6a7e655d7e5840e66733e9ee67cc69-Paper.pdf)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px4.p1.1)\.
- Rimskyet al\.\(2024\)N\. Rimsky, N\. Gabrieli, J\. Schulz, M\. Tong, E\. Hubinger, and A\. TurnerSteering llama 2 via contrastive activation addition\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 15504–15522\.External Links:[Link](https://aclanthology.org/2024.acl-long.828/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.828)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px5.p1.1)\.
- Savageet al\.\(2024\)T\. Savage, A\. Nayak, R\. Gallo, E\. Rangan, and J\. H\. ChenDiagnostic reasoning prompts reveal the potential for large language model interpretability in medicine\.npj Digital Medicine7\(1\),pp\. 20\.External Links:ISSN 2398\-6352,[Document](https://dx.doi.org/10.1038/s41746-024-01010-1),[Link](https://doi.org/10.1038/s41746-024-01010-1)Cited by:[Table 5](https://arxiv.org/html/2609.03148#A2.T5.2.7.1.1.1),[§2\.2](https://arxiv.org/html/2609.03148#S2.SS2.SSS0.Px1.p1.1)\.
- Shiet al\.\(2024\)W\. Shi, X\. Han, M\. Lewis, Y\. Tsvetkov, L\. Zettlemoyer, and W\. YihTrusting your evidence: hallucinate less with context\-aware decoding\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 2: Short Papers\),K\. Duh, H\. Gomez, and S\. Bethard \(Eds\.\),Mexico City, Mexico,pp\. 783–791\.External Links:[Link](https://aclanthology.org/2024.naacl-short.69/),[Document](https://dx.doi.org/10.18653/v1/2024.naacl-short.69)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.03148#S1.p1.1)\.
- Singhet al\.\(2025\)A\. Singh, A\. Fry, A\. Perelman, A\. Tart, A\. Ganesh, A\. El\-Kishky, A\. McLaughlin, A\. Low, A\. Ostrow, A\. Ananthram, A\. Nathan, A\. Luo, A\. Helyar, A\. Madry, A\. Efremov, A\. Spyra, A\. Baker\-Whitcomb, A\. Beutel, A\. Karpenko, A\. Makelov, A\. Neitz, A\. Wei, A\. Barr, A\. Kirchmeyer, A\. Ivanov, A\. Christakis, A\. Gillespie, A\. Tam, A\. Bennett, A\. Wan, A\. Huang, A\. M\. Sandjideh, A\. Yang, A\. Kumar, A\. Saraiva, A\. Vallone, A\. Gheorghe, A\. G\. Garcia, A\. Braunstein, A\. Liu, A\. Schmidt, A\. Mereskin, A\. Mishchenko, A\. Applebaum, A\. Rogerson, A\. Rajan, A\. Wei, A\. Kotha, A\. Srivastava, A\. Agrawal, A\. Vijayvergiya, A\. Tyra, A\. Nair, A\. Nayak, B\. Eggers, B\. Ji, B\. Hoover, B\. Chen, B\. Chen, B\. Barak, B\. Minaiev, B\. Hao, B\. Baker, B\. Lightcap, B\. McKinzie, B\. Wang, B\. Quinn, B\. Fioca, B\. Hsu, B\. Yang, B\. Yu, B\. Zhang, B\. Brenner, C\. R\. Zetino, C\. Raymond, C\. Lugaresi, C\. Paz, C\. Hudson, C\. Whitney, C\. Li, C\. Chen, C\. Cole, C\. Voss, C\. Ding, C\. Shen, C\. Huang, C\. Colby, C\. Hallacy, C\. Koch, C\. Lu, C\. Kaplan, C\. Kim, C\. Minott\-Henriques, C\. Frey, C\. Yu, C\. Czarnecki, C\. Reid, C\. Wei, C\. Decareaux, C\. Scheau, C\. Zhang, C\. Forbes, D\. Tang, D\. Goldberg, D\. Roberts, D\. Palmie, D\. Kappler, D\. Levine, D\. Wright, D\. Leo, D\. Lin, D\. Robinson, D\. Grabb, D\. Chen, D\. Lim, D\. Salama, D\. Bhattacharjee, D\. Tsipras, D\. Li, D\. Yu, D\. Strouse, D\. Williams, D\. Hunn, E\. Bayes, E\. Arbus, E\. Akyurek, E\. Y\. Le, E\. Widmann, E\. Yani, E\. Proehl, E\. Sert, E\. Cheung, E\. Schwartz, E\. Han, E\. Jiang, E\. Mitchell, E\. Sigler, E\. Wallace, E\. Ritter, E\. Kavanaugh, E\. Mays, E\. Nikishin, F\. Li, F\. P\. Such, F\. de Avila Belbute Peres, F\. Raso, F\. Bekerman, F\. Tsimpourlas, F\. Chantzis, F\. Song, F\. Zhang, G\. Raila, G\. McGrath, G\. Briggs, G\. Yang, G\. Parascandolo, G\. Chabot, G\. Kim, G\. Zhao, G\. Valiant, G\. Leclerc, H\. Salman, H\. Wang, H\. Sheng, H\. Jiang, H\. Wang, H\. Jin, H\. Sikchi, H\. Schmidt, H\. Aspegren, H\. Chen, H\. Qiu, H\. Lightman, I\. Covert, I\. Kivlichan, I\. Silber, I\. Sohl, I\. Hammoud, I\. Clavera, I\. Lan, I\. Akkaya, I\. Kostrikov, I\. Kofman, I\. Etinger, I\. Singal, J\. Hehir, J\. Huh, J\. Pan, J\. Wilczynski, J\. Pachocki, J\. Lee, J\. Quinn, J\. Kiros, J\. Kalra, J\. Samaroo, J\. Wang, J\. Wolfe, J\. Chen, J\. Wang, J\. Harb, J\. Han, J\. Wang, J\. Zhao, J\. Chen, J\. Yang, J\. Tworek, J\. Chand, J\. Landon, J\. Liang, J\. Lin, J\. Liu, J\. Wang, J\. Tang, J\. Yin, J\. Jang, J\. Morris, J\. Flynn, J\. Ferstad, J\. Heidecke, J\. Fishbein, J\. Hallman, J\. Grant, J\. Chien, J\. Gordon, J\. Park, J\. Liss, J\. Kraaijeveld, J\. Guay, J\. Mo, J\. Lawson, J\. McGrath, J\. Vendrow, J\. Jiao, J\. Lee, J\. Steele, J\. Wang, J\. Mao, K\. Chen, K\. Hayashi, K\. Xiao, K\. Salahi, K\. Wu, K\. Sekhri, K\. Sharma, K\. Singhal, K\. Li, K\. Nguyen, K\. Gu\-Lemberg, K\. King, K\. Liu, K\. Stone, K\. Yu, K\. Ying, K\. Georgiev, K\. Lim, K\. Tirumala, K\. Miller, L\. Ahmad, L\. Lv, L\. Clare, L\. Fauconnet, L\. Itow, L\. Yang, L\. Romaniuk, L\. Anise, L\. Byron, L\. Pathak, L\. Maksin, L\. Lo, L\. Ho, L\. Jing, L\. Wu, L\. Xiong, L\. Mamitsuka, L\. Yang, L\. McCallum, L\. Held, L\. Bourgeois, L\. Engstrom, L\. Kuhn, L\. Feuvrier, L\. Zhang, L\. Switzer, L\. Kondraciuk, L\. Kaiser, M\. Joglekar, M\. Singh, M\. Shah, M\. Stratta, M\. Williams, M\. Chen, M\. Sun, M\. Cayton, M\. Li, M\. Zhang, M\. Aljubeh, M\. Nichols, M\. Haines, M\. Schwarzer, M\. Gupta, M\. Shah, M\. Huang, M\. Dong, M\. Wang, M\. Glaese, M\. Carroll, M\. Lampe, M\. Malek, M\. Sharman, M\. Zhang, M\. Wang, M\. Pokrass, M\. Florian, M\. Pavlov, M\. Wang, M\. Chen, M\. Wang, M\. Feng, M\. Bavarian, M\. Lin, M\. Abdool, M\. Rohaninejad, N\. Soto, N\. Staudacher, N\. LaFontaine, N\. Marwell, N\. Liu, N\. Preston, N\. Turley, N\. Ansman, N\. Blades, N\. Pancha, N\. Mikhaylin, N\. Felix, N\. Handa, N\. Rai, N\. Keskar, N\. Brown, O\. Nachum, O\. Boiko, O\. Murk, O\. Watkins, O\. Gleeson, P\. Mishkin, P\. Lesiewicz, P\. Baltescu, P\. Belov, P\. Zhokhov, P\. Pronin, P\. Guo, P\. Thacker, Q\. Liu, Q\. Yuan, Q\. Liu, R\. Dias, R\. Puckett, R\. Arora, R\. T\. Mullapudi, R\. Gaon, R\. Miyara, R\. Song, R\. Aggarwal, R\. Marsan, R\. Yemiru, R\. Xiong, R\. Kshirsagar, R\. Nuttall, R\. Tsiupa, R\. Eldan, R\. Wang, R\. James, R\. Ziv, R\. Shu, R\. Nigmatullin, S\. Jain, S\. Talaie, S\. Altman, S\. Arnesen, S\. Toizer, S\. Toyer, S\. Miserendino, S\. Agarwal, S\. Yoo, S\. Heon, S\. Ethersmith, S\. Grove, S\. Taylor, S\. Bubeck, S\. Banesiu, S\. Amdo, S\. Zhao, S\. Wu, S\. Santurkar, S\. Zhao, S\. R\. Chaudhuri, S\. Krishnaswamy, Shuaiqi, Xia, S\. Cheng, S\. Anadkat, S\. P\. Fishman, S\. Tobin, S\. Fu, S\. Jain, S\. Mei, S\. Egoian, S\. Kim, S\. Golden, S\. Mah, S\. Lin, S\. Imm, S\. Sharpe, S\. Yadlowsky, S\. Choudhry, S\. Eum, S\. Sanjeev, T\. Khan, T\. Stramer, T\. Wang, T\. Xin, T\. Gogineni, T\. Christianson, T\. Sanders, T\. Patwardhan, T\. Degry, T\. Shadwell, T\. Fu, T\. Gao, T\. Garipov, T\. Sriskandarajah, T\. Sherbakov, T\. Kaftan, T\. Hiratsuka, T\. Wang, T\. Song, T\. Zhao, T\. Peterson, V\. Kharitonov, V\. Chernova, V\. Kosaraju, V\. Kuo, V\. Pong, V\. Verma, V\. Petrov, W\. Jiang, W\. Zhang, W\. Zhou, W\. Xie, W\. Zhan, W\. McCabe, W\. DePue, W\. Ellsworth, W\. Bain, W\. Thompson, X\. Chen, X\. Qi, X\. Xiang, X\. Shi, Y\. Dubois, Y\. Yu, Y\. Khakbaz, Y\. Wu, Y\. Qian, Y\. T\. Lee, Y\. Chen, Y\. Zhang, Y\. Xiong, Y\. Tian, Y\. Cha, Y\. Bai, Y\. Yang, Y\. Yuan, Y\. Li, Y\. Zhang, Y\. Yang, Y\. Jin, Y\. Jiang, Y\. Wang, Y\. Wang, Y\. Liu, Z\. Stubenvoll, Z\. Dou, Z\. Wu, and Z\. WangOpenAI gpt\-5 system card\.External Links:2601\.03267,[Link](https://arxiv.org/abs/2601.03267)Cited by:[§2\.2](https://arxiv.org/html/2609.03148#S2.SS2.p2.1)\.
- Suet al\.\(2024\)Z\. Su, J\. Zhang, X\. Qu, T\. Zhu, Y\. Li, J\. Sun, J\. Li, M\. Zhang, and Y\. ChengConflictBank: a benchmark for evaluating the influence of knowledge conflicts in llms\.InAdvances in Neural Information Processing Systems,A\. Globerson, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. Tomczak, and C\. Zhang \(Eds\.\),Vol\.37,pp\. 103242–103268\.External Links:[Document](https://dx.doi.org/10.52202/079017-3280),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/file/baf4b960d118f838ad0b2c08247a9ebe-Paper-Datasets_and_Benchmarks_Track.pdf)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px1.p1.1),[Table 5](https://arxiv.org/html/2609.03148#A2.T5.2.18.1.1.1),[§1](https://arxiv.org/html/2609.03148#S1.p2.1),[§2\.2](https://arxiv.org/html/2609.03148#S2.SS2.SSS0.Px4.p1.1),[§2\.2](https://arxiv.org/html/2609.03148#S2.SS2.p2.1)\.
- Tenneyet al\.\(2019\)I\. Tenney, D\. Das, and E\. PavlickBERT rediscovers the classical NLP pipeline\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,A\. Korhonen, D\. Traum, and L\. Màrquez \(Eds\.\),Florence, Italy,pp\. 4593–4601\.External Links:[Link](https://aclanthology.org/P19-1452/),[Document](https://dx.doi.org/10.18653/v1/P19-1452)Cited by:[§4\.1](https://arxiv.org/html/2609.03148#S4.SS1.p2.1)\.
- Waddenet al\.\(2020\)D\. Wadden, S\. Lin, K\. Lo, L\. L\. Wang, M\. van Zuylen, A\. Cohan, and H\. HajishirziFact or fiction: verifying scientific claims\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),B\. Webber, T\. Cohn, Y\. He, and Y\. Liu \(Eds\.\),Online,pp\. 7534–7550\.External Links:[Link](https://aclanthology.org/2020.emnlp-main.609/),[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.609)Cited by:[Table 5](https://arxiv.org/html/2609.03148#A2.T5.2.16.1.1.1),[§2\.2](https://arxiv.org/html/2609.03148#S2.SS2.SSS0.Px3.p1.1)\.
- Wanget al\.\(2025\)H\. Wang, A\. Prasad, E\. Stengel\-Eskin, and M\. BansalRetrieval\-augmented generation with conflicting evidence\.InSecond Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=z1MHB2m3V9)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px2.p1.1)\.
- Xieet al\.\(2024\)J\. Xie, K\. Zhang, J\. Chen, R\. Lou, and Y\. SuAdaptive chameleon or stubborn sloth: revealing the behavior of large language models in knowledge conflicts\.InInternational Conference on Learning Representations,B\. Kim, Y\. Yue, S\. Chaudhuri, K\. Fragkiadaki, M\. Khan, and Y\. Sun \(Eds\.\),Vol\.2024,pp\. 35623–35646\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/file/99261adc8a6356b38bcf999bba9a26dc-Paper-Conference.pdf)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.03148#S1.p2.1)\.
- Xuet al\.\(2024\)R\. Xu, Z\. Qi, Z\. Guo, C\. Wang, H\. Wang, Y\. Zhang, and W\. XuKnowledge conflicts for LLMs: a survey\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 8541–8565\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.486/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.486)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px1.p1.1),[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2609.03148#S1.p2.1)\.
- Ye and Yoganarasimhan \(2025\)Z\. Ye and H\. YoganarasimhanFair document valuation in llm summaries via shapley values\.External Links:2505\.23842,[Link](https://arxiv.org/abs/2505.23842)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px3.p1.1)\.
- Zhanget al\.\(2024\)Z\. Zhang, R\. Chen, S\. Liu, Z\. Yao, O\. Ruwase, B\. Chen, X\. Wu, and Z\. WangFound in the middle: how language models use long contexts better via plug\-and\-play positional encoding\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=fPmScVB1Td)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px5.p1.1)\.
- Zouet al\.\(2023\)A\. Zou, L\. Phan, S\. Chen, J\. Campbell, P\. Guo, R\. Ren, A\. Pan, X\. Yin, M\. Mazeika, A\. Dombrowski, S\. Goel, N\. Li, M\. J\. Byun, Z\. Wang, A\. Mallen, S\. Basart, S\. Koyejo, D\. Song, M\. Fredrikson, J\. Z\. Kolter, and D\. HendrycksRepresentation engineering: a top\-down approach to ai transparency\.CoRRabs/2310\.01405\.External Links:[Link](https://doi.org/10.48550/arXiv.2310.01405)Cited by:[Appendix A](https://arxiv.org/html/2609.03148#A1.SS0.SSS0.Px5.p1.1)\.

## Appendix AExtended Related Work

##### Knowledge Conflicts and Datasets\.

Large language models often produce inconsistent or hallucinatory outputs when confronted with conflicting knowledge[Xie et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib13);[Xu et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib7)\. Prior datasets primarily focus on context\-memory conflicts[Longpre et al\. \(2021\)](https://arxiv.org/html/2609.03148#bib.bib1), often constructed through entity replacement[Su et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib10)\. However, these resources exhibit four major limitations\. They rely heavily on synthetic conflicts with limited real\-world complexity, emphasize explicit surface\-level contradictions while under\-covering implicit multi\-step conflicts, provide limited domain diversity, and suffer from severe class imbalance that weakens category\-level analysis\. In contrast, we propose six fine\-grained conflict categories spanning multiple domains, with explicit and implicit coverage across reasoning and summarization task types and type\-specific implicit proportions\. This design addresses key gaps in conflict complexity, diversity, and evaluation fairness\. Existing datasets such as ConflictBank[Su et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib10)focus on entity\-substitution\-based factual conflicts, and AmbigDocs[Lee et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib23)targets a single ambiguity type\. To the best of our knowledge, few existing datasets jointly cover all six conflict types and two task paradigms within one multi\-domain dataset\.

##### Conflict Mitigation in RAG\.

Recent work addresses multi\-source conflicts through several approaches\. These include multi\-agent deliberation[Li et al\. \(2025\)](https://arxiv.org/html/2609.03148#bib.bib34);[Wang et al\. \(2025\)](https://arxiv.org/html/2609.03148#bib.bib35), conflict\-driven summarization[Chen et al\. \(2025\)](https://arxiv.org/html/2609.03148#bib.bib32), and adversarial\-trained assessors[Choi et al\. \(2025\)](https://arxiv.org/html/2609.03148#bib.bib31);[Javadi et al\. \(2026\)](https://arxiv.org/html/2609.03148#bib.bib16)\. A separate line of work uses contrastive decoding to amplify the contribution of context relative to parametric memory, most notably context\-aware decoding \(CAD\)[Shi et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib9)\. CAD contrastslog⁡p⁡\(y∣ctx\)\\log p\(y\\mid\\text\{ctx\}\)withlog⁡p⁡\(y∣∅\)\\log p\(y\\mid\\emptyset\), where the counterfactual baseline is the empty context\. This binary contrast treats theKKevidence pieces as a single block: it amplifies trust in context as a whole but cannot redistribute attention or representational contribution among the individual evidence pieces\. The position bias we identify is precisely an intra\-context asymmetry \(b1\(l\)≫bi\>1\(l\)b\_\{1\}^\{\(l\)\}\\gg b\_\{i\>1\}^\{\(l\)\}\), which CAD’s reference frame cannot express; the two methods therefore address orthogonal problems and cannot be directly compared as alternatives\. These methods improve QA performance but largely treat conflict resolution as a black\-box problem, often requiring additional models or training and offering limited insight into internal mechanisms\. We instead focus on internal conflict representations and propose a training\-free activation steering method without external models\. More importantly, to the best of our knowledge, few dataset\-based studies provide a comparably broad mechanistic account of when and why LLMs fail in contextual conflict resolution\.

##### Evidence Attribution for Multi\-Document Summarization\.

Shapley values have been used to quantify document importance in LLM\-generated summaries[Ye and Yoganarasimhan \(2025\)](https://arxiv.org/html/2609.03148#bib.bib14)\. Recent work improves efficiency through semantic clustering and decomposable utility functions[Ye and Yoganarasimhan \(2025\)](https://arxiv.org/html/2609.03148#bib.bib14);[Patel et al\. \(2025\)](https://arxiv.org/html/2609.03148#bib.bib15)\. However, these methods rely on LLM\-as\-a\-judge scoring and focus on content provider compensation scenarios rather than conflict analysis\. We introduce Shapley attribution specifically for fairness analysis in conflict scenarios, proposing the Balance metric to quantify evidence\-integration bias via a normalized Gini coefficient\.

##### Mechanistic Interpretability of LLMs\.

Prior work has probed model representations using concept activation vectors[Kim et al\. \(2018\)](https://arxiv.org/html/2609.03148#bib.bib11), spectral decomposition[Raghu et al\. \(2017\)](https://arxiv.org/html/2609.03148#bib.bib17), and sparse autoencoders[Huben et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib12), primarily to study factual recall, reasoning, or sentiment\. Yet mechanistic analysis of how LLMs internally handle contextual knowledge conflicts remains scarce[Xu et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib7)\. To the best of our knowledge, few dataset\-based studies jointly analyze conflict awareness, representational geometry, shared feature organization, and evidence\-integration bias across conflict types and tested model architectures\.

##### Positional Bias in LLMs\.

LLMs exhibit positional bias over long contexts[Liu et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib8), which has been addressed through positional\-encoding adjustments[Zhang et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib18), document reordering[Jin et al\. \(2025\)](https://arxiv.org/html/2609.03148#bib.bib19), or training\-time augmentation[An et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib20)\. However, prior studies mainly examine retrieval under noisy context, where most evidence is irrelevant and positional bias determines whether one relevant item is recovered from surrounding noise\. In conflict scenarios, by contrast, all evidence is relevant but semantically competing and often logically incompatible\. Positional bias therefore becomes a selective\-integration problem, reflecting the model’s tendency to favor specific evidence positions under genuine competition\. To our knowledge, this setting remains underexplored\. We provide a systematic analysis of positional bias in this regime and propose a training\-free activation steering method[Zou et al\. \(2023\)](https://arxiv.org/html/2609.03148#bib.bib22);[Rimsky et al\. \(2024\)](https://arxiv.org/html/2609.03148#bib.bib21)that improves reasoning performance and maintains strong summarization evidence\-integration balance on tested models\.

## Appendix BDataset Details

### B\.1Construction Process

Table 5:Base datasets used to construct our dataset\.Table[5](https://arxiv.org/html/2609.03148#A2.T5)lists the ten base datasets used to construct our dataset\. The construction procedure for each conflict type is described in Section[2\.2](https://arxiv.org/html/2609.03148#S2.SS2); here we provide the GPT\-5 prompts used for semi\-synthetic data generation\. Figure[6](https://arxiv.org/html/2609.03148#A2.F6)shows the two\-step prompt for misinformation conflicts \(SciFact subset\): question generation followed by conflicting evidence generation\. Figure[7](https://arxiv.org/html/2609.03148#A2.F7)shows the prompt for inferential conflicts \(FOLIO and ENTAILMENTBANK subsets\), which augments original premises with conflicting reasoning branches\. The annotation interface for manual re\-annotation of FOLIO instances is shown in Figure[8](https://arxiv.org/html/2609.03148#A2.F8)\. Figure[9](https://arxiv.org/html/2609.03148#A2.F9)shows the prompt for temporal conflicts \(ConflictBank subset\), which attaches explicit temporal markers to generate time\-dependent questions\. Figure[10](https://arxiv.org/html/2609.03148#A2.F10)shows the prompt for generating factual questions for other ConflictBank conflict types\.

Misinformation: Question GenerationClaim:\{claim\} Original Evidence:\{evidence\_context\} Generate a Clear Question: Create a direct, focused question based on the claim that can be answered using scientific evidence\. The question should be specific and answerable\.

Misinformation: Conflicting Evidence GenerationTask:Generate conflicting evidence statements that oppose the original findings\. Requirements: 1\.Quantity & Style:Generate the exact same number of statements as provided\. Match the language style, professional tone, and complexity perfectly\. 2\.Conflict Strategy:Create plausible rebuttals\. Do not use simple “not” negations; instead: modify findings, cite different methodological outcomes, or focus on alternative variables that lead to opposite conclusions\. 3\.Diversity:Ensure each statement uses a unique sentence structure\. 4\.Output:Return ONLY the numbered list of conflicting evidence statements\. Claim:\{claim\} Original Evidence:\{evidence\_context\}

Figure 6:Prompts for misinformation conflict construction\.FOLIO: Conflicting Premise GenerationYour task:
1\.Decision:With 50% probability, choose either to:
\-Option A:Add 1–2 plausible statements that create a conflicting reasoning branch \(requires at least one inference step; avoid trivial “X vs\. not\-X”\)\.
\-Option B:Keep the premises as they are\.
2\.Output:Return ONLY the final numbered list of statements\.
3\.Constraints:No meta\-talk, no “source\_\#” references, no explanations\.
Premises:\{premises\_raw\_json\}
Hypothesis:\{conclusion\}Figure 7:Prompt for FOLIO inferential conflict construction\.![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/folio_ann_ui.png)Figure 8:Annotation interface for FOLIO manual verification\. Annotators select True/False if reasoning chains lead to a unique answer, or Uncertain if chains produce conflicting conclusions\.Temporal Conflicts: Question GenerationBased on two conflicting claims about the same subject, generate ONE concise open\-ended question with a temporal marker\.
Inputs:Subject, default claim, temporal claim, default evidence, temporal evidence
Requirements:
1\. Extract ALL timestamps/dates from both evidences\.
2\. Identify the attribute being described \(e\.g\., employer, award, title\)\.
3\. Randomly choose to ask about a SPECIFIC YEAR before, between, or after identified timestamps\.
4\. Use patterns like: “What was \{subject\}’s \[attribute\] in \[year\]?” or “Who was \{subject\}’s \[attribute\] as of \[year\]?”
5\. Prefer specific years over time ranges\.
6\. Do NOT use yes/no questions or multiple\-choice formats\.
7\. Self\-contained, one sentence,≤\\leq20 words\.
8\. Output only the question \(no explanations, no metadata\)\.Figure 9:Prompt for temporal conflict question generation\.General ConflictBank: Question GenerationGiven a subject with a default claim \(assumed correct\) and a conflicting claim, generate ONE concise factual question about the subject without revealing either claim explicitly\.
Inputs:Subject, default claim, conflicting claim
Requirements:
\- Use simple WH\-questions like “Where did \{subject\} work?”, “When was \{subject\} born?”, “Who is \{subject\} married to?”
\- Do NOT include claim contents in the question\.
\- Self\-contained, one sentence\.Figure 10:Prompt for general ConflictBank question generation\.
### B\.2Decision Rules for Borderline Conflict\-Type Overlaps

Our taxonomy is defined by operational, structural criteria rather than surface intuition\. We document the decision rule for each pair of conflict types whose surface descriptions could otherwise be confused\.

##### Temporal vs\. Misinformation\.

The criterion is whether each evidence piece holds true at some time point\. Intemporalconflicts, every evidence piece is true at its own time point; the conflict arises because claim validity changes over time\. Inmisinformationconflicts, some evidence is factually incorrect, and each piece carries an accuracy label in the dataset metadata\.

##### Ambiguity vs\. Inferential\.

The criterion is the task goal, not merely the number of reasoning steps\. Inambiguityconflicts, each evidence piece describes a different real entity that shares the same name \(e\.g\., “Jordan” can refer to the basketball player or to a university teacher with the same name\); each piece is understandable on its own, no cross\-evidence reasoning is needed, and the goal is to integrate and present every entity fairly rather than let the more popular referent dominate\. Ambiguity instances are identified directly from the entity\-disambiguation metadata of the source dataset\. Ininferentialconflicts, the conflict is not limited to entities: it only emerges after combining multiple pieces of evidence through reasoning, and the goal is to derive the single correct conclusion\.

##### Perspective vs\. Granularity\.

The criterion is compatibility\.Perspectiveconflicts involve incompatible stances on the same question\.Granularityconflicts involve compatible answers at different levels of specificity\.

### B\.3Data Samples

We provide representative examples for each conflict type in our dataset\. Figure[11](https://arxiv.org/html/2609.03148#A2.F11)illustrates an inferential conflict where logical reasoning leads to an uncertain conclusion\. Figure[12](https://arxiv.org/html/2609.03148#A2.F12)presents a misinformation conflict involving contradictory scientific claims with accuracy labels\. Figure[13](https://arxiv.org/html/2609.03148#A2.F13)demonstrates a temporal conflict requiring temporal reasoning across events\. Figure[14](https://arxiv.org/html/2609.03148#A2.F14)shows an ambiguity conflict where one name refers to multiple distinct entities\. Figure[15](https://arxiv.org/html/2609.03148#A2.F15)provides a granularity conflict example with varying levels of detail, and Figure[16](https://arxiv.org/html/2609.03148#A2.F16)highlights a perspective conflict involving diverse viewpoints on a policy issue\.

Inferential Conflict ExampleQuestion:Based on the given evidence, is the hypothesis ’Miroslav Venhoda loved music\.’ true, false, or uncertain? The final answer should be ’True’, ’False’, or ’Uncertain’\. Only answer ’Uncertain’ if the reasoning results from different evidence pieces conflict with each other\.Evidence 1:Miroslav Venhoda was a Czech choral conductor\.Evidence 2:Miroslav Venhoda specialized in the performance of Renaissance and Baroque music\.Evidence 3:All choral conductors are musicians\.Evidence 4:Some musicians love music\.Evidence 5:In 1946, Miroslav Venhoda published a book titled Method of Studying Gregorian Chant\.Evidence 6:Authors of instructional books on musical practice often have strong enthusiasm for music\.Evidence 7:A scholar’s focus on liturgical chant can be motivated by historical or religious interest rather than a personal love of music\.Ground Truth:UncertainFigure 11:Example of an Inferential Conflict, demonstrating logical uncertainty\.Misinformation Conflict ExampleQuestion:In a bleomycin\-induced mouse model of pulmonary fibrosis, does pharmacological activation of AMPK \(e\.g\., with metformin\) reduce established lung fibrosis in an AMPK\-dependent manner? Answer with Yes or No\.Evidence 1:In a bleomycin model of lung fibrosis in mice, metformin therapeutically accelerates the resolution of well\-established fibrosis in an AMPK\-dependent manner\.Evidence 2:In primary myofibroblasts isolated from IPF lungs, pharmacological activation of AMPK did not attenuate fibrotic activity; instead, it was accompanied by diminished mitochondrial biogenesis and persistence of apoptosis resistance\.Evidence 3:Contrary to expectations, these investigations indicate that heightened AMPK signaling underlies non\-resolving, pathologic fibrotic remodeling, and argue that metformin \(and related AMPK agonists\) are unlikely to reverse established fibrosis because they maintain myofibroblasts in an activated, apoptosis\-resistant state\.Accuracy Labels:\[True, False, False\]Ground Truth:YesFigure 12:Example of a Misinformation Conflict with specific accuracy labels for each evidence\.Temporal Conflict ExampleQuestion:Which university did Alvaro Mata attend in 2030?Note: Evidence snippets are truncated for readability; we use \[…\] as the truncation marker\.Evidence 1:In the heart of Cleveland, Ohio\[…\] lies Cleveland State University\. It is here that Alvaro Mata, a researcher of unyielding curiosity and dedication, embarked on a transformative journey\[…\] It is the early 2000s, and Cleveland State University is thriving\[…\] Alvaro Mata emerges as a bright and ambitious individual\[…\] Evidence of Alvaro’s attendance at Cleveland State University can be found in the university’s library archives, where academic papers and theses reveal his prolific output as a student researcher\.Evidence 2:Renowned Researcher Alvaro Mata Celebrates 14\-Year Milestone at Macquarie University\. Sydney, Australia \- December 15, 2038\. Today, Macquarie University in Sydney marked a significant milestone as esteemed researcher Alvaro Mata completed an impressive 14\-year tenure at the institution\[…\] "It’s hard to believe it’s been 14 years since I first stepped onto this campus," Mata reflected\.Ground Truth:Macquarie UniversityFigure 13:Example of a Temporal Conflict requiring chronological reasoning\.Ambiguity Conflict ExampleQuestion:What is Peter Cusack known for?Evidence 1:Peter Cusack \(musician\) is an artist and musician who is a member of CRiSAP \(Creative Research in Sound Arts Practice\), and is a research staff member at the London College of Communication\. He was a founding member and director of the London Musicians’ Collective\. He is best known as a member of the avant garde musical quartet, Alterations \(1978–1986; with Steve Beresford, David Toop, and Terry Day\), and the creator of field and wildlife recording\-based albums\.Evidence 2:Peter Cusack made his premiership début with the Sydney Roosters in the 1998 NRL season\. A front\-rower, Cusack was one of the last remaining top level league players to hold a job outside football, working part\-time as a plumber\. He played in the 2000 NRL Grand Final loss to the Brisbane Broncos\. Cusack was awarded the 2002 Sydney Roosters season’s "Clubman of the Year" and played in their 2002 NRL Grand Final victory over the New Zealand Warriors\.Figure 14:Example of an Ambiguity Conflict involving two different individuals with the same name\.Granularity Conflict ExampleTask:Summarize the following evidence in 2\-3 sentences\.Evidence 1:I’m only here for taste and texture, and the recent cups haven’t hit the same mark\. The broth turns thin unless I dump the entire seasoning in, and even then I reach for soy sauce to wake it up\. The noodles rehydrate fine—springy enough after five minutes under a kettle—but the base lacks that savory backbone it used to have\[…\] If you’re chasing a rich sip straight from the cup, this version feels watered down\.Evidence 2:The "lower sodium" splash on the front sent me straight to the Nutrition Facts, and the numbers tell a different story than I expected\. The serving size is smaller than my older cups\[…\] Net weight is down, the water line looks lower, and the sodium per serving drops mostly because the serving itself is lighter\. It reads like less soup rather than a smarter recipe\.Evidence 3:As an overall lunch option, these are still easy: boil water, fill to the line, lid on for five minutes, stir hard, and you’ve got something warm at your desk\. The flavor is dependable once you learn your routine\[…\] The convenience is great for busy days, yet the front\-of\-pack claims don’t line up neatly with what I taste and how much I’m actually eating\.Evidence 4:I prefer the large McDougall cups because one can carry me through an afternoon of meetings without snacks\. Filled to the line, it’s closer to a bowl than a snack cup, and sometimes I can’t finish it in one sitting\.Evidence 5:For anyone sensitive to salt, using about three\-quarters of the seasoning packet has been the sweet spot\. I stir in the dry mix gradually, taste, and stop right before it tips into that heavy\-salty zone; a squeeze of lemon and some scallions round it out\.Figure 15:Example of a Granularity Conflict where evidence varies in detail and focus\.Perspective Conflict ExampleQuestion:Should the school day be extended?Evidence 1:Evidence from Chicago Public Schools shows that assigning underprepared 9th graders to a double\-period \(extended time\) algebra course raised math achievement, increased credit accumulation, and led to higher graduation and college enrollment rates—indicating targeted extensions of the school day for core instruction can produce substantial gains for struggling students\.Evidence 2:The AAP recommends middle and high schools start at 8:30 a\.m\. or later because adolescents need 8–10 hours of sleep; insufficient sleep is linked to worse grades, depression, and accident risk\. Extending the school day without shifting to later start times may exacerbate sleep deprivation and harm learning and health\.Evidence 3:National survey data show teachers report substantially higher job\-related stress and burnout than other working adults, with time pressure and workload as leading contributors\. Lengthening the school day risks worsening burnout and turnover unless paired with added staffing, planning time, and compensation\.Evidence 4:On school days, juvenile crime and victimization rates peak in the hours immediately after school \(roughly 2–6 p\.m\.\)\. Extending the school day to provide structured supervision during these hours is argued to reduce delinquency and enhance community safety\.Evidence 5:A package of reforms that included substantially increased instructional time produced large gains in math and reading\. The study suggests extended time can be effective when embedded in a broader model, implying time alone is unlikely to deliver similar results\.Evidence 6:The EEF estimates extending school time yields a small average impact on learning \(around \+2 months’ progress per year\) at moderate cost; benefits are larger when time is tightly targeted \(e\.g\., tutoring\), raising questions about the cost\-effectiveness of blanket school\-day extensions\.Figure 16:Example of a Perspective Conflict featuring diverse viewpoints on school hour extensions\.

## Appendix CEvaluation Details

### C\.1Evaluation via Different Scorer Models

Our Balance metric relies on a scorer model to compute the value function\. As defined in Eq\. 2 of Section[3\.1](https://arxiv.org/html/2609.03148#S3.SS1), this function maps each evidence subset to a normalized utility score\. To verify the robustness of our metric, we test two different scorer models: Llama\-3\.1\-8B and Gemma\-2B\. These models differ in both scale and pretraining approach\.

Table[6](https://arxiv.org/html/2609.03148#A3.T6)shows Balance scores computed using both scorer models\. While absolute values differ between scorers, relative performance patterns remain highly consistent across tested models and conflict types\. Crucially, model rankings by Balance remain nearly identical across scorers\. For instance,gemini\-2\.5\-proconsistently achieves the best Balance scores in Ambiguity under both scorers, whilellama\-3\.1\-8b\-instructconsistently shows the highest Balance scores across conflict types\. These results suggest that our Shapley\-based Balance metric is robust to scorer choice and less likely to be driven by scorer\-specific artifacts\.

Table 6:Balance scores under two different scorer models\. The relative rankings and performance patterns remain consistent across scorers, demonstrating robustness of the Shapley\-based Balance metric to scorer choice\.
### C\.2Human Annotation and Metric Validation

To validate our Shapley\-based metric, we conduct a human annotation study\. Two independent annotators evaluate evidence contributions for a subset of samples from summarization tasks\. Figure[17](https://arxiv.org/html/2609.03148#A3.F17)shows the annotation interface\. Annotators rate each evidence source’s contribution to the response on a 1–5 scale\. We provide detailed guidelines in Figure[18](https://arxiv.org/html/2609.03148#A3.F18)\. Table[2](https://arxiv.org/html/2609.03148#S3.T2)reports Cohen’s Kappa coefficients across 108 annotated samples\. Human annotators achieve moderate agreement \(κ=0\.5199\\kappa=0\.5199\)\. Our automatic metric also agrees with both annotators \(κ=0\.5208\\kappa=0\.5208and0\.50140\.5014\)\. These results suggest that the metric is broadly consistent with human judgments of evidence contribution\. Table[7](https://arxiv.org/html/2609.03148#A3.T7)reports per\-model Balance scores from both human raters and the LLM\-as\-a\-Judge across the three summarization tasks\.

Table 7:Balance scores \(normalized Gini %, lower is better\) from human evaluation and LLM\-as\-a\-Judge on three models spanning diverse capability levels\.![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/human_eval_ui.png)Figure 17:Annotation interface for rating evidence contributions on a 1–5 scale\.Evidence Contribution Annotation GuidelinesTask Overview:Evaluate the contribution of each evidence source to the model’s response for a given question\. Rate each source on a 1–5 scale\.
Rating Scale:
1 – Minimal Contribution:Evidence barely reflected in response; removing it would not affect core content\.
2 – Minor Contribution:Some information mentioned but peripheral; plays supporting role\.
3 – Moderate Contribution:Key information clearly reflected; important for response completeness\.
4 – Major Contribution:Core content prominently featured; significant impact on main arguments\.
5 – Dominant Contribution:Response primarily derived from this evidence; plays decisive role\.
Core Principles:
1\. Focus on relative distribution, not absolute values\.Ratings should reflect*relative contribution differences*between sources\.
Example A \(Significant disparity\):Source 1: 5, Source 2: 1→\\rightarrowResponse heavily relies on Source 1\.
Example B \(Balanced\):Source 1: 3, Source 2: 3→\\rightarrowBoth sources contribute equally\.
Example C \(Low but balanced\):Source 1: 1, Source 2: 1→\\rightarrowResponse relies on model’s own knowledge\.
2\. Base judgment on content matching:
–Direct quotation:Response uses verbatim or paraphrased content from evidence\.
–Information correspondence:Facts, data, viewpoints in response traceable to evidence\.
–Structural influence:Response organization influenced by evidence structure\.
3\. Evaluate each evidence independently,then use score distribution to reflect relative relationships\.
Annotation Steps:
1\. Read theQuestionto understand the core inquiry\.
2\. Read allSourcesto understand the information each provides\.
3\. Read theResponsecarefully, analyzing its content and structure\.
4\.Rate each source:Identify which parts of the response come from each evidence, assess the weight of that information, and assign a 1–5 score\.
5\.Check consistency:Ensure score distribution reasonably reflects relative contributions\.
Important Notes:
–Avoid position bias:Do not favor evidence appearing earlier\.
–Distinguish quality from contribution:High\-quality evidence unused in response should receive a low score\.
–Handle conflicting evidence:Focus on which evidence’s viewpoint dominates in the response\.Figure 18:Annotation guidelines for evaluating evidence contribution to model responses\.

## Appendix DMechanistic Analysis Details

### D\.1Conflict\-Consistent Pair Construction and Cross\-Validation Statistics

The concept\-vector analysis in Section[4](https://arxiv.org/html/2609.03148#S4)relies on pairing each conflict instancexconf\(t\)x\_\{\\text\{conf\}\}^\{\(t\)\}with a consistent counterpartxcons\(t\)x\_\{\\text\{cons\}\}^\{\(t\)\}in which the evidence no longer produces a conflict\. We construct each consistent counterpart by preserving the question and surrounding context and replacing or filtering only the evidence set, following per\-type rules:

- •Misinformation\.We retain only the evidence pieces labeled factually correct in the source annotation and discard the conflicting ones, so that every remaining piece supports the same gold conclusion\.
- •Inferential \(FOLIO and EntailmentBank\)\.We keep the original entailment chain and remove the GPT\-5\-generated conflicting branch, so that all premises jointly support the original hypothesis label\.
- •Temporal\.We anchor on a single timestamp and retain only the evidence pieces consistent with that anchor; we introduce no new content\.
- •Granularity\.We keep evidence pieces drawn from the same diagnostic specificity level \(either all broad\-syndrome or all specific\-disease\), without mixing levels\.
- •Perspective\.We retain evidence pieces that share the same stance label and discard those expressing the opposing stance\.
- •Ambiguity\.We retain evidence pieces that refer to a single entity, using the source dataset’s disambiguation metadata\.

Each conflict instance is paired with exactly one consistent counterpart, and we use these conf\-versus\-cons pairs as the labeled inputs to the linear probes in Section[4](https://arxiv.org/html/2609.03148#S4)\. Table[8](https://arxiv.org/html/2609.03148#A4.T8)reports per\-type sample counts and 5\-fold stratified cross\-validation statistics\. Probes are linear logistic\-regression classifiers with inverse\-regularization strengthC=1\.0C\{=\}1\.0and a maximum of 1000 iterations, fit on residual\-stream activations at each layer\.

Table 8:Conflict\-consistent pair counts and 5\-fold stratified cross\-validation statistics per conflict type\. Each conflict instance is paired with one consistent counterpart, so\|xconf\|=\|xcons\|\|x\_\{\\text\{conf\}\}\|=\|x\_\{\\text\{cons\}\}\|by construction; per\-fold training and validation sizes are computed from the combined pool\|xconf\|\+\|xcons\|\|x\_\{\\text\{conf\}\}\|\+\|x\_\{\\text\{cons\}\}\|with an 80/20 stratified split\.
### D\.2Concept Vector Analysis on Additional Models and Implicit Conflicts

We extend concept vector analysis to additional models and examine implicit versus explicit conflicts\.

##### Analysis on Additional Models\.

Figure[19](https://arxiv.org/html/2609.03148#A4.F19)shows that GPT\-OSS\-20B exhibits awareness trends similar to those of Llama models despite having fewer layers\. Temporal and ambiguity conflicts reach saturation quickly, while other conflict types show gradual emergence\. The model exhibits strong conflict awareness across all types\. These findings suggest that our observations generalize across the tested model scales and architectures\.

![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/auc_comparison_gpt20b.png)Figure 19:Layer\-wise AUC for conflict awareness in GPT\-OSS\-20B\. Despite fewer layers, the model demonstrates consistent awareness patterns across conflict types\.
##### Implicit vs\. Explicit Conflicts\.

We analyze perspective and inferential conflicts from different source datasets\. We operationalize the explicit/implicit distinction using the structural criterion in Section[2\.1](https://arxiv.org/html/2609.03148#S2.SS1)\. A conflict isexplicitif it requires at most one inferential step to detect, andimplicitif it requires at least two cross\-evidence inferential steps\.

Inferential conflicts\.ENTAILMENTBANK \(explicit\): Each reasoning chain presents a step\-by\-step entailment that directly yields a stated conclusion\. When a second evidence piece asserts an incompatible conclusion, the contradiction is identifiable by direct comparison of their final claims, requiring a single inferential step\. FOLIO \(implicit\): GPT\-5\-generated premises introduce a conflicting reasoning branch by altering conditional or logical dependencies\. To identify the conflict, a reader must trace causal relationships through multiple premises and compare derivations across evidence chains\. This process requires combining at least two evidence pieces through conditional logic before the incompatibility surfaces, satisfying our implicit criterion\.

Perspective conflicts\.Perspectrum \(explicit\): Evidence pieces contain explicit stance sentences that directly affirm or negate the same claim \(e\.g\., “X is beneficial” vs\. “X is harmful”\)\. The viewpoint conflict is identifiable by direct comparison of these surface propositions in a single inferential step\. AllSides \(implicit\): Evidence pieces describe the same event through selective emphasis, differential fact selection, and divergent rhetorical framing, without any single statement directly contradicting another\. Detecting the underlying viewpoint conflict requires integrating implicit stances across multiple documents\. The process demands multi\-step cross\-document synthesis to identify what each article implies but does not state, which satisfies our implicit criterion on structural grounds independent of the data source\.

Figure[20](https://arxiv.org/html/2609.03148#A4.F20)compares inferential conflicts\. ENTAILMENTBANK \(explicit\) maintains consistently high AUC throughout all layers\. FOLIO \(implicit\) shows lower AUC in early layers and continues to decline in final layers\. The gap suggests that models rely heavily on surface\-level signals for conflict detection and struggle when contradiction requires multi\-step cross\-evidence inference\.

Figure[21](https://arxiv.org/html/2609.03148#A4.F21)compares perspective conflicts\. The difference is even more striking: Perspectrum \(explicit\) achieves stable high AUC across layers, whereas AllSides \(implicit\) fluctuates near chance level\. This suggests that model representations are less sensitive to conflicts that require multi\-step cross\-document synthesis, regardless of whether those conflicts arise from logical structure \(FOLIO\) or selective framing \(AllSides\)\. The shared structural factor is the number of inferential steps required\.

![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/subfolder_auc_comparison_infer.png)Figure 20:Concept vector AUC comparison for inferential conflicts across datasets\. EntailmentBank \(explicit\) shows consistently higher AUC than FOLIO \(implicit\), indicating stronger awareness of surface\-level logical conflicts\.![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/subfolder_auc_comparison_per.png)Figure 21:Concept vector AUC comparison for perspective conflicts across datasets\. Perspectrum \(explicit\) achieves stable high AUC, while AllSides \(implicit\) fluctuates near chance level, indicating reduced sensitivity to subtle viewpoint differences\.

### D\.3Spectral Energy Analysis Implementation Notes

The three\-stage pipeline and the ER andΔ\\DeltaER formulas are defined in Section[4\.2](https://arxiv.org/html/2609.03148#S4.SS2)\. We list here the numerical and software details omitted from the main text\. We compute the top\-kksingular values with PyTorch’ssvd\_lowrankon the centered activation matrix𝐇~\\tilde\{\\mathbf\{H\}\}\. We setk=10k\{=\}10andϵER=10−12\\epsilon\_\{\\mathrm\{ER\}\}=10^\{\-12\}for numerical stability\. Activations are taken at the last non\-padding token of each sample, matching the protocol used in Section[4\.2](https://arxiv.org/html/2609.03148#S4.SS2)\.

### D\.4Spectral Energy Analysis in Other Models

Figure[22](https://arxiv.org/html/2609.03148#A4.F22)shows delta energy\-ratio patterns inopenai/gpt\-oss\-20b\. The type\-specific geometric patterns remain consistent with Llama\-3\.1\-8B, suggesting that similar geometric trends appear across tested models\.

![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/delta_er_comparison_gpt20b.png)Figure 22:Delta energy ratio inopenai/gpt\-oss\-20bacross layers\.Δ​ER\>0\\Delta\\text\{ER\}\>0indicates conflict states are more concentrated \(low\-rank\);Δ​ER<0\\Delta\\text\{ER\}<0indicates more dispersed \(high\-rank\)\.
### D\.5Evidence Position Bias Across Tested Models

We analyze evidence\-position bias across seven tested models spanning different scales and training approaches\. Figures[23](https://arxiv.org/html/2609.03148#A4.F23)–[29](https://arxiv.org/html/2609.03148#A4.F29)show evidence\-contribution distributions for these tested models\. The bias is pervasive within this dataset: earlier evidence consistently receives larger contributions\. This pattern appears in both small models \(Llama\-3\.1\-8B\) and large models \(GPT\-5, Claude\-4\.5\-Sonnet\), and in both proprietary and open\-source systems\. In our experiments, this bias is not removed by model scale or training approach\.

![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/evidence_order_bias_pie_chart_llama-3_1-8b-instruct.png)Figure 23:Evidence contribution distribution from output\-level analysis\. As described in Section[3\.1](https://arxiv.org/html/2609.03148#S3.SS1), the distribution is highly non\-uniform, with earlier evidence often dominating\.![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/evidence_order_bias_pie_chart_gpt-5.png)Figure 24:Evidence position bias \(pie chart\) forgpt\-5\.![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/evidence_order_bias_pie_chart_claude-4_5-sonnet.png)Figure 25:Evidence position bias \(pie chart\) forclaude\-4\.5\-sonnet\.![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/evidence_order_bias_pie_chart_gemini-2_5-pro.png)Figure 26:Evidence position bias \(pie chart\) forgemini\-2\.5\-pro\.![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/evidence_order_bias_pie_chart_gpt-oss-120b.png)Figure 27:Evidence position bias \(pie chart\) forgpt\-oss\-120b\.![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/evidence_order_bias_pie_chart_gpt-oss-20b.png)Figure 28:Evidence position bias \(pie chart\) forgpt\-oss\-20b\.
### D\.6Bias Measurement Implementation Notes

The measurement setup, neutral centerμ\(l\)\\mu^\{\(l\)\}, and normalized directional projectionbi\(l\)b\_\{i\}^\{\(l\)\}are defined in Section[4\.2](https://arxiv.org/html/2609.03148#S4.SS2)’s sibling subsection on representation\-level position bias\. We list here the numerical and edge\-case details omitted from the main text\.

##### Numerical constants\.

We setϵb=10−8\\epsilon\_\{b\}=10^\{\-8\}inside the projection denominator for numerical stability\. If∥d\(l\)∥\\lVert d^\{\(l\)\}\\rVertor∥vi\(l\)∥\\lVert v\_\{i\}^\{\(l\)\}\\rVertfalls below the degeneracy thresholdτb=10−12\\tau\_\{b\}=10^\{\-12\}, we setbi\(l\)=0b\_\{i\}^\{\(l\)\}=0for that sample\-layer pair and log it as a skipped projection\.

##### Interpretation\.

The projectionbi\(l\)∈\[−1,1\]b\_\{i\}^\{\(l\)\}\\in\[\-1,1\]measures cosine alignment between the deviation directiond\(l\)d^\{\(l\)\}and the direction toward evidenceii\. A value of11meansc\(l\)c^\{\(l\)\}aligns perfectly with evidenceii,00means orthogonality, and−1\-1means opposing alignment\. Because cosine projections can be negative and do not sum to one, we treatbi\(l\)b\_\{i\}^\{\(l\)\}as directional alignment strength rather than probability mass\. Larger gaps \(e\.g\.,b1\(l\)≫b2\(l\)b\_\{1\}^\{\(l\)\}\\gg b\_\{2\}^\{\(l\)\}\) indicate stronger positional asymmetry\.

##### Layer\-wise computation\.

We computebi\(l\)b\_\{i\}^\{\(l\)\}across all layersl∈\{1,…,L\}l\\in\\\{1,\\ldots,L\\\}to track how bias evolves through depth, allowing us to distinguish bias that emerges in lower layers from bias that accumulates gradually across the network\.

![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/evidence_order_bias_pie_chart_llama-3_1-70b-instruct.png)Figure 29:Evidence position bias \(pie chart\) forllama\-3\.1\-70b\-instruct\.

### D\.7Representation\-Level Position Bias in GPT\-OSS\-20B

To assess whether representation\-level positional asymmetry is specific to the Llama architecture, we apply the same directional bias attribution analysis to GPT\-OSS\-20B\. The full methodology is provided in Appendix[D\.6](https://arxiv.org/html/2609.03148#A4.SS6)\. Figure[30](https://arxiv.org/html/2609.03148#A4.F30)shows layer\-wise directional projections for all six conflict types on GPT\-OSS\-20B\. The geometric bias toward earlier evidence persists from the lowest to the highest layers, closely mirroring the pattern observed in Llama\-3\.1\-8B\-Instruct\. As shown in Figure[3](https://arxiv.org/html/2609.03148#S4.F3), this cross\-layer tendency is already evident in the Llama model\. This cross\-model consistency suggests a shared pattern on tested models under our dataset: directional asymmetry in combined\-evidence representations appears across both architectures, rather than being limited to a Llama\-specific artifact\.

![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/bias_stacked_area_all_conflicts_gpt20b.png)Figure 30:Layer\-wise directional projection in GPT\-OSS\-20B via directional bias attribution\. The combined representation consistently aligns with earlier evidence directions across layers and conflict types, replicating the pattern in Llama\-3\.1\-8B and suggesting cross\-model consistency of representation\-level positional bias on tested models\.
### D\.8Position Shuffling Experiment

To test whether randomly reordering evidence input can mitigate position bias, we permute evidence order at inference time and re\-measure evidence contribution distributions using our Shapley\-based attribution framework\. Tables[9](https://arxiv.org/html/2609.03148#A4.T9)and[10](https://arxiv.org/html/2609.03148#A4.T10)show that Evidence 1 continues to dominate after shuffling on both Llama\-3\.1\-8B\-Instruct and GPT\-OSS\-20B\. This result suggests that position bias is mechanistically stable and is not fully resolved by simple input\-ordering heuristics in our experiments\.

Table 9:Evidence contribution distribution after position shuffling \(Llama\-3\.1\-8B\-Instruct\)\.Table 10:Evidence contribution distribution after position shuffling \(GPT\-OSS\-20B\)\.
### D\.9Steering Strength Sensitivity Analysis

To assess sensitivity to steering strength, we evaluate Llama\-3\.1\-8B\-Instruct acrossα∈\{0\.5,1\.0,1\.5,2\.0\}\\alpha\\in\\\{0\.5,1\.0,1\.5,2\.0\\\}on all six conflict categories\. Figure[31](https://arxiv.org/html/2609.03148#A4.F31)reports Balance scores \(summarization tasks\) and Accuracy \(reasoning tasks\) for each value ofα\\alpha\. The curves remain stable across this range\. Accuracy improves fromα=0\.5\\alpha=0\.5toα=1\.5\\alpha=1\.5for all three reasoning conflict types under both steering variants, and changes only modestly atα=2\.0\\alpha=2\.0\. Temporal conflicts show the largest gain, especially underfirst\_generatedsteering, while inferential and misinformation conflicts follow the same overall trend with smaller variation\. These results indicate that the method is robust to the choice ofα\\alpha, with strong performance throughout the tested range and a reliable operating region aroundα∈\[1\.0,1\.5\]\\alpha\\in\[1\.0,1\.5\]\.

![Refer to caption](https://arxiv.org/html/2609.03148v1/figures/sensitivity_analysis.png)Figure 31:Sensitivity of activation steering to the coefficientα\\alphaon Llama\-3\.1\-8B\-Instruct\. Balance scores \(lower is better\) and Accuracy \(higher is better\) remain stable across a wide range ofα\\alpha, suggesting robustness to this hyperparameter within the tested range\.

## Appendix EMitigation Method Details

### E\.1Prompts

We use different prompts for summarization and reasoning tasks\. Each prompt consists of a system message and a user message\. All prompts are designed to be concise and task\-appropriate\.

#### E\.1\.1Summarization Tasks

For summarization tasks \(ambiguity, granularity, perspective conflicts\), we use three prompt variants shown in Figures[32](https://arxiv.org/html/2609.03148#A5.F32),[33](https://arxiv.org/html/2609.03148#A5.F33), and[34](https://arxiv.org/html/2609.03148#A5.F34)\.

Simple Prompt for SummarizationSystem:You are a helpful assistant that answers questions based on provided evidence\. Answer concisely in 2–3 sentences\.
User:\{evidence\_text\}
\[Question\]:\{question\}Figure 32:Simple prompt for summarization tasks\.Prompt for SummarizationSystem:You are a neutral information synthesis assistant\. Analyze all sources with equal importance and provide unbiased, balanced answers\.
User:\{evidence\_text\}
\[Question\]:\{question\}Figure 33:Prompt for summarization tasks\.Single\-Evidence Prompt for SummarizationSystem:You are a helpful assistant\. Answer the question based ONLY on the provided evidence\. Be concise and factual\.
User:\[Evidence\]:\{source\_content\}
\[Question\]:\{question\}Figure 34:Single\-evidence prompt for summarization tasks\.
#### E\.1\.2Reasoning Tasks

For reasoning tasks \(inferential, misinformation, temporal conflicts\), we use prompts shown in Figures[35](https://arxiv.org/html/2609.03148#A5.F35)and[36](https://arxiv.org/html/2609.03148#A5.F36)\.

Single\-Evidence Prompt for ReasoningSystem:You are a helpful assistant that provides step\-by\-step reasoning\.
User:\[Evidence\]:\{source\_content\}
\[Question\]:\{question\}
Based on the evidence above, think step by step and provide your final answer inside<answer\></answer\>tags\.Figure 35:Single\-evidence prompt for reasoning tasks\.Combined\-Evidence Prompt for ReasoningSystem:You are a helpful assistant that provides step\-by\-step reasoning\.
User:\[Evidence 1\]:\{source\_1\}
\[Evidence 2\]:\{source\_2\}
…
\[Question\]:\{question\}
Analyze all evidence above and answer the question\. Think step by step, then provide your final answer inside<answer\></answer\>tags\.Figure 36:Combined\-evidence prompt for reasoning tasks\.