Shapley Context Pruning: A Cooperative Game Perspective for Context Reranking and Pruning

# ShapleyContextPruning: A Cooperative Game Perspective for Context Reranking and Pruning
Source: [https://arxiv.org/html/2607.16209](https://arxiv.org/html/2607.16209)
Yanqiao Chen∗, Dongsheng Hou∗, Yuhan Rui∗, Zhen Cao, Yepang Liu† Southern University of Science and Technology \{12412115, 12410421\}@mail\.sustech\.edu\.cn ruiyuhan0110@gmail\.com,12410102@mail\.sustech\.edu\.cn liuyp1@sustech\.edu\.cn ∗Equal contribution\.†Corresponding author

###### Abstract

Context reranking and pruning have become essential for improving the efficiency of modernRetrieval\-Augmented Generation \(RAG\)systems, yet aninterpretable and unified frameworkremains underexplored\. Previous work has primarily emphasized lexical retrieval, cross\-encoder architectures, model distillation, and Low\-Rank Adaptation \(LoRA\), mostly relying on heuristic loss functions and empirical attribution\. This paper presentsShapley Context Pruning \(SCP\), a novel framework for context reranking that establishes acooperative\-game\-theory perspectivefor importance attribution by modeling the context as a cooperative game\. Balancing the trade\-off between fine\-grained and coarse\-grained representations, we employ a Deep Sets architecture to approximate a permutation\-invariant value function at the sentence level, utilizing pre\-trained language models as sentence embedders and optimizing via a pairwise margin ranking loss\. To ensure practical scalability without sacrificing mathematical rigor, we leverageMonte\-Carlo samplingfor efficient training and inference, providing formaltheoretical error boundsandsample complexity guaranteesfor preserving Top\-KKsubset rankings\. Furthermore, we conductcomprehensive experiments—spanning supporting\-sentence recall, Needle\-in\-the\-Haystack \(NIAH\) evaluations, long\-context QA, and multi\-hop reasoning—alongside rigorous ablation studies on embedding quality and attribution strategies\. The model achieves competitive downstream QA performance against robust baselines\. Notably, our framework decouples the attribution logic into an exceptionally lightweight3M\-parameter value networkprototype that operates on top of standard sentence embeddings, underscoring its modular efficiency and potential deployment feasibility as a scalable context filter\. Additionally, we provide multiple case studies, including robustness justifications, analyzing real\-world application scenarios\. Beyond practical contributions, we propose an intuitive theoretical blueprint for context analysis—the*“landscape of the context”*—which offers a structural perspective on information aggregation\. Finally, based on our empirical observations, we formulate several hypotheses and open research questions for future investigation\.

## 1Introduction

![Refer to caption](https://arxiv.org/html/2607.16209v1/x1.png)Figure 1:An overview of concept evolution and how SCP different from the previous methods\. SCP explicitly models interactions between components as a cooperative game rather than simple sets or sequences\. Inspired by explainable AI \(XAI\) theory, SCP extend the restrictive methodology to a wider range of scenarios, i\.e\., context reranking\.![Refer to caption](https://arxiv.org/html/2607.16209v1/x2.png)Figure 2:Architecture Overview\. The SCP framework consists of three main components: \(i\) Embedder: Can be any pre\-trained embedding model\. \(ii\) Value Function: Evaluating the value of a subset of sentences\. \(iii\) Shapley Estimator: Estimating the Shapley value of each sentence based on the value function\. The output is a ranked list of sentences based on their Shapley values, which can be used for pruning\. Despite post\-hoc explanation in XAI, SCP actively learns a value function that directly optimizes the ranking of supporting sentences, thus ensuring that the Shapley values are meaningful for the downstream task\.Retrieval\-Augmented Generation \(RAG\)\[[24](https://arxiv.org/html/2607.16209#bib.bib24)\]has emerged as a dominant paradigm for mitigating hallucinations and expanding the knowledge boundary of Large Language Models \(LLMs\)\. Modern RAG systems generally partition retrieval, reranking, and pruning into two stages: first, applying a coarse\-grained model \(e\.g\., BM25\[[37](https://arxiv.org/html/2607.16209#bib.bib37)\], Bi\-Encoders\[[36](https://arxiv.org/html/2607.16209#bib.bib36)\]\) to efficiently filter candidate components; second, utilizing a fine\-grained model \(e\.g\., Cross\-Encoders\[[33](https://arxiv.org/html/2607.16209#bib.bib33)\]\) to rerank and prune\. However, as retrieved documents scale in length and complexity, processing expansive contexts incurs prohibitively high computational costs and often distracts models with irrelevant noise\. Existing approaches frequently cascade language models with Low\-Rank Adaptation \(LoRA\) or distillation techniques; however, their loss functions and attribution mechanisms predominantly rely on heuristics and empirical observations\. Consequently, the absence of a systematic theoretical background leads to empirically driven designs, constraining the analytical rigor essential for transparent architectural improvements\. In a recent survey\[[30](https://arxiv.org/html/2607.16209#bib.bib30)\], researchers proposed an abstract theoretical formulation for context modeling, wherein context components are aggregated via an assembly function𝒜​\(c1,c2,…,cn\)\\mathcal\{A\}\(c\_\{1\},c\_\{2\},\\ldots,c\_\{n\}\)\. The objective is to identify a context\-generating function that maximizes the expected quality of the LLM’s ensuing output\. This formulation intuitively parallels the concept of a “coalition” in game theory, directly motivating our integration of the Shapley value into the reranking stage\.

In this paper, we propose Shapley Context Pruning \(SCP\), a novel framework that transcends the interpretability limitations of existing heuristic methods and leverages cooperative game theory to improve coarse\-grained reranking accuracy\. Our primary contributions are summarized as follows:

##### A Novel Attribution Perspective for Context Engineering

Deviating from conventional heuristic attribution techniques, we introduce the Shapley value at the sentence level to establish a robust, theory\-inspired attribution mechanism\. We naturally adopt a setwise value function with pairwise ranking supervision and listwise pruning output with the help of the Shapley value\. Beyond implementation, this work provides a conceptual way to bridge engineering and theory via cooperative game theory\. See Figure[1](https://arxiv.org/html/2607.16209#S1.F1)for overview\. We discuss the broader vision and future directions for context engineering in Section[A](https://arxiv.org/html/2607.16209#A1)\.

##### A Coarse\-Grained Reranker Prototype for RAG Systems

We propose an operational Shapley\-value\-based reranking and pruning framework tailored for the preliminary context pruning stage—where the raw text is exceedingly long and demands an aggressive, coarse\-grained filter\. By reranking sentences according to their computed Shapley values, the mechanism systematically retains the most informative “coalition” of sentences that maximally contribute to downstream generation tasks\. This approach offers a novel, highly scalable perspective on the efficiency–accuracy trade\-off, prioritizing structural integrity over merely pursuing state\-of\-the\-art benchmarks\. See Figure[2](https://arxiv.org/html/2607.16209#S1.F2)for the architecture overview\.

##### Engineering Implementation and Comprehensive Experimental Validation

We validate SCP across a spectrum of rigorous benchmarks, encompassing supporting\-sentence recall, Needle\-in\-the\-Haystack \(NIAH\) tasks, long\-context QA, and multi\-hop reasoning\. Our results confirm that SCP maintains competitive overall performance, validating its feasibility on datasets such as MuSiQue, 2WikiMH, and HotpotQA\. To further validate our approach, we present extensive ablation studies analyzing the impacts of embedding choices, Monte\-Carlo sampling budgets, and comparative attribution strategies against classical methods \(e\.g\., Leave\-One\-Out\)\. Additionally, we provide case studies and empirical robustness analyses\. Crucially, to isolate and certify the efficacy of SCP ablation, we conduct evaluations independent of end\-to\-end RAG confounders\.

Ultimately, the SCP framework introduces a versatile methodology applicable beyond context pruning—extending to general component reranking, feature selection, and software test\-case optimization\. By integrating rigorous theoretical grounding with scalable implementation, this framework contributes a robust analytical tool for the broader scientific community\.

## 2Related Work

Recent context pruning methods fall into four categories\.*\(i\)*Token\-level prompt compression \(LLMLingua\-2\[[20](https://arxiv.org/html/2607.16209#bib.bib20),[35](https://arxiv.org/html/2607.16209#bib.bib35)\]and Selective Context\[[25](https://arxiv.org/html/2607.16209#bib.bib25)\]\) compresses prompts via surprisal\-based selection, but relies on local token signals without cross\-sentence cooperative modeling; we use LLMLingua\-2 as our baseline\.*\(ii\)*BERT\-based sentence\-level pruning \(Provence\[[5](https://arxiv.org/html/2607.16209#bib.bib5)\]\) filters content using transformer\-style adaptation\.*\(iii\)*Extractive and abstractive compressors \(RECOMP\[[49](https://arxiv.org/html/2607.16209#bib.bib49)\]\) condense passages into new representations, diverging from strict sentence\-selection reranking\.*\(iv\)*Task\-specialized systems \(SWE\-Pruner\[[48](https://arxiv.org/html/2607.16209#bib.bib48)\]\) show that structure\-aware pruning yields gains in domain\-specific settings, emphasizing the need to align pruning with downstream task structure\.

In standard RAG pipelines, retrieval and reranking dictate the effective context length\. Sparse retrieval \(BM25\[[37](https://arxiv.org/html/2607.16209#bib.bib37)\]\) remains a strong lexical baseline, while dense retrieval \(DPR\[[22](https://arxiv.org/html/2607.16209#bib.bib22)\]\) and cross\-encoder rerankers\[[33](https://arxiv.org/html/2607.16209#bib.bib33)\]improve semantic matching at higher computational cost\. Models like FiD\[[19](https://arxiv.org/html/2607.16209#bib.bib19)\]demonstrate that inter\-passage interactions are critical for multi\-hop reasoning\. Meanwhile, “Lost in the Middle”\[[27](https://arxiv.org/html/2607.16209#bib.bib27)\]and related surveys\[[26](https://arxiv.org/html/2607.16209#bib.bib26)\]reveal that LLMs underutilize long contexts, motivating selective, high\-quality pruning before generation\. Feature attribution studies also highlight a gap between local relevance and holistic contribution: attention weights\[[6](https://arxiv.org/html/2607.16209#bib.bib6)\]do not equate to feature importance, and gradient methods such as Integrated Gradients\[[41](https://arxiv.org/html/2607.16209#bib.bib41)\]lack direct mappings to discrete, set\-valued context selection\.

Learning\-to\-rank work spans pointwise\[[7](https://arxiv.org/html/2607.16209#bib.bib7)\], pairwise\[[16](https://arxiv.org/html/2607.16209#bib.bib16)\], setwise\[[53](https://arxiv.org/html/2607.16209#bib.bib53)\], and listwise\[[3](https://arxiv.org/html/2607.16209#bib.bib3)\]paradigms\. OptiSet\[[21](https://arxiv.org/html/2607.16209#bib.bib21)\]pursues set\-wise modeling by selecting optimal component combinations, aligning with our objective of optimizing discrete coalitions\. Closer to our setting, LooComp\[[9](https://arxiv.org/html/2607.16209#bib.bib9)\]applies a Leave\-One\-Out \(LOO\) objective with LoRA for context pruning\. We differ in two key ways: \(i\) we train an independent 3M\-parameter Deep Sets value function from scratch, decoupling attribution from the embedder; \(ii\) our analysis \(Appendix[C\.4\.2](https://arxiv.org/html/2607.16209#A3.SS4.SSS2)\) shows LOO collapses under semantic redundancy, assigning near\-zero scores to duplicated sentences, whereas Shapley attribution—averaging marginal contributions across all coalitions—inherently resists this failure mode\. Applying Shapley attribution to subsets requires a permutation\-invariant value function, for which Deep Sets\[[52](https://arxiv.org/html/2607.16209#bib.bib52)\]is the canonical choice; we keep the estimator lightweight to avoid representation bottlenecks under finite latent dimensions\[[46](https://arxiv.org/html/2607.16209#bib.bib46)\]\.

The Shapley value\[[40](https://arxiv.org/html/2607.16209#bib.bib40)\], originating in cooperative game theory, quantifies a component’s expected marginal contribution across all coalitions—an attribution scheme that aligns naturally with context pruning, where a sentence inconsequential in isolation may prove critical when synergized with bridging evidence\. Yet existing Shapley\-based frameworks each sacrifice one key property: Data Shapley\[[11](https://arxiv.org/html/2607.16209#bib.bib11)\]achieves interpretability and redundancy robustness but is fundamentally unscalable, since each evaluation ofv​\(S\)v\(S\)requires retraining the underlying model; SHAP\[[29](https://arxiv.org/html/2607.16209#bib.bib29)\]explains a fixed model by attributing along feature dimensions rather than samples, and thus does not rank context units; TokenSHAP\[[12](https://arxiv.org/html/2607.16209#bib.bib12)\]operates at an overly fine token granularity with a hard\-coded value function \(e\.g\., TF\-IDF cosine similarity\), discarding richer semantic signals\. SCP closes this gap by*learning*a dedicated, permutation\-invariant value function at the sentence level, jointly delivering scalability, interpretability, and redundancy robustness\. Beyond pruning, this learned value function opens new directions for context engineering—hierarchical context structures, bridging\-sentence analysis, and tree\-based context management—which we explore in Appendix[A](https://arxiv.org/html/2607.16209#A1)\.

In summary, while SCP builds upon established theoretical ingredients—Shapley values, Deep Sets, and cooperative games—its contribution lies in answering, for the first time, how Shapley attribution can be feasibly operationalized for context reranking, and in providing a novel blueprint toward more systematic and interpretable context pruning and management\. See Table[1](https://arxiv.org/html/2607.16209#A1.T1)for detailed comparison\.

## 3Shapley Context Pruning \(SCP\)

### 3\.1The Shapley value

We introduce the Shapley value\[[40](https://arxiv.org/html/2607.16209#bib.bib40)\]from cooperative game theory as the theoretical foundation of our context pruning method\. The Shapley value is a solution concept in cooperative game theory that aims to fairly distribute the total gains \(or costs\) among players based on their individual contributions to the overall outcome\.

###### Definition 3\.1\(Shapley Value\)\.

Consider a cooperative game with a set of playersNNand a characteristic functionv:2N→ℝv:2^\{N\}\\rightarrow\\mathbb\{R\}that assigns a value to each coalition of players\. The Shapley valueϕi​\(v\)\\phi\_\{i\}\(v\)for playeri∈Ni\\in Nis defined as:

ϕi​\(v\)=∑S⊆N∖\{i\}\|S\|\!​\(\|N\|−\|S\|−1\)\!\|N\|\!​\[v​\(S∪\{i\}\)−v​\(S\)\]\\phi\_\{i\}\(v\)=\\sum\_\{S\\subseteq N\\setminus\\\{i\\\}\}\\frac\{\|S\|\!\(\|N\|\-\|S\|\-1\)\!\}\{\|N\|\!\}\[v\(S\\cup\\\{i\\\}\)\-v\(S\)\]where the sum is taken over all subsetsSSofNNthat do not include playerii\.

##### Rationale

We draw inspiration from the Shapley value to design a ranking\-based importance estimator\. Although our learned value functionvθv\_\{\\theta\}does not strictly satisfy all Shapley axioms due to its data\-driven rather than axiomatic construction, the Shapley formulation serves as the conceptual foundation for our coalition\-based importance aggregation\. While we relax these axiomatic constraints to maximize representational capacity, reintroducing specific theoretical constraints via targeted training regularization presents a promising avenue for future research to bridge empirical flexibility with formal guarantees\. See Section[A\.1](https://arxiv.org/html/2607.16209#A1.SS1)for a formal description of the four Shapley properties and how these properties correspond to real context reranking\.

### 3\.2The Value Function

Constructing an accurate and tractable value functionv​\(S\)v\(S\)is critical for deriving Shapley values\. Specifically, we use a supervised learning paradigm to map any chosen subset of contextual passages to an expected retrieval/generation performance metric \(e\.g\., QA accuracy or span recall\)\. Given thatSSis an unordered set, the value function should satisfy permutation invariance\. We therefore adopt the Deep Sets architecture\.

###### Definition 3\.2\(Deep Sets\[[52](https://arxiv.org/html/2607.16209#bib.bib52)\]\)\.

A functionf:2𝒳→𝒴f:2^\{\\mathcal\{X\}\}\\to\\mathcal\{Y\}acting on sets is permutation invariant iff​\(\{x1,…,xM\}\)=f​\(\{xπ​\(1\),…,xπ​\(M\)\}\)f\(\\\{x\_\{1\},\\ldots,x\_\{M\}\\\}\)=f\(\\\{x\_\{\\pi\(1\)\},\\ldots,x\_\{\\pi\(M\)\}\\\}\)for any permutationπ\\pi\. The Deep Sets theorem states that a functionffoperating on a setXXhaving permutation invariance can be universally approximated by decomposing into the form:

f​\(X\)=ρ​\(∑x∈Xψ​\(x\)\)f\(X\)=\\rho\\left\(\\sum\_\{x\\in X\}\\psi\(x\)\\right\)whereψ:𝒳→ℝd\\psi:\\mathcal\{X\}\\to\\mathbb\{R\}^\{d\}andρ:ℝd→𝒴\\rho:\\mathbb\{R\}^\{d\}\\to\\mathcal\{Y\}are continuous transformations\.

In our context, the parameterized value function operates asv​\(S;θ\)=ρ​\(∑ci∈Sψ​\(ci\)\)v\(S;\\theta\)=\\rho\\left\(\\sum\_\{c\_\{i\}\\in S\}\\psi\(c\_\{i\}\)\\right\)where the feature extractorψ\\psiindependently maps each context componentcic\_\{i\}\(the original sentence and the query\) to a latent embedding, and the regressorρ\\rhomaps the additively aggregated representations to a scalar performance value\. Both components are Multi\-Layer Perceptrons \(MLPs\) parameterized byθ\\theta\.

The training paradigm is structured as follows:*\(i\)*Input: The embedded sentence setS⊆CS\\subseteq Cpaired with a specific reasoning query\.*\(ii\)*Output: A scalar prediction indicating the inferred value \(relative importance\) of the subsetSSfor answering the given query\. Since obtaining precise real\-valued targets requires excessive LLM queries, we supervise the optimization using a pairwise margin ranking loss\. This objective only forces the network to correctly rank optimal evidence passages above distractors, rather than memorize exact performance figures:

L​\(θ\)=1\|P\|⋅\|N\|​∑i∈P∑j∈Nmax⁡\(0,ϵ−\(ϕi​\(vθ\)−ϕj​\(vθ\)\)\)L\(\\theta\)=\\frac\{1\}\{\|P\|\\cdot\|N\|\}\\sum\_\{i\\in P\}\\sum\_\{j\\in N\}\\max\(0,\\epsilon\-\(\\phi\_\{i\}\(v\_\{\\theta\}\)\-\\phi\_\{j\}\(v\_\{\\theta\}\)\)\)whereP=\{k:yk=1\}P=\\\{k:y\_\{k\}=1\\\}is the set of positive supporting facts,N=\{k:yk=0\}N=\\\{k:y\_\{k\}=0\\\}is the set of negative distractors, andϵ\>0\\epsilon\>0is the margin hyperparameter\. In real training datasets, these sentences are annotated by human annotators\. This formulation directly binds the Shapley value calculation into the backward pass, ensuring that relevant documents maintain a Shapley value margin ofϵ\\epsilonover irrelevant ones:ϕi\>ϕj\\phi\_\{i\}\>\\phi\_\{j\}wheneveryi\>yjy\_\{i\}\>y\_\{j\}\. See Section[D\.4](https://arxiv.org/html/2607.16209#A4.SS4)for justification\.

### 3\.3Efficient Shapley Estimation via Monte\-Carlo Sampling

Scaling exact Shapley value derivation poses substantial latency challenges for long\-context scenarios, as it mandates𝒪​\(2n\)\\mathcal\{O\}\(2^\{n\}\)model forward passes\. To mitigate this bottleneck, we use an optimized Monte\-Carlo sampling approximation\[[4](https://arxiv.org/html/2607.16209#bib.bib4)\]\.

Rather than exhaustively querying the combinatorial space, the algorithm uniformly samples independent context permutations and accumulates the marginal delta contributions dynamically\. We provide Algorithm[1](https://arxiv.org/html/2607.16209#alg1)in pseudocode form for clarity\.

Algorithm 1Algorithm 1Monte\-Carlo Shapley Estimation1:A context

C=\{c1,…,cn\}C=\\\{c\_\{1\},\\ldots,c\_\{n\}\\\}, optimized value function

vθv\_\{\\theta\}, permutation count

MM
2:Estimated Shapley values

\{ϕ^1,ϕ^2,…,ϕ^n\}\\\{\\hat\{\\phi\}\_\{1\},\\hat\{\\phi\}\_\{2\},\\ldots,\\hat\{\\phi\}\_\{n\}\\\}
3:Initialize Shapley estimates

ϕ^i=0\\hat\{\\phi\}\_\{i\}=0for all

i∈\{1,…,n\}i\\in\\\{1,\\dots,n\\\}
4:foreach sample

m=1m=1to

MMdo

5:Sample a uniform random permutation

π\\piof the set

CC
6:Initialize an expanding coalition subset

S=∅S=\\emptyset
7:foreach component

cπ​\(j\)c\_\{\\pi\(j\)\}according to

π\\pido

8:Compute marginal contribution:

Δ​v←vθ​\(S∪\{cπ​\(j\)\}\)−vθ​\(S\)\\Delta v\\leftarrow v\_\{\\theta\}\(S\\cup\\\{c\_\{\\pi\(j\)\}\\\}\)\-v\_\{\\theta\}\(S\)
9:Accumulate onto attribution:

ϕ^π​\(j\)←ϕ^π​\(j\)\+Δ​vM\\hat\{\\phi\}\_\{\\pi\(j\)\}\\leftarrow\\hat\{\\phi\}\_\{\\pi\(j\)\}\+\\frac\{\\Delta v\}\{M\}
10:Augment coalition:

S←S∪\{cπ​\(j\)\}S\\leftarrow S\\cup\\\{c\_\{\\pi\(j\)\}\\\}
11:endfor

12:endfor

13:return

\{ϕ^1,ϕ^2,…,ϕ^n\}\\\{\\hat\{\\phi\}\_\{1\},\\hat\{\\phi\}\_\{2\},\\ldots,\\hat\{\\phi\}\_\{n\}\\\}

This estimator converges to the analytical Shapley value as the number of sampled permutations increases, while capping forward evaluations at𝒪​\(M​n\)\\mathcal\{O\}\(Mn\)\. We prove the following error bound for the Shapley value estimation, under the assumption stated below\. See Appendix A\.4 for the proof\.

###### Assumption 3\.1\(Bounded Value Function\)\.

The trained value functionvθv\_\{\\theta\}and the true value functionv∗v^\{\*\}are uniformly bounded: there exists a constantB\>0B\>0such that for every coalitionS⊆NS\\subseteq N,

0≤vθ​\(S\)≤Band0≤v∗​\(S\)≤B\.0\\leq v\_\{\\theta\}\(S\)\\leq B\\quad\\text\{and\}\\quad 0\\leq v^\{\*\}\(S\)\\leq B\.In practice, this can be enforced by a sigmoid output layer or by clipping the predictions to\[0,B\]\[0,B\]\.

###### Theorem 3\.1\(Error Bound for Pruning Attribution\)\.

Letv∗v^\{\*\}be the true value function andvθv\_\{\\theta\}a trained approximation satisfying Assumption[3\.1](https://arxiv.org/html/2607.16209#S3.Thmassumption1)\. Assume the point\-wise approximation error is uniformly bounded byϵapprox\\epsilon\_\{\\mathrm\{approx\}\}, i\.e\.,\|v∗​\(S\)−vθ​\(S\)\|≤ϵapprox,∀S⊆N\.\|v^\{\*\}\(S\)\-v\_\{\\theta\}\(S\)\|\\leq\\epsilon\_\{\\mathrm\{approx\}\},\\quad\\forall S\\subseteq N\.Letϕ^i\\hat\{\\phi\}\_\{i\}be the Monte\-Carlo Shapley estimate of playeriiobtained fromMMindependent uniform permutations \(Algorithm 1\)\. Then, with probability at least1−δ1\-\\deltaover the random permutations, the following holds simultaneously for alli∈Ni\\in N:

\|ϕ^i​\(vθ\)−ϕi​\(v∗\)\|≤B​2​ln⁡\(2​n/δ\)M\+2​ϵapprox\.\\bigl\|\\hat\{\\phi\}\_\{i\}\(v\_\{\\theta\}\)\-\\phi\_\{i\}\(v^\{\*\}\)\\bigr\|\\;\\leq\\;B\\sqrt\{\\frac\{2\\ln\(2n/\\delta\)\}\{M\}\}\\;\+\\;2\\epsilon\_\{\\mathrm\{approx\}\}\.

The theorem shows that the training error and the estimation error are decoupled: they are additive, not multiplicative\. See[6](https://arxiv.org/html/2607.16209#A1.F6)for a visualization\. From the error\-bound theorem, asnnis around 10 for multi\-hop reasoning tasks, we applyM=50M=50for efficient training and inference\. However, higher sampling counts are recommended\. See the ablation study for further details\.

### 3\.4The General Shapley Context Pruning Algorithms

The objective of context pruning is to preserve the useful components and filter the distractors\.

###### Definition 3\.3\(Context Pruning Problem \(sentence\-level\)\)\.

Given a set of context components \(e\.g\., sentences or paragraphs\)C=\{c1,c2,…,cn\}C=\\\{c\_\{1\},c\_\{2\},\\ldots,c\_\{n\}\\\}, the objective is to learn a binary decision vectorD∈\{0,1\}nD\\in\\\{0,1\\\}^\{n\}, whereDi=1D\_\{i\}=1denotes retaining the componentcic\_\{i\}andDi=0D\_\{i\}=0denotes discarding it\.

Once the robust value functionvθv\_\{\\theta\}is empirically optimized, calculating the contribution of any constituent part relies on establishing the Shapley valueϕi​\(vθ\)\\phi\_\{i\}\(v\_\{\\theta\}\)\. Subsequently, pruning reduces to a straightforward filtration process based on these generated utility scores\. We outline the Shapley Context Pruning procedure via empirical criteria: an absolute top\-K constraint depicted in Algorithm[2](https://arxiv.org/html/2607.16209#alg2)\.

##### Rationale

Since the algorithm operates at a coarse\-grained level, the pruner must reliably filter out the most irrelevant sentences while retaining the most informative ones; we therefore use a less aggressive pruning ratio for this preliminary stage\. We provide a bound for Monte\-Carlo sampling number for Top\-K pruning in the appendix\. See Theorem[A\.3](https://arxiv.org/html/2607.16209#A1.Thmtheorem3)\.

## 4Experiment

Our experiments are designed as feasibility studies: we seek to validate that a cooperative\-game\-theoretic formulation can produce meaningful, computationally tractable context rankings, rather than to claim state\-of\-the\-art compression rates and accuracy\.

### 4\.1Experiment Setup

Our standard Shapley Context Pruning \(SCP\) model uses a computationally lightweight Deep Sets architecture with a hidden dimension of 1024 and a latent dimension of 512, totalling only 3\.03M parameters \(do not contain embedding parameters, by default the total parameters in the pipeline are 25\.3M\)\. The default embedding model for the SCP model is MiniLM\[[47](https://arxiv.org/html/2607.16209#bib.bib47)\]\. To be specific, for baseline "Cross Encoder", we usems\-marco\-MiniLM\-L6\-v2\[[32](https://arxiv.org/html/2607.16209#bib.bib32)\]as Cross Encoder, which is different from our embedderall\-MiniLM\-L6\-v2\. We train the network on the MS MARCO\[[1](https://arxiv.org/html/2607.16209#bib.bib1)\], HotpotQA\[[51](https://arxiv.org/html/2607.16209#bib.bib51)\], 2WikiMultiHop\[[17](https://arxiv.org/html/2607.16209#bib.bib17)\], MuSiQue\[[44](https://arxiv.org/html/2607.16209#bib.bib44)\]and FEVER\[[43](https://arxiv.org/html/2607.16209#bib.bib43)\]datasets for 10 epochs using an AdamW\[[28](https://arxiv.org/html/2607.16209#bib.bib28)\]optimizer \(learning rate 1e\-3, cosine schedule\)\. By default, models are trained on the corresponding dataset \(80% for training, 10% for validation, 10% for testing\)\. The pairwise marginϵ\\epsilonis set to 0\.15\. Compression rate is computed by token count, while keep rate is computed by the number of sentences; therefore, they are not exactly the same\. The PairwiseAUC\(Area Under Curve\) metric, or Pairwise Ranking Accuracy, evaluates the performance of ranking the supporting sentences before the distractors\.R@0\.Xmetric represents the percentage of supporting sentences preserved, i\.e\.,Preserved Supporting SentencesTotal Supporting Sentences\\frac\{\\text\{Preserved Supporting Sentences\}\}\{\\text\{Total Supporting Sentences\}\}\.EMmetric is short for "Exact Match", computed by strict string matching of the LLM\-output and true answers\.F1metric is the harmonic mean of the recall and precision\. The datasets, baseline methods \(and the exact version we use\), and inference LLMs are listed in Table[2](https://arxiv.org/html/2607.16209#A3.T2)\. The LLM QA pipeline is as follows: the context is embedded at the sentence level, the sentences are reranked and pruned, then fed to the LLM for inference\. After that, we apply an EM/F1 evaluation script or use an LLM\-as\-judge to output a performance score\.

### 4\.2Main Result

Figure[3](https://arxiv.org/html/2607.16209#S4.F3)summarizes the empirical evaluation of SCP across four dimensions: reranking quality, downstream LLM QA accuracy, cross\-model robustness, and long\-context needle recall\.

![Refer to caption](https://arxiv.org/html/2607.16209v1/x3.png)Figure 3:Overview of SCP empirical results\. \(a\) Reranking AUC \(Pairwise Ranking Accuracy\) comparison; SCP achieves the highest AUC on MuSiQue, 2WikiMH, and FEVER\. \(b\) Downstream LLM QA F1 under 50% compression\. \(c\) Cross\-model judge\-score heatmap on HotpotQA; highest score per model is bolded and boxed\. \(d\) NIAH needle\-recall curves across context lengths \(top\) and insertion depths \(bottom\)\. BM25 dominates fact recall in long contexts since the "needle" typically contains query keywords, a scenario well\-suited to BM25’s sparse retrieval mechanism\.The original main result is shown in Table[3](https://arxiv.org/html/2607.16209#A3.T3), Table[4](https://arxiv.org/html/2607.16209#A3.T4), Table[5](https://arxiv.org/html/2607.16209#A3.T5), Table[13](https://arxiv.org/html/2607.16209#A3.T13)and Table[14](https://arxiv.org/html/2607.16209#A3.T14)\. We report the AUC of the reranking stage and the downstream QA performance of the LLMs\. The AUC metric directly reflects the quality of the reranking stage, while the downstream QA performance reflects the end\-to\-end effectiveness of the pruning method\.

For the MuSiQue rows of Table[4](https://arxiv.org/html/2607.16209#A3.T4), we test four SCP variants: trained at paragraph level and pruning at paragraph level \(paragraph\*\); trained at sentence level and pruning at sentence level \(sentlevel split\); trained at sentence level and pruning at paragraph level \(sentlevel no split\); and trained on MS MARCO and pruning at sentence level \(MS MARCO, cross\)\. We observe a misalignment between AUC and downstream LLM QA accuracy on MuSiQue for in\-domain trained models, which is caused by the coarse\-grained annotation from the dataset itself\. The Cross Encoder used here is a variant of MiniLM\. See Section[C\.3\.3](https://arxiv.org/html/2607.16209#A3.SS3.SSS3)for a detailed justification\.

We also conduct experiments on CL\-Bench and RULER \(a benchmark for Needle\-In\-A\-Haystack experiment, NIAH\) to test the performance of SCP on long\-context tasks\. See Tables[9](https://arxiv.org/html/2607.16209#A3.T9),[10](https://arxiv.org/html/2607.16209#A3.T10),[7](https://arxiv.org/html/2607.16209#A3.T7),[6](https://arxiv.org/html/2607.16209#A3.T6),[8](https://arxiv.org/html/2607.16209#A3.T8)and Figure[7](https://arxiv.org/html/2607.16209#A3.F7)for CL\-Bench and Tables[14](https://arxiv.org/html/2607.16209#A3.T14),[15](https://arxiv.org/html/2607.16209#A3.T15),[13](https://arxiv.org/html/2607.16209#A3.T13),[16](https://arxiv.org/html/2607.16209#A3.T16)for NIAH\. The result shows that SCP outperforms BM25 on CL\-Bench and equal to BM25 on RULER, showing the effectiveness of our approach in long context fact recalling and key sentence preservation\. We provide the detail results and analysis in the appendix\. We also provide a case study in real world in Tables[23](https://arxiv.org/html/2607.16209#A5.T23),[25](https://arxiv.org/html/2607.16209#A5.T25),[24](https://arxiv.org/html/2607.16209#A5.T24),[26](https://arxiv.org/html/2607.16209#A5.T26)\.

The main results show that SCP reaches competitive performance with fewer parameters \(even when accounting for the embedder parameters\), and outperforms baselines on specific LLMs and datasets\. These experiments validate our method as an engineering contribution; further improvements to fully realize SCP’s potential in real\-world tasks are left to future work\.

### 4\.3Ablations

Figure[4](https://arxiv.org/html/2607.16209#S4.F4)presents the ablation studies, covering embedding quality robustness, Monte\-Carlo sampling budget, Shapley versus LOO attribution, and a concrete redundancy failure mode\.

![Refer to caption](https://arxiv.org/html/2607.16209v1/x4.png)Figure 4:Ablation studies\. \(a\) Embedding mechanism comparison; the default SentenceTransformer and BERT outperform shallow embeddings\. SCP performs robustness under lower parameters embedder, i\.e\., MiniLM\. Model Size: MiniLM\(22M\), BERT\(110M\)\. The model size of SCP will change slightly for embedding dimension alignment, Word2Vec\(300dim\), MiniLM\(384dim\), BERT\(768dim\)\. The total parameters of the value function are \(3M±\\pm0\.5M\) in this experiment\. \(b\) Recall improves with more Monte\-Carlo samples; the defaultM=50M=50is marked\. \(c\) SCP mean recall approaches and crosses LOO exact mean aroundM≈400M\\approx 400\. \(d\) LOO under\-credits duplicated supporting facts, whereas SCP preserves positive attribution\.#### 4\.3\.1Embedders

A natural concern is whether SCP’s effectiveness stems merely from the pre\-trained embedding model that explicitly encodes queries\. To rigorously decouple the source of SCP’s robustness, we ablate the initial sentence\-embedding input mechanism\. We implement experiments on MiniLM,bert\-base\-uncased\[[8](https://arxiv.org/html/2607.16209#bib.bib8)\],word2vec\-google\-news\-300\[[31](https://arxiv.org/html/2607.16209#bib.bib31)\]and TF\-IDF\[[38](https://arxiv.org/html/2607.16209#bib.bib38)\]\. Table[17](https://arxiv.org/html/2607.16209#A3.T17)shows the results\. The results show that SCP is robust across modern embedding models\. Though embedding quality does impact performance as expected, the similar AUC performance on MiniLM and Word2Vec shows that embedding dimension rather than embedding mechanism is the factor that determines the performance of SCP\. See Section[D\.3](https://arxiv.org/html/2607.16209#A4.SS3)for more analysis\.

#### 4\.3\.2Attribution Strategies Comparison

In this ablation, we compare the performance of SCP using the Shapley value attribution mechanism against a simpler Leave\-One\-Out \(LOO\) attribution method\. The LOO method evaluates the marginal contribution by simply removing one element from the full coalition, without averaging over all permutations\. We shows that LOO strategy is reasonable choice for low computational budget, but not optimal for higher accuracy requirements and robust performance in Table[16](https://arxiv.org/html/2607.16209#A3.T16),[18](https://arxiv.org/html/2607.16209#A3.T18)\. See Section[C\.4\.2](https://arxiv.org/html/2607.16209#A3.SS4.SSS2)for the full analysis and Table[19](https://arxiv.org/html/2607.16209#A3.T19)for a concrete example of the redundancy failure mode of LOO\.

#### 4\.3\.3Monte\-Carlo Sampling Number

Table[20](https://arxiv.org/html/2607.16209#A3.T20),[16](https://arxiv.org/html/2607.16209#A3.T16)shows that SCP’s accuracy improves as the Monte\-Carlo sampling count increases\. Table[20](https://arxiv.org/html/2607.16209#A3.T20)indicates that SCP is appropriate as a coarse\-grained reranker at higher keep ratios, where it filters most low\-value noise and lets a finer\-grained reranker/pruner handle the preliminary processed context\. However, it is worth noting that we do not take the embedding time into consideration\.

## 5Discussion and Future Work

### 5\.1Limitations

Although the SCP framework shows competitive performance in our validation experiments and provides a promising step toward a theoretical blueprint, several limitations remain to be addressed:

Suboptimal Pruning Criteria:In the current implementation, we predominantly rely on Top\-K heuristics as the final pruning criterion; an adaptive threshold for SCP is not proposed in this paper\.

Estimation Error and High Variance in Different Scenarios:Although the framework’s error is bounded by Theorem[3\.1](https://arxiv.org/html/2607.16209#S3.Thmtheorem1), an appropriate sampling count for different scenarios has not yet been characterized in this work\. We did not implement multiple runs in LLM QA tasks due to API budget constraints\. However, experiments in different scenarios have shown the feasibility of SCP framework\.

The Gap between theoretical structure and engineering performance:We introduce the concept “Landscape of Context”, which is a theoretical structure for modeling the context\. See Figure[5](https://arxiv.org/html/2607.16209#A1.F5)for an illustration\. However, such structures needs supermodular value functions\. As an initial step toward closing this gap, we proposeConvexDeepSetsV2in Appendix[A\.5](https://arxiv.org/html/2607.16209#A1.SS5)and Table[22](https://arxiv.org/html/2607.16209#A4.T22)\. to enforce input convexity in the value function\. However, fully reconciling theoretical guarantees and empirical performance remains an open question\.

### 5\.2Future Work

Our study bridges cooperative game theory with context reranking, providing several promising direction for future research:

- •Advanced Interaction\-Aware Pruning Algorithms:Addressing the limitations outlined above, future work could explore self\-adapting thresholding pruning algorithms that may dynamically changeKKunder different scenarios\. Furthermore, theoretical insights in Section[A](https://arxiv.org/html/2607.16209#A1)may provide structural priors for designing more efficient pruning algorithms\.
- •Theoretical Structure implications:The cooperative game formulation admits a rich analytical toolkit: "core" provides structural insights into stable context configurations, the supermodularity suggests a hierarchical multi\-granularity pruning strategy \(Appendix[A](https://arxiv.org/html/2607.16209#A1)\)\. Future work could empirically validate how these structures manifest in real\-world retrieval scenarios\.See more in Section[D\.7](https://arxiv.org/html/2607.16209#A4.SS7)\.
- •Extending Beyond Context Engineering:The SCP framework’s generality suggests applications beyond context pruning, such as feature selection in software test\-case prioritization\. Exploring these extensions could further demonstrate the versatility of cooperative game\-theoretic approaches\.

## 6Conclusion

This paper introduces Shapley Context Pruning \(SCP\), framing context reranking as a cooperative game\. A lightweight Deep Sets network with pairwise ranking and Monte\-Carlo sampling makes Shapley attribution scalable\. Empirically, SCP outperforms baselines on multi\-hop benchmarks while remaining interpretable via approximately axiomatic attribution\. Beyond, this paper provides a conceptual guideline towards future work in context engineering\. These results bridge cooperative game theory with practical context engineering for scalable, interpretable long\-context processing\.

## References

- Bajaj et al\. \[2016\]Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang\.MS MARCO: A human generated machine reading comprehension dataset\.*arXiv preprint arXiv:1611\.09268*, 2016\.
- Blei et al\. \[2003\]David M\. Blei, Andrew Y\. Ng, and Michael I\. Jordan\.Latent dirichlet allocation\.*J\. Mach\. Learn\. Res\.*, 3\(null\):993–1022, March 2003\.ISSN 1532\-4435\.
- Cao et al\. \[2007\]Zhe Cao, Tao Qin, Tie\-Yan Liu, Ming\-Feng Tsai, and Hang Li\.Learning to rank: from pairwise approach to listwise approach\.In*Proceedings of the 24th International Conference on Machine Learning*, ICML ’07, page 129–136, New York, NY, USA, 2007\. Association for Computing Machinery\.ISBN 9781595937933\.doi:10\.1145/1273496\.1273513\.URL[https://doi\.org/10\.1145/1273496\.1273513](https://doi.org/10.1145/1273496.1273513)\.
- Castro et al\. \[2009\]Javier Castro, Daniel Gómez, and Juan Tejada\.Polynomial calculation of the Shapley value based on sampling\.*Computers and Operations Research*, 36\(1\):303–314, 2009\.
- Chirkova et al\. \[2025\]Nadezhda Chirkova, Thibault Formal, Vassilina Nikoulina, and Stéphane Clinchant\.Provence: Efficient and robust context pruning for retrieval\-augmented generation\.In*International Conference on Learning Representations \(ICLR\)*, 2025\.arXiv:2501\.16214\.
- Clark et al\. \[2019\]Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D\. Manning\.What does BERT look at? an analysis of BERT’s attention\.In*Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP*, 2019\.arXiv:1906\.04341\.
- Cossock and Zhang \[2006\]David Cossock and Tong Zhang\.Subset ranking using regression\.In Gábor Lugosi and Hans Ulrich Simon, editors,*Learning Theory*, pages 605–619, Berlin, Heidelberg, 2006\. Springer Berlin Heidelberg\.ISBN 978\-3\-540\-35296\-9\.
- Devlin et al\. \[2019\]Jacob Devlin, Ming\-Wei Chang, Kenton Lee, and Kristina Toutanova\.Bert: Pre\-training of deep bidirectional transformers for language understanding, 2019\.URL[https://arxiv\.org/abs/1810\.04805](https://arxiv.org/abs/1810.04805)\.
- Do et al\. \[2026\]Thao Do, Dinh Phu Tran, An Vo, Seon Kwon Kim, and Daeyoung Kim\.Loocomp: Leverage leave\-one\-out strategy to encoder\-only transformer for efficient query\-aware context compression, 2026\.URL[https://arxiv\.org/abs/2603\.09222](https://arxiv.org/abs/2603.09222)\.
- Dou et al\. \[2026\]Shihan Dou et al\.CL\-bench: A benchmark for context learning\.*arXiv preprint arXiv:2602\.03587*, 2026\.
- Ghorbani and Zou \[2019\]Amirata Ghorbani and James Zou\.Data Shapley: Equitable valuation of data for machine learning\.In*International Conference on Machine Learning \(ICML\)*, 2019\.arXiv:1904\.02868\.
- Goldshmidt and Horovicz \[2024\]Roni Goldshmidt and Miriam Horovicz\.Tokenshap: Interpreting large language models with monte carlo shapley value estimation, 2024\.URL[https://arxiv\.org/abs/2407\.10114](https://arxiv.org/abs/2407.10114)\.
- Google DeepMind \[2026\]Google DeepMind\.Gemini 3\.[https://deepmind\.google](https://deepmind.google/), 2026\.The experiment corresponding to Gemini 3 Pro/Flash is implemented within 27 March 2026 to 15 April 2026\.
- Guo et al\. \[2025\]Daya Guo, Dejian Yang, and Haowei et al\. Zhang\.Deepseek\-r1 incentivizes reasoning in llms through reinforcement learning\.*Nature*, 645\(8081\):633–638, 2025\.ISSN 1476\-4687\.doi:10\.1038/s41586\-025\-09422\-z\.URL[http://dx\.doi\.org/10\.1038/s41586\-025\-09422\-z](http://dx.doi.org/10.1038/s41586-025-09422-z)\.
- Harsanyi \[1982\]John C\. Harsanyi\.*A Simplified Bargaining Model for the n\-Person Cooperative Game*, pages 44–70\.Springer Netherlands, Dordrecht, 1982\.ISBN 978\-94\-017\-2527\-9\.doi:10\.1007/978\-94\-017\-2527\-9\_3\.URL[https://doi\.org/10\.1007/978\-94\-017\-2527\-9\_3](https://doi.org/10.1007/978-94-017-2527-9_3)\.
- Herbrich et al\. \[1999\]R\. Herbrich, T\. Graepel, and K\. Obermayer\.Support vector learning for ordinal regression\.In*1999 Ninth International Conference on Artificial Neural Networks ICANN 99\. \(Conf\. Publ\. No\. 470\)*, volume 1, pages 97–102 vol\.1, 1999\.doi:10\.1049/cp:19991091\.
- Ho et al\. \[2020\]Xanh Ho, Anh\-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa\.Constructing a multi\-hop QA dataset for comprehensive evaluation of reasoning steps\.In*Proceedings of the 28th International Conference on Computational Linguistics \(COLING\)*, 2020\.arXiv:2011\.01060\.
- Hsieh et al\. \[2024\]Cheng\-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg\.Ruler: What’s the real context size of your long\-context language models?, 2024\.URL[https://arxiv\.org/abs/2404\.06654](https://arxiv.org/abs/2404.06654)\.
- Izacard and Grave \[2021\]Gautier Izacard and Edouard Grave\.Leveraging passage retrieval with generative models for open domain question answering\.In*International Conference on Learning Representations \(ICLR\)*, 2021\.arXiv:2007\.01282\.
- Jiang et al\. \[2023\]Huiqiang Jiang, Qianhui Wu, Chin\-Yew Lin, Yuqing Yang, and Lili Qiu\.LLMLingua: Compressing prompts for accelerated inference of large language models\.In*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)*, 2023\.arXiv:2310\.05736\.
- Jiang et al\. \[2026\]Yi Jiang, Sendong Zhao, Jianbo Li, Bairui Hu, Yanrui Du, Haochun Wang, and Bing Qin\.Optiset: Unified optimizing set selection and ranking for retrieval\-augmented generation, 2026\.URL[https://arxiv\.org/abs/2601\.05027](https://arxiv.org/abs/2601.05027)\.
- Karpukhin et al\. \[2020\]Vladimir Karpukhin et al\.Dense passage retrieval for open\-domain question answering\.In*Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)*, 2020\.arXiv:2004\.04906\.
- Lee et al\. \[2019\]Juho Lee, Yoonho Lee, Jungtaek Kim, Adam R\. Kosiorek, Seungjin Choi, and Yee Whye Teh\.Set transformer: A framework for attention\-based permutation\-invariant neural networks, 2019\.URL[https://arxiv\.org/abs/1810\.00825](https://arxiv.org/abs/1810.00825)\.
- Lewis et al\. \[2021\]Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela\.Retrieval\-augmented generation for knowledge\-intensive nlp tasks, 2021\.URL[https://arxiv\.org/abs/2005\.11401](https://arxiv.org/abs/2005.11401)\.
- Li et al\. \[2023\]Yucheng Li, Bo Dong, Chenghua Lin, and Frank Guerin\.Compressing context to enhance inference efficiency of large language models\.In*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)*, 2023\.arXiv:2310\.06201\.
- Liu et al\. \[2025\]Jiaheng Liu et al\.A comprehensive survey on long context language modeling\.*arXiv preprint arXiv:2503\.17407*, 2025\.
- Liu et al\. \[2024\]Nelson F\. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang\.Lost in the middle: How language models use long contexts\.*Transactions of the Association for Computational Linguistics \(TACL\)*, 12:157–173, 2024\.arXiv:2307\.03172\.
- Loshchilov and Hutter \[2019\]Ilya Loshchilov and Frank Hutter\.Decoupled weight decay regularization, 2019\.URL[https://arxiv\.org/abs/1711\.05101](https://arxiv.org/abs/1711.05101)\.
- Lundberg and Lee \[2017\]Scott Lundberg and Su\-In Lee\.A unified approach to interpreting model predictions\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2017\.arXiv:1705\.07874\.
- Mei et al\. \[2025\]Lingrui Mei, Jiayu Yao, Yuyao Ge, Yiwei Wang, Baolong Bi, Yujun Cai, Jiazhi Liu, Mingyu Li, Zhong\-Zhi Li, Duzhen Zhang, Chenlin Zhou, Jiayi Mao, Tianze Xia, Jiafeng Guo, and Shenghua Liu\.A survey of context engineering for large language models, 2025\.URL[https://arxiv\.org/abs/2507\.13334](https://arxiv.org/abs/2507.13334)\.
- Mikolov et al\. \[2013\]Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean\.Distributed representations of words and phrases and their compositionality, 2013\.URL[https://arxiv\.org/abs/1310\.4546](https://arxiv.org/abs/1310.4546)\.
- Nils Reimers, Tom Aarsen \[2021\]Nils Reimers, Tom Aarsen\.cross\-encoder/ms\-marco\-MiniLM\-L6\-v2\.[https://huggingface\.co/cross\-encoder/ms\-marco\-MiniLM\-L6\-v2](https://huggingface.co/cross-encoder/ms-marco-MiniLM-L6-v2), 2021\.Accessed: 2026\-03\-27\.
- Nogueira and Cho \[2020\]Rodrigo Nogueira and Kyunghyun Cho\.Passage re\-ranking with bert, 2020\.URL[https://arxiv\.org/abs/1901\.04085](https://arxiv.org/abs/1901.04085)\.
- OpenAI \[2026\]OpenAI\.GPT\-5\.4\.[https://openai\.com](https://openai.com/), 2026\.The experiment corresponding to GPT\-5\.4 \(Medium\) is implemented within 27 March 2026 to 15 April 2026\.
- Pan et al\. \[2024\]Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, et al\.LLMLingua\-2: Data distillation for efficient and faithful task\-agnostic prompt compression\.In*Findings of the Association for Computational Linguistics: ACL 2024*, 2024\.
- Reimers and Gurevych \[2019\]Nils Reimers and Iryna Gurevych\.Sentence\-bert: Sentence embeddings using siamese bert\-networks, 2019\.URL[https://arxiv\.org/abs/1908\.10084](https://arxiv.org/abs/1908.10084)\.
- Robertson and Zaragoza \[2009\]Stephen Robertson and Hugo Zaragoza\.The probabilistic relevance framework: BM25 and beyond\.*Foundations and Trends in Information Retrieval*, 3\(4\):333–389, 2009\.
- Salton and Buckley \[1988\]Gerard Salton and Christopher Buckley\.Term\-weighting approaches in automatic text retrieval\.*Inf\. Process\. Manage\.*, 24\(5\):513–523, August 1988\.ISSN 0306\-4573\.doi:10\.1016/0306\-4573\(88\)90021\-0\.URL[https://doi\.org/10\.1016/0306\-4573\(88\)90021\-0](https://doi.org/10.1016/0306-4573(88)90021-0)\.
- Schlatt et al\. \[2025\]Ferdinand Schlatt, Maik Fröbe, Harrisen Scells, Shengyao Zhuang, Bevan Koopman, Guido Zuccon, Benno Stein, Martin Potthast, and Matthias Hagen\.*Set\-Encoder: Permutation\-Invariant Inter\-Passage Attention for Listwise Passage Re\-Ranking with Cross\-Encoders*, pages 1–19\.Springer Nature Switzerland, 2025\.ISBN 9783031887116\.doi:10\.1007/978\-3\-031\-88711\-6\_1\.arXiv:2404\.06912\.
- Shapley \[1953\]Lloyd S\. Shapley\.A value for n\-person games\.In*Contributions to the Theory of Games*\. Princeton University Press, 1953\.
- Sundararajan et al\. \[2017\]Mukund Sundararajan, Ankur Taly, and Qiqi Yan\.Axiomatic attribution for deep networks\.In*International Conference on Machine Learning \(ICML\)*, 2017\.
- Tateno \[2025\]Yuichi Tateno\.Openprovence: An open\-source implementation of efficient and robust context pruning for retrieval\-augmented generation, 2025\.URL[https://github\.com/hotchpotch/open\_provence](https://github.com/hotchpotch/open_provence)\.
- Thorne et al\. \[2018\]James Thorne et al\.FEVER: a large\-scale dataset for fact extraction and VERification\.In*Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics \(NAACL\)*, 2018\.arXiv:1803\.05355\.
- Trivedi et al\. \[2022\]Harsh Trivedi et al\.MuSiQue: Multihop questions via single\-hop question composition\.*Transactions of the Association for Computational Linguistics \(TACL\)*, 10:569–585, 2022\.arXiv:2108\.00573\.
- Vaswani et al\. \[2023\]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N\. Gomez, Lukasz Kaiser, and Illia Polosukhin\.Attention is all you need, 2023\.URL[https://arxiv\.org/abs/1706\.03762](https://arxiv.org/abs/1706.03762)\.
- Wagstaff et al\. \[2019\]Edward Wagstaff, Fabian B\. Fuchs, Martin Engelcke, Ingmar Posner, and Michael Osborne\.On the limitations of representing functions on sets\.In*International Conference on Machine Learning \(ICML\)*, 2019\.arXiv:1901\.09006\.
- Wang et al\. \[2020\]Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou\.Minilm: Deep self\-attention distillation for task\-agnostic compression of pre\-trained transformers, 2020\.URL[https://arxiv\.org/abs/2002\.10957](https://arxiv.org/abs/2002.10957)\.
- Wang et al\. \[2026\]Yuhang Wang et al\.SWE\-Pruner: Self\-adaptive context pruning for coding agents\.*arXiv preprint arXiv:2601\.16746*, 2026\.
- Xu et al\. \[2023\]Fangyuan Xu, Weijia Shi, and Eunsol Choi\.Recomp: Improving retrieval\-augmented lms with compression and selective augmentation, 2023\.URL[https://arxiv\.org/abs/2310\.04408](https://arxiv.org/abs/2310.04408)\.
- Yang and et al\. \[2025\]An Yang and Anfeng Li et al\.Qwen3 technical report, 2025\.URL[https://arxiv\.org/abs/2505\.09388](https://arxiv.org/abs/2505.09388)\.
- Yang et al\. \[2018\]Zhilin Yang et al\.HotpotQA: A dataset for diverse, explainable multi\-hop question answering\.In*Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)*, 2018\.arXiv:1809\.09600\.
- Zaheer et al\. \[2018\]Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan Salakhutdinov, and Alexander Smola\.Deep sets, 2018\.URL[https://arxiv\.org/abs/1703\.06114](https://arxiv.org/abs/1703.06114)\.
- Zhuang et al\. \[2024\]Shengyao Zhuang, Honglei Zhuang, Bevan Koopman, and Guido Zuccon\.A setwise approach for effective and highly efficient zero\-shot ranking with large language models\.In*Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval*, SIGIR 2024, page 38–47\. ACM, 2024\.doi:10\.1145/3626772\.3657813\.URL[http://dx\.doi\.org/10\.1145/3626772\.3657813](http://dx.doi.org/10.1145/3626772.3657813)\.

## Appendix ATowards a Theoretical Framework: A Cooperative Game\-Theoretic Perspective

In this section, we outline a promising conceptual blueprint beyond the empirical experimental validation of this paper\. Our primary goal here is to inspire future researchers by demonstrating how the rich mathematical machinery of cooperative game theory can provide a theoretically grounded perspective for understanding the hidden structure of context\. While the empirical sections of this paper operationalize these concepts using Deep Sets and neural approximations, this appendix serves as a theoretical sandbox\. We aim to show how concepts like the "Core" and "Convex Games" can reframe the way the NLP community views context synergy, offering the formal intuition beneath the "Landscape of Context", i\.e\., how context looks like under certain value function\. Before that, we first provide a more detailed comparison of different Shapley\-based attribution frameworks in Table[1](https://arxiv.org/html/2607.16209#A1.T1), to clarify the unique position of SCP in the broader landscape of Shapley applications\.

Table 1:Comparison of Shapley\-Based Attribution FrameworksTable[1](https://arxiv.org/html/2607.16209#A1.T1)summarizes the comparison of different Shapley\-based attribution frameworks that answer completely different questions\. Notably, SCP is the only trainable, scalable framework that directly attributes at the context unit level, making it uniquely suited for context pruning applications\. The other frameworks fall into explainable AI \(XAI\) paradigms, where the Shapley value is used to attribute feature importance for a fixed model, rather than to rank discrete context units for pruning\. SCP’s direct optimization of a value function that maps subsets of context to performance metrics allows it to leverage the full theoretical properties of Shapley values in a practical, scalable manner\.

It is worthy of notice that we have not designed any specific algorithm or training paradigm using the following concepts, since any validation of these algorithms and paradigms would require a separate paper\. The following discussion is purely conceptual and theoretical, but we hope that these discussions can inspire future research in this direction\.

### A\.1Formulating Context as a Cooperative Game

Instead of viewing a long prompt as a flat sequence of tokens, we model the context as a grand coalition of semantic components \(e\.g\., sentences or paragraphs\)\. The objective of the LLM reading this context can be abstracted as a value function\.

###### Definition A\.1\(Context Value Game\)\.

LetN=\{c1,c2,…,cn\}N=\\\{c\_\{1\},c\_\{2\},\\ldots,c\_\{n\}\\\}be the set of context components\. We define a fundamental value functionv:2N→ℝv:2^\{N\}\\rightarrow\\mathbb\{R\}withv​\(∅\)=0v\(\\emptyset\)=0, wherev​\(S\)v\(S\)represents the utility or information completeness \(e\.g\., the likelihood of generating the correct answer\) provided by a subset of componentsS⊆NS\\subseteq N\. The pair\(N,v\)\(N,v\)constitutes a cooperative game\.

In natural language, sentences rarely contribute value independently\. Information is riddled with logical prerequisites, anaphora, and multi\-hop reasoning chains\. Therefore, the marginal contribution of a componentcic\_\{i\}, defined asv​\(S∪\{ci\}\)−v​\(S\)v\(S\\cup\\\{c\_\{i\}\\\}\)\-v\(S\), is highly dependent on what informationSSis already present\.

To assign a fair "importance score" to each component without bias toward any specific reading order or subset, we compute the expected marginal contribution over all possible information\-accumulation sequences, which is precisely the Shapley value:

###### Definition A\.2\(Shapley Importance Score\)\.

The attribution scoreϕi​\(v\)\\phi\_\{i\}\(v\)for componentcic\_\{i\}in contextNNunder value functionvvis the Shapley value:

ϕi​\(v\)=∑S⊆N∖\{ci\}\|S\|\!​\(\|N\|−\|S\|−1\)\!\|N\|\!​\[v​\(S∪\{ci\}\)−v​\(S\)\]\\phi\_\{i\}\(v\)=\\sum\_\{S\\subseteq N\\setminus\\\{c\_\{i\}\\\}\}\\frac\{\|S\|\!\(\|N\|\-\|S\|\-1\)\!\}\{\|N\|\!\}\[v\(S\\cup\\\{c\_\{i\}\\\}\)\-v\(S\)\]

The score obeys the four axiomatic properties:

- •Efficiency: The sum of the Shapley values of all players equals the total value of the grand coalition, i\.e\.,∑i∈Nϕi​\(v\)=v​\(N\)\\sum\_\{i\\in N\}\\phi\_\{i\}\(v\)=v\(N\)\.
- •Symmetry: If two playersiiandjjcontribute equally to all coalitions, then they receive the same Shapley value, i\.e\., ifv​\(S∪\{i\}\)=v​\(S∪\{j\}\)v\(S\\cup\\\{i\\\}\)=v\(S\\cup\\\{j\\\}\)for allS⊆N∖\{i,j\}S\\subseteq N\\setminus\\\{i,j\\\}, thenϕi​\(v\)=ϕj​\(v\)\\phi\_\{i\}\(v\)=\\phi\_\{j\}\(v\)\.
- •Dummy Player: If a playeriidoes not contribute to any coalition \(i\.e\.,v​\(S∪\{i\}\)=v​\(S\)v\(S\\cup\\\{i\\\}\)=v\(S\)for allS⊆N∖\{i\}S\\subseteq N\\setminus\\\{i\\\}\), then their Shapley value is zero, i\.e\.,ϕi​\(v\)=0\\phi\_\{i\}\(v\)=0\.
- •Additivity: For two games with characteristic functionsvvandww, the Shapley value of the combined gamev\+wv\+wis the sum of the Shapley values of the individual games, i\.e\.,ϕi​\(v\+w\)=ϕi​\(v\)\+ϕi​\(w\)\\phi\_\{i\}\(v\+w\)=\\phi\_\{i\}\(v\)\+\\phi\_\{i\}\(w\)for alli∈Ni\\in N\.

These four axioms are not merely mathematical conveniences; they correspond to minimal desiderata that any reasonable context\-reranking mechanism should satisfy:

- •Efficiencyguaranteesinformation completeness: the sum of all sentence scores equals the total value of the full context, so no utility is left unexplained or double\-counted\.
- •Symmetryenforcesfunctional equivalence: if two sentences provide identical marginal contributions across all subsets, the reranker must treat them identically—this naturally handles semantically duplicated retrievals\.
- •Dummy Playerensuresnoise suppression: any sentence that never changes the answer quality \(a pure distractor\) receives zero score and is pruned first\.
- •Additivityenablesmulti\-objective decomposition: if the total value is the sum of independent sub\-tasks \(e\.g\., correctness \+ faithfulness\), the attribution of each sub\-task can be computed separately and then linearly combined without retraining the value function\.

Because these four properties are natural requirements for a fair and interpretable reranking system, the following classical result acquires a contextual justification: the Shapley value is the*only*attribution mechanism compatible with all of them\.

###### Theorem A\.1\(Axiomatic Characterisation of Context Attribution\[[40](https://arxiv.org/html/2607.16209#bib.bib40)\]\)\.

Let𝒢\\mathcal\{G\}be the set of all cooperative games on a fixed player setNN\. The Shapley valueϕ:𝒢→ℝn\\phi:\\mathcal\{G\}\\to\\mathbb\{R\}^\{n\}is the unique attribution mechanism that simultaneously satisfies Efficiency, Symmetry, Dummy Player, and Additivity\.

### A\.2Synergy, Stability, and the Core

If we simply truncate the context by selecting the top\-KKsentences with the highest Shapley values, how do we know the resulting subset is "stable"? In cooperative game theory, stability means that no unselected group of players can form a coalition that yields a higher value than their current assigned payoff\. This concept is encapsulated by theCore\.

###### Definition A\.3\(The Contextual Core\)\.

Given a context game\(N,v\)\(N,v\), the CoreC​\(v\)C\(v\)is the set of all payoff allocationsx∈ℝNx\\in\\mathbb\{R\}^\{N\}that efficiently distribute the total context value \(∑i∈Nxi=v​\(N\)\\sum\_\{i\\in N\}x\_\{i\}=v\(N\)\) while ensuring no subset is "undervalued" relative to its intrinsic synergy:

C​\(v\)=\{x∈ℝN\|∑i∈Sxi≥v​\(S\)for all​S⊆N\}C\(v\)=\\left\\\{x\\in\\mathbb\{R\}^\{N\}\\;\|\\;\\sum\_\{i\\in S\}x\_\{i\}\\geq v\(S\)\\quad\\text\{for all \}S\\subseteq N\\right\\\}

In applied context engineering, the core ensures that if a subset of sentencesSScontains a highly synergistic logical chain \(e\.g\., a multi\-hop bridge\), the individual scoresxix\_\{i\}assigned to those sentences must sum to at leastv​\(S\)v\(S\)\. If the core is empty, it means the context contains conflicting synergies that cannot be linearly decomposed\.

We call a subsetSStightat a core allocationxxwhen its scores sum to*exactly*its value, i\.e\.,∑i∈Sxi=v​\(S\)\\sum\_\{i\\in S\}x\_\{i\}=v\(S\)\. In context\-pruning terms, this means the importance scores of the sentences inSSadd up to precisely the worth ofSSas a whole—no surplus, no deficit\. A tight set is therefore a “natural information block”: its internal attribution is fully explained by its member scores, suggesting a coherent, self\-contained unit of meaning\. We will see that these tight sets form the nodes of the context hierarchy\.

To accommodate the noisy nature of LLM generation, we can relax this to theϵ\\epsilon\-Core:

###### Definition A\.4\(ϵ\\epsilon\-Core\)\.

Cϵ​\(v\)=\{x∈ℝN\|∑i∈Nxi=v​\(N\),∑i∈Sxi≥v​\(S\)−ϵfor all​S⊆N\}C\_\{\\epsilon\}\(v\)=\\left\\\{x\\in\\mathbb\{R\}^\{N\}\\;\|\\;\\sum\_\{i\\in N\}x\_\{i\}=v\(N\),\\sum\_\{i\\in S\}x\_\{i\}\\geq v\(S\)\-\\epsilon\\quad\\text\{for all \}S\\subseteq N\\right\\\}

A non\-emptyϵ\\epsilon\-core suggests that the attribution mechanism can be stable under bounded noise, providing theoretical motivation for using convex approximations in our neural network design\.

### A\.3Convex Games and the Context Hierarchy

To deeply understand the structural topology of information, we study a specific class of games where the value function exhibits supermodularity \(often termed "convex games" in cooperative game theory\)\. For non\-convex scenario, we provide a possible idea, see paragraph[D\.7](https://arxiv.org/html/2607.16209#A4.SS7.SSS0.Px2)\.

###### Definition A\.5\(Convex Context Game \(Supermodularity\)\)\.

A context game\(N,v\)\(N,v\)is convex if its value functionvvis supermodular: for any two subsetsS,T⊆NS,T\\subseteq N,

v​\(S\)\+v​\(T\)≤v​\(S∪T\)\+v​\(S∩T\)v\(S\)\+v\(T\)\\leq v\(S\\cup T\)\+v\(S\\cap T\)

In a convex game, a "snowball effect" can occur: the marginal utility of a sentence is non\-decreasing as the surrounding context grows\. This provides a natural abstraction for multi\-hop reasoning, where a bridge entity may be uninformative alone but valuable when joined with its paired premise\.

A foundational property of a convex game is that its CoreC​\(v\)C\(v\)is large and non\-empty\. The extreme points of this Core induce structured families of subsets calledLaminar Families\.

###### Definition A\.6\(Laminar Family\)\.

A family of setsℱ\\mathcal\{F\}is called laminar if for any two setsA,B∈ℱA,B\\in\\mathcal\{F\}, eitherA⊆BA\\subseteq B,B⊆AB\\subseteq A, orA∩B=∅A\\cap B=\\emptyset\.

![Refer to caption](https://arxiv.org/html/2607.16209v1/x5.png)Figure 5:Landscape of Context: What Will This Look Like? In this figure, we show an example of a context hierarchy induced by a convex value function\. The specific value function and the value in the figure is not important, since they will vary across different algorithms and scenarios\. The key point is that the tree structure itself is a direct consequence of the convexity of the value function, which ensures that the tight sets form a laminar family\.Every extreme pointxxin the core of a convex game defines a specific nested sequence of tight subsetsTv​\(x\)=\{S⊆N∣∑i∈Sxi=v​\(S\)\}T\_\{v\}\(x\)=\\\{S\\subseteq N\\mid\\sum\_\{i\\in S\}x\_\{i\}=v\(S\)\\\}\. Because this set system is laminar, each parent set either remains a leaf or splits into mutually disjoint child subsets\. Concretely, a setSSsplits whenever there exist disjoint proper subsetsT1,T2⊊ST\_\{1\},T\_\{2\}\\subsetneq Sthat are also tight and together coverSS, meaningv​\(S\)v\(S\)decomposes additively across those children with no residual cross\-subset synergy\. If no such decomposition exists,SSremains a leaf \(typically an individual sentence or an irreducible cohesive group\)\. This yields a hierarchical tree: the root is the entire context, intermediate nodes branch into non\-overlapping topical paragraphs or logical chains, and the leaves are individual sentences \(Figure[5](https://arxiv.org/html/2607.16209#A1.F5)\)\.

##### From Context to Tree: A Concrete Procedure\.

To see how a concrete context unfolds into a tree, fix a core allocationxx\(e\.g\., the Shapley value\)\. The construction proceeds in three steps\.\(1\) Identify tight sets\.For every subsetS⊆NS\\subseteq N, check whether∑i∈Sxi=v​\(S\)\\sum\_\{i\\in S\}x\_\{i\}=v\(S\)\. The collection𝒯​\(x\)\\mathcal\{T\}\(x\)of all such sets always containsNN\(by efficiency\) and all singletons whose standalone worth equals their allocated score\.\(2\) Uncross\.If two tight setsA,B∈𝒯​\(x\)A,B\\in\\mathcal\{T\}\(x\)cross \(i\.e\.,A∩B≠∅A\\cap B\\neq\\emptysetbut neither contains the other\), replace them byA∪BA\\cup BandA∩BA\\cap B, which are also tight by convexity\. SinceA∩B⊆A∪BA\\cap B\\subseteq A\\cup B, this pair is laminar\. Repeating this yields a maximal laminar familyℒ⊆𝒯​\(x\)\\mathcal\{L\}\\subseteq\\mathcal\{T\}\(x\)\.\(3\) Build the tree\.For eachS∈ℒS\\in\\mathcal\{L\}, its children are the maximal proper subsets ofSSthat also belong toℒ\\mathcal\{L\}\. These children are pairwise disjoint and coverSSwheneverSSis not a leaf\. The root isNN; leaves are sets with no children\. This tree reveals the latent hierarchical structure of the context: each node is a self\-contained information block, and each split marks a boundary where the internal synergy is fully explained by the children\.

If there are multiple tight sets that can split a parent, the resulting tree is not unique\. This non\-uniqueness reflects future design for hieararchy construction algorithms, which may select splits based on additional criteria \(e\.g\., greedy strategies\)\.

Of course, not every node in this tree can be split further: a leaf has no tight proper subsets, meaning its internal synergy cannot be decomposed into smaller self\-contained blocks\. This irreducible synergy is not attributable to any single sentence, but arises from the*cooperation*among the members of that leaf\. To quantify exactly how much value is created by this cooperation itself, beyond what the sub\-components already contribute: we apply Möbius inversion, which yields theHarsanyi Dividends\.

###### Definition A\.7\(Harsanyi Dividends on Subsets\[[15](https://arxiv.org/html/2607.16209#bib.bib15)\]\)\.

Given a convex context game\(N,v\)\(N,v\), the Möbius inversion of the value function yields the Harsanyi dividendΔv​\(S\)\\Delta\_\{v\}\(S\)for a coalitionSS:

Δv​\(S\)=∑T⊆S\(−1\)\|S\|−\|T\|​v​\(T\)\\Delta\_\{v\}\(S\)=\\sum\_\{T\\subseteq S\}\(\-1\)^\{\|S\|\-\|T\|\}v\(T\)

The Harsanyi dividend captures precisely the value generatedonlywhen all members ofSSare present simultaneously\. Through this, any context value function can be orthogonally decomposed:

v​\(S\)=∑T⊆SΔv​\(T\)v\(S\)=\\sum\_\{T\\subseteq S\}\\Delta\_\{v\}\(T\)

##### Rationale

In the theoretical analysis, we are interested in how much value is created by the cooperation itself, beyond the value derived from sub\-coalitions\. Harsanyi dividends provide a tool that may be applied to identify essential “bridge” sentences rather than only supporting facts\.

We highlight the theoretical discussion around laminar families as a foundation for future context hierarchy design and context management systems\. Future work may develop clever decomposition strategies and efficient tight set identification methods to accelerate the construction of the landscape\. See Section[D\.7](https://arxiv.org/html/2607.16209#A4.SS7)for more discussion on future work in this direction\.

### A\.4Landscape of Context

Building on the preceding concepts, the “Landscape of Context” may be defined as a forest of game trees, each induced by a convex value function\. We can further explore how to design neural architectures that can effectively capture this structure while remaining computationally tractable\. This motivates our proposal of Convex Deep Sets in the next section\.

###### Definition A\.8\(Landscape of Context\)\.

The Landscape of Context is a forest induced by a family of convex functions\{vi\}i=1n\\\{v\_\{i\}\\\}\_\{i=1\}^\{n\}, where each tree corresponds to a convex value function defined over the same set of context components\. Each node in the tree represents a coalition of sentences, and the edges represent the hierarchical relationships among these coalitions\.

We note that each value functionviv\_\{i\}does not determine a unique tree on its own: each extreme point of the coreC​\(vi\)C\(v\_\{i\}\)induces a distinct laminar chain, with up ton\!n\!such chains in total\. Selecting a specific tree therefore requires an auxiliary rule—for example, a prior permutation, or a partition that maximizes Harsanyi dividends\. We leave the design of such selection rules to future work\.

### A\.5Convex Value Functions

While standard Deepsets act as universal approximators, the resulting baseline value function typically lacks explicit analytical structure\. Building upon the formalized framework in Appendix A, we emphasize that certain mathematical properties, particularly convexity and supermodularity, are useful when mapping a combinatorial hierarchy among contextual elements\. To merge deep learning representations with this structural prior, we propose a specialized subset configuration termedConvex Deep Setsutilizing convex neural networks\.

###### Definition A\.9\(Convex Deep Sets\)\.

We define the Convex Deep Sets architecture as:

f​\(X\)=ρc​o​n​v​e​x​\(∑x∈Xψc​o​n​v​e​x​\(x\)\)f\(X\)=\\rho\_\{convex\}\\left\(\\sum\_\{x\\in X\}\\psi\_\{convex\}\(x\)\\right\)where, in the context of this paper, the functionally constrained value distribution becomes:

v​\(S;θ\)=ρc​o​n​v​e​x​\(∑ci∈Sψc​o​n​v​e​x​\(ci\)\)v\(S;\\theta\)=\\rho\_\{convex\}\\left\(\\sum\_\{c\_\{i\}\\in S\}\\psi\_\{convex\}\(c\_\{i\}\)\\right\)

By architecturally instilling this constraint, we introduce a convexity\-based structural prior that is conceptually connected to supermodular \(convex\) games \(discussed in Appendix A\)\. While imposing convexity may restrict the hypothesis space, it can serve as an inductive bias for exploring hierarchical information aggregation in the proposed “Landscape of Context\.” See Section[D\.2](https://arxiv.org/html/2607.16209#A4.SS2)for implementation experiments\.

### A\.6The error bound for Monte\-Carlo Sampling

![Refer to caption](https://arxiv.org/html/2607.16209v1/x6.png)Figure 6:The error bound for pruning attribution can be decoupled into two independent sources: the approximation error of the trained value function and the sampling error from Monte\-Carlo estimation\.In our practical implementation, we use Monte\-Carlo sampling to estimate the Shapley values\. A natural question is: how does the estimation error of the Shapley values affect the downstream performance after pruning? We now define the trained value function with error and the corresponding Shapley value estimation, and then we can derive a theoretical error bound for the pruning performance based on the estimation error\.

###### Theorem A\.2\(Error Bound for Pruning Attribution \(Recall\)\)\.

Letv∗v^\{\*\}be the true value function andvθv\_\{\\theta\}a trained approximation satisfying Assumption[3\.1](https://arxiv.org/html/2607.16209#S3.Thmassumption1)\. Assume the point\-wise approximation error is uniformly bounded byϵapprox\\epsilon\_\{\\mathrm\{approx\}\}, i\.e\.,

\|v∗​\(S\)−vθ​\(S\)\|≤ϵapprox,∀S⊆N\.\|v^\{\*\}\(S\)\-v\_\{\\theta\}\(S\)\|\\leq\\epsilon\_\{\\mathrm\{approx\}\},\\quad\\forall S\\subseteq N\.Letϕ^i\\hat\{\\phi\}\_\{i\}be the Monte\-Carlo Shapley estimate of playeriiobtained fromMMindependent uniform permutations \(Algorithm 1\)\. Then, with probability at least1−δ1\-\\deltaover the random permutations, the following holds simultaneously for alli∈Ni\\in N:

\|ϕ^i​\(vθ\)−ϕi​\(v∗\)\|≤B​2​ln⁡\(2​n/δ\)M\+2​ϵapprox\.\\bigl\|\\hat\{\\phi\}\_\{i\}\(v\_\{\\theta\}\)\-\\phi\_\{i\}\(v^\{\*\}\)\\bigr\|\\;\\leq\\;B\\sqrt\{\\frac\{2\\ln\(2n/\\delta\)\}\{M\}\}\\;\+\\;2\\epsilon\_\{\\mathrm\{approx\}\}\.

###### Proof\.

We decompose the total error into two independent sources\.

Define the residualζ​\(S\)=v∗​\(S\)−vθ​\(S\)\\zeta\(S\)=v^\{\*\}\(S\)\-v\_\{\\theta\}\(S\)\. By the additivity axiom of the Shapley value,

ϕi​\(v∗\)=ϕi​\(vθ\+ζ\)=ϕi​\(vθ\)\+ϕi​\(ζ\)\.\\phi\_\{i\}\(v^\{\*\}\)=\\phi\_\{i\}\(v\_\{\\theta\}\+\\zeta\)=\\phi\_\{i\}\(v\_\{\\theta\}\)\+\\phi\_\{i\}\(\\zeta\)\.By the definition of the Shapley value,ϕi​\(ζ\)\\phi\_\{i\}\(\\zeta\)is a convex combination of the marginal contributions\{ζ​\(S∪\{i\}\)−ζ​\(S\)\}S⊆N∖\{i\}\\\{\\zeta\(S\\cup\\\{i\\\}\)\-\\zeta\(S\)\\\}\_\{S\\subseteq N\\setminus\\\{i\\\}\}\. The uniform bound\|ζ​\(S\)\|≤ϵapprox\|\\zeta\(S\)\|\\leq\\epsilon\_\{\\mathrm\{approx\}\}implies\|ζ​\(S∪\{i\}\)−ζ​\(S\)\|≤2​ϵapprox\|\\zeta\(S\\cup\\\{i\\\}\)\-\\zeta\(S\)\|\\leq 2\\epsilon\_\{\\mathrm\{approx\}\}for everySS, and therefore

\|ϕi​\(vθ\)−ϕi​\(v∗\)\|=\|ϕi​\(ζ\)\|≤2​ϵapprox\.\\bigl\|\\phi\_\{i\}\(v\_\{\\theta\}\)\-\\phi\_\{i\}\(v^\{\*\}\)\\bigr\|=\\bigl\|\\phi\_\{i\}\(\\zeta\)\\bigr\|\\leq 2\\epsilon\_\{\\mathrm\{approx\}\}\.
Fix a playerii\. For each sampled permutationπ\(m\)\\pi^\{\(m\)\}\(m=1,…,Mm=1,\\dots,M\), letSπ,j\(m\)S^\{\(m\)\}\_\{\\pi,j\}denote the coalition of elements precedingiiinπ\(m\)\\pi^\{\(m\)\}\. The marginal contribution ofiialong this permutation is

Xm=vθ​\(Sπ,j\(m\)∪\{i\}\)−vθ​\(Sπ,j\(m\)\)\.X\_\{m\}=v\_\{\\theta\}\\bigl\(S^\{\(m\)\}\_\{\\pi,j\}\\cup\\\{i\\\}\\bigr\)\-v\_\{\\theta\}\\bigl\(S^\{\(m\)\}\_\{\\pi,j\}\\bigr\)\.Becausevθv\_\{\\theta\}takes values in\[0,B\]\[0,B\]\(Assumption[3\.1](https://arxiv.org/html/2607.16209#S3.Thmassumption1)\), eachXmX\_\{m\}lies in\[−B,B\]\[\-B,B\]\. The variables\{Xm\}m=1M\\\{X\_\{m\}\\\}\_\{m=1\}^\{M\}are i\.i\.d\. with𝔼​\[Xm\]=ϕi​\(vθ\)\\mathbb\{E\}\[X\_\{m\}\]=\\phi\_\{i\}\(v\_\{\\theta\}\)\. Consequently, the Monte\-Carlo estimatorϕ^i=1M​∑m=1MXm\\hat\{\\phi\}\_\{i\}=\\frac\{1\}\{M\}\\sum\_\{m=1\}^\{M\}X\_\{m\}satisfies, by Hoeffding’s inequality,

ℙ​\(\|ϕ^i−ϕi​\(vθ\)\|≥t\)≤2​exp⁡\(−M​t22​B2\),∀t\>0\.\\mathbb\{P\}\\Bigl\(\\bigl\|\\hat\{\\phi\}\_\{i\}\-\\phi\_\{i\}\(v\_\{\\theta\}\)\\bigr\|\\geq t\\Bigr\)\\;\\leq\\;2\\exp\\\!\\left\(\-\\frac\{Mt^\{2\}\}\{2B^\{2\}\}\\right\),\\quad\\forall t\>0\.Choosingt=B​2​ln⁡\(2​n/δ\)/Mt=B\\sqrt\{2\\ln\(2n/\\delta\)/M\}yields

ℙ​\(\|ϕ^i−ϕi​\(vθ\)\|≥B​2​ln⁡\(2​n/δ\)M\)≤δn\.\\mathbb\{P\}\\Bigl\(\\bigl\|\\hat\{\\phi\}\_\{i\}\-\\phi\_\{i\}\(v\_\{\\theta\}\)\\bigr\|\\geq B\\sqrt\{\\frac\{2\\ln\(2n/\\delta\)\}\{M\}\}\\Bigr\)\\;\\leq\\;\\frac\{\\delta\}\{n\}\.Applying a union bound over allnnplayers gives the simultaneous high\-probability guarantee\.

Combining the two bounds,

\|ϕ^i−ϕi​\(v∗\)\|≤\|ϕ^i−ϕi​\(vθ\)\|⏟sampling error\+\|ϕi​\(vθ\)−ϕi​\(v∗\)\|⏟approximation error≤B​2​ln⁡\(2​n/δ\)M\+2​ϵapprox\.\\bigl\|\\hat\{\\phi\}\_\{i\}\-\\phi\_\{i\}\(v^\{\*\}\)\\bigr\|\\;\\leq\\;\\underbrace\{\\bigl\|\\hat\{\\phi\}\_\{i\}\-\\phi\_\{i\}\(v\_\{\\theta\}\)\\bigr\|\}\_\{\\text\{sampling error\}\}\\;\+\\;\\underbrace\{\\bigl\|\\phi\_\{i\}\(v\_\{\\theta\}\)\-\\phi\_\{i\}\(v^\{\*\}\)\\bigr\|\}\_\{\\text\{approximation error\}\}\\;\\leq\\;B\\sqrt\{\\frac\{2\\ln\(2n/\\delta\)\}\{M\}\}\+2\\epsilon\_\{\\mathrm\{approx\}\}\.∎

###### Theorem A\.3\(Sample Complexity for Top\-K Ranking Preservation\)\.

Under the assumptions of Theorem[A\.2](https://arxiv.org/html/2607.16209#A1.Thmtheorem2), letΔ\(K\)=ϕ\(K\)​\(v∗\)−ϕ\(K\+1\)​\(v∗\)\\Delta\_\{\(K\)\}=\\phi\_\{\(K\)\}\(v^\{\*\}\)\-\\phi\_\{\(K\+1\)\}\(v^\{\*\}\)denote the gap between theKK\-th and\(K\+1\)\(K\+1\)\-th true Shapley values\. IfΔ\(K\)\>4​ϵapprox\\Delta\_\{\(K\)\}\>4\\epsilon\_\{\\mathrm\{approx\}\}, then with

M≥8​B2​ln⁡\(2​n/δ\)\(Δ\(K\)−4​ϵapprox\)2M\\;\\geq\\;\\frac\{8B^\{2\}\\,\\ln\(2n/\\delta\)\}\{\\bigl\(\\Delta\_\{\(K\)\}\-4\\epsilon\_\{\\mathrm\{approx\}\}\\bigr\)^\{2\}\}independent permutations, the estimated Top\-KKsetS^K=\{i:ϕ^i≥ϕ^\(K\)\}\\hat\{S\}\_\{K\}=\\\{i:\\hat\{\\phi\}\_\{i\}\\geq\\hat\{\\phi\}\_\{\(K\)\}\\\}equals the true Top\-KKsetSK∗=\{i:ϕi​\(v∗\)≥ϕ\(K\)​\(v∗\)\}S\_\{K\}^\{\*\}=\\\{i:\\phi\_\{i\}\(v^\{\*\}\)\\geq\\phi\_\{\(K\)\}\(v^\{\*\}\)\\\}with probability at least1−δ1\-\\delta\.

###### Proof\.

By Theorem[A\.2](https://arxiv.org/html/2607.16209#A1.Thmtheorem2), with probability at least1−δ1\-\\deltawe have\|ϕ^i−ϕi​\(v∗\)\|≤η\|\\hat\{\\phi\}\_\{i\}\-\\phi\_\{i\}\(v^\{\*\}\)\|\\leq\\etasimultaneously for allii, whereη=B​2​ln⁡\(2​n/δ\)/M\+2​ϵapprox\\eta=B\\sqrt\{2\\ln\(2n/\\delta\)/M\}\+2\\epsilon\_\{\\mathrm\{approx\}\}\. For anyi∈SK∗i\\in S\_\{K\}^\{\*\}andj∉SK∗j\\notin S\_\{K\}^\{\*\}, we haveϕi​\(v∗\)≥ϕ\(K\)​\(v∗\)\>ϕj​\(v∗\)\+Δ\(K\)\\phi\_\{i\}\(v^\{\*\}\)\\geq\\phi\_\{\(K\)\}\(v^\{\*\}\)\>\\phi\_\{j\}\(v^\{\*\}\)\+\\Delta\_\{\(K\)\}\. Ifη<Δ\(K\)/2\\eta<\\Delta\_\{\(K\)\}/2, then

ϕ^i≥ϕi​\(v∗\)−η\>ϕ\(K\)​\(v∗\)−Δ\(K\)2=ϕ\(K\+1\)​\(v∗\)\+Δ\(K\)2\>ϕj​\(v∗\)\+η≥ϕ^j,\\hat\{\\phi\}\_\{i\}\\;\\geq\\;\\phi\_\{i\}\(v^\{\*\}\)\-\\eta\\;\>\\;\\phi\_\{\(K\)\}\(v^\{\*\}\)\-\\frac\{\\Delta\_\{\(K\)\}\}\{2\}\\;=\\;\\phi\_\{\(K\+1\)\}\(v^\{\*\}\)\+\\frac\{\\Delta\_\{\(K\)\}\}\{2\}\\;\>\\;\\phi\_\{j\}\(v^\{\*\}\)\+\\eta\\;\\geq\\;\\hat\{\\phi\}\_\{j\},so the ordering betweenSK∗S\_\{K\}^\{\*\}and its complement is preserved\. The conditionη<Δ\(K\)/2\\eta<\\Delta\_\{\(K\)\}/2is equivalent toB​2​ln⁡\(2​n/δ\)/M<Δ\(K\)/2−2​ϵapproxB\\sqrt\{2\\ln\(2n/\\delta\)/M\}<\\Delta\_\{\(K\)\}/2\-2\\epsilon\_\{\\mathrm\{approx\}\}, which yields the stated bound onMMafter rearrangement \(requiringΔ\(K\)\>4​ϵapprox\\Delta\_\{\(K\)\}\>4\\epsilon\_\{\\mathrm\{approx\}\}\)\. ∎

## Appendix BAlternative Pruning Strategies

### B\.1Top\-K Pruning

We provide the pruning algorithm in pseudo\-code format in Algorithm[2](https://arxiv.org/html/2607.16209#alg2)\. The algorithm is simple but effective in terms of coarse\-grained pruning\.

Algorithm 2Algorithm 2Shapley Pruning Algorithm \(Top\-K Version\)1:A context set

C=\{c1,c2,…,cn\}C=\\\{c\_\{1\},c\_\{2\},\\ldots,c\_\{n\}\\\}, optimized value function

vθv\_\{\\theta\}, budget

KK
2:A pruned context subset

C′⊆CC^\{\\prime\}\\subseteq C
3:Initialize an empty subset

C′=∅C^\{\\prime\}=\\emptyset
4:Compute the Shapley attribution

ϕi​\(vθ\)\\phi\_\{i\}\(v\_\{\\theta\}\)for each

ci∈Cc\_\{i\}\\in Cwith Monte Carlo sampling

5:Identify the index set

IT​o​p​KI\_\{TopK\}corresponding to the

KKhighest

ϕi\\phi\_\{i\}scores

6:foreach part

ci∈Cc\_\{i\}\\in Cdo

7:if

i∈IT​o​p​Ki\\in I\_\{TopK\}then

8:

C′←C′∪\{ci\}C^\{\\prime\}\\leftarrow C^\{\\prime\}\\cup\\\{c\_\{i\}\\\}
9:endif

10:endfor

11:return

C′C^\{\\prime\}

### B\.2Average Pruning \(Tried, but not as effective as Top\-K\)

In an early version of this work, we tried a straightforward pruning strategy based on the average Shapley value as a threshold\. The algorithm is shown in Algorithm[3](https://arxiv.org/html/2607.16209#alg3)\. We found that this simple strategy could not meet stable compression requirements, so we switched to the Top\-K version used in the main paper\. We still include this version here as a reference for future research on more advanced pruning strategies\. In future work, adaptive thresholding strategies may be applied to achieve better compression rates across diverse scenarios, but this is far beyond the scope of this paper, and we leave it as an open question for future research\.

Algorithm 3Algorithm 3Shapley Pruning Algorithm \(Average\-Threshold Version\)1:A context set

C=\{c1,c2,…,cn\}C=\\\{c\_\{1\},c\_\{2\},\\ldots,c\_\{n\}\\\}, optimized value function

vθv\_\{\\theta\}
2:A pruned context subset

C′⊆CC^\{\\prime\}\\subseteq C
3:Initialize an empty subset

C′=∅C^\{\\prime\}=\\emptyset
4:Compute the Shapley attribution

ϕi​\(vθ\)\\phi\_\{i\}\(v\_\{\\theta\}\)for each

ci∈Cc\_\{i\}\\in C
5:Calculate threshold

ϕ¯←max⁡\(0,1n​∑i=1nϕi\)\\bar\{\\phi\}\\leftarrow\\max\(0,\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\phi\_\{i\}\)
6:foreach part

ci∈Cc\_\{i\}\\in Cdo

7:if

ϕi≥ϕ¯\\phi\_\{i\}\\geq\\bar\{\\phi\}then

8:

C′←C′∪\{ci\}C^\{\\prime\}\\leftarrow C^\{\\prime\}\\cup\\\{c\_\{i\}\\\}
9:endif

10:endfor

11:return

C′C^\{\\prime\}

## Appendix CExperiment Details

This appendix provides comprehensive experimental data that complements the main results presented in Section 4\. We include detailed performance metrics across all datasets, ablation studies, and cross\-domain generalization analysis\. "sentlevel" means that SCP is trained on sentence\-level annotations, while "paragraph" means that SCP is trained on paragraph\-level annotations\. "split" means that the model is able to prune at the sentence level, while "no split" means that the model can only prune at the paragraph level\.

### C\.1Experiment Setup Details

During inference, we apply our parallel Monte\-Carlo estimator batched on an Nvidia A800 GPU\. We use the SCP model trained on MS MARCO by default for inference; the use of models trained on other datasets is specified before each test case\. Inference time for each model is measured as the wall\-clock cost of the pruning process, including the forward passes of the value function and Monte\-Carlo sampling\. Specifically, the reported inference time excludes sentence\-transformer encoding time, since we assume that all sentences have been encoded before the pruning process \(i\.e\., a static environment\) and the encoding time is the same for all methods\. We report the average inference time per sample across the evaluated datasets\.

Table 2:Overview of datasets, baselines, and inference LLMs used in our experiments\.#### C\.1\.1Experimental Setup and Implementation Details

To ensure reproducibility, we outline the precise model architecture, training configuration, and deployment hardware used in our experiments\.

##### Architecture

We instantiate the value function using a Deep Sets neural network\. The feature extractorψ\\psiis a symmetric 3\-layer Multi\-Layer Perceptron \(MLP\) mapping the input embedding dimension down to a latent dimension, while the regressorρ\\rhois a 3\-layer MLP mapping the latent sum back to a scalar value\. Specifically, we set the intermediate hidden dimension to 1024 and the latent dimension of the Deep Sets bottleneck to 512\. Across all layers of bothψ\\psiandρ\\rho, we utilize Layer Normalization, ReLU activations, and a Dropout rate of0\.10\.1\. The resulting model comprises exactly 3\.03M parameters\. Standard textual representations are injected intoψ\\psiusing SentenceTransformer variants \(e\.g\., MiniLM, 384 dimensions\) precomputed for training efficiency\.

##### Training Paradigm

The network is optimized via the Pairwise Margin Ranking Loss with the margin set toϵ=0\.15\\epsilon=0\.15\. We employ the AdamW optimizer with a weight decay of1​e\-41\\text\{e\-4\}and a peak learning rate of1​e\-31\\text\{e\-3\}\. Learning rate scheduling utilizes cosine annealing, preceded by a warmup period equal to roughly10%10\\%of total epochs\. Training runs for1010epochs across all variants\. Training batches gather context states from datasets configured strictly to an80%/10%/10%80\\%/10\\%/10\\%train/validation/test split rule\.

##### Hardware

Due to the aggressive3​M3\\text\{M\}parameter capacity and cached contextual representation paradigm, an entire 10\-epoch training procedure converges uniformly in merely several hours on two NVIDIA A800 80GB GPUs\. Furthermore, the memory overhead associated directly with the SCP framework is extremely low \(averaging less than 4GB of active VRAM requirement outside standard embedding storage\), indicating the need of the development of the better training algorithm\. At inference time, as shown previously, evaluatingM=50M=50Monte\-Carlo samples operates highly efficiently, however, we recommend higher sampling number in practice for better performance, as it can be parallelized computed on GPU\.

##### LLM API Version

All experiments with LLM usage are done within 27 March 2026 and 15 April 2026 with Official API\. This information is mentioned for reproduction\.

### C\.2Main Results

Main result is shown in Table[3](https://arxiv.org/html/2607.16209#A3.T3), Table[4](https://arxiv.org/html/2607.16209#A3.T4)and Table[5](https://arxiv.org/html/2607.16209#A3.T5)\. We report the AUC performance of all methods across all datasets in Table[3](https://arxiv.org/html/2607.16209#A3.T3), and the downstream LLM QA performance \(EM and F1\) in Table[4](https://arxiv.org/html/2607.16209#A3.T4)\. We also report the multi\-model, multi\-pruner results on HotpotQA in Table[5](https://arxiv.org/html/2607.16209#A3.T5)\.

Table 3:Complete AUC performance comparison across all datasets \(k=50% compression\)\. Best in bold\.Table 4:Downstream LLM QA results \(GPT\-5\.4\) across all datasets\. Best pruning result per dataset in bold\.Table 5:LLM QA \(Judge, GPT\-5\.4\) across Multi\-Model, Multi\-Pruners on HotpotQA\. Best per model in bold\.
### C\.3Fact Recalling Supplementary Experiments

#### C\.3\.1Cross\-Domain Experiment

Tables[20](https://arxiv.org/html/2607.16209#A3.T20)show how a model trained on MS MARCO generalizes to unseen datasets\.

#### C\.3\.2CL\-Bench Results

![Refer to caption](https://arxiv.org/html/2607.16209v1/x7.png)Figure 7:CL\-Bench overall results \(k=0\.5\)\. Judged by GPT\-5\.4\. GPT\-5\.4 performs not idealy on CL\-Bench, mainly due to its format errors\. We provide this result for supplementary reference, and we also report the ablation results with different judges in Table[7](https://arxiv.org/html/2607.16209#A3.T7)\. SCP improve the performance of GPT\-5\.4 by 0\.25%, while BM25 hurts the performance by 0\.39%\. However, this result is not statistically significant, and the performance of GPT\-5\.4 on CL\-Bench is generally low, which may be due to the strict judging criteria of GPT\-5\.4\. We include this result for completeness and reference, but we do not draw strong conclusions from it\.The main results for the CL\-Bench experiment are shown in Table[6](https://arxiv.org/html/2607.16209#A3.T6)and Figure[7](https://arxiv.org/html/2607.16209#A3.F7)\.

Table 6:CL\-Bench overall results \(k=0\.5\)\. Judged by GPT\-5\.4\.Different from the official paper, we use GPT\-5\.4 as the judge, which is more strict than GPT\-5\.1 \(Official used\)\. See table[7](https://arxiv.org/html/2607.16209#A3.T7)\.

Table 7:Performance of SCP on CL\-Bench with Different Evaluating LLMs, Inference Model: GPT\-5\.4\(not High\), Not compressedAccording to the official paper, the pair \(GPT\-5\.2 High inference, GPT\-5\.1 judge\) yields 18\.1±\\pm0\.8% case success, 33\.9% format error, 59\.3% context\-ignore, 65\.4% context\-misuse, and 2\.4% refusal\. GPT\-5\.4 is more likely to produce format errors but less likely to produce context\-ignore or context\-misuse, which suggests that GPT\-5\.4 enforces context content more strictly but is less strict about output format\. The behavior of the GPT series depends on the training objective set by OpenAI, which may change across versions\. We speculate that GPT\-5\.4 places stronger emphasis on factual accuracy and context adherence, leading to more format errors when the model is forced to follow the context strictly, whereas GPT\-5\.1 takes a more lenient approach to context adherence, leading to more context\-ignore and misuse cases\.

Table[8](https://arxiv.org/html/2607.16209#A3.T8)shows how SCP outperforms BM25—another popular coarse\-grained retriever—on long\-context QA, e\.g\. CL\-Bench\. We also report rubric\-satisfaction bands\. Detailed data comparing SCP and the unpruned context are given in Table[9](https://arxiv.org/html/2607.16209#A3.T9)\.

Table 8:CL\-Bench requirement\-satisfaction distribution \(BM25 vs SCP, GPT\-5\.4 judge,k=0\.5k=0\.5\)\.Table 9:CL\-Bench results by rubric satisfaction bands\.\(GPT\-5\.4, k=0\.5\)\.The “Pass” column reports how many tasks in each band were graded 100% correct by the LLM\. In the 0–80% bands no task is fully correct, so the rate is automatically 0\.

These tables show that SCP successfully reduces the number of answers in the low\-satisfaction bands of the rubric and, as expected, occasionally prunes important information, which slightly lowers QA accuracy\.

Table[10](https://arxiv.org/html/2607.16209#A3.T10)reports the error\-type distribution of the different methods\.

Table 10:Heuristic error\-type ratios on failed CL\-Bench samples \(keyword tags over grading rationales, multi\-label\)\. Refusal is reported with a strict hard\-refusal matcher; a broader inability\-phrase rate is shown only as reference\.
#### C\.3\.3AUC Experiment Justification

Table[11](https://arxiv.org/html/2607.16209#A3.T11)shows that SCP attains very high AUC on MuSiQue, FEVER, and 2WikiMH against the human\-annotated supporting labels\. However, on MuSiQue this advantage does not translate to a downstream win in LLM QA \(Table[4](https://arxiv.org/html/2607.16209#A3.T4)\)\. We attribute this to a cascading issue in the official MuSiQue annotation:

- •Paragraph\-level supervision\.MuSiQue provides supporting labels at paragraph granularity\. To train a sentence\-level reranker we must propagate each paragraph label to every sentence inside it\.
- •Distractor leakage after propagation\.A “supporting paragraph” typically contains only 1–2 bridge sentences and several filler sentences that are not themselves supporting\. The propagation therefore inserts a substantial number of non\-supporting sentences into the positive set of our pairwise margin loss\.
- •Overfitting to noisy labels\.Training directly on MuSiQue fits a corrupted ranking objective\. A model trained on MS MARCO, which carries cleaner sentence\-level supervision, transfers back to MuSiQue better than either MuSiQue\-trained variant \(EM 0\.466 vs\. 0\.441 / 0\.389 / 0\.377; see Table[4](https://arxiv.org/html/2607.16209#A3.T4)\)\.

The broader insight is thatAUC against human annotations is correlated with downstream LLM QA performance, but the two are not equivalent\. When the human labels themselves are noisy, fitting them more tightly \(higher AUC\) does not yield more useful pruning\. We therefore report downstream EM/F1 in Table[4](https://arxiv.org/html/2607.16209#A3.T4)as the primary metric and treat AUC as a diagnostic\.

Table 11:Extended baseline analysis with ranking and performance gaps\.
#### C\.3\.4Compression Rate Study for SCP on HotpotQA

Table[12](https://arxiv.org/html/2607.16209#A3.T12)shows how different compression rates affect SCP’s performance\.

Table 12:SCP performance under different compression rates on HotpotQA \(GPT\-5\.4\)\.
#### C\.3\.5NIAH Supplementary Details

Tables[13](https://arxiv.org/html/2607.16209#A3.T13)and[14](https://arxiv.org/html/2607.16209#A3.T14)show how SCP’s recall performance changes under the single\-needle setting\.

Table 13:NIAH Performance Across Different Context LengthsTable 14:NIAH Depth Curve ResultsTable[15](https://arxiv.org/html/2607.16209#A3.T15)reports the performance of reranking and pruning when there are two needles in the context\.

Table 15:NIAH Key Retrieval Results \(Two Sentences\)We further test SCP with five needles under different keep ratios to check whether performance degrades\. See Table[16](https://arxiv.org/html/2607.16209#A3.T16)\.

Table 16:5 needle NIAH: all\-needle recall under different keep ratios\.

### C\.4Ablations

#### C\.4\.1Embedders

Table[17](https://arxiv.org/html/2607.16209#A3.T17)compares the performance of SCP when using different embedding mechanisms\. The default setting uses Sentence Transformer\[[36](https://arxiv.org/html/2607.16209#bib.bib36)\]with MiniLM\[[47](https://arxiv.org/html/2607.16209#bib.bib47)\], which is a common choice for sentence\-level embeddings\. We also test BERT, Word2Vec, TF\-IDF with SVD dimensionality reduction, and random embeddings as baselines\.

Table 17:Ablation: Evaluating SCP Performance under Different Embedding Constraints on HotpotQA
#### C\.4\.2Attribution Strategies Supplementary Details

This section is a controlled study of the LOO attribution*strategy*itself, not a re\-implementation of LooComp\[[9](https://arxiv.org/html/2607.16209#bib.bib9)\]\. LooComp couples its LOO signal to a LoRA adapter over a pretrained language model, whereas we hold the value function and embedding backbone fixed and only swap the attribution rule\. This isolates the contribution of the attribution principle \(LOO vs\. Shapley\) from confounding factors such as backbone capacity or LoRA fine\-tuning\.

Table[18](https://arxiv.org/html/2607.16209#A3.T18)reports the matched\-setting comparison between SCP \(MC\-Shapley\) and pure LOO attribution, where both methods use the same learned set\-value model and embedding backbone\. See Table[16](https://arxiv.org/html/2607.16209#A3.T16)for the number of Monte\-Carlo samples needed to match or exceed the performance of the LOO strategy on the NIAH experiment\.

Table 18:SCP vs\. LOO attribution under matched value\-function settings\.Using the LOO strategy at inference can be faster and more effective on these benchmarks because of its low variance \(SCP relies on Monte\-Carlo sampling\)\. In real\-world RAG systems, however, semantically duplicated supporting sentences are not guaranteed to be filtered, so the speed gain is obtained at the cost of robustness\. LOO mistakes a duplicated supporting sentence for a dummy player becausef​\(S∪\{i\}\)−f​\(S\)=0f\(S\\cup\\\{i\\\}\)\-f\(S\)=0when an equivalent sentence already lies inSS\. Using the trained value function we instead computev​\(S∪\{i\}\)−v​\(S\)v\(S\\cup\\\{i\\\}\)\-v\(S\)to score a sentence directly\. We provide an adversarial example for this strategy below; scores are rounded to one decimal place\.

##### Updated dose \+ issuer with old\-version distractor\.

Q: According to the latest protocol, what is the standard analgesic dose, and who issued it?The results are reported in Table[19](https://arxiv.org/html/2607.16209#A3.T19)\.

Table 19:Per\-sentence LOO and SCP scores\#SentenceRoleLOOSCPS1The old manual states: standard analgesic dose is 5 mg\.distractor \(old\)\-1\-0\.8S2Update notice: the standard dose has been adjusted to 8 mg\.G1†00\.7S3The 8 mg standard dose is currently in force\.G1†0\.10\.8S4This dose update was issued by Medical Officer Gu Ning\.G2†00\.5S5Gu Ning issued the present 8 mg adjustment\.G2†\-0\.10\.5S6The old 5 mg clause has been rescinded\.G31\.21\.3S7Pharmacy stock cap remains 200 vials\.noise00S8Hallway lights enter energy\-saving mode after 22:00\.noise\-0\.1\-0\.1S9A lost access card must be reported within 24 hours\.noise00S10Morning meeting moved from 08:00 to 08:30\.noise00We observe that LOO significantly underestimates duplicated supporting sentences, which makes its selection degenerate toward random\. Furthermore, the accuracy of LOO is fixed since the computation is deterministic, whereas SCP can improve with more samples\.

#### C\.4\.3Monte\-Carlo Sampling Number

Table[20](https://arxiv.org/html/2607.16209#A3.T20)shows how the number of Monte\-Carlo samples \(MC\) affects SCP’s performance on HotpotQA when trained on MS MARCO\. Spped here only refers to the inference time of SCP itself, the embedding time is not included\. In practical implementation, we recommend using batch pre\-embedding and GPU\-parallelized sampling, which can be easily optimized in comparison to the Provence, LLMLingua2 and other methods\.

Table 20:Sampling Number Ablation for SCP trained on MS MARCO, tested on HotpotQA \(dev subset, 612 samples\)\.

### C\.5Prompt Usage in Inference

Table[21](https://arxiv.org/html/2607.16209#A3.T21)lists the system prompts and generation settings used for LLM\-side evaluation across QA datasets\. HotpotQA and 2WikiMultiHopQA share the same system prompt; MuSiQue uses a stricter variant with a tighter output budget; CL\-Bench reuses the dataset\-releasedmessagesverbatim\.

Table 21:System prompts and generation settings across QA datasets\.

## Appendix DJustifications, Hypotheses and More Future Works

### D\.1Why 3M parameters?

We intentionally design the SCP model to be substantially smaller than competing models in order to show that the performance gains do not stem from raw model capacity alone but also from the evaluation paradigm\. With only 3M parameters, a lightweight model can still capture useful interdependencies among context components under the Shapley\-based training objective\. Our rationale is to decouple attribution complexity from the neural representation: we delegate the attribution computation to the Monte\-Carlo sampling mechanism, while the neural value function approximates holistic subset values—a simpler prediction task\. Practically, this design reduces inference time, since the value function only needs lightweight forward passes and the Monte\-Carlo aggregation can be parallelized on the GPU\.

### D\.2Impact of the Convexity Constraint

As described in appendix A, modeling supermodular valuations enforces a coherent mathematical hierarchy—the foundation for analysing the “Landscape of Context”\. To empirically transition from heuristic sets to a rigorous core, we implement the structurally restrictedConvexDeepSetsV2\.

Imposing continuous convexity bounds the hypothesis space\. Historically this can reduce predictive accuracy on elementary lexical tasks \(e\.g\., a slight degradation on MS MARCO\)\. However, on combinatorial tasks \(HotpotQA, MuSiQue\), the convex prior is more aligned with our conjecture that some bridging components exhibit complementarity \(e\.g\.,v​\(A∪B\)\>v​\(A\)\+v​\(B\)v\(A\\cup B\)\>v\(A\)\+v\(B\)\)\. In this sense, enforcing convexity acts as an inductive bias for capturing synergy in multi\-hop chains and provides empirical support for the proposed hierarchical Context\-Landscape perspective\.

Table[22](https://arxiv.org/html/2607.16209#A4.T22)reveals nuanced trade\-offs between the unconstrained SCP and ConvexV2\. Rather than positioning ConvexV2 as a universally superior variant, these results illuminate a fundamental alignment between architectural inductive biases and the underlying combinatorial structure of different reasoning tasks\.

Table 22:ConvexV2 vs\. Original SCP: Best AUC Comparison
### D\.3Ablation Explanation for Feature Extraction

Notably, downgrading the embedding input from a deep bidirectional contextual model \(SentenceTransformer\) to a shallow lexical model \(Word2Vec\) induces only a∼\\sim3% drop in AUC on HotpotQA while retaining strong multi\-hop performance\. This suggests that SCP’s effectiveness is not solely due to a powerful sentence embedding model, and that the cooperative aggregation mechanism \(theρ\\rholayer and the Shapley allocations\) plays an important role\.

Nevertheless, a baseline level of embedding quality is still required to capture basic semantic relevance, as evidenced by the substantial performance drop when using TF\-IDF \+ SVD or random embeddings\. This indicates that while SCP’s Shapley\-based training objective contributes to its robustness, it still relies on a certain level of semantic encoding in the input features to model context dependencies effectively\. The ablation supports the view that at the in\-sentence\-token level the embedding requires a fine\-grained understanding of the query and the context—especially the position of each token in the sentence—while at the in\-document\-sentence level a more coarse\-grained evaluation of the overall semantic relevance of the sentence to the query is sufficient for SCP to capture the interdependencies among sentences\.

### D\.4Architecture Justification for the Permutation\-Invariant Design and training\-inference alignment

In Natural Language Processing, positional encodings and order information are usually presumed to be necessary\. However, although positional encodings are essential at smaller scales, in context pruning—especially in the long\-context regime—the well\-known “lost in the middle” phenomenon is widely reported, and set modeling has proven effective in many long\-context scenarios, particularly in RAG\. In RAG, retrieved sentences are not organized in any fixed order, and the retrieval system may return sentences in arbitrary order\. We hypothesize that sentences are not simply organized in a sequence but rather form a complex logical hierarchy, which we call the “Landscape of Context”\.

##### Design Rationale: Why Pairwise Ranking Suffices for Coarse\-Grained Pruning

We deliberately employ a pairwise margin ranking loss rather than a regression\-based objective for three reasons rooted in the practical realities of RAG context pruning\. First,Dataset Annotation Constraints: Standard RAG datasets \(e\.g\., HotpotQA, MuSiQue, MS MARCO\) provide only binary supporting\-sentence labels; human annotators mark whether a sentence is relevant, but not its degree of relevance relative to other supporting sentences\. Obtaining such fine\-grained comparative labels would require prohibitively expensive additional annotation\. Second,Task\-level Requirements: Our goal is not to select the single most important sentence, but to filter out non\-supporting sentences while retaining as many supporting sentences as possible\. This is a coarse\-grained pruning objective: by operating at a conservative compression ratio, we prioritize recall over precision at the pruning stage, ensuring that even if the ranking among supporting sentences is imperfect, critical evidence is not lost\. The top\-KKpruning operation is mathematically a ranking truncation: it depends only on the relative orderingϕ\(1\)≥ϕ\(2\)≥⋯≥ϕ\(n\)\\phi\_\{\(1\)\}\\geq\\phi\_\{\(2\)\}\\geq\\dots\\geq\\phi\_\{\(n\)\}, not on absolute magnitudes\. Formally, for any strictly increasing monotonic transformationff, the top\-KKset satisfies\{i:ϕi≥ϕ\(K\)\}=\{i:f​\(ϕi\)≥f​\(ϕ\(K\)\)\}\\\{i:\\phi\_\{i\}\\geq\\phi\_\{\(K\)\}\\\}=\\\{i:f\(\\phi\_\{i\}\)\\geq f\(\\phi\_\{\(K\)\}\)\\\}\. Since Shapley values are homogeneous of degree one in the value function—ϕi​\(α​v\)=α⋅ϕi​\(v\)\\phi\_\{i\}\(\\alpha v\)=\\alpha\\cdot\\phi\_\{i\}\(v\)—the absolute scale is inherently arbitrary and only relative comparisons are structurally meaningful in cooperative game theory\. Third,Generalization across LLMs: The absolute importance of a sentence is inherently dependent on the downstream LLM’s internal knowledge and reasoning patterns\. Attempting to learn LLM\-specific absolute importance scores would harm cross\-model generalization\. A relative ranking objective, by contrast, learns an LLM\-agnostic ordering that transfers robustly across diverse generators\.

### D\.5Top\-K Justification

While Top\-K and Average\-Threshold are direct, empirically effective implementations, we hope that future work will explore more sophisticated pruning criteria that leverage the full distribution of Shapley values, such as clustering\-based methods or dynamic thresholding based on the variance of attributions\. The mathematical flexibility of Shapley values supports a wide range of pruning strategies beyond simple cutoffs, which can be tailored to specific downstream tasks and computational constraints, as much prior work has explored\. In our setting, however, given the low parameter count of our value function and the use of sentence\-level pruning, the simple Top\-K and Average\-Threshold methods already achieve competitive performance across the evaluated datasets, since SCP is designed primarily as a preliminary reranker and pruner for the retrieved context, after which the downstream LLM can further filter and use the pruned context to generate the final answer\.

### D\.6The Landscape of Context: Scale Change May Means Paradigm Shift

In most popular NLP settings, the necessity of positional encoding is taken for granted\. Historically, the absence of sequence awareness in the original self\-attention mechanism \(e\.g\., its inability to distinguish “dog chases cat” from “cat chases dog”\) prompted the adoption of positional embeddings in Transformer architectures\. However, as context length scales exponentially in the LLM era, the prevalent “lost in the middle” phenomenon suggests that models struggle to allocate attention uniformly across long, linear sequences\.

This awareness is not new\. In earlier years, researchers developed techniques such as Latent Dirichlet Allocation \(LDA\)\[[2](https://arxiv.org/html/2607.16209#bib.bib2)\]to classify documents into a limited set of topics, Set Transformers\[[23](https://arxiv.org/html/2607.16209#bib.bib23)\]to process unordered sets, and Deep Sets to model permutation\-invariant functions\. The advent of the Transformer architecture\[[45](https://arxiv.org/html/2607.16209#bib.bib45)\], however, encouraged the assumption that any context could be effectively processed through an adapted attention mechanism\. As a result, work that adapts attention mechanisms has received the bulk of attention, while other theoretical perspectives have been overlooked\.

We propose Shapley Context Pruning \(SCP\) as an attempt to construct an analytical framework based on cooperative game theory, a field thoroughly studied in economics and mathematics\. The theory provides a new perspective on the structure of context and the interactions between sentences, which can be used to guide the design of more effective context\-reranking methods and to provide a principled tool for analysing the structure of the context\. This raises the question of whether macro\-level sequence order may be fundamentally less important than micro\-level token order in certain contexts, especially in RAG systems where retrieved passages are returned in non\-deterministic orders without strict sequential causality\.

This leads to a paradigm\-shifting hypothesis:In extreme\-long\-context Retrieval\-Augmented Generation \(RAG\)\[[24](https://arxiv.org/html/2607.16209#bib.bib24)\]settings, macro\-level sequence order is fundamentally less important than micro\-level token order, though sequence modeling in sentence level is essential\.While positional information is undeniably crucial intra\-sentence to preserve semantic and syntactic integrity, inter\-sentence organization in RAG systems and complex user prompts is often an unordered “Bag\-of\-Evidence”\. Retrieved passages are frequently provided in non\-deterministic orders without strict sequential causality\.

The hypothesis suggests a possible paradigm: sequential modeling and unordered set modeling are two methods operating at different scales\. The former is more suitable for modeling token\-level interactions within a sentence, while the latter is more suitable for modeling sentence\-level interactions within a long context\. Recent work such as Set\-Encoder\[[39](https://arxiv.org/html/2607.16209#bib.bib39)\]applies a Set Transformer at the document level to implement cross\-document attention while still using BERT\-based embeddings, researchers start to explore setwise modeling rather than simple pairwise, pointwise, listwise modeling\. Combining set\-based methods at the larger scale with position\-aware methods at the smaller scale may be a more effective way to model the context, particularly in the RAG scenario\.

We therefore argue against the forced sequential projection of long contexts\. We instead propose the “Landscape of Context”[A\.4](https://arxiv.org/html/2607.16209#A1.SS4), a theoretical abstraction that conceptualises context not as a flat sequence but as a hierarchical topology of subsets\.

We propose this not as a closed problem but as a conceptual genesis\. We leave the rigorous mathematical formulation of this landscape’s exact boundaries, its adaptation to non\-convex empirical settings, and its broader applications in LLM alignment as open invitations to the research community\. By liberating context modeling from the strict inductive biases of linear positional embeddings, we hope that future theoretical work will explore the “Landscape of Context” architecture in depth and its fundamental implications for next\-generation context engineering\.

### D\.7More Future Work and Hypothesis

We believe that, as context grows larger and more complex, SCP can offer a less fine\-grained \(compared with LLM\- and BERT\-based methods\) but effective, fast, and interpretable pruning method for preliminary context compression in RAG systems and edge\-device deployment, saving tokens and reducing the LLM\-hallucination risk induced by distractors\. We suggest that future work explore a multi\-layer context\-pruning pipeline, where SCP serves as a fast and interpretable first stage that filters out the most irrelevant components, followed by more fine\-grained methods \(e\.g\., BERT\-based\) for further pruning\. Such a multi\-layer approach can balance efficiency and precision, leveraging the strengths of different pruning techniques at different stages of the pipeline\. SCP can also be deployed on local devices to perform preliminary context compression before sending the pruned context to the cloud for LLM inference, saving bandwidth and reducing latency in real\-world applications\.

##### Towards Incremental Context Engineering\.

The current SCP framework computes Shapley values in batch\. In real\-world RAG, however, retrieved sentences arrive incrementally, and recomputing all attributions from scratch is wasteful\. A promising direction is to exploit the structural properties of convex games to derive*incremental update rules*\. Future work could develop online algorithms that maintain a laminar tree dynamically, inserting new nodes and rebalancing only along the affected path rather than reconstructing the entire hierarchy\.

##### Forest Models for Multi\-Topic Contexts\.

A single convex game assumes that the entire context is one large cooperative structure\. Real documents, however, typically contain multiple semantically disjoint topics or reasoning chains\. We propose modelling such contexts as a*forest of convex games*: each tree corresponds to an independent topic or multi\-hop chain \(a local convex game\), while the forest captures the weak or negligible cross\-topic interactions\. Under the idealised assumption that cross\-topic synergy vanishes, the value function becomes block\-additive across a partition\{Nj\}\\\{N\_\{j\}\\\}ofNN:

v​\(S\)=∑jvj​\(S∩Nj\),∀S⊆N\.v\(S\)\\;=\\;\\sum\_\{j\}v\_\{j\}\\bigl\(S\\cap N\_\{j\}\\bigr\),\\qquad\\forall\\,S\\subseteq N\.To see that attributions can then be computed locally, define the extensionv~j​\(S\)=vj​\(S∩Nj\)\\tilde\{v\}\_\{j\}\(S\)=v\_\{j\}\(S\\cap N\_\{j\}\)for allS⊆NS\\subseteq N, so thatv=∑jv~jv=\\sum\_\{j\}\\tilde\{v\}\_\{j\}\. By Additivity,

ϕi​\(v\)=∑jϕi​\(v~j\)\.\\phi\_\{i\}\(v\)\\;=\\;\\sum\_\{j\}\\phi\_\{i\}\(\\tilde\{v\}\_\{j\}\)\.For any blockNjN\_\{j\}not containingii, playeriiis a dummy inv~j\\tilde\{v\}\_\{j\}\(addingiidoes not changeS∩NjS\\cap N\_\{j\}\); henceϕi​\(v~j\)=0\\phi\_\{i\}\(\\tilde\{v\}\_\{j\}\)=0by the Dummy\-Player axiom\. It remains to relateϕi​\(v~j∗\)\\phi\_\{i\}\(\\tilde\{v\}\_\{j^\{\*\}\}\)to the block\-level Shapley value for the unique blockNj∗∋iN\_\{j^\{\*\}\}\\ni i\. Expanding the definition, consider anyS⊆N∖\{i\}S\\subseteq N\\setminus\\\{i\\\}and defineT=S∩Nj∗T=S\\cap N\_\{j^\{\*\}\}andU=S∩\(N∖Nj∗\)U=S\\cap\(N\\setminus N\_\{j^\{\*\}\}\), so thatS=T∪US=T\\cup UandT∩U=∅T\\cap U=\\emptyset\.

ϕi​\(v~j∗\)\\displaystyle\\phi\_\{i\}\(\\tilde\{v\}\_\{j^\{\*\}\}\)=∑S⊆N∖\{i\}\|S\|\!​\(\|N\|−\|S\|−1\)\!\|N\|\!​\[v~j∗​\(S∪\{i\}\)−v~j∗​\(S\)\]\\displaystyle=\\sum\_\{S\\subseteq N\\setminus\\\{i\\\}\}\\frac\{\|S\|\!\\,\(\|N\|\-\|S\|\-1\)\!\}\{\|N\|\!\}\\bigl\[\\tilde\{v\}\_\{j^\{\*\}\}\(S\\cup\\\{i\\\}\)\-\\tilde\{v\}\_\{j^\{\*\}\}\(S\)\\bigr\]=∑T⊆Nj∗∖\{i\}\(∑U⊆N∖Nj∗\(\|T\|\+\|U\|\)\!​\(\|N\|−\|T\|−\|U\|−1\)\!\|N\|\!\)​\[vj∗​\(T∪\{i\}\)−vj∗​\(T\)\]\.\\displaystyle=\\sum\_\{T\\subseteq N\_\{j^\{\*\}\}\\setminus\\\{i\\\}\}\\Biggl\(\\sum\_\{U\\subseteq N\\setminus N\_\{j^\{\*\}\}\}\\frac\{\(\|T\|\+\|U\|\)\!\\,\(\|N\|\-\|T\|\-\|U\|\-1\)\!\}\{\|N\|\!\}\\Biggr\)\\bigl\[v\_\{j^\{\*\}\}\(T\\cup\\\{i\\\}\)\-v\_\{j^\{\*\}\}\(T\)\\bigr\]\.The inner sum overUUsatisfies the combinatorial identity

∑U⊆N∖Nj∗\(\|T\|\+\|U\|\)\!​\(\|N\|−\|T\|−\|U\|−1\)\!\|N\|\!=\|T\|\!​\(\|Nj∗\|−\|T\|−1\)\!\|Nj∗\|\!,\\sum\_\{U\\subseteq N\\setminus N\_\{j^\{\*\}\}\}\\frac\{\(\|T\|\+\|U\|\)\!\\,\(\|N\|\-\|T\|\-\|U\|\-1\)\!\}\{\|N\|\!\}\\;=\\;\\frac\{\|T\|\!\\,\(\|N\_\{j^\{\*\}\}\|\-\|T\|\-1\)\!\}\{\|N\_\{j^\{\*\}\}\|\!\},which collapses the global Shapley formula to the local one:

ϕi​\(v~j∗\)=∑T⊆Nj∗∖\{i\}\|T\|\!​\(\|Nj∗\|−\|T\|−1\)\!\|Nj∗\|\!​\[vj∗​\(T∪\{i\}\)−vj∗​\(T\)\]=ϕiNj∗​\(vj∗\)\.\\phi\_\{i\}\(\\tilde\{v\}\_\{j^\{\*\}\}\)\\;=\\;\\sum\_\{T\\subseteq N\_\{j^\{\*\}\}\\setminus\\\{i\\\}\}\\frac\{\|T\|\!\\,\(\|N\_\{j^\{\*\}\}\|\-\|T\|\-1\)\!\}\{\|N\_\{j^\{\*\}\}\|\!\}\\bigl\[v\_\{j^\{\*\}\}\(T\\cup\\\{i\\\}\)\-v\_\{j^\{\*\}\}\(T\)\\bigr\]\\;=\\;\\phi\_\{i\}^\{N\_\{j^\{\*\}\}\}\(v\_\{j^\{\*\}\}\)\.Thusϕi​\(v\)=ϕiNj∗​\(vj∗\)\\phi\_\{i\}\(v\)=\\phi\_\{i\}^\{N\_\{j^\{\*\}\}\}\(v\_\{j^\{\*\}\}\): each player’s global attribution equals its attribution within its own block\. This decomposition holds for*any*block\-additive game, convex or not; convexity is only needed to guarantee that each block induces a laminar hierarchy with non\-negative Harsanyi dividends\. The deeper challenge is therefore three\-fold: \(i\) enforcing block\-additivity, which is unlikely to emerge from standard training and must be imposed architecturally or via regularization; \(ii\) preserving convexity within each block so that the resulting trees remain interpretable; and \(iii\) discovering the partition\{Nj\}\\\{N\_\{j\}\\\}from data\. We leave the joint realization of these three desiderata as an exciting direction for future work\.

##### Transfer Learning and Multi\-objective Learning\.

The additivity of Shapley values raises a natural question: can two value functions trained on different datasets or under different loss objectives be directly composed into a single attribution? When such direct composition fails, an equally informative question is to characterize the gap—the representational discrepancy between the constituent value functions, expressed through their induced Shapley values\. A principled answer would clarify when transfer and multi\-objective training of context value functions reduce to a Shapley\-additive combination, and when they instead require explicit interaction modeling between the underlying objectives\.

##### Designing Context Management System

The “Landscape of Context” perspective \(Appendix[A\.4](https://arxiv.org/html/2607.16209#A1.SS4)\) suggests a new design for context management systems in RAG\. Instead of treating retrieved sentences as a flat sequence, we may organize them into the hierarchical structure induced by the laminar families of a convex context game, with Shapley values and Harsanyi dividends quantifying per\-node synergy\. This may allow a system to dynamically insert newly retrieved sentences into the existing hierarchy, evaluating and pruning them based on their localized contribution within the tree rather than via a flat ranking\. We expect the main practical benefit to be amortization via decomposition, perhaps utilizing additivity of Shapley values across the forest of value functions\{vi\}\\\{v\_\{i\}\\\}defined in Appendix[A\.4](https://arxiv.org/html/2607.16209#A1.SS4)\.

## Appendix ESimulated Real World Case Study for Context Reranking: A Research Scenario in Cognitive Neuroscience

We construct a real\-world case in cognitive neuroscience to demonstrate how context attribution behaves under multimodal data fusion\. The scenario involves an fMRI–EEG concurrent experiment on working\-memory load\. This case generated by GPT\-5\.4\[[34](https://arxiv.org/html/2607.16209#bib.bib34)\]is based on the authors’ real\-world experience in neuroimaging research, and is designed to illustrate the practical challenges and decision\-making processes that arise during the preparation, acquisition, and analysis phases of a complex multimodal experiment\. The dialogue captures the interactions between a principal investigator \(PI\), a PhD student, a research assistant \(RA\), and an engineer, highlighting how technical considerations, data quality issues, and methodological choices are navigated in a real\-world research setting\. The case study is structured into three phases: preparation, acquisition and incident, and analysis, with each phase containing specific dialogues that reflect the critical moments in the experimental workflow\.

Table[23](https://arxiv.org/html/2607.16209#A5.T23)records the preparation phase; Table[24](https://arxiv.org/html/2607.16209#A5.T24)covers acquisition and the incident; Table[25](https://arxiv.org/html/2607.16209#A5.T25)contains the methodological discussion and research question\.

Table 23:Neuroimaging case study—Phase 1: Preparation\.Table 24:Neuroimaging case study—Phase 2: Acquisition and incident\.Table 25:Neuroimaging case study—Phase 3: Analysis discussion and research question\.##### SCP pruning diagnosis on the neuroimaging case\.

When the full dialogue \(29 sentences\) is fed to SCP, the model retains 12 sentences \(40% compression\)\. Table[26](https://arxiv.org/html/2607.16209#A5.T26)summarises the pruning outcome\.

Table 26:SCP attribution ranking on the neuroimaging case study\.The result reveals a clear pattern: SCP, trained on MS MARCO, strongly favoursaction\-orientedsentences \(emergency procedures, equipment checks\) while deprioritisingmethodological discussion\(DCM, GLM, wavelet transforms\)\. This aligns with the observation in Section[D\.3](https://arxiv.org/html/2607.16209#A4.SS3)that the value function is sensitive to concrete operational cues but less attuned to domain\-specific methodological terminology\. The retained sliding\-window sentence \(rank 11\) survives only because it explicitly mentions “temporal resolution mismatch”—a phrase structurally similar to the technical details seen in MS MARCO passages\. This case therefore illustrates both a strength \(robust handling of procedural urgency\) and a limitation \(methodological nuance is pruned\) of the current SCP model, motivating the multi\-layer pipeline proposed in Section[D\.5](https://arxiv.org/html/2607.16209#A4.SS5)\.