More Is Not Always Better: Cross-Component Interference in LLM Agent Scaffolding

# Cross-Component Interference in LLM Agent Scaffolding
Source: [https://arxiv.org/html/2605.05716](https://arxiv.org/html/2605.05716)
## More Is Not Always Better: Cross\-Component Interference in LLM Agent Scaffolding

###### Abstract

LLM agent systems are typically constructed by stacking scaffolding components—planning modules, tool interfaces, memory, self\-reflection, and retrieval—on the assumption that more components improve performance\. We challenge this assumption with a systematic empirical study ofcross\-component interference \(CCI\): the degradation that arises when scaffolding components interact in ways that undermine each other\.

We conduct a full factorial experiment over all25=322^\{5\}\{=\}32subsets of five standard agent components on two benchmarks \(HotpotQA, GSM8K\) across three model families and five scales \(Llama\-3\.1\-8B/70B, Qwen2\.5\-3B/7B, Claude Haiku 4\.5\), yielding 118 controlled configurations with up to 10 random seeds on the primary setting\. We find that in every setting tested, the best proper subset matches or exceeds the five\-component All\-In system: on HotpotQA the single\-tool agent surpasses All\-In by 32% \(F1F\_\{1\}0\.2330\.233vs\.0\.1770\.177, pairedtt\-testp=0\.023p\{=\}0\.023, Wilcoxonp=0\.014p\{=\}0\.014, Cohen’sdz=0\.87d\_\{z\}\{=\}0\.87, 10 seeds\), while on GSM8K a three\-component subset achieves 79% higher accuracy than All\-In \(0\.430\.43vs\.0\.240\.24,p=0\.010p\{=\}0\.010\)\. The optimal component count is task\-dependent \(k∗=1k^\{\*\}\{=\}1–44\) and scale\-sensitive: CCI follows a capability gradient—strong at 8B \(gap=32%\{=\}32\\%\), attenuated at 70B \(gap=19%\{=\}19\\%\), and saturating to within\-noise at Claude Haiku \(gap≈0%\{\\approx\}0\\%\)—but across all scales, the simplest adequate scaffold matches or beats the All\-In agent\.

To characterize the interaction structure, we fit a main\-effects regression \(R2=0\.916R^\{2\}\{=\}0\.916, adjustedR2=0\.899R^\{2\}\{=\}0\.899, LOOCVR2=0\.872R^\{2\}\{=\}0\.872\) that decisively outperforms a 16\-parameter pairwise model \(Δ\\DeltaBIC=25\.3\{=\}25\.3\), compute exact Shapley values showing Tool Use captures 70% of total scaffold value \(ϕ=\+0\.177\\phi\{=\}\{\+\}0\.177,z=9\.1z\{=\}9\.1\) while Planning has a*significantly negative*Shapley value, and document 183 violations of submodularity out of 325 testable triples \(56\.3%; median submodularity ratioγmed=0\.52\\gamma\_\{\\mathrm\{med\}\}\{=\}0\.52, bootstrap 95% CI\[0\.23,0\.71\]\[0\.23,0\.71\], well below the submodular threshold of 1\)—with 90% of top violations exhibiting sign\-flipping rather than diminishing returns\. We identify a higher\-order synergy among Tool Use, Self\-Reflection, and Retrieval on retrieval tasks \(Harsanyi dividendINT3=\+0\.175\\text\{INT\}\_\{3\}\{=\}\{\+\}0\.175, BCa 95% CI:\[\+0\.004,\+0\.352\]\[\+0\.004,\+0\.352\]\), reported as an exploratory observation pending multi\-seed confirmation\. The directional CCI pattern replicates across model families \(Qwen2\.5\-3B/7B\), extends to a closed\-source API model \(Claude Haiku 4\.5 via AWS Bedrock, where the qualitative pattern—T dominates, Memory harmful, All\-In not optimal—is preserved\), and is robust to template paraphrasing\.

Our findings suggest that practitioner defaults of maximally\-equipped agents should be replaced by task\-specific subset selection informed by interaction\-aware analysis\.

More Is Not Always Better: Cross\-Component Interference in LLM Agent Scaffolding

Ming LiuAmazonData Scientistmlliuz@amazon\.com

## 1Introduction

The standard recipe for building capable LLM agents is*scaffolding*: augmenting a base model with planning modules, tool interfaces, working memory, chain\-of\-thought prompting, and self\-reflection loops\(Yao et al\.,[2023](https://arxiv.org/html/2605.05716#bib.bib32); Shinn et al\.,[2023](https://arxiv.org/html/2605.05716#bib.bib23); Wang et al\.,[2023](https://arxiv.org/html/2605.05716#bib.bib26)\)\. Frameworks like LangChain\(Chase,[2022](https://arxiv.org/html/2605.05716#bib.bib3)\)encourage developers to compose these components freely, with no systematic guidance on which subsets to include\. The implicit assumption is*more is better*\.

We present evidence that this assumption is often wrong for prompt\-based scaffolding, and wrong in a specific, measurable, and nuanced way\.

Consider Llama\-3\.1\-8B on HotpotQA multi\-hop reasoning\. Across 10 random seeds, tool use alone scoresF1=0\.233F\_\{1\}\{=\}0\.233; the full five\-component agent scoresF1=0\.177F\_\{1\}\{=\}0\.177—a32%degradation \(pairedtt\-testp=0\.023p\{=\}0\.023, Wilcoxon signed\-rankp=0\.014p\{=\}0\.014, Cohen’sdz=0\.87d\_\{z\}\{=\}0\.87; Bayesian:BF10=3\.2\\text\{BF\}\_\{10\}\{=\}3\.2, 95% HDI on difference\[\+0\.010,\+0\.102\]\[\+0\.010,\+0\.102\]\)\. We call thisCross\-Component Interference \(CCI\): the operational phenomenon of negative marginal returns under component composition\.

But the story is more nuanced than “less is always more\.” On GSM8K math reasoning, the optimal 8B configuration uses*three*components \(k∗=3k^\{\*\}\{=\}3, accuracy=0\.43=0\.43\), not one\. At 70B scale on HotpotQA, adding components to tool use*helps*—a reversal of the 8B pattern—though the full suite \(F1=0\.372F\_\{1\}\{=\}0\.372\) still trails the best subset \(F1=0\.441F\_\{1\}\{=\}0\.441\) by 19%\. CCI is not a universal “more is less” law; it is a task\- and scale\-dependent interaction effect that, in every setting we test, leaves the all\-in configuration outperformed by some proper subset\.

#### Contributions\.

1. 1\.Full factorial characterization\.We evaluate all 32 subsets of five agent components on two benchmarks across three model families and five scales \(118 configurations, 32,000\+ evaluations\), eliminating selection bias and exposing the full interaction landscape\. \(§[4](https://arxiv.org/html/2605.05716#S4)\)
2. 2\.CCI is pervasive, task\-dependent, and scale\-sensitive\.On Llama\-8B/HotpotQA \(10 seeds\), every expansion from the best scaffold degrades performance—5/6 significantly atp<0\.05p\{<\}0\.05\(4/6 survive Holm–Bonferroni correction; Cohen’sd=0\.87d\{=\}0\.87–1\.441\.44, all large\)\. The optimalk∗k^\{\*\}is task\-dependent \(1 vs\. 3\); at 70B, CCI direction reverses but the best proper subset matches or outperforms All\-In in all 6 model×\\timesbenchmark conditions tested\. \(§[5](https://arxiv.org/html/2605.05716#S5)\)
3. 3\.Structural parsimony and non\-submodularity\.A 6\-parameter main\-effects model explains 91\.6% of variance \(LOOCVR2=0\.872R^\{2\}\{=\}0\.872\), decisively outperforming a 16\-parameter pairwise model \(Δ\\DeltaBIC=25\.3\{=\}25\.3\)\. Shapley decomposition reveals Tool Use captures 70% of scaffold value while Planning is significantly negative\. 183/325 submodularity violations \(median ratioγ=0\.52\\gamma\{=\}0\.52, bootstrap CI\[0\.23,0\.71\]\[0\.23,0\.71\], below the submodular threshold\) exhibit sign\-flipping—components harmful in isolation become beneficial in specific combinations—making greedy selection empirically unreliable\. \(§[6](https://arxiv.org/html/2605.05716#S6)\)
4. 4\.Robustness and generality\.CCI replicates across model families \(Qwen2\.5\), extends to a closed\-source API model \(Claude Haiku 4\.5, where the qualitative pattern is preserved but CCI saturates to within\-noise\), is robust to template paraphrasing \(All\-In suboptimal in all 3 variants\), and is not a context\-length artifact \(length\-matched control: 6–9×\\timesgap\)\. \(§[7](https://arxiv.org/html/2605.05716#S7)\)

## 2Related Work

#### LLM Agent Scaffolding\.

ReAct\(Yao et al\.,[2023](https://arxiv.org/html/2605.05716#bib.bib32)\)interleaves reasoning and action; Reflexion\(Shinn et al\.,[2023](https://arxiv.org/html/2605.05716#bib.bib23)\)adds self\-reflection; Voyager\(Wang et al\.,[2023](https://arxiv.org/html/2605.05716#bib.bib26)\)combines planning, skill memory, and self\-verification\. Cognitive architecture frameworks\(Sumers et al\.,[2024](https://arxiv.org/html/2605.05716#bib.bib24)\)compose multiple such components into complex agents\.Kapoor et al\. \([2024](https://arxiv.org/html/2605.05716#bib.bib13)\)question whether complex scaffolding outperforms simple baselines\. These works demonstrate individual components’ value but do not systematically study*interaction effects*when components are combined—they use one\-at\-a\-time ablation at best, never full factorial designs that reveal higher\-order interactions\.

#### Prompt Sensitivity and Optimization\.

LLMs are sensitive to prompt format\(Sclar et al\.,[2024](https://arxiv.org/html/2605.05716#bib.bib22)\), information position\(Liu et al\.,[2024](https://arxiv.org/html/2605.05716#bib.bib17); He et al\.,[2024](https://arxiv.org/html/2605.05716#bib.bib10)\), and paraphrase wording\(Mizrahi et al\.,[2024](https://arxiv.org/html/2605.05716#bib.bib19)\)\.Battle and Gollapudi \([2024](https://arxiv.org/html/2605.05716#bib.bib2)\)test 60 system message combinations and find that optimal prompting is model\-size\-dependent\. DSPy\(Khattab et al\.,[2023](https://arxiv.org/html/2605.05716#bib.bib14)\), OPRO\(Yang et al\.,[2024](https://arxiv.org/html/2605.05716#bib.bib30)\), and TextGrad\(Yuksekgonul et al\.,[2024](https://arxiv.org/html/2605.05716#bib.bib34)\)optimize prompts automatically but treat the prompt as an atomic string, ignoring internal component structure\. Our work asks which*combination*of components to activate\.

#### Instruction Interference and Capacity Limits\.

Recent work documents that instruction\-following degrades as constraint count increases\(Wen et al\.,[2024](https://arxiv.org/html/2605.05716#bib.bib28); Jaroslawicz et al\.,[2025](https://arxiv.org/html/2605.05716#bib.bib12)\)\.Qi et al\. \([2026](https://arxiv.org/html/2605.05716#bib.bib21)\)identify “paradoxical interference” where adding format constraints hurts task solving, attributing this to attention being diverted from task\-relevant tokens\.Li et al\. \([2025](https://arxiv.org/html/2605.05716#bib.bib16)\)show that chain\-of\-thought degrades instruction\-following accuracy\. These works observe interference between pairs of objectives but study one interaction at a time\. We provide the full*interaction landscape*across all252^\{5\}component combinations\.

#### Prompt Component Regression and Interaction Models\.

Most closely related to our analytical framework,Lauziere et al\. \([2026](https://arxiv.org/html/2605.05716#bib.bib15)\)fit a pairwise interaction modelf​\(S\)=c\+∑iwi​si\+∑i​jJi​j​si​sjf\(S\)=c\+\\sum\_\{i\}w\_\{i\}s\_\{i\}\+\\sum\_\{ij\}J\_\{ij\}s\_\{i\}s\_\{j\}to prompt component effects on arithmetic tasks, explaining 72–77% of variance\. We adopt the same model class—treating the coupling matrixJJas a compact, interpretable summary of component interactions—but fit it to agent scaffolding configurations rather than generic prompt features\. Applied to our 32 meanF1F\_\{1\}values \(8B, HotpotQA\), the main\-effects model achievesR2=0\.916R^\{2\}\{=\}0\.916\(adjustedR2=0\.899R^\{2\}\{=\}0\.899, LOOCVR2=0\.872R^\{2\}\{=\}0\.872\); adding pairwise terms yieldsR2=0\.937R^\{2\}\{=\}0\.937but*lower*LOOCVR2=0\.748R^\{2\}\{=\}0\.748due to overfitting with 15 parameters on 32 data points\. We go beyond pairwise terms to identify an exploratory three\-body residual \(Harsanyi dividend INT=3\+0\.175\{\}\_\{3\}\{=\}\{\+\}0\.175, BCa 95% CI:\[\+0\.004,\+0\.352\]\[\+0\.004,\+0\.352\]\) that cannot be recovered from pairwise terms alone\.

#### Shapley Values and Higher\-Order Interactions\.

Shapley values have been applied to prompt\-level attribution\(Mohammadi,[2024](https://arxiv.org/html/2605.05716#bib.bib20)\)and in\-context example valuation\(Xie et al\.,[2024](https://arxiv.org/html/2605.05716#bib.bib29)\)\. SHAP\-IQ\(Fumagalli et al\.,[2023](https://arxiv.org/html/2605.05716#bib.bib9)\)provides a unified framework for computing any\-order Shapley interaction indices; HarsanyiNet\(Chen et al\.,[2023](https://arxiv.org/html/2605.05716#bib.bib4)\)grounds irreduciblekk\-way interactions in the Harsanyi dividend\. We apply these concepts at the*instruction component*level: our five scaffolding components admit exact computation over all25=322^\{5\}\{=\}32coalitions without approximation\.

#### Scale\-Dependent Phenomena and Non\-Submodularity\.

The Inverse Scaling Prize\(McKenzie et al\.,[2023](https://arxiv.org/html/2605.05716#bib.bib18)\)documents tasks where performance worsens with scale\.Huang et al\. \([2025](https://arxiv.org/html/2605.05716#bib.bib11)\)prove that LLM ensemble selection is non\-submodular\. Our finding that CCI reverses between 8B and 70B is a new instance of scale\-dependent behavior, and our 183 empirically observed submodularity violations demonstrate that greedy component selection is unreliable in this domain\.

## 3Problem Setup

### 3\.1Scaffolding Components

We study five canonical scaffolding components present across modern agent frameworks\(Sumers et al\.,[2024](https://arxiv.org/html/2605.05716#bib.bib24)\):

- •Planning \(P\): System\-level instruction to decompose tasks into sub\-goals\.
- •Tool Use \(T\): Function\-calling interface with tool descriptions\.
- •Memory \(M\): Structured working memory persisting observations across steps\.
- •Structured Reasoning \(SR\): Chain\-of\-thought formatting instructions\(Wei et al\.,[2022](https://arxiv.org/html/2605.05716#bib.bib27)\)\.
- •Reflection \(R\): Self\-evaluation prompt appended after each step\.

These five cover the principal functional modules identified in language agent taxonomies\(Sumers et al\.,[2024](https://arxiv.org/html/2605.05716#bib.bib24)\): profiling \(P\), reasoning \(T \+ SR\), memory \(M\), and self\-regulation \(SR \+ R\)\. A*configuration*C⊆\{P,T,M,S​R,R\}C\\subseteq\\\{P,T,M,SR,R\\\}is a subset of active components, withK=\|C\|K=\|C\|\. There are25=322^\{5\}=32possible configurations\. A full factorial design overkkbinary components requires2k2^\{k\}configurations; our 32\-configuration sweep already represents a substantial computational commitment, and each additional component doubles this cost\.

### 3\.2CCI Definition

###### Definition 1\(Cross\-Component Interference\)\.

CCI occurs for a pair\(C,s\)\(C,s\)wheres∉Cs\\notin Cif addingsstoCCreduces performance:ϕ​\(C∪\{s\}\)<ϕ​\(C\)\\phi\(C\\cup\\\{s\\\}\)<\\phi\(C\)\.

CCI is an*operational*definition describing the observable phenomenon of negative marginal returns, without committing to a specific causal mechanism\. We say CCI is*widespread*if it holds for many\(C,s\)\(C,s\)pairs across the252^\{5\}lattice\.

###### Definition 2\(Optimal Component Count\)\.

k∗=arg⁡maxK⁡max\|C\|=K⁡ϕ​\(C\)k^\{\*\}=\\arg\\max\_\{K\}\\max\_\{\|C\|=K\}\\phi\(C\)\.

### 3\.3Experimental Protocol

Models\.Llama\-3\.1\-8B\-Instruct and Llama\-3\.1\-70B\-Instruct\(Dubey et al\.,[2024](https://arxiv.org/html/2605.05716#bib.bib7)\), Qwen2\.5\-3B/7B\-Instruct\(Team,[2025](https://arxiv.org/html/2605.05716#bib.bib25)\)for cross\-family replication, and Claude Haiku 4\.5\(Anthropic,[2024](https://arxiv.org/html/2605.05716#bib.bib1)\)via AWS Bedrock as a closed\-source API model validation\. The 70B model uses 4\-bit NF4 quantization\(Dettmers et al\.,[2023](https://arxiv.org/html/2605.05716#bib.bib6)\)\.

Benchmarks\.\(1\) HotpotQA\(Yang et al\.,[2018](https://arxiv.org/html/2605.05716#bib.bib31)\)\(token\-levelF1F\_\{1\}\), and \(2\) GSM8K\(Cobbe et al\.,[2021](https://arxiv.org/html/2605.05716#bib.bib5)\)\(exact\-match accuracy\)\. Each configuration uses up to 4 reasoning steps\. All models use temperature=0\.1=0\.1, top\-p=0\.9p=0\.9, max 256 new tokens per step\.

Configurations\.Our main experiment evaluates all 32 subsets on 100 questions per benchmark\. We replicate key configurations across 10 random seeds on HotpotQA \(16,000 evaluations\)\. Total: 32,000\+ evaluations\.

Statistical methodology\.Single\-seed results report bootstrap 95% CIs \(2,000 resamples\)\. Multi\-seed comparisons use pairedtt\-tests, confirmed with non\-parametric Wilcoxon signed\-rank tests; exploratory comparisons use Benjamini–Hochberg correction\. We report Cohen’sdzd\_\{z\}\(paired\) as primary effect size measure and complement frequentist tests with Bayesian analysis \(JZS Cauchy prior,r=0\.707r\{=\}0\.707\)\. BCa bootstrap \(50,000 resamples\) is used for interaction\-term CIs\.

## 4Main Result: CCI in the Combinatorial Space

### 4\.1Tool Use Dominates; Adding Components Hurts

Table[8](https://arxiv.org/html/2605.05716#A1.T8)\(Appendix[A](https://arxiv.org/html/2605.05716#A1)\) presents all 32 configurations on HotpotQA with Llama\-3\.1\-8B\. The clearest pattern is a sharp partition based on whether tool use \(T\) is included:

- •With T:F1F\_\{1\}ranges from0\.1420\.142to0\.2840\.284\(mean0\.2040\.204\)\.
- •Without T:F1F\_\{1\}ranges from0\.0000\.000to0\.0990\.099\(mean0\.0430\.043\)\.

In the seed\-42 sweep,*every multi\-component configuration containing T scores below T alone*\(Table[1](https://arxiv.org/html/2605.05716#S4.T1)\)\. The mean degradation from T \(F1=0\.284F\_\{1\}\{=\}0\.284\) when adding 1–4 components ranges from−\-26% to−\-33%\.

Table 1:Top T\-containing configurations on HotpotQA \(8B, seed 42\)\. Adding components to T generally reduces performance\.
### 4\.2Multi\-Seed Validation

We replicate 16 key configurations across 10 random seeds \(16,000 total evaluations\)\. T alone achievesF1=0\.233±0\.039F\_\{1\}\{=\}0\.233\\pm 0\.039\[95% CI: 0\.209, 0\.257\], while All\-In achievesF1=0\.177±0\.049F\_\{1\}\{=\}0\.177\\pm 0\.049\[0\.147, 0\.207\]\. T outperforms All\-In in 8/10 seeds \(mean difference\+0\.056\+0\.056, median\+0\.059\+0\.059\):

Statistical tests \(T vs\. All\-In\):

- •Pairedtt\-test:t​\(9\)=2\.74t\(9\)\{=\}2\.74,p=0\.023p\{=\}0\.023, Cohen’sdz=0\.87d\_\{z\}\{=\}0\.87\(large\)
- •Wilcoxon signed\-rank:W=6W\{=\}6,p=0\.014p\{=\}0\.014
- •Bayesian \(JZS prior, Cauchyr=0\.707r\{=\}0\.707\):BF10=3\.2\\text\{BF\}\_\{10\}\{=\}3\.2\(moderate evidence on Jeffreys scale\), posterior direction probabilityP​\(δ\>0∣data\)=0\.98P\(\\delta\{\>\}0\\mid\\text\{data\}\)\{=\}0\.98
- •95% HDI on difference:\[\+0\.010,\+0\.102\]\[\+0\.010,\+0\.102\]

The 8 seeds favoring T average\+0\.062\+0\.062F1 gains, while the 2 reversals average only−0\.035\-0\.035\(one nearly tied at−0\.007\-0\.007\)\. Excluding the discovery seed \(42\):p=0\.046p\{=\}0\.046\.

Table 2:Multi\-seed validation \(10 seeds×\\times100 HotpotQA tasks, 8B\)\.Every measured expansion from T degrades performance—5 of 6 comparisons reach significance at uncorrectedp<0\.05p\{<\}0\.05\(4/6 survive Holm–Bonferroni correction for the family of 6 tests\), with effect sizes uniformly large \(Cohen’sdz=0\.87d\_\{z\}\{=\}0\.87–1\.441\.44\)\. These exceed typical effect sizes in the NLP prompt engineering literature \(d≈0\.2d\{\\approx\}0\.2–0\.50\.5\) by a factor of 2–4×\\times, establishing CCI as a practically significant phenomenon, not merely a statistically detectable one\.

### 4\.3Shapley Decomposition

With all25=322^\{5\}\{=\}32coalition values available, we compute exact Shapley values for each component on HotpotQA \(8B\):

Table 3:Exact Shapley valuesϕ​\(i\)\\phi\(i\)— average marginal contribution of each component across all possible coalitions\.On HotpotQA, T dominates \(ϕ=\+0\.177\\phi\{=\}\+0\.177,z=9\.1z\{=\}9\.1\), capturing70% of total absolute Shapley mass—the remaining four components contribute 30% combined\. Planning has a*statistically significant negative*Shapley value \(ϕ=−0\.029\\phi\{=\}\{\-\}0\.029, 95% bootstrap CI:\[−0\.055,−0\.003\]\[\-0\.055,\-0\.003\], entirely below zero\): on average, adding planning to*any*coalition reduces performance\. Memory is directionally negative \(ϕ=−0\.016\\phi\{=\}\{\-\}0\.016\) though not individually significant\. Together, P and M impose a 22% “CCI tax” on the positive value created by T and SR\. On GSM8K, R becomes strongly positive \(ϕ=\+0\.065\\phi\{=\}\+0\.065\), consistent with thek∗=3k^\{\*\}\{=\}3finding where R is part of the optimal subset\.

## 5CCI Varies by Task and Scale

### 5\.1Task\-Dependence: GSM8K

The GSM8K results reveal a fundamentally different CCI pattern \(Table[4](https://arxiv.org/html/2605.05716#S5.T4)\):

Table 4:Top configurations on GSM8K \(8B, 100 questions\)\.k∗=3k^\{\*\}\{=\}3, not 1\.1. 1\.Optimal subset size differs\.k∗=3k^\{\*\}\{=\}3on GSM8K \(T\+SR\+R:0\.4300\.430\) vs\.k∗=1k^\{\*\}\{=\}1on HotpotQA \(T alone\)\. SR and R*help*on math tasks but hurt on retrieval QA\.
2. 2\.CCI still exists\.Best subset outperforms All\-In by 79% \(McNemarp=0\.010p\{=\}0\.010\)\.
3. 3\.Different interference source\.CCI here manifests as a penalty from adding P and M to an already\-effective three\-component core, not from adding*any*component to T\.

### 5\.2Capacity\-Dependence: 70B Scale

Table 5:CCI across scale \(HotpotQA\)\. At 70B, adding components*helps*T\.CCI severity varies with model scale:

1. 1\.CCI direction reverses\.At 8B, adding R to T hurts \(Δ=−0\.072\\Delta\{=\}\{\-\}0\.072\); at 70B, it helps \(Δ=\+0\.072\\Delta\{=\}\{\+\}0\.072\)\.
2. 2\.CCI is reduced but not eliminated\.Best\-vs\-All\-In gap narrows from 32% \(8B, 10\-seed validated\) to 19% \(70B, single seed,p=0\.13p\{=\}0\.13\)\.
3. 3\.Important caveats\.Our two models differ in architecture and training compute in addition to parameter count\. Two data points cannot establish a scaling law; these results are suggestive evidence that larger models tolerate wider component combinations\.

### 5\.3Cross\-Family Replication

The directional CCI pattern—best achievable subset matches or outperforms All\-In—replicates inall six model×\\timesbenchmark conditionstested \(three families, five scales\), with gaps ranging from<<1% to 79% \(Table[6](https://arxiv.org/html/2605.05716#S5.T6)\)\. The gap narrows with scale within each family \(Qwen: 40% at 3B→\\rightarrow8% at 7B; Llama: 32% at 8B→\\rightarrow19% at 70B\), reaching a saturation plateau on Claude Haiku \(<<1%\)\. CCI extends to aclosed\-source API model: Claude Haiku 4\.5 \(via AWS Bedrock\) shows a*saturation regime*where all configurations fromK=1K\{=\}1toK=5K\{=\}5cluster within 0\.25 percentage points \(F1∈\[0\.395,0\.398\]F\_\{1\}\\in\[0\.395,0\.398\], CIs fully overlapping\)\. The qualitative pattern is preserved—T alone matches All\-In, Memory is harmful in combination \(P\+T\+SR→\\rightarrowP\+T\+M\+SR:−5\-5%\)—but the CCI signal attenuates to within\-noise, consistent with the capability gradient \(strong at 8B, attenuated at 70B, saturated at Claude\)\. Only Llama\-8B is backed by multi\-seed validation \(p=0\.023p\{=\}0\.023\); other results are directional\.

Table 6:Cross\-family CCI replication \(HotpotQA, 100 tasks, single seed except Llama\-8B\)\.The Claude Haiku result is notable: while All\-In is not*worse*than simpler subsets \(unlike Llama\-8B\), the four additional components beyond Tool Use contribute virtually nothing \(\+0\.2\+0\.2percentage points\)\. This “plateau” pattern—where additional components neither help nor harm—represents a milder form of CCI where the interference cancels any potential benefit, yielding zero marginal return on increased system complexity\.

## 6Analysis Framework

### 6\.1Component Interaction Regression

To characterize the interaction structure, we fit two models to the 32 meanF1F\_\{1\}values \(8B HotpotQA\):

#### Main\-effects model

\(6 parameters: intercept \+ 5 binary predictors\):

f​\(S\)=c\+∑i=15wi​sif\(S\)=c\+\\sum\_\{i=1\}^\{5\}w\_\{i\}s\_\{i\}This achievesR2=0\.916R^\{2\}\{=\}0\.916, adjustedR2=0\.899R^\{2\}\{=\}0\.899, and LOOCVR2=0\.872R^\{2\}\{=\}0\.872\.

#### Pairwise interaction model

\(16 parameters: \+ 10 pairwise terms\):

f​\(S\)=c\+∑iwi​si\+∑i<jJi​j​si​sjf\(S\)=c\+\\sum\_\{i\}w\_\{i\}s\_\{i\}\+\\sum\_\{i<j\}J\_\{ij\}s\_\{i\}s\_\{j\}This achievesR2=0\.937R^\{2\}\{=\}0\.937, but adjustedR2R^\{2\}drops to0\.8780\.878and LOOCVR2R^\{2\}drops to0\.7480\.748\. The BIC strongly favors the simpler main\-effects model \(Δ\\DeltaBIC=25\.3\{=\}25\.3\)\.

The main\-effects model’s superior generalization indicates that CCI in our data is primarily*additive*: each component imposes an approximately constant marginal cost \(or benefit\) regardless of context\. The pairwise model’s LOOCVR2R^\{2\}drops by 14 percentage points \(0\.872→\\rightarrow0\.748\), and theΔ\\DeltaBIC of 25\.3 constitutes “decisive evidence” favoring the simpler model on the Kass & Raftery scale\. This is a key structural finding: CCI is*predictable*from individual component effects alone, without modeling interaction terms—practitioners can assess component value independently\. This finding contrasts withLauziere et al\. \([2026](https://arxiv.org/html/2605.05716#bib.bib15)\)’s 72–77% fit; our higher explanatory power \(adj\-R2=0\.899R^\{2\}\{=\}0\.899\) suggests that structured scaffolding components interact more systematically than generic prompt features\.

TheJJmatrix of the pairwise model reveals positive couplings \(SR×\\timesR:\+0\.031\+0\.031\) and negative couplings \(T×\\timesM:−0\.019\-0\.019\), but these should be interpreted with caution given the overfitting evidence\.

### 6\.2Non\-Submodularity

We test the standard submodularity condition across all valid\(S,T,i\)\(S,T,i\)triples whereS⊊TS\\subsetneq Tandi∉Ti\\notin T\. These 325 triples are not statistically independent: they are derived from only 32 underlyingf​\(S\)f\(S\)measurements, each entering an average of 40\.6 triples\. We therefore report descriptive statistics together with a task\-level cluster bootstrap \(5,000 resamples over 100 tasks\) that preserves this dependence structure\.

- •Total triples evaluated: 325
- •Violations \(gain at superset\>\>gain at subset\):183 \(56\.3%\); cluster\-bootstrap 95% CI on violation rate:\[0\.43,0\.66\]\[0\.43,0\.66\]
- •Violations with gap\>\>0\.05: 84; with gap\>\>0\.10: 17
- •Median submodularity ratio:γmed=0\.52\\gamma\_\{\\mathrm\{med\}\}\{=\}0\.52\(bootstrap 95% CI\[0\.23,0\.71\]\[0\.23,0\.71\]\), strictly below the submodular thresholdγ=1\\gamma\{=\}1

The bootstrap CI on the violation rate overlaps 50%, so we do not claim the rate itself differs from chance\. The robust evidence against submodularity comes from the marginal\-ratio distribution:γmed=0\.52\\gamma\_\{\\mathrm\{med\}\}\{=\}0\.52with CI well below 1\.

The*structure*of violations is more revealing than the count\. Among the 20 violations with the largest absolute gap \(all gaps\>0\.096\>0\.096\), 18 \(90%\) exhibitsign\-flipping: a component that*reduces*performance when added to a small scaffold becomes*beneficial*in a specific larger combination\. For example, adding SR to\{T\}\\\{T\\\}hurts \(marginal=−0\.068=\-0\.068\) but adding SR to\{P,R,T\}\\\{P,R,T\\\}helps \(marginal=\+0\.094=\+0\.094\)—a sign reversal of\+0\.161\+0\.161\. T is present in 95% of the contexts where top violations occur, acting as the enabling catalyst\. We characterize these structurally rather than testing each triple individually; none of the top\-20 violations is individually significant after Bonferroni correction over 325 tests\.

These sign\-flipping cases show that scalar “component quality” rankings cannot be reliable in general: whether a component helps or hurts depends on what is already present\. On HotpotQA, greedy from T terminates atK=1K\{=\}1\(all marginals negative\), coincidentally finding the optimum\. On GSM8K wherek∗=3k^\{\*\}\{=\}3, greedy from T also terminates atK=1K\{=\}1\(accuracy0\.2200\.220\) while the true optimumT\+S​R\+RT\{\+\}SR\{\+\}Rachieves0\.4300\.430—a 95% improvement that greedy cannot reach\. We therefore characterize greedy as empirically unreliable on this benchmark suite, though not provably suboptimal in general\.

### 6\.3Exploratory: Three\-Body Catalysis

Beyond pairwise analysis, we compute the three\-body Harsanyi dividend for the triplet\(T,S​R,R\)\(T,SR,R\)on HotpotQA:

INT3​\(T,S​R,R\)=\+0\.175\\text\{INT\}\_\{3\}\(T,SR,R\)=\+0\.175
This is the largest single interaction effect in our data: SR and R individually hurt T \(marginals:−0\.068\-0\.068,−0\.072\-0\.072\), but together they produce a surplus \(T\+S​R\+R=0\.271T\{\+\}SR\{\+\}R=0\.271vs\. pairwise prediction≈0\.145\\approx 0\.145\)\.

BCa bootstrap 95% CI \(50,000 resamples over 100 tasks\):\[\+0\.004,\+0\.352\]\[\+0\.004,\+0\.352\],p=0\.027p\{=\}0\.027\(one\-sidedtt\-test\)\. The CI excludes zero but is wide, reflecting high per\-task heterogeneity \(std=0\.89=0\.89across tasks\)\.

We report this as anexploratory observation: the effect is statistically significant at the per\-task level but the wide CI warrants caution\. INT3is positive on HotpotQA but near\-zero on GSM8K \(\+0\.01\+0\.01\), indicating task\-specificity\.

## 7Robustness

### 7\.1Length\-Matched Control

We compareKKreal components against length\-matched meaningless padding:

Table 7:Length\-matched control \(3\-seed mean\)\. Real components massively outperform padding\.CCI is driven by semantic content, not token count: real components outperform padding by 6–9×\\times\.

### 7\.2Template Paraphrase Robustness

Three semantically equivalent template variants confirm the core CCI pattern:

- •T\>\>All\-In inall three variants\(gaps: \+3%, \+20%, \+13%\)\.
- •Individual configuration rankings are template\-sensitive \(Spearmanρ=0\.14\\rho=0\.14–0\.490\.49across variant pairs over the 6 tested configurations\), meaning fine\-grained ordering is not reproducible across paraphrases\.
- •The*component\-level*finding \(T dominates, All\-In is suboptimal\) is robust;*configuration\-level*rankings are template\-dependent\.

This rules out single\-template artifact while establishing an important boundary condition: CCI is a reliable phenomenon at the level of component attribution \(Shapley rankings are stable\), even though fine\-grained configuration rankings depend on prompt wording\.

### 7\.3Closed\-Source API Validation: Capability Saturation

To probe how CCI scales with model capability, we evaluate 8 key configurations on Claude Haiku 4\.5 via AWS Bedrock \(100 HotpotQA tasks, seed 42\)\.

The results reveal a*saturation regime*that completes the capability gradient established by the 8B and 70B experiments\. All configurations fromK=1K\{=\}1toK=5K\{=\}5cluster within a 0\.25\-point band \(F1∈\[0\.395,0\.398\]F\_\{1\}\\in\[0\.395,0\.398\], 95% CIs fully overlapping\)\. Tool Use alone reachesF1=0\.396F\_\{1\}\{=\}0\.396, while All\-In reachesF1=0\.398F\_\{1\}\{=\}0\.398—a difference of 0\.2% that is far below the noise floor\. Memory remains the only detectably harmful component \(P\+T\+SR→\\rightarrowP\+T\+M\+SR:−5\-5%\), consistent with Llama\-8B, but even this effect is not statistically significant at the single\-seed level\. The configuration rank ordering shows a moderate positive correlation with Llama\-8B \(Spearmanρ=0\.54\\rho\{=\}0\.54,n=7n\{=\}7shared configurations; not statistically significant given the small sample,p=0\.22p\{=\}0\.22\)\.

We interpret this as evidence for acapability\-saturation hypothesis: as model capability increases, the marginal cost of redundant components shrinks—the CCI signal that dominates at 8B \(32% gap\) and persists at 70B \(19% gap\) flattens to within\-noise at Claude Haiku \(≈\\approx0% gap\)\. The practical implication is unchanged: at every capability level tested, the simplest single\-component scaffold \(T alone\) matches or outperforms the full agent, with the cost of unnecessary components ranging from severe \(8B\) to negligible \(Claude\)\.

### 7\.4Component Ordering

In a preliminary test of 5 orderings×\\times5 configurations \(200 HotpotQA tasks\), ordering effects are substantial \(best vs\. worst: 62% for P\+T\)\. However, no single ordering dominates, and optimal ordering only partially recovers CCI losses\.

## 8Error Analysis

Among the 19 HotpotQA tasks where T succeeds \(F1\>0\.3F\_\{1\}\{\>\}0\.3\) but All\-In fails \(F1<0\.1F\_\{1\}\{<\}0\.1\):

- •Component\-specific interference\(68%\): Some T\+X pairs work while others collapse—interference depends on which component interacts with which task\.
- •Threshold collapse\(21%\): Any perturbation from T is fatal\.
- •Fragile T\-only success\(11%\): T itself is marginal\.

No single component is the universal culprit: P disrupts 84% of CCI tasks, M 68%, R 68%, SR 58%\. The*identity*of the harmful component varies across tasks—on 13/19 tasks, at least one component can be safely added while another cannot\.

## 9Discussion

### 9\.1Implications for Agent Design

1. 1\.Evaluate subsets, not just the full suite\.In every setting tested, some proper subset outperforms all\-in \(validated at 8B, directionally consistent elsewhere\)\.
2. 2\.Expect task\-dependent optima\.k∗k^\{\*\}shifts from 1 \(HotpotQA\) to 3 \(GSM8K\)\. Developers should evaluate per\-setting\.
3. 3\.Scale helps but may not eliminate CCI\.The best\-vs\-All\-In gap narrows from 32% \(8B\) to 19% \(70B\) but does not close\.
4. 4\.Greedy selection is unreliable\.With 56\.3% submodularity violations \(median ratioγ=0\.52\\gamma\{=\}0\.52, below the submodular threshold\), practitioners cannot simply add components one by one and stop when marginals become negative\. On GSM8K, greedy from T terminates atK=1K\{=\}1\(accuracy 0\.220\) because all single\-component marginals are negative—missing the true optimum T\+SR\+R \(accuracy 0\.430, a 95% improvement\)\.

### 9\.2Relation to Multi\-Task Learning

CCI bears a functional resemblance to negative transfer in MTL, but the two are mechanistically distinct\. In MTL, interference arises during*training*via gradient conflict over shared parameters\(Yu et al\.,[2020](https://arxiv.org/html/2605.05716#bib.bib33)\)\. In CCI, model weights are frozen; interference arises during*inference*through the shared context window\. We term this*context\-window crowding*: the inference\-time analogue of gradient conflict\.

Despite this mechanistic gap, MTL task\-grouping heuristics\(Fifty et al\.,[2021](https://arxiv.org/html/2605.05716#bib.bib8)\)inspire component\-grouping: find the subset of scaffolding components that do not crowd each other out\. Developing an inference\-time*component affinity*metric—analogous to gradient cosine similarity—is a promising direction\.

### 9\.3Hypothesized CCI Patterns \(Exploratory\)

Across our three primary settings, we observe three qualitatively distinct interference regimes\. We tentatively hypothesize—pending validation on additional task–model combinations—that these correspond to three mechanisms:

- •Type I — Bandwidth Competition\(observed: HotpotQA/8B\): Retrieval\-bottlenecked tasks are disrupted by non\-retrieval components;k∗=1k^\{\*\}\{=\}1\.
- •Type II — Signal Dilution\(observed: GSM8K/8B\): Reasoning tasks tolerate coordinated support \(SR\+R\) but not orthogonal overhead \(P, M\);k∗=3k^\{\*\}\{=\}3\.
- •Type III — Capacity\-Resolved\(observed: HotpotQA/70B\): Sufficient capacity transforms competition into synergy for compatible components\.

We emphasize that each regime is currently supported by a single \(task, model\) cell; the labels are descriptive rather than predictive\. An*a priori*prediction would be: a small model on a coding task \(retrieval\-light, reasoning\-heavy\) should resemble Type II rather than Type I\. Validating—or falsifying—these hypotheses on coding, embodied, and additional QA benchmarks is the central direction for follow\-up work\.

## Limitations

Model coverage\.We validate across two open\-weight families \(Llama, Qwen\) with models≤\\leq70B, plus one closed\-source API model \(Claude Haiku 4\.5\)\. CCI may differ for larger frontier models \(GPT\-4, Claude Opus\) with stronger instruction\-following, where the saturation pattern observed for Haiku might shift toward genuine component complementarity\.

Tool\-use protocol confound\.The Tool Use component includes theFinish\[answer\]submission protocol; configurations without T lack this protocol, partially explaining the zero\-performance baseline\. Shapley attributions for T should be interpreted as reflecting both tool capability and protocol access\.

Joint prompt optimization\.Our prompts are hand\-written and composed independently per component\. Joint optimization via methods like DSPy or OPRO might attenuate CCI by finding component wordings that avoid mutual interference\. However, our paraphrase robustness experiment \(§[7](https://arxiv.org/html/2605.05716#S7)\) shows directional CCI is preserved across three stylistically diverse template variants, and the finding replicates across four independent model checkpoints, arguing against single\-template artifact\.

Multi\-seed coverage\.Only 8B HotpotQA has 10\-seed validation \(pairedtt\-testp=0\.023p\{=\}0\.023, Wilcoxonp=0\.014p\{=\}0\.014\)\. The 70B \(p=0\.13p\{=\}0\.13, explicitly non\-significant\), Qwen, and GSM8K experiments use single seeds\.

70B quantization\.The 70B experiments use 4\-bit NF4 quantization\. CCI effects may be slightly attenuated or amplified\.

Two benchmarks\.HotpotQA and GSM8K cover retrieval QA and math but not multi\-step agentic tasks \(SWE\-bench, WebArena\)\. The tool\-use component differs across benchmarks, confounding cross\-task comparison\.

Step budget\.All experiments use 4 reasoning steps\. Planning and Memory may show different patterns in longer\-horizon tasks\.

Prompt\-level only\.We study static prompt\-based scaffolding\. Non\-prompt implementations \(tool APIs, external memory\) might exhibit different patterns\.

No confirmed mechanism\.Our regression shows CCI is primarily additive at the aggregate level\. A definitive causal account of why components interfere remains open\.

## Ethical Considerations

This work studies composition of publicly available techniques on open\-weight models and established benchmarks\. It introduces no new capabilities, collects no human data, and poses no foreseeable dual\-use risks\. Our findings may reduce unnecessary computation from suboptimal configurations\.

## 10Conclusion

We have presented systematic evidence that prompt\-based scaffolding components in LLM agents exhibit Cross\-Component Interference—negative marginal returns when combined beyond a task\-optimal subset\. In full factorial studies across two benchmarks, three model families \(including a closed\-source API model\), and five scales \(32,000\+ evaluations\), we find:

1. 1\.CCI is statistically confirmed at 8B with converging evidence: pairedtt\-test \(p=0\.023p\{=\}0\.023\), Wilcoxon \(p=0\.014p\{=\}0\.014\), Bayesian \(BF10=3\.2\\text\{BF\}\_\{10\}\{=\}3\.2, moderate\), and large effect size \(Cohen’sdz=0\.87d\_\{z\}\{=\}0\.87\)\. On Llama\-8B/HotpotQA, 5/6 expansions from T degrade performance \(4/6 survive Holm correction\)\.
2. 2\.The optimal subset is task\-dependent \(k∗=1k^\{\*\}\{=\}1vs\.k∗=3k^\{\*\}\{=\}3\) and scale\-sensitive \(gap narrows 32%→\\rightarrow19%→\\rightarrow≈\\approx0% from 8B to 70B to Claude Haiku\), but the best proper subset matches or outperforms All\-In in all 6 conditions tested\.
3. 3\.A 6\-parameter main\-effects model captures 91\.6% of variance \(LOOCVR2=0\.872R^\{2\}\{=\}0\.872\), decisively outperforming pairwise models \(Δ\\DeltaBIC=25\.3\{=\}25\.3\)\. Shapley decomposition reveals Tool Use captures 70% of scaffold value\. 183/325 submodularity violations \(median ratioγ=0\.52\\gamma\{=\}0\.52, CI below 1\) exhibit sign\-flipping, making greedy selection empirically unreliable\.
4. 4\.The qualitative pattern \(T dominates, Memory harmful, All\-In not optimal\) replicates across model families, extends to a closed\-source API model \(with capability\-dependent attenuation\), and is robust to template variants\.

Our recommendation:evaluate scaffolding subsets per task and model before assuming more is better\.

## References

- Anthropic \(2024\)Anthropic\. 2024\.The Claude model family\.[https://www\.anthropic\.com/claude](https://www.anthropic.com/claude)\.
- Battle and Gollapudi \(2024\)Rick Battle and Teja Gollapudi\. 2024\.The unreasonable effectiveness of eccentric automatic prompts\.*arXiv preprint arXiv:2402\.10949*\.
- Chase \(2022\)Harrison Chase\. 2022\.Langchain\.[https://github\.com/langchain\-ai/langchain](https://github.com/langchain-ai/langchain)\.
- Chen et al\. \(2023\)Lu Chen, Siyu Lou, Keyan Zhang, Jin Huang, and Quanshi Zhang\. 2023\.Harsanyinet: Computing accurate shapley values in a single forward propagation\.In*International Conference on Machine Learning \(ICML\)*\.
- Cobbe et al\. \(2021\)Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heather Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman\. 2021\.Training verifiers to solve math word problems\.*arXiv preprint arXiv:2110\.14168*\.
- Dettmers et al\. \(2023\)Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer\. 2023\.Qlora: Efficient finetuning of quantized language models\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.
- Dubey et al\. \(2024\)Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al\-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others\. 2024\.The llama 3 herd of models\.*arXiv preprint arXiv:2407\.21783*\.
- Fifty et al\. \(2021\)Chris Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, and Chelsea Finn\. 2021\.Efficiently identifying task groupings for multi\-task learning\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.
- Fumagalli et al\. \(2023\)Fabian Fumagalli, Maximilian Muschalik, Patrick Kolpaczki, Eyke Hüllermeier, and Barbara Hammer\. 2023\.SHAP\-IQ: Unified approximation of any\-order shapley interactions\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.
- He et al\. \(2024\)Zhiyuan He, Huiqiang Jiang, Zilong Wang, Yuqing Yang, Luna K\. Qiu, and Lili Qiu\. 2024\.Position engineering: Boosting large language models through positional information manipulation\.In*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)*\.
- Huang et al\. \(2025\)Keke Huang, Yimin Shi, Dujian Ding, Yifei Li, Yang Fei, Laks Lakshmanan, and Xiaokui Xiao\. 2025\.Thriftllm: On cost\-effective selection of large language models for classification queries\.*Proceedings of the VLDB Endowment*, 18\.
- Jaroslawicz et al\. \(2025\)Daniel Jaroslawicz, Brendan Whiting, Parth Shah, and Karime Maamari\. 2025\.How many instructions can llms follow at once?*arXiv preprint arXiv:2507\.11538*\.
- Kapoor et al\. \(2024\)Sayash Kapoor, Benedikt Stroebl, Zachary S\. Siegel, Nitya Nadgir, and Arvind Narayanan\. 2024\.Ai agents that matter\.*arXiv preprint arXiv:2407\.01502*\.
- Khattab et al\. \(2023\)Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T\. Joshi, Hanna Mober, and 1 others\. 2023\.Dspy: Compiling declarative language model calls into self\-improving pipelines\.*arXiv preprint arXiv:2310\.03714*\.
- Lauziere et al\. \(2026\)Andrew Lauziere, Jonathan Daugherty, and Taisa Kushner\. 2026\.A regression framework for understanding prompt component impact on llm performance\.*arXiv preprint arXiv:2603\.26830*\.
- Li et al\. \(2025\)Xiaomin Li, Zhou Yu, Zhiwei Zhang, Xupeng Chen, Ziji Zhang, Yingying Zhuang, Narayanan Sadagopan, and Anurag Beniwal\. 2025\.When thinking fails: The pitfalls of reasoning for instruction\-following in llms\.*arXiv preprint arXiv:2505\.11423*\.
- Liu et al\. \(2024\)Nelson F\. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang\. 2024\.Lost in the middle: How language models use long contexts\.*Transactions of the Association for Computational Linguistics \(TACL\)*\.
- McKenzie et al\. \(2023\)Ian R\. McKenzie, Alexander Lyzhov, Michael Pieler, Alicia Parrish, Aaron Mueller, Ameya Prabhu, Euan McLean, Aaron Kirtland, Alexis Rofin, Matthew Groh, and 1 others\. 2023\.Inverse scaling: When bigger isn’t better\.*Transactions on Machine Learning Research \(TMLR\)*\.
- Mizrahi et al\. \(2024\)Moran Mizrahi, Guy Kaplan, Dan Malkin, Rotem Dror, Dafna Shahaf, and Gabriel Stanovsky\. 2024\.State of what art? a call for multi\-prompt llm evaluation\.*Transactions of the Association for Computational Linguistics \(TACL\)*\.
- Mohammadi \(2024\)Behnam Mohammadi\. 2024\.Explaining large language models decisions using shapley values\.*arXiv preprint arXiv:2404\.01332*\.
- Qi et al\. \(2026\)Yunjia Qi, Hao Peng, Xintong Shi, Amy Xin, Xiaozhi Wang, Bin Xu, Lei Hou, and Juanzi Li\. 2026\.On the paradoxical interference between instruction\-following and task solving\.*arXiv preprint arXiv:2601\.22047*\.
- Sclar et al\. \(2024\)Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr\. 2024\.Quantifying language models’ sensitivity to spurious features in prompt design\.In*International Conference on Learning Representations \(ICLR\)*\.
- Shinn et al\. \(2023\)Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao\. 2023\.Reflexion: Language agents with verbal reinforcement learning\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.
- Sumers et al\. \(2024\)Theodore R\. Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L\. Griffiths\. 2024\.Cognitive architectures for language agents\.*arXiv preprint arXiv:2309\.02427*\.
- Team \(2025\)Qwen Team\. 2025\.Qwen2\.5 technical report\.*arXiv preprint arXiv:2412\.15115*\.
- Wang et al\. \(2023\)Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar\. 2023\.Voyager: An open\-ended embodied agent with large language models\.*arXiv preprint arXiv:2305\.16291*\.
- Wei et al\. \(2022\)Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V\. Le, and Denny Zhou\. 2022\.Chain\-of\-thought prompting elicits reasoning in large language models\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.
- Wen et al\. \(2024\)Bosi Wen, Pei Ke, Xiaotao Gu, Lindong Wu, Hao Huang, Jinfeng Zhou, Wenchuan Li, Bin Hu, Wen Gao, Jiaxin Xu, Yiming Liu, Jie Tang, Hongning Wang, and Minlie Huang\. 2024\.Benchmarking complex instruction\-following with multiple constraints composition\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.
- Xie et al\. \(2024\)Shan Xie, Man Luo, Chadly Daniel Stern, Mengnan Du, and Lu Cheng\. 2024\.Demoshapley: Valuation of demonstrations for in\-context learning\.*arXiv preprint arXiv:2410\.07523*\.
- Yang et al\. \(2024\)Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V\. Le, Denny Zhou, and Xinyun Chen\. 2024\.Large language models as optimizers\.In*International Conference on Learning Representations \(ICLR\)*\.
- Yang et al\. \(2018\)Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W\. Cohen, Ruslan Salakhutdinov, and Christopher D\. Manning\. 2018\.Hotpotqa: A dataset for diverse, explainable multi\-hop question answering\.In*Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)*\.
- Yao et al\. \(2023\)Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao\. 2023\.React: Synergizing reasoning and acting in language models\.In*International Conference on Learning Representations \(ICLR\)*\.
- Yu et al\. \(2020\)Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn\. 2020\.Gradient surgery for multi\-task learning\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.
- Yuksekgonul et al\. \(2024\)Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou\. 2024\.Textgrad: Automatic “differentiation” via text\.*arXiv preprint arXiv:2406\.07496*\.

## Appendix AFull 32\-Configuration Results

Table 8:All 32 configurations on HotpotQA \(100 tasks, Llama\-3\.1\-8B,F1F\_\{1\}\)\.
## Appendix BFull 70B Results

Table 9:All 32 configurations on HotpotQA \(100 tasks, Llama\-3\.1\-70B\-4bit,F1F\_\{1\}\)\.
## Appendix CPairwise Interaction Model Details

The pairwise interaction regression uses±1\\pm 1spin encoding for component presence/absence\. With intercept \+ 5 main effects \+ 10 pairwise couplings = 16 parameters fit to 32 observations:

- •RawR2=0\.937R^\{2\}=0\.937; AdjustedR2=0\.878R^\{2\}=0\.878; LOOCVR2=0\.748R^\{2\}=0\.748
- •Strongest positive coupling:JJ\(SR, R\)=\+0\.031=\+0\.031
- •Strongest negative coupling:JJ\(T, M\)=−0\.019=\-0\.019
- •JJeigenvalue structure: 3 negative \+ 2 positive eigenvalues→\\rightarrownon\-convex landscape

The LOOCVR2R^\{2\}dropping to 0\.748 \(vs\. 0\.872 for main\-effects\-only\) indicates the pairwise terms overfit\. AIC: main\-effects−130\.5\-130\.5vs\. pairwise−119\.9\-119\.9; BIC: main\-effects−120\.2\-120\.2vs\. pairwise−95\.0\-95\.0\. Both information criteria favor the simpler model\.

We retain the pairwise model in this appendix because the coupling structure provides interpretive value \(which component pairs synergize vs\. conflict\), even though it does not improve out\-of\-sample prediction\.

## Appendix DComponent Prompt Templates

Each component adds a labeled block to the system prompt\. The HotpotQA templates \(character counts in parentheses\) are:

- •Planning \(295 chars\):“Before each action, plan your information\-gathering strategy: \(1\) What information do I still need? \(2\) Most efficient way to find it? \(3\) How does current info connect?”
- •Tool Use \(447 chars\):Defines Search\[query\], Lookup\[keyword\], Finish\[answer\] with usage instructions\.
- •Memory \(302 chars\):“Track information gathered: facts confirmed, entities searched, connections identified, information missing\.”
- •Structured Reasoning \(309 chars\):“For each step: Evidence, Gap Analysis, Reasoning Chain, Next Step\.”
- •Reflection \(258 chars\):“After each observation: Did I get useful info? Am I closer? Should I search differently? Time to submit?”

## Appendix ENegative Mechanistic Results

For transparency, we report experiments that did not produce positive results:

- •Hidden\-state drift:Linear probes achieved near\-chance accuracy predictingKKfrom hidden states\.
- •Information entropy:Output token entropy did not correlate with per\-question performance\.
- •Theoretical CCI prediction:Component dataflow graph predictions showed near\-zero correlation with empirical outcomes\.
- •Non\-monotone decline:Performance\-vs\-KKis better fit by a smooth decline than a piecewise threshold model\.