When Does Multi-Agent Collaboration Help? An Entropy Perspective

# When Does Multi-Agent Collaboration Help? An Entropy Perspective
Source: [https://arxiv.org/html/2602.04234](https://arxiv.org/html/2602.04234)
Yuxuan Zhao1,2Sijia Chen2,Ningxin Su2 1Yantai Research Institute of Harbin Engineering University 2The Hong Kong University of Science and Technology \(Guangzhou\) Project page:[https://multiagent\-entropy\.github\.io/](https://multiagent-entropy.github.io/)

###### Abstract

Multi\-agent systems \(MAS\) have emerged as a prominent paradigm for leveraging large language models \(LLMs\) to tackle complex tasks\. However, the mechanisms governing the effectiveness of MAS built upon publicly available LLMs, specifically the underlying rationales for their success or failure, remain largely unexplored\. In this paper, we revisit MAS through the perspective ofentropy, considering both intra\- and inter\-agent dynamics by investigating entropy transitions during problem\-solving across various topologies, six reasoning benchmarks, and two agentic tasks\. By analyzing 245 features spanning token\-, agent\-, and round\-level entropy, we counterintuitively find that a single agent outperforms MAS in approximately 43\.3% of cases, and that entropy dynamics are largely determined during the first round of interaction\. Furthermore, we provide three key observations: 1\)Certainty Preference: peak entropy directly harms and stable entropy directly benefits MAS correctness; 2\)Base Entropy: base models with lower entropy during problem\-solving causally drive MAS performance; and 3\)Task Awareness: entropy dynamics of MAS play varying roles across different tasks\. Building on these insights, we introduce a simple yet effective algorithm, theEntropy Judger, to select solutions from MAS’s pass@kkresults, leading to consistent accuracy improvements across all MAS configurations and tasks\. Our source code is available at[this https URL](https://github.com/AgenticFinLab/multiagent-entropy)\.

## 1Introduction

Multi\-agent systems \(MAS\), with each agent built upon large language models \(LLMs\), are broadly applied in diverse domains\(Duet al\.,[2023](https://arxiv.org/html/2602.04234#bib.bib33); Honget al\.,[2024](https://arxiv.org/html/2602.04234#bib.bib36); Liuet al\.,[2024](https://arxiv.org/html/2602.04234#bib.bib58); Paglieriet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib60); Danget al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib32)\)and even regarded as the only choice for problem\-solving\(Zhanget al\.,[2024a](https://arxiv.org/html/2602.04234#bib.bib37)\)\. However, it remains largely unexplored whether MAS, particularly those built upon open\-source LLMs, can outperform their single\-agent counterparts, and what underlies their effectiveness\.

Existing work has observed that single\-agent systems \(SAS\) can match or even surpass MAS on certain tasks\(Gaoet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib34); Tran and Kiela,[2026](https://arxiv.org/html/2602.04234#bib.bib25)\), and the failures of MAS often stem from communication breakdowns, inter\-agent misalignment, and insufficient verification\(Cemriet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib30)\)\. In particular, recent work establishes scaling principles for MAS through quantitative analysis\(Kimet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib27)\)\. However, these studies are primarily conditioned on simple metrics such as accuracy, latency, and cost without providing a deeper understanding of the underlying mechanisms\.

Entropy has emerged as a key perspective for understanding LLM reasoning, ranging from reinforcement learning \(RL\) analysis\(Cuiet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib12)\)to training algorithms that balance exploration and exploitation\(Zhanget al\.,[2025a](https://arxiv.org/html/2602.04234#bib.bib20); Zhuet al\.,[2025b](https://arxiv.org/html/2602.04234#bib.bib19)\)\. Notably, even individual observations of correlations between entropy and accuracy have spurred further research in regularization, advantage shaping, and token updates\(Chenet al\.,[2025a](https://arxiv.org/html/2602.04234#bib.bib6); Chenget al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib9); Wanget al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib8)\)\.

Therefore, for LLM\-based MAS, which are inherently complex and exhibit uncertainty at the token, agent, and other levels, it is crucial to build a comprehensive relationship between entropy and reasoning reliability\. Although recent works approach it from an information\-theoretic perspective\(Heet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib26); Tran and Kiela,[2026](https://arxiv.org/html/2602.04234#bib.bib25)\), they are limited to specific architectures or token\-budget comparisons\.

In this paper, we revisit MAS by investigating the entire lifecycle of entropy across diverse levels, steps, and phases involved in reasoning, under varying MAS topologies and tasks\. Specifically, by mining the fine\-grained relationship between large\-scale information entropy, derived from both intra\-agent and inter\-agent interactions, and MAS performance, we demonstrate that MAS effectiveness is largely determined by early\-round entropy dynamics, with peak entropy universally harmful\. We further validate through causal inference that these entropy\-performance associations reflect genuine causal mechanisms rather than mere correlations\. In summary, our key contributions are:

- •A systematic study of entropy dynamics across six reasoning tasks and two agentic tasks, under four MAS topologies, analyzing 245 features at token, agent, and interaction\-round levels;
- •The counterintuitive finding that SAS outperform MAS in approximately 43\.3% of cases, underscoring the trade\-off between system complexity and performance;
- •Three insights on entropy behavior: Certainty Preference \(peak entropy directly harms and stable low entropy directly benefits MAS correctness\), Base Entropy \(lower base model entropy causally drives MAS performance\), and Task Awareness \(entropy patterns differ across tasks\); these findings generalize to agentic settings, where tool\-call entropy and first\-round inter\-agent dispersion jointly constrain MAS correctness;
- •TheEntropy Judger, which selects high\-quality outputs from MAS pass@kkresults and consistently boosts accuracy across all configurations and tasks\.

## 2Related Work

Large Language Model\-based Multi\-Agent Systems \(MAS\)decompose complex tasks into specialized agents that interact through diverse coordination topologies\(Chenet al\.,[2024](https://arxiv.org/html/2602.04234#bib.bib31); Zhanget al\.,[2024b](https://arxiv.org/html/2602.04234#bib.bib28); Danget al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib32); Zhanget al\.,[2025d](https://arxiv.org/html/2602.04234#bib.bib54)\), thereby enhancing problem\-solving capabilities\. Beyond improving accuracy and efficiency of MAS, several studies examine the conditions under which MAS are effective\. Scaling analyses show that gains from adding agents or varying coordination structures are often offset by communication overhead and inter\-agent misalignment, yielding diminishing or even negative returns\(Kimet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib27)\)\. Recent work identifies key failure modes in MAS, such as insufficient verification and communication breakdowns\(Cemriet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib30)\)\. More recently, SAS equipped with rich skill libraries have been shown to match or surpass MAS in both accuracy and efficiency\(Li,[2026](https://arxiv.org/html/2602.04234#bib.bib29)\)\. While these works inform MAS practice, they rely on simple metrics that fail to capture system complexity and therefore cannot uncover the underlying rationales governing MAS effectiveness\. Recent information\-theoretic analysis\(Heet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib26); Tran and Kiela,[2026](https://arxiv.org/html/2602.04234#bib.bib25)\)offers deeper insight but is limited to specific architectures or token budget comparison\.

However, analyzingentropy in LLM reasoninghas deepened our understanding of the underlying reasoning process\(Cuiet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib12); Zhuet al\.,[2025b](https://arxiv.org/html/2602.04234#bib.bib19)\)\. In RL training, reducing entropy sharpens output distribution and improve accuracy\(Agarwalet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib11); Karan and Du,[2026](https://arxiv.org/html/2602.04234#bib.bib3)\), but this benefit is highly dependent on the base model capabilities, and aggressive entropy reduction may triggerentropy collapse, wherein the model becomes overconfident and converges to suboptimal policies\(Yueet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib21); Zhanget al\.,[2025c](https://arxiv.org/html/2602.04234#bib.bib7)\)\. To mitigate this, recent work proposes entropy\-intervention methods, including entropy regularization\(Zhanget al\.,[2025b](https://arxiv.org/html/2602.04234#bib.bib16); Chenet al\.,[2025a](https://arxiv.org/html/2602.04234#bib.bib6)\), entropy\-based advantage shaping\(Chenget al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib9)\), and entropy\-guided token updates\(Wanget al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib8)\)\. At test time, entropy also signals uncertainty to dynamically adjust reasoning depth and direction\(Liet al\.,[2026](https://arxiv.org/html/2602.04234#bib.bib14); Yanget al\.,[2026](https://arxiv.org/html/2602.04234#bib.bib2)\)\. While these studies enhance single\-agent reasoning by probing entropy, they largely overlook how entropy propagates across multiple interacting agents\. To address this gap, we analyze entropy dynamics in LLM\-based MAS to explain when and why they succeed\.

## 3Preliminaries

LetMbaseM\_\{\\text\{base\}\}be an LLM with parametersθ\\theta, defining a distributionπθ​\(v∣s\)\\pi\_\{\\theta\}\(v\\mid s\)over tokensv∈𝒱v\\in\\mathcal\{V\}given statess\. We formalize a Multi\-Agent System \(MAS\) as𝖬=\(A,G,R\)\\mathsf\{M\}=\(A,G,R\), whereAAis a set of agents each instantiated fromMbaseM\_\{\\text\{base\}\},G=\(A,E\)G=\(A,E\)is a directed interaction graph, andRRis the number of interaction rounds\. In each roundrr, agentaagenerates a reasoning trajectoryτa\(r\)∼πθ\(⋅∣ℋa\(r\)\)\\tau\_\{a\}^\{\(r\)\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid\\mathcal\{H\}\_\{a\}^\{\(r\)\}\), where the contextℋa\(r\)\\mathcal\{H\}\_\{a\}^\{\(r\)\}is determined byGGas follows, and the system outputs a final predictiony^\\hat\{y\}\.

Single Agent System \(SAS\)\.A=\{a\}A=\\\{a\\\},G=\(\{a\},∅\)G=\(\\\{a\\\},\\emptyset\),ℋa\(r\)=\{τa\(r′\)\}r′<r\\mathcal\{H\}\_\{a\}^\{\(r\)\}=\\\{\\tau\_\{a\}^\{\(r^\{\\prime\}\)\}\\\}\_\{r^\{\\prime\}<r\},y^=τa\(R\)\\hat\{y\}=\\tau\_\{a\}^\{\(R\)\}\.

Sequential\.A=\{a1,…,aN\}A=\\\{a\_\{1\},\\ldots,a\_\{N\}\\\},GGis a directed patha1→⋯→aNa\_\{1\}\\to\\cdots\\to a\_\{N\}:ℋaj\(r\)=\{τa\(r′\)\}a∈A,r′<r\\mathcal\{H\}\_\{a\_\{j\}\}^\{\(r\)\}=\\\{\\tau\_\{a\}^\{\(r^\{\\prime\}\)\}\\\}\_\{a\\in A,\\,r^\{\\prime\}<r\}ifj=1j=1, elseℋaj\(r\)=\{τaj−1\(r\)\}\\mathcal\{H\}\_\{a\_\{j\}\}^\{\(r\)\}=\\\{\\tau\_\{a\_\{j\-1\}\}^\{\(r\)\}\\\};y^=τaN\(R\)\\hat\{y\}=\\tau\_\{a\_\{N\}\}^\{\(R\)\}\.

Centralized\.A=Awork∪\{ao\}A=A\_\{\\text\{work\}\}\\cup\\\{a\_\{o\}\\\},GGis a star centered at orchestratoraoa\_\{o\}:ℋwi\(r\)=\{τao\(r−1\)\}\\mathcal\{H\}\_\{w\_\{i\}\}^\{\(r\)\}=\\\{\\tau\_\{a\_\{o\}\}^\{\(r\-1\)\}\\\},ℋao\(r\)=\{τwi\(r\)\}wi∈Awork\\mathcal\{H\}\_\{a\_\{o\}\}^\{\(r\)\}=\\\{\\tau\_\{w\_\{i\}\}^\{\(r\)\}\\\}\_\{w\_\{i\}\\in A\_\{\\text\{work\}\}\},y^=τao\(R\)\\hat\{y\}=\\tau\_\{a\_\{o\}\}^\{\(R\)\}\.

Debate\.A=\{a1,…,aN\}A=\\\{a\_\{1\},\\ldots,a\_\{N\}\\\},GGis a directed acyclic graph overAAwith total ordera1≺⋯≺aNa\_\{1\}\\prec\\cdots\\prec a\_\{N\}, where eachaja\_\{j\}observes all predecessorsaj′a\_\{j^\{\\prime\}\}\(j′<jj^\{\\prime\}<j\) within the current round and all agents from prior rounds:ℋaj\(r\)=\{τaj′\(r\)\}j′<j∪\{τa\(r′\)\}a∈A,r′<r\\mathcal\{H\}\_\{a\_\{j\}\}^\{\(r\)\}=\\\{\\tau\_\{a\_\{j^\{\\prime\}\}\}^\{\(r\)\}\\\}\_\{j^\{\\prime\}<j\}\\cup\\\{\\tau\_\{a\}^\{\(r^\{\\prime\}\)\}\\\}\_\{a\\in A,\\,r^\{\\prime\}<r\};y^\\hat\{y\}is determined by majority voting over\{τa\(R\)\}a∈A\\\{\\tau\_\{a\}^\{\(R\)\}\\\}\_\{a\\in A\}\.

Hybrid\.A=Awork∪\{ao\}A=A\_\{\\text\{work\}\}\\cup\\\{a\_\{o\}\\\}as in Centralized, with expanded worker context:ℋwi\(r\)=\{τao\(r−1\)\}∪\{τw\(r−1\)\}w∈Awork∪\{τwj\(r\)\}j<i\\mathcal\{H\}\_\{w\_\{i\}\}^\{\(r\)\}=\\\{\\tau\_\{a\_\{o\}\}^\{\(r\-1\)\}\\\}\\cup\\\{\\tau\_\{w\}^\{\(r\-1\)\}\\\}\_\{w\\in A\_\{\\text\{work\}\}\}\\cup\\\{\\tau\_\{w\_\{j\}\}^\{\(r\)\}\\\}\_\{j<i\},ℋao\(r\)=\{τw\(r\)\}w∈Awork\\mathcal\{H\}\_\{a\_\{o\}\}^\{\(r\)\}=\\\{\\tau\_\{w\}^\{\(r\)\}\\\}\_\{w\\in A\_\{\\text\{work\}\}\},y^=τao\(R\)\\hat\{y\}=\\tau\_\{a\_\{o\}\}^\{\(R\)\}\.

##### Entropy Metrics\.

Building on the established link between entropy dynamics and LLM reasoning\(Zhanget al\.,[2025a](https://arxiv.org/html/2602.04234#bib.bib20); Cuiet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib12)\), we extend these insights to MAS through hierarchical entropy metrics\. Specifically, letst\(i,a,r\)s\_\{t\}^\{\(i,a,r\)\}denote the decoding state preceding thett\-th token of agentaain roundrrfor sampleii\. The token\-level entropy isHt\(i,a,r\)=−∑v∈𝒱πθ​\(v∣st\(i,a,r\)\)​log⁡πθ​\(v∣st\(i,a,r\)\)H\_\{t\}^\{\(i,a,r\)\}=\-\\sum\_\{v\\in\\mathcal\{V\}\}\\pi\_\{\\theta\}\(v\\mid s\_\{t\}^\{\(i,a,r\)\}\)\\log\\pi\_\{\\theta\}\(v\\mid s\_\{t\}^\{\(i,a,r\)\}\)\. During experiments, we logHt\(i,a,r\)H\_\{t\}^\{\(i,a,r\)\}, latency, and token costs for every agent to analyze hierarchical entropy dynamics\.

## 4Exploring MAS with Entropy Dynamics

While prior work investigates failure modes of MAS\(Cemriet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib30); Kimet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib27)\), it relies on proprietary models and aggregate metrics, such as accuracy, latency, and cost, overlooking how entropy evolves within and across agents\. We address this gap by analyzing entropy dynamics in open\-source LLM\-based MAS across diverse topologies and tasks\.

### 4\.1Evaluation Protocol

Small, open\-source LLMs enable cost\-effective multi\-agent collaboration through specialized task allocation and coordination\(Shenet al\.,[2024](https://arxiv.org/html/2602.04234#bib.bib66)\)\. Their open access to token\-level probabilities enables direct entropy computation, which is critical for agent decision\-making\. More discussion can be found in Appendix[A](https://arxiv.org/html/2602.04234#A1)\. To this end, in contrast to prior work which evaluates only proprietary API\-based models\(Kimet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib27)\), our study focuses on publicly available LLMs, including the LLaMA series \(3\.1\-8B\-Instruct, 3\.2\-3B\-Instruct\)\(Grattafioriet al\.,[2024](https://arxiv.org/html/2602.04234#bib.bib71)\)and the Qwen3 series \(0\.6B, 4B, 8B\)\(Yanget al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib72)\)\. We evaluate across six benchmarks:GSM8K\(Cobbeet al\.,[2021](https://arxiv.org/html/2602.04234#bib.bib74)\),MATH500\(Hendryckset al\.,[2021b](https://arxiv.org/html/2602.04234#bib.bib75)\),AIME2024, andAIME2025for mathematics;HumanEval\(Chen and others,[2021](https://arxiv.org/html/2602.04234#bib.bib77)\)for code generation; andMMLU\(Hendryckset al\.,[2021a](https://arxiv.org/html/2602.04234#bib.bib76)\)for knowledge question\-answering \(Q&A\)\. All systems use the sameMbaseM\_\{\\text\{base\}\}withR=2R=2interaction rounds\. Additional experimental details are provided in Appendix[B](https://arxiv.org/html/2602.04234#A2)\. We further extend the evaluation to two agentic benchmarks:GAIA\(Mialonet al\.,[2024](https://arxiv.org/html/2602.04234#bib.bib55)\)andFinanceAgent\(Bigeardet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib61)\)in Appendix[E](https://arxiv.org/html/2602.04234#A5)\.

### 4\.2Measuring Reasoning with Entropy and Beyond

Entropy has proven effective for analyzing single\-agent reasoning\(Zhuet al\.,[2025b](https://arxiv.org/html/2602.04234#bib.bib19); Zhanget al\.,[2025a](https://arxiv.org/html/2602.04234#bib.bib20)\)\. Recent work has even leveraged entropy\-based features to train simple machine learning models that predict LLM correctness\(Buffa and Del Corro,[2026](https://arxiv.org/html/2602.04234#bib.bib22)\)\. However, this approach is limited to a single LLM and computes entropy only from the top\-20 token probabilities, yielding just 11 entropy\-related features\. We extend entropy analysis to MAS by designing hierarchical entropy features that capture how entropy evolves across agents and rounds\. Specifically, for each sample, we extract 254 features by aggregating the logged traces across all agents and rounds:

Entropy features\(ℱE\\mathcal\{F\}\_\{E\}, 239 features\) measure entropy across hierarchical levels:

- •Agent\-level statisticscapture per\-agent reasoning trajectories, including their statistical properties, variations across rounds, and inter\-agent entropy divergence;
- •Round\-level dynamicstrack aggregate entropy metrics for each round and their relative changes;
- •Sample\-level statisticsaggregate entropy measures across all agents for a given sample;
- •System\-level aggregationprovides global entropy measures across different topologies;
- •Base\-model entropy\(ℱbase\-E\\mathcal\{F\}\_\{\\text\{base\-E\}\}, 17 features\) additionally captures entropy characteristics ofMbaseM\_\{\\text\{base\}\}and quantifies shifts in entropy betweenMbaseM\_\{\\text\{base\}\}and MAS;

Computational metrics\(ℱC\\mathcal\{F\}\_\{C\}, 15 features\) capture non\-entropy\-related quantities, including reasoning time, token usage, inference counts, andMbaseM\_\{\\text\{base\}\}correctness \(ℱbase\-C\\mathcal\{F\}\_\{\\text\{base\-C\}\}\), at the same hierarchical levels\.

Excluding 9 experimental identifier columns yields 245 trainable features\. We define three feature groups to isolate the influence ofMbaseM\_\{\\text\{base\}\}on MAS performance: \(1\)MAS only\(𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\},d=224d=224\):𝒢MAS=\(ℱE∪ℱC\)∖\(ℱbase\-E∪ℱbase\-C\)\\mathcal\{G\}\_\{\\text\{MAS\}\}=\(\\mathcal\{F\}\_\{E\}\\cup\\mathcal\{F\}\_\{C\}\)\\setminus\(\\mathcal\{F\}\_\{\\text\{base\-E\}\}\\cup\\mathcal\{F\}\_\{\\text\{base\-C\}\}\), capturing entropy dynamics intrinsic to multi\-agent interaction; \(2\)Base entropy\(𝒢base\-H\\mathcal\{G\}\_\{\\text\{base\-H\}\},d=241d=241\):𝒢base\-H=𝒢MAS∪ℱbase\-E\\mathcal\{G\}\_\{\\text\{base\-H\}\}=\\mathcal\{G\}\_\{\\text\{MAS\}\}\\cup\\mathcal\{F\}\_\{\\text\{base\-E\}\}, evaluating the impact ofMbaseM\_\{\\text\{base\}\}entropy on MAS performance; \(3\)Base full\(𝒢base\-full\\mathcal\{G\}\_\{\\text\{base\-full\}\},d=245d=245\):𝒢base\-full=𝒢base\-H∪ℱbase\-C\\mathcal\{G\}\_\{\\text\{base\-full\}\}=\\mathcal\{G\}\_\{\\text\{base\-H\}\}\\cup\\mathcal\{F\}\_\{\\text\{base\-C\}\}, directly measuring howMbaseM\_\{\\text\{base\}\}’s reasoning capability conditions or limits MAS effectiveness\. Detailed feature descriptions, together with a discussion of feature redundancy, are provided in Appendix[C](https://arxiv.org/html/2602.04234#A3)\.

### 4\.3Mining Effectiveness of MAS

MAS built on LLMs inherently exhibit uncertainty during individual reasoning, and their interactions among agents compound this effect\(Cemriet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib30)\)\. Prior work largely relies on simple metrics that fail to capture internal entropy dynamics or identify the factors underlying MAS failure\.

To address this, we leverage the hierarchical entropy features defined in Section[4\.2](https://arxiv.org/html/2602.04234#S4.SS2)to reformulate MAS evaluation as a supervised learning problem: predicting per\-sample correctnessyi∈\{0,1\}y\_\{i\}\\in\\\{0,1\\\}from entropy traces𝐱i∈ℝd\\mathbf\{x\}\_\{i\}\\in\\mathbb\{R\}^\{d\}, where𝐱i\\mathbf\{x\}\_\{i\}is the feature vector extracted from sampleii’s MAS execution logs according to one of the three feature groups\. We train an ensemble classifier, theEntropy Judger, by averaging the predicted probabilities of XGBoost\(Chen,[2016](https://arxiv.org/html/2602.04234#bib.bib78)\)and LightGBM\(Keet al\.,[2017](https://arxiv.org/html/2602.04234#bib.bib79)\), yielding a final predictionf​\(𝐱i\)∈\[0,1\]f\(\\mathbf\{x\}\_\{i\}\)\\in\[0,1\]\. This formulation enables data\-driven discovery of fine\-grained factors governing MAS effectiveness\. Beyond binary prediction, theEntropy Judgersupports label\-free pass@kkselection by choosing the candidate with the highest predicted correctness; details are in Appendix[I](https://arxiv.org/html/2602.04234#A9)\. To interpret learned patterns, we perform SHAP analysis\(Lundberg and Lee,[2017](https://arxiv.org/html/2602.04234#bib.bib80)\)on both XGBoost and LightGBM\. For each featurejj, we extract two metrics: \(1\)mean feature importanceI¯j\\bar\{I\}\_\{j\}, the average min\-max\-normalized importance across both models; and \(2\)SHAP correlationρj\\rho\_\{j\}, the average Pearson correlation between feature values and their SHAP attributions\. A positiveρj\\rho\_\{j\}indicates that higher feature values increase predicted correctness\. Formal definitions are in Appendix[B\.4](https://arxiv.org/html/2602.04234#A2.SS4)\. To distinguish correlation from causation, we further conduct a causal inference analysis in Section[5\.4](https://arxiv.org/html/2602.04234#S5.SS4)to validate these findings\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/x1.png)Figure 1:Accuracy comparison of SAS and MAS across models and datasets\. For brevity, LLaMA\-3\.2\-3B\-Instruct and LLaMA\-3\.1\-8B\-Instruct are denoted as L\-3 and L\-8, while Qwen3\-0\.6B/4B/8B are denoted as Q\-0\.6/Q\-4/Q\-8\. Thebasedenotes the accuracy of a singleMbaseM\_\{\\text\{base\}\}\.
### 4\.4Examining Entropy Impacts on MAS

##### MAS does not always outperform SAS\.

Conventional assumptions suggest that more agents improve MAS performance\(Zhanget al\.,[2024a](https://arxiv.org/html/2602.04234#bib.bib37)\)\. However, consistent with recent findings\(Kimet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib27); Tran and Kiela,[2026](https://arxiv.org/html/2602.04234#bib.bib25)\), we show that MAS does not universally surpass SAS, and we substantially extend this observation to open\-source LLMs across a broader range of tasks\. As shown in Figure[1](https://arxiv.org/html/2602.04234#S4.F1), SAS matches or exceeds the performance of at least one MAS topology in 26/30 cases across 5 models and 6 datasets\. Specifically, SAS achieves the highest accuracy in 13 cases \(43\.3%\), surpassing the average MAS accuracy by 6\.28%, particularly on math tasks and with smaller models\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/x2.png)Figure 2:Base\-model entropy limits MAS effectiveness\. The left two subfigures show results for LLaMA; the right two for Qwen\. \(a\) Relationship between feature values and SHAP values for the most important entropy features on𝒢base\-H\\mathcal\{G\}\_\{\\text\{base\-H\}\}, sorted byI¯j\\bar\{I\}\_\{j\}and annotated withρj\\rho\_\{j\}\. \(b\) MAS performance across deciles ofMbaseM\_\{\\text\{base\}\}entropy:MbaseM\_\{\\text\{base\}\}entropy is partitioned into ten equal\-sized bins, and average MAS accuracy, aggregated over datasets and model sizes, is computed per bin\. Additionally, the averageMbaseM\_\{\\text\{base\}\}entropy and accuracy across all datasets are overlaid as markers\.
##### Base model entropy limits MAS effectiveness\.

Prior work shows MAS performance depends onMbaseM\_\{\\text\{base\}\}capability\(Zhanget al\.,[2025d](https://arxiv.org/html/2602.04234#bib.bib54)\), a trend we also observe in Figure[1](https://arxiv.org/html/2602.04234#S4.F1); moreover, we find thatMbaseM\_\{\\text\{base\}\}entropy further constrains MAS effectiveness\. On𝒢base\-H\\mathcal\{G\}\_\{\\text\{base\-H\}\}, the top predictors areMbaseM\_\{\\text\{base\}\}’s total token\-level entropy and answer length: for LLaMA, total token count \(ρ≈−0\.47\\rho\\approx\-0\.47,I¯=1\.0\\bar\{I\}=1\.0\) and entropy \(ρ≈−0\.73\\rho\\approx\-0\.73,I¯≈0\.72\\bar\{I\}\\approx 0\.72\); for Qwen3, answer token count \(ρ≈−0\.18\\rho\\approx\-0\.18,I¯≈0\.58\\bar\{I\}\\approx 0\.58\) and entropy \(ρ≈−0\.64\\rho\\approx\-0\.64,I¯≈0\.36\\bar\{I\}\\approx 0\.36\)\. Critically, higherMbaseM\_\{\\text\{base\}\}entropy consistently reduces MAS accuracy across both families, as shown in Figure[2](https://arxiv.org/html/2602.04234#S4.F2)b,d, with performance dropping sharply when entropy exceeds 100; additional results appear in Appendix[F\.1](https://arxiv.org/html/2602.04234#A6.SS1)\. Notably, despite this shared trend, entropy scales differ: LLaMA operates in low\-entropy ranges \(0\-100\) but achieves lower accuracy, while Qwen uses higher entropy \(100\-1,000\) yet performs better\. This reflects divergent reasoning styles, as Qwen verifies and refines its answers before finalizing, generating self\-correcting trajectories that suppress error propagation in MAS despite higher entropy, whereas LLaMA tends to reuse others’ answers without verification, leading to uncontrolled error propagation\. We use an example question to illustrate this contrast in Appendix[H](https://arxiv.org/html/2602.04234#A8)\. We treat entropy primarily as a predictive feature for MAS correctness rather than a universal measure of uncertainty; detailed calibration analyses showing its dependence on model family and task difficulty are provided in Appendix[J](https://arxiv.org/html/2602.04234#A10)\. We further examine how RL fine\-tuning reshapes this entropy\-performance relationship in Appendix[G](https://arxiv.org/html/2602.04234#A7)\.

##### MAS mainly fails on inter\-agent misalignment\.

Prior work identifies inter\-agent misalignment as a key cause of MAS failure\(Cemriet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib30)\)\. We deepen this understanding by analyzing fine\-grained entropy dynamics within MAS\. On𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}, as shown in Figure[3](https://arxiv.org/html/2602.04234#S4.F3), we examine the top entropy\-related features ranked byI¯\\bar\{I\}\. For Qwen, failure is driven by high entropy variance across agents during problem solving \(ρ≈−0\.92\\rho\\approx\-0\.92,I¯≈0\.83\\bar\{I\}\\approx 0\.83\) and strong agent disagreement in round 1 \(ρ≈−0\.87\\rho\\approx\-0\.87,I¯≈0\.47\\bar\{I\}\\approx 0\.47\), indicating that early divergence leads to increasingly incompatible reasoning trajectories\. Figure[3](https://arxiv.org/html/2602.04234#S4.F3)d further shows that correctly solved MAS samples \(MAS Positive\) cluster at low sample\-level entropy variance and low round\-1 per\-agent entropy variance, whereas SAS solves samples correctly even at higher entropy variance, indicating thatMAS imposes a stricter entropy constraint than SAS for successful problem\-solving\. In contrast, LLaMA failures are characterized by verbose and uncertain final answers, with answer\-token count \(ρ≈−0\.63\\rho\\approx\-0\.63,I¯≈0\.78\\bar\{I\}\\approx 0\.78\) and minimum answer\-token entropy \(ρ≈−0\.78\\rho\\approx\-0\.78,I¯≈0\.56\\bar\{I\}\\approx 0\.56\) as dominant predictors\. Notably, LLaMA round\-2 entropy positively correlates with correctness, unlike Qwen, which relies on early convergence\. More analysis can be found in the Appendix[F\.2](https://arxiv.org/html/2602.04234#A6.SS2)\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/x3.png)Figure 3:MAS mainly fails on inter\-agent misalignment\. The left two subfigures show results for LLaMA; the right two for Qwen\. \(a\) Same as Figure[2](https://arxiv.org/html/2602.04234#S4.F2)a, but for entropy features in𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}\. \(b\) Impact of these features on sample predicted correctness: we plot feature values against the average predicted probability of correctness from XGBoost and LightGBM\.

## 5Deep Analysis

### 5\.1Effective MAS Requires Stable Deliberation

We categorize mathematical reasoning tasks by average MAS accuracy into easy \(GSM8K\), medium \(Math500\), and hard \(AIME24/25\)\. We then analyze entropy features in𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}across these levels to understand how entropy shapes MAS effectiveness under varying task difficulty\.

Stable and Low Entropy for Simple Problems\.OnGSM8K, the top features include final answer length and dispersion of early\-round agent entropy \(withI¯∈\[0\.51,0\.56\]\\bar\{I\}\\in\[0\.51,0\.56\]and\|ρ\|≤0\.15\|\\rho\|\\leq 0\.15\), indicating only a mild influence on predicted correctness\. In contrast, overall round\-1 entropy \(I¯≈0\.47\\bar\{I\}\\approx 0\.47,ρ≈−0\.64\\rho\\approx\-0\.64\) and the stability index which measures consistency of entropy across agents \(I¯≈0\.44\\bar\{I\}\\approx 0\.44,ρ≈−0\.79\\rho\\approx\-0\.79\), are both highly predictive and strongly negatively correlated with success\. This suggests thatsimple problems are solved when agents converge quickly to low\-entropy, stable answers\.

Balanced Exploration for Medium Problems\.OnMath500, high average per\-agent reasoning entropy \(I¯≈0\.77,ρ≈0\.63\\bar\{I\}\\approx 0\.77,\\rho\\approx 0\.63\) and longer reasoning time in round 1 \(I¯≈0\.15,ρ≈0\.71\\bar\{I\}\\approx 0\.15,\\rho\\approx 0\.71\) correlate positively with MAS success, suggesting that medium\-difficulty problems benefit from sustained exploration with moderate entropy\. Conversely, excessive early entropy, measured by maximum total entropy across agents in round 1 \(ρ≈−0\.73\\rho\\approx\-0\.73\), and verbose final answers \(ρ≈−0\.48\\rho\\approx\-0\.48\) strongly predict failure\. Notably, median round\-1 entropy also shows a negative correlation \(ρ≈−0\.36\\rho\\approx\-0\.36\), indicating that while some entropy aids discovery, uncontrolled divergence hinders consensus\. The divergence between mean and median correlations suggests a right\-skewed entropy distribution: moderate sustained entropy benefits MAS, whereas rare extreme spikes dominate the median signal and predict failure\. Together, these findings show thaton medium\-difficulty problems, MAS succeeds when agents explore long enough with moderate entropy, but avoid excessive early divergence, and converge on a concise answer\.

Structured Deliberation for Hard Problems\.OnAIME24/25, round\-1 total reasoning time is the top predictor \(I¯=1\.0\\bar\{I\}=1\.0,ρ≈0\.73\\rho\\approx 0\.73\), confirming that olympiad\-level problems require substantial early effort\. Crucially, entropy\-related features reveal a sharp trade\-off: excessive early entropy harms performance, evident in strong negative correlations for round\-1 max entropy \(ρ≤−0\.70\\rho\\leq\-0\.70\) and per\-token entropy \(ρ≈−0\.37\\rho\\approx\-0\.37\), while moderate average output entropy shows a positive correlation \(ρ≥0\.37\\rho\\geq 0\.37\)\. Entropy dispersion in later rounds also degrades accuracy, with high inter\-agent variance in round\-2 strongly predicting failure \(\|ρ\|≥0\.66\|\\rho\|\\geq 0\.66\)\. Together, these results show thatsuccess on hard problems demands not just long reasoning, but controlled entropy: early exploration must be bounded, and inter\-agent reasoning must remain aligned\.

Stable Entropy for Code Generation\.OnHumanEval, total reasoning time is the strongest predictor, and moderate average entropy improves performance, while both overconfident and erratic entropy profiles degrade it\. This echoes the principle observedin mathematical reasoning: deliberation benefits MAS only when entropy remains structured and stable\.

Inter\-Agent Agreement for Knowledge Q&A\.OnMMLU, MAS success hinges on inter\-agent agreement rather than extended deliberation, unlike in math and code tasks, where longer reasoning improves performance\. High entropy variance harms accuracy, and larger teams worsen consensus, confirming thatconsensus, not duration, drives success in knowledge\-intensive Q&A\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/x4.png)Figure 4:Entropy dynamics in MAS exerts distinct effects depending on task difficulty and the coordination architecture\. \(a, c\) Feature\-SHAP relationships for top entropy features in𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}, grouped by dataset \(a\) and architecture \(c\)\. \(b, d\) Corresponding box plots across all models, annotated with average MAS correctness per dataset \(b\) or per architecture \(d\)\.Overall,entropy\-performance relationships are task\-aware: simple tasks demand rapid convergence to low, stable entropy, whereas medium and hard tasks benefit from moderate average entropy but are harmed by peak or dispersed entropy\. This reveals that harder problems require more exploratory reasoning, yet uncontrolled entropy spikes remain universally detrimental\. Additionally,extended reasoning consistently improves MAS performance on hard tasks, with trajectory length strongly predictive of success\. This aligns with findings in LLM reinforcement learning, where longer reasoning chains reflect stronger underlying reasoning capabilities\(Yeoet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib24)\)\.

Beyond per\-dataset analyses, we examine the two most important entropy features across all models and datasets: round\-1 maximum agent entropy and average entropy of agents’ outputs during problem solving\. Figure[4](https://arxiv.org/html/2602.04234#S5.F4)a shows that harder tasks exhibit wider SHAP value distributions and benefit from moderate average entropy, whereas high early entropy consistently harms performance\. In contrast, easier datasets gain little from higher entropy\. Figure[4](https://arxiv.org/html/2602.04234#S5.F4)b further reveals that as dataset accuracy declines, from 82% onGSM8Kto 25% onAIME25, both entropy features increase in magnitude and dispersion, with round\-1 max entropy showing the strongest sensitivity to task difficulty\. This suggests that difficult problems not only induce higher entropy, but also amplify inter\-agent disagreement, making early entropy control increasingly critical as task complexity grows\. Finally, these results show thatMAS succeeds by reasoning more while keeping entropy low and consistent across agents\. We provide further analysis in Appendix[F\.3](https://arxiv.org/html/2602.04234#A6.SS3), and visualize token\-level entropy trajectories across different model families and datasets in Appendix[L](https://arxiv.org/html/2602.04234#A12)\. These patterns generalize to agentic tasks with external tools: tool\-call entropy negatively predicts accuracy \(mirroring sample\-level entropy in reasoning\), while step\-level features show that the*first reasoning step*dominates the round\-1 predictive signal, thus underscoring the critical role of early\-stage entropy control \(Appendix[E](https://arxiv.org/html/2602.04234#A5)\)\.

### 5\.2Peak Entropy Is Universally Harmful in MAS

We analyze how entropy features influence MAS effectiveness across five architectures on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}\.Centralizedsystems are highly sensitive to early\-round entropy: peak agent entropy in round 1 and peak answer entropy strongly predict failure, as the orchestrator’s single\-context aggregation allows any erratic agent to contaminate the entire reasoning process\.Debatearchitectures depend critically on early consensus: high initial agent divergence reflects how initial divergence amplifies across rounds, preventing convergence; yet once aligned, cumulative entropy becomes beneficial, indicating productive exploration\.Hybridsystems balance robustness and depth: early peak entropy remains harmful, but dual feedback from peers and orchestrator enables recovery through extended deliberation\.Sequentialsystems are most fragile: answer\-level entropy dominates the top predictors, reflecting error propagation through strict role chaining with no cross\-checking\.Single \(SAS\)prioritize brevity and penalize both high answer entropy and entropy variance, indicating that internal consistency is critical for success\.

Architecture determines which entropy matters\.Aggregation\-based systems \(centralized, debate\) fail on inter\-agent dispersion, as noisy inputs corrupt shared contexts\. Sequential systems fail on answer\-level entropy, where specialized roles propagate errors without cross\-verification\. Hybrid systems are most robust, reconciling conflicts through dual feedback\.Universally, peak entropy harms, while cumulative entropy helps, showing that MAS succeeds by shaping entropy distribution, not eliminating entropy\. This mirrors recent findings in multi\-turn agent training\(Xuet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib1)\), where bounding policy entropy variance within historical averages, rather than maximizing absolute entropy, is what stabilizes learning and prevents collapse, suggesting the principle that*entropy stability*, not absolute entropy level, governs effective deliberation, which we extend from single\-agent training to multi\-agent inference\.

Beyond per\-architecture analysis, we compare two key features across all architectures: upper\-quartile agent peak entropy and maximum entropy dispersion across agents in round 1\. Figure[4](https://arxiv.org/html/2602.04234#S5.F4)c shows that both negatively predict correctness within each architecture\. Surprisingly, Figure[4](https://arxiv.org/html/2602.04234#S5.F4)d reveals an inverse trend across architectures: sequential \(lowest\-accuracy\) shows the lowest feature averages, while single \(highest\-accuracy\) shows the highest\. This indicates thatthe relationship between entropy and performance depends on architectural capacity to control entropy, not merely on minimizing entropy\. Further details are provided in Appendix[F\.3](https://arxiv.org/html/2602.04234#A6.SS3)\. Causal discovery in Section[5\.4](https://arxiv.org/html/2602.04234#S5.SS4)subsequently confirms that maximum answer\-token entropy is a consensus direct cause of MAS correctness, elevating the peak\-entropy harm finding from correlation to causation\.

### 5\.3More Rounds Are Not Always Better

![Refer to caption](https://arxiv.org/html/2602.04234v6/x5.png)Figure 5:More rounds do not necessarily improve MAS performance\. \(a\) Accuracy and token consumption for different MAS architectures withR=2R=2andR=5R=5on two benchmarks\. \(b\) Evolution of three key entropy metrics across rounds\. \(c\) The impact of two prominent entropy features, notable for their high importance \(I¯\\bar\{I\}\) and strong correlation \(\|ρ\|\|\\rho\|\) with sample correctness\.To investigate whether additional rounds improve performance, we extend our analysis from fixedR=2R=2toR=5R=5using Qwen3\-0\.6/4B onMATH500andAIME2025, expanding the feature space from 224 to 494 dimensions\. ComparingR=2R=2andR=5R=5, we find thatextending deliberation rarely improves performance and often harms it, even at the cost of higher token consumption, as shown in Figure[5](https://arxiv.org/html/2602.04234#S5.F5)a\. On challenging benchmarks likeAIME25andMATH500, most architectures, including debate and hybrid, exhibit performance degradation with additional rounds, especially for smaller models\. The only consistent gains occur in centralized systems, where strong orchestration enables effective aggregation over longer trajectories\. In contrast, peer\-based architectures \(debate, hybrid\) appear to suffer from prolonged disagreement, as repeated interactions without convergence amplify noise rather than refine reasoning\. This is further supported by entropy dynamics: as Figure[5](https://arxiv.org/html/2602.04234#S5.F5)b shows, key entropy metrics, maximum, mean, and total entropy, drop sharply from round 1 to round 2, but remain nearly flat from round 2 to round 5, indicating that agents largely stabilize after the second round\. Together, these results demonstrate that simply increasing the number of rounds does not enhance MAS performance; instead,the benefit of extended deliberation depends critically on an architecture’s ability to align agents and stabilize entropy early in the process\.

Despite the expanded feature space withR=5R=5, early\-round entropy remains the dominant failure mode, as shown in Figure[5](https://arxiv.org/html/2602.04234#S5.F5)c\. Round\-1 features dominate the top predictors: peak cumulative agent entropy in round 1 ranks second \(I¯≈0\.60\\bar\{I\}\\approx 0\.60,ρ≈−0\.91\\rho\\approx\-0\.91\), underscoring the critical role of early consensus\. Even cumulative sample\-level entropy is strongly harmful \(ρ≈−0\.73\\rho\\approx\-0\.73\), reinforcing that uncontrolled entropy, not just its timing, is detrimental\. These results confirm a central principle:MAS effectiveness is largely determined in the first round, and additional deliberation cannot reliably recover from initial misalignment\. See Appendix[F\.4](https://arxiv.org/html/2602.04234#A6.SS4)for details\. Causal discovery in Section[5\.4](https://arxiv.org/html/2602.04234#S5.SS4)directly supports this: round\-1 total entropy is a consensus direct cause of MAS correctness across both PC and FCI algorithms, confirming that first\-round entropy has a genuine causal footprint on the outcome rather than being merely predictive\. In addition, controlled experiments in Appendix[K](https://arxiv.org/html/2602.04234#A11)confirm that inter\-agent interaction rarely yields genuine accuracy improvements, further validating that MAS outcomes are predominantly fixed by round\-1 dynamics rather than subsequent deliberation\.

### 5\.4Entropy Causally Drives MAS Correctness

To elevate SHAP\-based correlational findings to causal mechanisms, we employ PC and FCI algorithms with temporal constraints, followed by DoWhy effect estimation\.Both algorithms identify three consensus direct causal factors of MAS correctness: base\-model average per\-token entropy \(ATEPS=−0\.12\\text\{ATE\}\_\{\\text\{PS\}\}=\-0\.12,p<10−21p<10^\{\-21\}\), round\-1 total entropy \(p<10−19p<10^\{\-19\}\), and maximum answer\-token entropy \(ATEPS=−0\.31\\text\{ATE\}\_\{\\text\{PS\}\}=\-0\.31,p<10−28p<10^\{\-28\}\), all refutation tests pass; the consensus DAG is shown in Figure[6\(a\)](https://arxiv.org/html/2602.04234#S5.F6.sf1)\. Round\-1 maximum agent entropy dispersion is additionally a direct cause under PC but not FCI, suggesting a possible latent confounder \(e\.g\., problem difficulty\)\. Propensity\-based estimators \(PS/IPW\) serve as the primary quantitative evidence; for base\-model entropy they yield−0\.12\-0\.12to−0\.15\-0\.15, and for maximum answer\-token entropy−0\.31\-0\.31to−0\.34\-0\.34, confirming consistent negative causal effects\. Round\-1 total entropy shows near\-zero propensity\-based effects \(PS=−0\.007=\-0\.007, IPW=\+0\.056=\+0\.056\), consistent with it acting primarily through downstream entropy nodes rather than as an independent linear driver\.Across all estimators, no treatment variable shows a robustly positive effect\(Figure[6\(b\)](https://arxiv.org/html/2602.04234#S5.F6.sf2)\), confirming that higher entropy does not causally benefit MAS correctness\.

Mediation analysis further reveals thatround\-1 inter\-agent entropy dispersion transmits 30–33% of its causal effect on correctness through round\-2 entropy\(Figure[6\(c\)](https://arxiv.org/html/2602.04234#S5.F6.sf3)\), causally supporting the first\-round dominance finding: early misalignment compounds into the subsequent round rather than self\-correcting\. Base\-model entropy exerts its causal effect primarily through direct pathways rather than through sample\-level mediators, indicating that base\-model uncertainty directly shapes the multi\-agent output distribution\. Together, these confirm thatentropy is a causal driver of MAS performance, operating through hierarchical, multi\-round mechanisms\. Full details in Appendix[D](https://arxiv.org/html/2602.04234#A4)\. Beyond this global analysis, per\-finding causal validations are provided in Appendix[F](https://arxiv.org/html/2602.04234#A6)\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/x6.png)\(a\)
![Refer to caption](https://arxiv.org/html/2602.04234v6/x7.png)\(b\)
![Refer to caption](https://arxiv.org/html/2602.04234v6/x8.png)\(c\)

Figure 6:Causal analysis of entropy and MAS correctness\. \(a\) Consensus causal DAG \(PC∩\\capFCI\): base\-model entropy, round\-1 total entropy, and maximum answer\-token entropy are the three consensus direct causes of MAS correctness; other entropy signals act through a layered cascade from round\-1 to round\-2 agent entropy\. \(b\) ATE forest plot \(LR / PS / IPW, sorted by PS\-ATE\): consensus direct causes \(red\) show tighter estimator spread than indirect causes \(blue\)\. \(c\) Mediation decomposition of significant round\-1 paths \(directc′c^\{\\prime\}in red, indirecta×ba\{\\times\}bin blue\)\.
### 5\.5Entropy Predicts MAS Correctness

TheEntropy Judgerachieves high cross\-validation accuracy using only MAS\-derived entropy features: 72\.6% / 79\.1% \(LLaMA / Qwen\) on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}, rising to 74\.5% / 80\.7% when base\-model entropy is added \(𝒢base\-H\\mathcal\{G\}\_\{\\text\{base\-H\}\}\), and 81\.2% / 91\.6% with base\-model correctness included \(𝒢base\-full\\mathcal\{G\}\_\{\\text\{base\-full\}\}\)\. This demonstrates that entropy dynamics alone are highly predictive of correctness, and thatMbaseM\_\{\\text\{base\}\}further influence MAS effectiveness\. Beyond binary prediction, theEntropy Judgerenables label\-free pass@kkselection and consistently improves accuracy across all MAS configurations; see details in Appendix[I](https://arxiv.org/html/2602.04234#A9)\.

## 6Conclusion

This study presents a comprehensive analysis of entropy dynamics in LLM\-based MAS, examining 245 entropy features across six reasoning tasks, two agentic tasks, and four MAS topologies\. Our findings challenge prevailing assumptions: single agents outperform MAS in 43\.3% of cases, and MAS effectiveness is largely determined by first\-round entropy dynamics\. We identify three principles governing MAS performance: \(1\)Certainty Preference, where peak entropy directly harms and stable low entropy directly benefits MAS correctness; \(2\)Base Entropy, where lower base model entropy is a direct causal driver of MAS performance; and \(3\)Task Awareness, where optimal entropy profiles vary by task difficulty\. These findings extend to agentic settings: tool\-call entropy and first\-round inter\-agent dispersion jointly constrain MAS performance\. Building on these insights, theEntropy Judgerleverages learned entropy patterns to select high\-quality outputs from MAS pass@kkcandidates, achieving consistent accuracy gains without ground\-truth labels\.

## References

- S\. Agarwal, Z\. Zhang, L\. Yuan, J\. Han, and H\. Peng \(2025\)The unreasonable effectiveness of entropy minimization in llm reasoning\.arXiv preprint arXiv:2505\.15134\.Cited by:[§2](https://arxiv.org/html/2602.04234#S2.p2.1)\.
- A\. Bigeard, L\. Nashold, R\. Krishnan, and S\. Wu \(2025\)Finance agent benchmark: benchmarking llms on real\-world financial research tasks\.arXiv preprint arXiv:2508\.00828\.Cited by:[Appendix M](https://arxiv.org/html/2602.04234#A13.SS0.SSS0.Px1.p1.1),[§B\.1](https://arxiv.org/html/2602.04234#A2.SS1.SSS0.Px2.p1.1),[Appendix E](https://arxiv.org/html/2602.04234#A5.p1.1),[§4\.1](https://arxiv.org/html/2602.04234#S4.SS1.p1.2)\.
- P\. M\. Buffa and L\. Del Corro \(2026\)Entropy sentinel: continuous llm accuracy monitoring from decoding entropy traces in stem\.arXiv preprint arXiv:2601\.09001\.Cited by:[Appendix A](https://arxiv.org/html/2602.04234#A1.SS0.SSS0.Px1.p1.3),[§4\.2](https://arxiv.org/html/2602.04234#S4.SS2.p1.1)\.
- M\. Cemri, M\. Z\. Pan, S\. Yang, L\. A\. Agrawal, B\. Chopra, R\. Tiwari, K\. Keutzer, A\. Parameswaran, D\. Klein, K\. Ramchandran,et al\.\(2025\)Why do multi\-agent LLM systems fail?\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2602.04234#S1.p2.1),[§2](https://arxiv.org/html/2602.04234#S2.p1.1),[§4\.3](https://arxiv.org/html/2602.04234#S4.SS3.p1.1),[§4\.4](https://arxiv.org/html/2602.04234#S4.SS4.SSS0.Px3.p1.10),[§4](https://arxiv.org/html/2602.04234#S4.p1.1)\.
- M\. Chenet al\.\(2021\)Evaluating large language models trained on code\.arXiv preprint arXiv:2107\.03374\.Cited by:[§B\.1](https://arxiv.org/html/2602.04234#A2.SS1.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2602.04234#S4.SS1.p1.2)\.
- P\. Chen, X\. Li, Z\. Li, W\. Yin, X\. Chen, and T\. Lin \(2025a\)Exploration vs exploitation: rethinking rlvr through clipping, entropy, and spurious reward\.InThe 5th Workshop on Mathematical Reasoning and AI at NeurIPS 2025,Cited by:[§1](https://arxiv.org/html/2602.04234#S1.p3.1),[§2](https://arxiv.org/html/2602.04234#S2.p2.1)\.
- T\. Chen \(2016\)XGBoost: a scalable tree boosting system\.Cornell University\.Cited by:[§4\.3](https://arxiv.org/html/2602.04234#S4.SS3.p2.10)\.
- W\. Chen, Y\. Su, J\. Zuo, C\. Yang, C\. Yuan, C\. Chan, H\. Yu, Y\. Lu, Y\. Hung, C\. Qian,et al\.\(2024\)Agentverse: facilitating multi\-agent collaboration and exploring emergent behaviors\.InThe Twelfth International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2602.04234#S2.p1.1)\.
- Z\. Chen, X\. Ma, S\. Zhuang, P\. Nie, K\. Zou, A\. Liu, J\. Green, K\. Patel, R\. Meng, M\. Su,et al\.\(2025b\)Browsecomp\-plus: a more fair and transparent evaluation benchmark of deep\-research agent\.arXiv preprint arXiv:2508\.06600\.Cited by:[Appendix M](https://arxiv.org/html/2602.04234#A13.SS0.SSS0.Px1.p1.1)\.
- D\. Cheng, S\. Huang, X\. Zhu, B\. Dai, W\. X\. Zhao, Z\. Zhang, and F\. Wei \(2025\)Reasoning with exploration: an entropy perspective\.InAnnual AAAI Conference on Artificial Intelligence,Cited by:[§1](https://arxiv.org/html/2602.04234#S1.p3.1),[§2](https://arxiv.org/html/2602.04234#S2.p2.1)\.
- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano,et al\.\(2021\)Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§B\.1](https://arxiv.org/html/2602.04234#A2.SS1.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2602.04234#S4.SS1.p1.2)\.
- G\. Cui, Y\. Zhang, J\. Chen, L\. Yuan, Z\. Wang, Y\. Zuo, H\. Li, Y\. Fan, H\. Chen, W\. Chen,et al\.\(2025\)The entropy mechanism of reinforcement learning for reasoning language models\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2602.04234#S1.p3.1),[§2](https://arxiv.org/html/2602.04234#S2.p2.1),[§3](https://arxiv.org/html/2602.04234#S3.SS0.SSS0.Px1.p1.7)\.
- Y\. Dang, C\. Qian, X\. Luo, J\. Fan, Z\. Xie, R\. Shi, W\. Chen, C\. Yang, X\. Che, Y\. Tian,et al\.\(2025\)Multi\-agent collaboration via evolving orchestration\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2602.04234#S1.p1.1),[§2](https://arxiv.org/html/2602.04234#S2.p1.1)\.
- Y\. Du, S\. Li, A\. Torralba, J\. B\. Tenenbaum, and I\. Mordatch \(2023\)Improving factuality and reasoning in language models through multiagent debate\.InForty\-first International Conference on Machine Learning,Cited by:[§1](https://arxiv.org/html/2602.04234#S1.p1.1)\.
- M\. Gao, Y\. Li, B\. Liu, Y\. Yu, P\. Wang, C\. Lin, and F\. Lai \(2025\)Single\-agent or multi\-agent systems? why not both?\.arXiv preprint arXiv:2505\.18286\.Cited by:[§1](https://arxiv.org/html/2602.04234#S1.p2.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.\(2024\)The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§B\.1](https://arxiv.org/html/2602.04234#A2.SS1.SSS0.Px1.p1.3),[§4\.1](https://arxiv.org/html/2602.04234#S4.SS1.p1.2)\.
- S\. He, A\. Narayan, I\. S\. Khare, S\. W\. Linderman, C\. Ré, and D\. Biderman \(2025\)An information theoretic perspective on agentic system design\.arXiv preprint arXiv:2512\.21720\.Cited by:[§1](https://arxiv.org/html/2602.04234#S1.p4.1),[§2](https://arxiv.org/html/2602.04234#S2.p1.1)\.
- D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. Steinhardt \(2021a\)Measuring massive multitask language understanding\.InInternational Conference on Learning Representations,Cited by:[§B\.1](https://arxiv.org/html/2602.04234#A2.SS1.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2602.04234#S4.SS1.p1.2)\.
- D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. Steinhardt \(2021b\)Measuring mathematical problem solving with the math dataset\.Advances in Neural Information Processing Systems\.Cited by:[§B\.1](https://arxiv.org/html/2602.04234#A2.SS1.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2602.04234#S4.SS1.p1.2)\.
- S\. Hong, M\. Zhuge, J\. Chen, X\. Zheng, Y\. Cheng, J\. Wang, C\. Zhang, Z\. Wang, S\. K\. S\. Yau, Z\. Lin,et al\.\(2024\)MetaGPT: meta programming for a multi\-agent collaborative framework\.InThe Twelfth International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2602.04234#S1.p1.1)\.
- HuggingFace \(2025\)Math\-verify\.External Links:[Link](https://github.com/huggingface/Math-Verify)Cited by:[§B\.1](https://arxiv.org/html/2602.04234#A2.SS1.SSS0.Px3.p1.1)\.
- A\. Karan and Y\. Du \(2026\)Reasoning with sampling: your base model is smarter than you think\.InThe International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2602.04234#S2.p2.1)\.
- G\. Ke, Q\. Meng, T\. Finley, T\. Wang, W\. Chen, W\. Ma, Q\. Ye, and T\. Liu \(2017\)Lightgbm: a highly efficient gradient boosting decision tree\.Advances in Neural Information Processing Systems30\.Cited by:[§4\.3](https://arxiv.org/html/2602.04234#S4.SS3.p2.10)\.
- Y\. Kim, K\. Gu, C\. Park, C\. Park, S\. Schmidgall, A\. A\. Heydari, Y\. Yan, Z\. Zhang, Y\. Zhuang, M\. Malhotra,et al\.\(2025\)Towards a science of scaling agent systems\.arXiv preprint arXiv:2512\.08296\.Cited by:[§E\.2\.1](https://arxiv.org/html/2602.04234#A5.SS2.SSS1.Px1.p2.1),[§1](https://arxiv.org/html/2602.04234#S1.p2.1),[§2](https://arxiv.org/html/2602.04234#S2.p1.1),[§4\.1](https://arxiv.org/html/2602.04234#S4.SS1.p1.2),[§4\.4](https://arxiv.org/html/2602.04234#S4.SS4.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2602.04234#S4.p1.1)\.
- X\. Li, E\. Callanan, A\. Ghassel, and X\. Zhu \(2026\)Entropy\-gated branching for efficient test\-time reasoning\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics,Cited by:[§2](https://arxiv.org/html/2602.04234#S2.p2.1)\.
- X\. Li \(2026\)When single\-agent with skills replace multi\-agent systems and when they fail\.arXiv preprint arXiv:2601\.04748\.Cited by:[§2](https://arxiv.org/html/2602.04234#S2.p1.1)\.
- X\. Liu, H\. Yu, H\. Zhang, Y\. Xu, X\. Lei, H\. Lai, Y\. Gu, H\. Ding, K\. Men, K\. Yang,et al\.\(2024\)Agentbench: evaluating LLMs as agents\.InInternational Conference on Learning Representations,Cited by:[Appendix M](https://arxiv.org/html/2602.04234#A13.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2602.04234#S1.p1.1)\.
- S\. M\. Lundberg and S\. Lee \(2017\)A unified approach to interpreting model predictions\.InAdvances in Neural Information Processing Systems,Vol\.30\.Cited by:[§4\.3](https://arxiv.org/html/2602.04234#S4.SS3.p2.10)\.
- G\. Mialon, C\. Fourrier, T\. Wolf, Y\. LeCun, and T\. Scialom \(2024\)Gaia: a benchmark for general ai assistants\.InThe Twelfth International Conference on Learning Representations,Cited by:[Appendix M](https://arxiv.org/html/2602.04234#A13.SS0.SSS0.Px1.p1.1),[§B\.1](https://arxiv.org/html/2602.04234#A2.SS1.SSS0.Px2.p1.1),[Appendix E](https://arxiv.org/html/2602.04234#A5.p1.1),[§4\.1](https://arxiv.org/html/2602.04234#S4.SS1.p1.2)\.
- D\. Paglieri, B\. Cupiał, S\. Coward, U\. Piterbarg, M\. Wolczyk, A\. Khan, E\. Pignatelli, Ł\. Kuciński, L\. Pinto, R\. Fergus,et al\.\(2025\)BALROG: benchmarking agentic LLM and VLM reasoning on games\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2602.04234#S1.p1.1)\.
- W\. Shen, C\. Li, H\. Chen, M\. Yan, X\. Quan, H\. Chen, J\. Zhang, and F\. Huang \(2024\)Small llms are weak tool learners: a multi\-llm agent\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 16658–16680\.Cited by:[Appendix A](https://arxiv.org/html/2602.04234#A1.SS0.SSS0.Px3.p1.1),[§4\.1](https://arxiv.org/html/2602.04234#S4.SS1.p1.2)\.
- Q\. Team, A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Tang, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. Qiu \(2025\)Qwen2\.5 technical report\.arXiv preprint arXiv:2412\.15115\.Cited by:[Appendix G](https://arxiv.org/html/2602.04234#A7.p1.1)\.
- D\. Tran and D\. Kiela \(2026\)Single\-agent llms outperform multi\-agent systems on multi\-hop reasoning under equal thinking token budgets\.arXiv preprint arXiv:2604\.02460\.Cited by:[§1](https://arxiv.org/html/2602.04234#S1.p2.1),[§1](https://arxiv.org/html/2602.04234#S1.p4.1),[§2](https://arxiv.org/html/2602.04234#S2.p1.1),[§4\.4](https://arxiv.org/html/2602.04234#S4.SS4.SSS0.Px1.p1.1)\.
- S\. Wang, L\. Yu, C\. Gao, C\. Zheng, S\. Liu, R\. Lu, K\. Dang, X\. Chen, J\. Yang, Z\. Zhang,et al\.\(2025\)Beyond the 80/20 rule: high\-entropy minority tokens drive effective reinforcement learning for llm reasoning\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2602.04234#S1.p3.1),[§2](https://arxiv.org/html/2602.04234#S2.p2.1)\.
- W\. Xu, W\. Zhao, Z\. Wang, Y\. Li, C\. Jin, M\. Jin, K\. Mei, K\. Wan, and D\. N\. Metaxas \(2025\)Epo: entropy\-regularized policy optimization for llm agents reinforcement learning\.arXiv preprint arXiv:2509\.22576\.Cited by:[§5\.2](https://arxiv.org/html/2602.04234#S5.SS2.p2.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§B\.1](https://arxiv.org/html/2602.04234#A2.SS1.SSS0.Px1.p1.3),[§4\.1](https://arxiv.org/html/2602.04234#S4.SS1.p1.2)\.
- Z\. Yang, M\. Zhang, F\. Chen, G\. Ding, L\. Hou, X\. Tao, and Y\. Chen \(2026\)Less is more: improving llm reasoning with minimal test\-time intervention\.InAssociation for Computational Linguistics,Cited by:[§2](https://arxiv.org/html/2602.04234#S2.p2.1)\.
- R\. Ye, X\. Liu, Q\. Wu, X\. Pang, Z\. Yin, L\. Bai, and S\. Chen \(2025\)X\-mas: towards building multi\-agent systems with heterogeneous llms\.arXiv preprint arXiv:2505\.16997\.Cited by:[Appendix M](https://arxiv.org/html/2602.04234#A13.SS0.SSS0.Px2.p1.1)\.
- E\. Yeo, Y\. Tong, X\. Niu, G\. Neubig, and X\. Yue \(2025\)Demystifying long chain\-of\-thought reasoning in llms\.InICLR 2025 Workshop on Deep Generative Model in Machine Learning: Theory, Principle and Efficacy,Cited by:[§5\.1](https://arxiv.org/html/2602.04234#S5.SS1.p7.1)\.
- Y\. Yue, Z\. Chen, R\. Lu, A\. Zhao, Z\. Wang, S\. Song, and G\. Huang \(2025\)Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?\.InAdvances in Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2602.04234#S2.p2.1)\.
- W\. Zeng, Y\. Huang, Q\. Liu, W\. Liu, K\. He, Z\. Ma, and J\. He \(2025\)SimpleRL\-Zoo: investigating and taming zero reinforcement learning for open base models in the wild\.InConference on Language Models,Cited by:[Appendix G](https://arxiv.org/html/2602.04234#A7.p1.1),[Appendix G](https://arxiv.org/html/2602.04234#A7.p2.8)\.
- J\. Zhang, X\. Wang, F\. Mo, Y\. Zhou, W\. Gao, and K\. Liu \(2025a\)Entropy\-based exploration conduction for multi\-step reasoning\.InAssociation for Computational Linguistics,Cited by:[§1](https://arxiv.org/html/2602.04234#S1.p3.1),[§3](https://arxiv.org/html/2602.04234#S3.SS0.SSS0.Px1.p1.7),[§4\.2](https://arxiv.org/html/2602.04234#S4.SS2.p1.1)\.
- Q\. Zhang, Y\. Yu, Q\. Fu, D\. Ye,et al\.\(2024a\)More agents is all you need\.Transactions on Machine Learning Research\.Cited by:[§1](https://arxiv.org/html/2602.04234#S1.p1.1),[§4\.4](https://arxiv.org/html/2602.04234#S4.SS4.SSS0.Px1.p1.1)\.
- X\. Zhang, X\. Yuan, D\. Huang, W\. You, C\. Hu, J\. Ruan, K\. Chen, and X\. Hu \(2025b\)Rediscovering entropy regularization: adaptive coefficient unlocks its potential for llm reinforcement learning\.arXiv preprint arXiv:2510\.10959\.Cited by:[§2](https://arxiv.org/html/2602.04234#S2.p2.1)\.
- Y\. Zhang, Z\. Zhang, H\. Guan, Y\. Cheng, Y\. Duan, C\. Wang, Y\. Wang, S\. Zheng, and J\. He \(2025c\)No free lunch: rethinking internal feedback for llm reasoning\.arXiv preprint arXiv:2506\.17219\.Cited by:[§2](https://arxiv.org/html/2602.04234#S2.p2.1)\.
- Y\. Zhang, X\. Liu, and C\. Xiao \(2025d\)MetaAgent: automatically constructing multi\-agent systems based on finite state machines\.InProceedings of the 42nd International Conference on Machine Learning,Cited by:[§2](https://arxiv.org/html/2602.04234#S2.p1.1),[§4\.4](https://arxiv.org/html/2602.04234#S4.SS4.SSS0.Px2.p1.13)\.
- Y\. Zhang, R\. Sun, Y\. Chen, T\. Pfister, R\. Zhang, and S\. Arik \(2024b\)Chain of agents: large language models collaborating on long\-context tasks\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 132208–132237\.Cited by:[§2](https://arxiv.org/html/2602.04234#S2.p1.1)\.
- K\. Zhu, H\. Du, Z\. Hong, X\. Yang, S\. Guo, D\. Z\. Wang, Z\. Wang, C\. Qian, R\. Tang, H\. Ji,et al\.\(2025a\)Multiagentbench: evaluating the collaboration and competition of llm agents\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 8580–8622\.Cited by:[Appendix M](https://arxiv.org/html/2602.04234#A13.SS0.SSS0.Px1.p1.1)\.
- Y\. Zhu, L\. Sun, G\. Zhao, W\. Lin, and X\. Zhang \(2025b\)Uncertainty under the curve: a sequence\-level entropy area metric for reasoning llm\.InAnnual AAAI Conference on Artificial Intelligence,Cited by:[§1](https://arxiv.org/html/2602.04234#S1.p3.1),[§2](https://arxiv.org/html/2602.04234#S2.p2.1),[§4\.2](https://arxiv.org/html/2602.04234#S4.SS2.p1.1)\.

###### Appendix Contents

1. [1Introduction](https://arxiv.org/html/2602.04234#S1)
2. [2Related Work](https://arxiv.org/html/2602.04234#S2)
3. [3Preliminaries](https://arxiv.org/html/2602.04234#S3)
4. [4Exploring MAS with Entropy Dynamics](https://arxiv.org/html/2602.04234#S4)1. [4\.1Evaluation Protocol](https://arxiv.org/html/2602.04234#S4.SS1) 2. [4\.2Measuring Reasoning with Entropy and Beyond](https://arxiv.org/html/2602.04234#S4.SS2) 3. [4\.3Mining Effectiveness of MAS](https://arxiv.org/html/2602.04234#S4.SS3) 4. [4\.4Examining Entropy Impacts on MAS](https://arxiv.org/html/2602.04234#S4.SS4)
5. [5Deep Analysis](https://arxiv.org/html/2602.04234#S5)1. [5\.1Effective MAS Requires Stable Deliberation](https://arxiv.org/html/2602.04234#S5.SS1) 2. [5\.2Peak Entropy Is Universally Harmful in MAS](https://arxiv.org/html/2602.04234#S5.SS2) 3. [5\.3More Rounds Are Not Always Better](https://arxiv.org/html/2602.04234#S5.SS3) 4. [5\.4Entropy Causally Drives MAS Correctness](https://arxiv.org/html/2602.04234#S5.SS4) 5. [5\.5Entropy Predicts MAS Correctness](https://arxiv.org/html/2602.04234#S5.SS5)
6. [6Conclusion](https://arxiv.org/html/2602.04234#S6)
7. [References](https://arxiv.org/html/2602.04234#bib)
8. [ARationale for Open\-Source Small LLMs](https://arxiv.org/html/2602.04234#A1)
9. [BExperimental Details](https://arxiv.org/html/2602.04234#A2)1. [B\.1Evaluation](https://arxiv.org/html/2602.04234#A2.SS1) 2. [B\.2MAS Architecture Details](https://arxiv.org/html/2602.04234#A2.SS2) 3. [B\.3Agent Prompts](https://arxiv.org/html/2602.04234#A2.SS3) 4. [B\.4Data Mining](https://arxiv.org/html/2602.04234#A2.SS4)
10. [CEntropy Features](https://arxiv.org/html/2602.04234#A3)1. [C\.1Feature Hierarchy](https://arxiv.org/html/2602.04234#A3.SS1) 2. [C\.2Feature Groups](https://arxiv.org/html/2602.04234#A3.SS2) 3. [C\.3Key Feature Definitions](https://arxiv.org/html/2602.04234#A3.SS3) 4. [C\.4Feature Redundancy Discussion](https://arxiv.org/html/2602.04234#A3.SS4)1. [C\.4\.1Pairwise Correlation Analysis](https://arxiv.org/html/2602.04234#A3.SS4.SSS1) 2. [C\.4\.2PCA Analysis](https://arxiv.org/html/2602.04234#A3.SS4.SSS2) 3. [C\.4\.3Feature Ablation Study](https://arxiv.org/html/2602.04234#A3.SS4.SSS3) 4. [C\.4\.4Cross\-Method Feature Importance Validation](https://arxiv.org/html/2602.04234#A3.SS4.SSS4)
11. [DCausal Discovery and Effect Estimation](https://arxiv.org/html/2602.04234#A4)1. [D\.1Feature Selection Pipeline](https://arxiv.org/html/2602.04234#A4.SS1) 2. [D\.2Causal Structure Discovery](https://arxiv.org/html/2602.04234#A4.SS2) 3. [D\.3Causal Effect Estimation](https://arxiv.org/html/2602.04234#A4.SS3) 4. [D\.4Mediation Analysis](https://arxiv.org/html/2602.04234#A4.SS4)
12. [EEntropy Patterns Generalize to Agentic Tasks](https://arxiv.org/html/2602.04234#A5)1. [E\.1GAIA: First\-Round Dispersion Mediates Tool Execution Failure](https://arxiv.org/html/2602.04234#A5.SS1)1. [E\.1\.1Experimental Setup](https://arxiv.org/html/2602.04234#A5.SS1.SSS1) 2. [E\.1\.2Main Results](https://arxiv.org/html/2602.04234#A5.SS1.SSS2) 3. [E\.1\.3Additional Feature Analysis](https://arxiv.org/html/2602.04234#A5.SS1.SSS3) 4. [E\.1\.4Causal Validation](https://arxiv.org/html/2602.04234#A5.SS1.SSS4) 5. [E\.1\.5Tool\-Failure Attribution by Base Model](https://arxiv.org/html/2602.04234#A5.SS1.SSS5) 2. [E\.2FinanceAgent: Architecture Overhead Dominates over Entropy Control](https://arxiv.org/html/2602.04234#A5.SS2)1. [E\.2\.1Experimental Setup](https://arxiv.org/html/2602.04234#A5.SS2.SSS1) 2. [E\.2\.2Accuracy Results](https://arxiv.org/html/2602.04234#A5.SS2.SSS2) 3. [E\.2\.3Feature Importance Analysis](https://arxiv.org/html/2602.04234#A5.SS2.SSS3)
13. [FMore Experimental Results](https://arxiv.org/html/2602.04234#A6)1. [F\.1Base Model Entropy Dominates MAS Prediction](https://arxiv.org/html/2602.04234#A6.SS1) 2. [F\.2Inter\-Agent Misalignment Causes MAS Failure](https://arxiv.org/html/2602.04234#A6.SS2) 3. [F\.3Task Difficulty Determines Optimal Entropy Dynamics](https://arxiv.org/html/2602.04234#A6.SS3) 4. [F\.4Round\-1 Entropy Dominates Despite Extended Deliberation](https://arxiv.org/html/2602.04234#A6.SS4) 5. [F\.5Temperature Variation Preserves Relative Entropy Patterns](https://arxiv.org/html/2602.04234#A6.SS5) 6. [F\.6Model Scaling Preserves Entropy Feature Dominance](https://arxiv.org/html/2602.04234#A6.SS6)
14. [GRL Training Inverts the Role of Entropy](https://arxiv.org/html/2602.04234#A7)
15. [HDivergent Reasoning Styles of Qwen and LLaMA](https://arxiv.org/html/2602.04234#A8)1. [H\.1Qwen3\-4B \(Sequential Architecture\)](https://arxiv.org/html/2602.04234#A8.SS1) 2. [H\.2LLaMA\-3\.2\-3B\-Instruct \(Sequential Architecture\)](https://arxiv.org/html/2602.04234#A8.SS2)
16. [IEntropy Judger](https://arxiv.org/html/2602.04234#A9)1. [I\.1Training Data Construction](https://arxiv.org/html/2602.04234#A9.SS1) 2. [I\.2Model Hyperparameters](https://arxiv.org/html/2602.04234#A9.SS2) 3. [I\.3Cross\-Validation Protocol](https://arxiv.org/html/2602.04234#A9.SS3) 4. [I\.4Per\-Dataset Classification Performance](https://arxiv.org/html/2602.04234#A9.SS4) 5. [I\.5Pass@kkSelection](https://arxiv.org/html/2602.04234#A9.SS5)
17. [JEntropy Calibration Analysis](https://arxiv.org/html/2602.04234#A10)1. [J\.1Calibration Methodology](https://arxiv.org/html/2602.04234#A10.SS1) 2. [J\.2Calibration Results](https://arxiv.org/html/2602.04234#A10.SS2) 3. [J\.3Overconfidence and Confidently Wrong Analysis](https://arxiv.org/html/2602.04234#A10.SS3) 4. [J\.4Calibration on Tool\-Augmented Tasks](https://arxiv.org/html/2602.04234#A10.SS4)
18. [KControlled SAS vs\. MAS Comparison](https://arxiv.org/html/2602.04234#A11)1. [K\.1Controlled Experiment Design](https://arxiv.org/html/2602.04234#A11.SS1) 2. [K\.2Role Assignment as a Causal Intervention](https://arxiv.org/html/2602.04234#A11.SS2) 3. [K\.3Inter\-Agent Interaction Dynamics](https://arxiv.org/html/2602.04234#A11.SS3) 4. [K\.4Genuine Improvement and Anchoring](https://arxiv.org/html/2602.04234#A11.SS4)
19. [LCase Study: Token\-Level Entropy Dynamics](https://arxiv.org/html/2602.04234#A12)
20. [MLimitations, Broader Impacts, and Future Work](https://arxiv.org/html/2602.04234#A13)

## Appendix ARationale for Open\-Source Small LLMs

Our study exclusively employs open\-source small LLMs from the LLaMA and Qwen3 series, rather than proprietary API\-based models\. This choice is driven not merely by practical considerations but by the necessity of accessing internal probability distributions for entropy\-based analysis\.

##### Full Probability Access\.

Proprietary APIs typically return only generated text or at most top\-kklogprobs withk≤20k\\leq 20\(Buffa and Del Corro,[2026](https://arxiv.org/html/2602.04234#bib.bib22)\)\. This truncation prevents accurate entropy computation, which requires the complete token\-level probability distributionP​\(xt∣x<t\)P\(x\_\{t\}\\mid x\_\{<t\}\)over the full vocabulary\. Open\-weight models provide full access to these probabilities, enabling the 245\-dimensional hierarchical entropy features that span token, agent, and round levels\. Without this access, our core entropy analysis would be impossible\.

##### Reproducibility and API Instability\.

Proprietary models often undergo silent updates that change their behavior without notice, making longitudinal studies unreliable\. In contrast, open\-source models with fixed weights guarantee that our experimental protocol can be exactly replicated\. This stability is essential for scientific validity, as entropy dynamics measured today must match those observed in future reproductions\.

##### Cost\-Effective Multi\-Agent Scaling\.

Exploring diverse MAS configurations across five architectures, five models, six datasets, and multiple rounds requires thousands of LLM calls\. Using proprietary APIs would incur prohibitive costs, whereas local inference with small models makes comprehensive evaluation feasible\. Furthermore, multiple specialized small agents can match or even exceed the performance of a single large model through task decomposition\(Shenet al\.,[2024](https://arxiv.org/html/2602.04234#bib.bib66)\)\.

##### Complementing Prior Work\.

Existing MAS studies rely on proprietary models and report only aggregate metrics such as accuracy, latency, or cost, thereby overlooking internal entropy dynamics\. Our focus on open\-source models reveals how entropy evolves within and across agents, providing mechanistic insights that remain inaccessible under API\-only evaluation\. This approach complements rather than duplicates prior findings\.

## Appendix BExperimental Details

### B\.1Evaluation

##### Models and Configurations\.

We evaluate five open\-source LLMs on reasoning benchmarks: LLaMA\-3\.1\-8B\-Instruct, LLaMA\-3\.2\-3B\-Instruct\(Grattafioriet al\.,[2024](https://arxiv.org/html/2602.04234#bib.bib71)\), and Qwen3\-0\.6B, Qwen3\-4B, Qwen3\-8B\(Yanget al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib72)\)\. All systems are built using the sameMbaseM\_\{\\text\{base\}\}, with the number of interaction rounds fixed toR=2R=2, temperature set to 0\.6, top\-ppto 0\.95, and maximum sequence length to 8,192 tokens\. On the challengingAIME24/25benchmarks, the maximum sequence length is set to 16,384 tokens\. For Qwen3 models, thinking mode is enabled by default \(enable\_thinking=True\), which activates extended chain\-of\-thought reasoning prior to the final response; the thinking tokens are included in the entropy computation\. We validate the robustness of our findings across varying temperatures \(Appendix[F\.5](https://arxiv.org/html/2602.04234#A6.SS5)\) and verify their generalizability to 14B\-parameter models \(Appendices[F\.6](https://arxiv.org/html/2602.04234#A6.SS6)and[E\.1](https://arxiv.org/html/2602.04234#A5.SS1)\)\.

##### Benchmarks\.

We evaluate on six reasoning benchmarks across three task types: for mathematical reasoning,GSM8K\(Cobbeet al\.,[2021](https://arxiv.org/html/2602.04234#bib.bib74)\),MATH500\(Hendryckset al\.,[2021b](https://arxiv.org/html/2602.04234#bib.bib75)\),AIME2024, andAIME2025; for code generation,HumanEval\(Chen and others,[2021](https://arxiv.org/html/2602.04234#bib.bib77)\); and for knowledge question\-answering \(Q&A\),MMLU\(Hendryckset al\.,[2021a](https://arxiv.org/html/2602.04234#bib.bib76)\)\. Each question is accompanied by a chain\-of\-thought prompt\. We further evaluate on two agentic benchmarks:GAIA\(Mialonet al\.,[2024](https://arxiv.org/html/2602.04234#bib.bib55)\)\(165 questions, full validation split\) andFinanceAgent\(Bigeardet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib61)\); see Appendix[E](https://arxiv.org/html/2602.04234#A5)for details\.

##### Evaluation and Infrastructure\.

For mathematics andMMLU, models enclose final answers in\\boxed\{\}, and correctness is verified using theMath VerifytoolHuggingFace \([2025](https://arxiv.org/html/2602.04234#bib.bib81)\)or exact string matching\. ForHumanEval, code blocks are extracted from markdown output and validated by executing the provided test cases with a 10\-second timeout per sample\. We exclude debate architecture onHumanEval, as majority voting is generally ineffective for code generation tasks\. For agentic benchmarks, each agent operates in a ReAct loop;GAIAuses the official answer\-matching protocol, andFinanceAgentuses exact answer matching against ground\-truth financial metrics\. All experiments are conducted on four RTX 5090 GPUs\.

### B\.2MAS Architecture Details

All architectures use LangGraph for workflow orchestration\. Each architecture runs forR=2R=2rounds by default\. Table[1](https://arxiv.org/html/2602.04234#A2.T1)summarizes the key differences\.

##### Single Agent \(SAS\)\.

A singleSingleSolveragent processes the input and iteratively refines its answer across rounds\. Each round receives the accumulated history from previous rounds\. LLM calls:R×1R\\times 1\.

##### Sequential\.

Four specialized agents form a pipeline:Planner→\\rightarrowSolver→\\rightarrowCritic→\\rightarrowJudger\. The planner generates step\-by\-step instructions \(no calculations\), the solver executes the plan, the critic reviews and identifies errors, and the judger produces the final answer\. The planner \(first agent\) receives the accumulated outputs of all agents from prior rounds; each subsequent agent receives only its immediate predecessor’s output within the current round\. LLM calls:R×4R\\times 4\.

##### Centralized\.

Three domain experts \(MathAgent,ScienceAgent,CodeAgent\) execute in parallel, and anOrchestratorAgentaggregates their outputs\. In roundsr<Rr<R, the orchestrator provides feedback to all workers; in roundRR, it produces the final answer\. LLM calls:R×3\+R×1=R×4R\\times 3\+R\\times 1=R\\times 4\.

##### Debate\.

Three debate agents \(Agent1,Agent2,Agent3\) execute sequentially\. Each agent observes all prior agents’ outputs from both current and previous rounds\. The final answer is determined by majority voting over\\boxed\{\}extractions, without additional LLM inference\. LLM calls:R×3R\\times 3\.

##### Hybrid\.

Combines centralized and debate structures\. Workers receive both orchestrator feedback and peer outputs, enabling dual feedback channels\. The orchestrator provides guidance while workers can observe how peers interpret that guidance\. LLM calls:R×3\+R×1=R×4R\\times 3\+R\\times 1=R\\times 4\.

Table 1:Comparison of MAS architectures\.NN: number of worker agents;RR: communication rounds\.ArchitectureNNLLM CallsOrchestratorDecision RuleAgent Roles & FunctionsSingle \(SAS\)1RRNoneLast round outputSingleSolver: solves and iteratively refines answerSequential44​R4RNoneJudger outputPlanner: generates plans; Solver: executes plans; Critic: reviews solutions; Judger: outputs final answerCentralized3\+14​R4RLLM\-basedOrchestratorMath / Science / CodeAgent: domain\-specific reasoning; Orchestrator: aggregates feedback and outputs final answerDebate33​R3RVotingMajority voteAgent 1–3: independent solvers observing all prior outputsHybrid3\+14​R4RLLM\-basedOrchestratorMath / Science / CodeAgent: domain\-specific reasoning; Orchestrator: aggregates \+ peer feedback

### B\.3Agent Prompts

The prompts for the Sequential architecture are detailed in Appendix[H](https://arxiv.org/html/2602.04234#A8)\. Here we present the prompts for the Centralized architecture on mathematical reasoning tasks\. Prompts for code generation and knowledge Q&A follow similar patterns and can be found in the source code\.

##### First\-Layer Expert Agents\.

Each expert agent receives a system prompt defining its role and a user prompt containing the question\. All agents may receive orchestrator feedback from previous rounds\.

MathAgent:

- •System: “You are the MathAgent\. Solve the given question with clear steps\. Your input may include feedback from the Orchestrator from the previous round\.”
- •User: “Question: \{question\} Provide a concise mathematical solution, showing key steps\.”

ScienceAgent:

- •System: “You are the ScienceAgent\. Analyze and solve the given question with scientific reasoning\. Your input may include feedback from the Orchestrator from the previous round\.”
- •User: “Question: \{question\} Explain your scientific reasoning and provide a final result\.”

CodeAgent:

- •System: “You are the CodeAgent\. Provide a self\-contained Python function that solves the problem\. Your input may include feedback from the Orchestrator from the previous round\.”
- •User: “Question: \{question\} Write a single self\-contained Python function in a markdown code block that solves the problem\.”

##### Orchestrator Agent\.

The orchestrator operates in two modes:

Feedback Mode \(intermediate rounds\):

- •System: “You are the Orchestrator Agent\. Your task is to review the solutions provided by the first\-layer agents in the current round\. Analyze the provided solutions, identify any issues or areas for improvement, and provide constructive feedback\. You may rewrite content, provide specific feedback, and offer improvement suggestions as needed\. Your feedback will be used by the agents in the next round to improve their solutions\.”
- •User: “Question: \{question\} Here are the solutions from the expert agents in the current round: === Solutions === \{block\} === Solutions === Review these solutions and provide feedback for the next round\. If corrections are needed, specify the issues and suggest improvements\. If the solutions are satisfactory, acknowledge them and provide guidance for further refinement\.”

Aggregation Mode \(final round\):

- •System: “You are the Orchestrator Agent\. Your task is to aggregate the solutions provided by the first\-layer agents and produce a final answer wrapped in \\boxed\{\}\.”
- •User: “Question: \{question\} Here are the solutions from the expert agents: === Solutions === \{block\} === Solutions === Based on these inputs, provide the final answer wrapped in \\boxed\{\}\.”

### B\.4Data Mining

We employ an ensemble of XGBoost and LightGBM rather than a single model to obtain more robust and stable feature importance estimates\. These two algorithms use different tree construction strategies: XGBoost grows trees level\-wise, whereas LightGBM grows trees leaf\-wise\. By averaging their attributions, we capture a broader range of feature interactions and reduce variance due to correlated features or random data splits, yielding rankings that better reflect true relevance\.

For each featurejj, we compute two key metrics\. Themean feature importanceI¯j\\bar\{I\}\_\{j\}is obtained by: \(1\) extracting raw feature importancesIj\(m\)I\_\{j\}^\{\(m\)\}from each modelm∈\{XGB,LGB\}m\\in\\\{\\text\{XGB\},\\text\{LGB\}\\\}; \(2\) applying min\-max normalization to obtainI~j\(m\)=\(Ij\(m\)−mink⁡Ik\(m\)\)/\(maxk⁡Ik\(m\)−mink⁡Ik\(m\)\)∈\[0,1\]\\tilde\{I\}\_\{j\}^\{\(m\)\}=\(I\_\{j\}^\{\(m\)\}\-\\min\_\{k\}I\_\{k\}^\{\(m\)\}\)/\(\\max\_\{k\}I\_\{k\}^\{\(m\)\}\-\\min\_\{k\}I\_\{k\}^\{\(m\)\}\)\\in\[0,1\]; and \(3\) averaging:I¯j=12​\(I~jXGB\+I~jLGB\)\\bar\{I\}\_\{j\}=\\frac\{1\}\{2\}\(\\tilde\{I\}\_\{j\}^\{\\text\{XGB\}\}\+\\tilde\{I\}\_\{j\}^\{\\text\{LGB\}\}\)\. TheSHAP correlationρj\\rho\_\{j\}is computed as the average Pearson correlation between feature values𝐱j=\(x1​j,…,xn​j\)\\mathbf\{x\}\_\{j\}=\(x\_\{1j\},\\ldots,x\_\{nj\}\)and their SHAP attributionsϕj\(m\)=\(ϕ1​j\(m\),…,ϕn​j\(m\)\)\\bm\{\\phi\}\_\{j\}^\{\(m\)\}=\(\\phi\_\{1j\}^\{\(m\)\},\\ldots,\\phi\_\{nj\}^\{\(m\)\}\)across both models:ρj=12​∑m∈ℳcorr​\(𝐱j,ϕj\(m\)\)\\rho\_\{j\}=\\frac\{1\}\{2\}\\sum\_\{m\\in\\mathcal\{M\}\}\\text\{corr\}\(\\mathbf\{x\}\_\{j\},\\bm\{\\phi\}\_\{j\}^\{\(m\)\}\)\. The magnitude\|ρj\|\|\\rho\_\{j\}\|quantifies the strength of featurejj’s influence on predicted correctness, whilesign​\(ρj\)\\text\{sign\}\(\\rho\_\{j\}\)indicates its direction\.

## Appendix CEntropy Features

We design a hierarchical feature set to capture entropy dynamics across agents and rounds in MAS\. This section provides formal definitions for all 254 features used in our analysis\.

### C\.1Feature Hierarchy

Our features are organized into four hierarchical levels, reflecting the nested structure of MAS execution:

##### Token Level\.

For each generated tokenttin agentaa’s output, we compute Shannon entropy from the softmax distribution over vocabulary𝒱\\mathcal\{V\}:Ht=−∑v∈𝒱p​\(v\|x<t\)​log⁡p​\(v\|x<t\)H\_\{t\}=\-\\sum\_\{v\\in\\mathcal\{V\}\}p\(v\|x\_\{<t\}\)\\log p\(v\|x\_\{<t\}\)\.

##### Agent Level\.

For each agenta∈Aa\\in Ain roundrr, we aggregate token\-level entropy into summary statistics: total entropyHa\(r\)=∑tHtH\_\{a\}^\{\(r\)\}=\\sum\_\{t\}H\_\{t\}, and distributional measures \(mean, max, min, std, variance, median, Q1, Q3\)\.

##### Round Level\.

For each roundr∈\{1,…,R\}r\\in\\\{1,\\ldots,R\\\}, we aggregate agent\-level statistics across all agents: total round entropyH~\(r\)=∑a∈AHa\(r\)\\tilde\{H\}^\{\(r\)\}=\\sum\_\{a\\in A\}H\_\{a\}^\{\(r\)\}and mean round entropyH¯\(r\)=H~\(r\)/\|A\|\\bar\{H\}^\{\(r\)\}=\\tilde\{H\}^\{\(r\)\}/\|A\|\.

##### Sample Level\.

For each input sampleii, we aggregate round\-level statistics and compute cross\-round dynamics\.

### C\.2Feature Groups

We organize features into semantically coherent groups for analysis\.

##### Entropy Features \(ℱE\\mathcal\{F\}\_\{E\}, 239 features\)\.

Hierarchical structure:

##### Agent\-level statistics \(156 features\)

capture per\-agent reasoning trajectories:

- •Per\-round agent entropy: For each roundrrand statistics∈\{max,mean,std,…\}s\\in\\\{\\text\{max\},\\text\{mean\},\\text\{std\},\\ldots\\\}, we computes​\(\{Ha\(r\)\}a∈A\)s\(\\\{H\_\{a\}^\{\(r\)\}\\\}\_\{a\\in A\}\), yielding features likesample\_round\_1\_max\_agent\_total\_entropy\.
- •Inter\-agent divergence: Variance and coefficient of variation across agents within each round\.

##### Round\-level dynamics \(27 features\)

track temporal evolution:

- •Round totals: Total entropy and token count per round\.
- •Cross\-round changes: First\-to\-last differenceΔ​H=H~\(R\)−H~\(1\)\\Delta H=\\tilde\{H\}^\{\(R\)\}\-\\tilde\{H\}^\{\(1\)\}, ratioH~\(R\)/H~\(1\)\\tilde\{H\}^\{\(R\)\}/\\tilde\{H\}^\{\(1\)\}, and slope per round\.
- •Volatility: Standard deviation of entropy across rounds\.

##### Sample\-level statistics \(29 features\)

aggregate across the full MAS execution:

- •Basic statistics:∑r∑aHa\(r\)\\sum\_\{r\}\\sum\_\{a\}H\_\{a\}^\{\(r\)\}, mean, max, min, std, variance, quartiles\.
- •Distribution shape: Range, IQR, Bowley skewness\(Q3\+Q1−2⋅median\)/IQR\(Q\_\{3\}\+Q\_\{1\}\-2\\cdot\\text\{median\}\)/\\text\{IQR\}, coefficient of variationσ/μ\\sigma/\\mu, tail weight\(max−Q3\)/IQR\(\\max\-Q\_\{3\}\)/\\text\{IQR\}\.
- •Stability index:1−σH/μH1\-\\sigma\_\{H\}/\\mu\_\{H\}, measuring consistency across agents\.
- •Answer token entropy: Statistics computed over tokens in the final\\boxed\{\}output\.

##### System\-level aggregation \(10 features\)

provides global measures:

- •Architecture\-specific: Number of agents, total inference count\.
- •Experiment totals: Aggregate entropy, average entropy per inference\.

##### Base\-model entropy \(ℱbase\-E\\mathcal\{F\}\_\{\\text\{base\-E\}\}, 17 features\)

captures the single\-agent baseline:

- •Base model statistics: Total token\-level entropyHbaseH\_\{\\text\{base\}\}, token count, average entropy per token ofMbaseM\_\{\\text\{base\}\}\.
- •Comparison between MAS and base model: Entropy ratioHMAS/HbaseH\_\{\\text\{MAS\}\}/H\_\{\\text\{base\}\}, reductionHbase−HMASH\_\{\\text\{base\}\}\-H\_\{\\text\{MAS\}\}\.
- •Answer entropy shift: Difference and ratio between base model and MAS final answer entropy\.

##### Computational Metrics \(ℱC\\mathcal\{F\}\_\{C\}, 15 features\)\.

Non\-entropy quantities at the same hierarchical levels:

- •Timing: Total reasoning time, per\-round time\.
- •Token usage: Total tokens generated, per\-agent token count, answer length\.
- •Inference counts: Number of LLM calls per round and total\.
- •Base\-model correctness \(ℱbase\-C\\mathcal\{F\}\_\{\\text\{base\-C\}\}, 4 features\): WhetherMbaseM\_\{\\text\{base\}\}answered correctly, format compliance\.

### C\.3Key Feature Definitions

Table[2](https://arxiv.org/html/2602.04234#A3.T2)lists representative features from each category with their formal definitions\.

Table 2:Representative entropy features and their definitions\.FeatureLevelDefinitionsample\_total\_entropySample∑r,aHa\(r\)\\sum\_\{r,a\}H\_\{a\}^\{\(r\)\}sample\_entropy\_stability\_indexSample1−σH/μH1\-\\sigma\_\{H\}/\\mu\_\{H\}sample\_entropy\_cvSampleσH/μH\\sigma\_\{H\}/\\mu\_\{H\}\(coefficient of variation\)sample\_entropy\_bowley\_skewnessSample\(Q3\+Q1−2⋅med\)/IQR\(Q\_\{3\}\+Q\_\{1\}\-2\\cdot\\text\{med\}\)/\\text\{IQR\}sample\_max\_answer\_token\_entropySamplemaxt⁡Ht\\max\_\{t\}H\_\{t\}fort∈t\\inanswer tokensround\_1\_total\_entropyRound∑a∈AHa\(1\)\\sum\_\{a\\in A\}H\_\{a\}^\{\(1\)\}round\_1\_2\_change\_entropyRound∑aHa\(2\)−∑aHa\(1\)\\sum\_\{a\}H\_\{a\}^\{\(2\)\}\-\\sum\_\{a\}H\_\{a\}^\{\(1\)\}sample\_round\_1\_max\_agent\_max\_entropyAgentmaxa∈A⁡\(maxt⁡Ht\(a,1\)\)\\max\_\{a\\in A\}\(\\max\_\{t\}H\_\{t\}^\{\(a,1\)\}\)sample\_round\_1\_mean\_agent\_std\_entropyAgent1\|A\|​∑aσ​\(Ht\(a,1\)\)\\frac\{1\}\{\|A\|\}\\sum\_\{a\}\\sigma\(H\_\{t\}^\{\(a,1\)\}\)sample\_round\_1\_variance\_agent\_total\_entropyAgentVar​\(\{Ha\(1\)\}a∈A\)\\text\{Var\}\(\\\{H\_\{a\}^\{\(1\)\}\\\}\_\{a\\in A\}\)base\_sample\_total\_entropyBaseHbaseH\_\{\\text\{base\}\}, trajectory\-level entropy ofMbaseM\_\{\\text\{base\}\}sample\_entropy\_ratio\_vs\_base\_totalBaseEntropy ratioHMAS/HbaseH\_\{\\text\{MAS\}\}/H\_\{\\text\{base\}\}
### C\.4Feature Redundancy Discussion

Our goal is to comprehensively explore which aspects of entropy influence MAS effectiveness, rather than to identify a minimal predictor set\. Achieving broad coverage requires summarizing the same token\-level entropy signal through related statistics \(mean and median, std and variance, quartiles and skewness\) and pairing each base\-model feature with its MAS counterpart, which inevitably introduces correlation among features\. We acknowledge this cost because different statistics expose different regimes: the median is robust to outlier tokens while the maximum captures peak entropy, and running SHAP over the full set reveals which one matters in which context\.

To verify that the resulting redundancy does not compromise our findings, we characterize it with four methods: pairwise correlation, principal component analysis, recursive feature elimination, and cross\-method importance validation\. The analyses below support two complementary conclusions\. First, the redundancy does not undermine our predictive or interpretive results\. The 183:1 sample\-to\-feature ratio \(44,780 to 245\), tree\-ensemble regularization, and consistent top features across four independent importance methods together show that the conclusions in Section[4\.4](https://arxiv.org/html/2602.04234#S4.SS4)are stable under correlated inputs\. Second, redundancy does matter for causal inference, where collinearity biases structure learning and confounds effect estimation\. We therefore prune redundancy in our causal pipeline, applying Borda\-fusion feature selection followed by a\|ρ\|\>0\.85\|\\rho\|\>0\.85threshold to obtain 28 non\-redundant features for PC and FCI graph discovery and DoWhy ATE estimation \(Appendix[D](https://arxiv.org/html/2602.04234#A4)\)\. The exploratory and causal feature sets thus play complementary roles: we keep redundancy where it aids exploration and remove it where it would distort causal claims\.

#### C\.4\.1Pairwise Correlation Analysis

To characterize feature redundancy, we analyze the pairwise Pearson correlation matrix of𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}\. Consistent with expectations, highly correlated features form distinct blocks \(e\.g\.,sample\_mean\_\*withsample\_median\_\*;std\_\*withvariance\_\*\)\. Rather than discarding these as redundant, we leverage this structure to observe which specific statistic SHAP prioritizes within each block, thereby offering finer\-grained interpretability than selecting a single representative feature\. More notably, base\-model entropy features show strong associations with MAS entropy dynamics\. For example,base\_model\_min\_answer\_token\_entropyis correlated withanswer\_token\_entropy\_change\_direction, andbase\_model\_is\_finally\_correctis correlated withis\_finally\_correct\. These cross\-tier correlations directly reinforce our finding in Section[4\.4](https://arxiv.org/html/2602.04234#S4.SS4)thatMbaseM\_\{\\text\{base\}\}entropy and correctness condition MAS effectiveness\. In addition,architectureexhibits strong correlations with numerous entropy features \(\|ρ\|\>0\.5\|\\rho\|\>0\.5\), confirming that different MAS topologies induce distinct entropy dynamics, consistent with Section[5\.2](https://arxiv.org/html/2602.04234#S5.SS2)\. The correlation map also informs the\|ρ\|\>0\.85\|\\rho\|\>0\.85pruning threshold subsequently applied in our causal pipeline \(Appendix[D](https://arxiv.org/html/2602.04234#A4)\)\.

#### C\.4\.2PCA Analysis

We apply principal component analysis to the standardized 245\-dimensional feature space computed over 44,780 samples\. As shown in Figure[7\(a\)](https://arxiv.org/html/2602.04234#A3.F7.sf1), the eigenvalue spectrum reveals a distributed variance structure\. The first principal component explains 21\.1% of total variance, the top three components account for 41\.3%, the top five for 54\.0%, and 43 components are required to reach 95% cumulative explained variance, an 82\.4% dimensionality reduction\.

We draw two operational implications, neither of which contradicts the existence of definitional redundancy\. First, the absence of a small dominant subspace \(the top three components explain only about 41%\) indicates that the underlying entropy signal is spread across many orthogonal directions\. Even where individual features overlap, the feature set as a whole captures diverse aspects of MAS entropy rather than a single dominant axis\. Second, although 43 components suffice for 95% variance, replacing the raw features with this PCA representation incurs a non\-trivial classification cost\. XGBoost F1 drops from 89\.16% to 86\.13% \(Δ\\DeltaF1 =−\-3\.03%\), LightGBM from 89\.10% to 86\.03% \(Δ\\DeltaF1 =−\-3\.07%\), and Random Forest from 88\.13% to 86\.06% \(Δ\\DeltaF1 =−\-2\.07%\)\. The raw features therefore carry complementary nonlinear information that a linear projection erases\. This is why we retain the full set for the Entropy Judger and reserve targeted redundancy removal for the causal analysis, where collinearity is genuinely harmful\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/feature_redundancy/pca_analysis.png)\(a\)
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/feature_redundancy/feature_ablation.png)\(b\)

Figure 7:PCA analysis and feature ablation study\. \(a\) PCA variance explained for the 245\-dimensional entropy feature space\. The curve shows cumulative explained variance as a function of the number of principal components\. Achieving 95% explained variance requires 43 components, indicating that information is distributed across many dimensions rather than concentrated in a few dominant directions\. \(b\) Recursive feature elimination \(RFE\) performance curve\. The optimal subset of 25 features achieves the highest accuracy \(86\.77%\) and F1 \(88\.92%\)\. Performance plateaus and slightly decreases as more features are added, indicating that the model does not overfit to the high\-dimensional space\.
#### C\.4\.3Feature Ablation Study

To assess how much unique predictive content the redundant feature set carries, we perform recursive feature elimination \(RFE\) using Random Forest as the base estimator with 5\-fold cross\-validation\. Figure[7\(b\)](https://arxiv.org/html/2602.04234#A3.F7.sf2)presents the performance trajectory as features are progressively removed\.

The results reveal a characteristic plateau pattern\. A compact subset of 25 features achieves near\-optimal performance \(accuracy 86\.77%, F1 88\.92%\), and adding the remaining 220 features only marginally changes performance \(full\-set accuracy 85\.85%, F1 88\.13%\)\. Performance does not continuously increase with more features\. It slightly decreases beyond the optimal 25, consistent with a regularized model that absorbs correlated inputs without overfitting\. This plateau is consistent with the redundancy we explicitly designed into the feature set\. A small core of features carries most of the predictive signal, while the remainder provides interpretive coverage across statistics and hierarchical levels at no cost to predictive performance\. The same observation directly motivates the targeted feature selection used in our causal pipeline \(Appendix[D](https://arxiv.org/html/2602.04234#A4)\), where keeping the redundant tail would distort structure learning rather than aid interpretation\.

We further examine individual feature contributions through leave\-one\-out ablation from the full 245\-feature set\. Removingbase\_sample\_token\_countcauses the largest single\-feature accuracy drop \(Δ\\DeltaAcc =−\-0\.0163,Δ\\DeltaF1 =−\-0\.0142\), followed bysample\_answer\_token\_count\(Δ\\DeltaAcc =−\-0\.0151,Δ\\DeltaF1 =−\-0\.0125\)\. Even within a high\-dimensional space with correlated features, removing specific features still produces measurable degradation, indicating that the redundancy is partial rather than total\. Each feature retains some unique residual signal beyond its correlated neighbors\.

The favorable sample\-to\-feature ratio of our dataset \(44,780 samples and 245 features, about 183 to 1\) substantially exceeds conventional overfitting thresholds \(typically 10:1 to 20:1\), and the 5\-fold cross\-validation standard deviation remains below 0\.02 across all settings\. Together with the plateau behavior above, this confirms that retaining the full redundant feature set does not destabilize the predictive conclusions reported in Section[4\.4](https://arxiv.org/html/2602.04234#S4.SS4)\.

#### C\.4\.4Cross\-Method Feature Importance Validation

A potential concern with SHAP\-based feature importance is that correlated features may cause importance to be distributed unreliably across redundant groups, yielding rankings that depend on the choice of estimator\. To check whether our interpretive conclusions are robust to this effect, we compare feature importance rankings produced by four independent methods with fundamentally different assumptions: tree\-based importance \(Random Forest\), logistic regression coefficients, chi\-square statistical association, and mutual information\.

As shown in Figure[8](https://arxiv.org/html/2602.04234#A3.F8), the methods produce remarkably consistent top\-feature rankings despite their distinct mechanisms\. The featurebase\_model\_is\_finally\_correctranks first in tree\-based importance, logistic regression, chi\-square, and F\-statistic methods, and third in mutual information\. Averaging tree\-based and logistic regression ranks gives a top\-5 ofbase\_model\_is\_finally\_correct\(avg\. rank 1\.0\),base\_model\_accuracy\(4\.0\),base\_sample\_avg\_entropy\_per\_token\(6\.5\),sample\_round\_1\_median\_agent\_max\_entropy\(23\.0\), andbase\_sample\_token\_count\(23\.5\)\.

These top\-5 features span three distinct categories: base\-model correctness indicators \(ranks 1 and 2\), base\-model entropy characteristics \(ranks 3 and 5\), and round\-level agent entropy statistics \(rank 4\)\. This category coverage matters for our argument\. Since the same high\-importance signals appear across base\-model quality, single\-agent entropy, and multi\-agent coordination, the conclusions in Section[4\.4](https://arxiv.org/html/2602.04234#S4.SS4)cannot be artifacts of one correlated cluster dominating a single ranker\.

The methods do reveal complementary secondary emphases\. Tree models prioritize correctness indicators and answer token counts, logistic regression highlights answer entropy and format compliance, chi\-square emphasizes quartile entropy statistics, and mutual information favors total entropy and round\-level features\. This divergence is exactly what the redundant feature design is meant to enable\. When several correlated features encode overlapping signals, different importance methods surface different members of each block, and retaining the full set lets us read off these complementary perspectives instead of committing to a single representative chosen up front\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/feature_redundancy/cross_method_importance.png)Figure 8:Cross\-method feature importance comparison across tree\-based \(Random Forest\), logistic regression, chi\-square, mutual information, and F\-statistic methods\. Despite fundamentally different mechanisms, the methods produce consistent top\-feature rankings, validating the robustness of our feature importance findings\.

## Appendix DCausal Discovery and Effect Estimation

Appendix[C](https://arxiv.org/html/2602.04234#A3)characterized the redundancy structure of the 245\-dimensional entropy feature set and noted that, while this redundancy is benign for SHAP\-based exploration, it would distort causal structure learning and effect estimation\. This appendix takes the next step: we apply Borda\-fusion feature selection with a\|ρ\|\>0\.85\|\\rho\|\>0\.85pruning threshold to obtain a 28\-feature non\-redundant subset, then run constraint\-based causal discovery \(PC and FCI\) and the DoWhy framework on the resulting variables to identify direct causal factors and mediation pathways for MAS correctness across 44,780 samples\. The pipeline is the global counterpart of the per\-finding causal validations summarized in Appendix[F](https://arxiv.org/html/2602.04234#A6), and it is complemented from a different angle by the controlled three\-way SAS/MAS\-Round 1/MAS\-Round 2 experiment in Appendix[K](https://arxiv.org/html/2602.04234#A11), which isolates the role\-assignment and inter\-agent\-interaction contributions to entropy dynamics rather than estimating the effect of entropy on correctness\.

### D\.1Feature Selection Pipeline

The original feature space comprises 245 token\-level entropy features\. To ensure identifiability and interpretability for causal discovery, we apply a multi\-method fusion pipeline that reduces dimensionality by 88\.6% while preserving coverage of all hierarchical feature levels\.

##### Ranking Methods\.

Four complementary methods contribute to a weighted reciprocal\-rank \(Borda Count\) fusion:

- •Combined Tree \+ Logistic Regression importance\(weight=3\.0=3\.0\): captures both non\-linear and linear predictive relevance\.
- •Mutual Information\(weight=1\.5=1\.5\): measures general statistical dependence with the outcome\.
- •Chi\-squared test\(weight=1\.0=1\.0\): assesses categorical association strength\.
- •ANOVA F\-test\(weight=1\.0=1\.0\): evaluates between\-group variance for continuous features\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/x9.png)
Figure 9:Borda fusion scores for the 28 selected features, colored by hierarchical tier\. Features are sorted by score; abbreviated names follow the conventions used in the main text\.Table 3:Selected features ranked by Borda score, grouped by hierarchical feature level\.
RankFeatureScoreBase\-Model Entropyℱbase\-E\\mathcal\{F\}\_\{\\text\{base\-E\}\}\(2 features\)3base\_sample\_total\_entropy0\.020310base\_sample\_avg\_entropy\_per\_token0\.0092Agent\-Level Statistics, Round 1 \(11 features\)1sample\_round\_1\_max\_agent\_std\_entropy0\.03235sample\_round\_1\_agent\_mean\_entropy\_spread0\.01476sample\_round\_1\_std\_agent\_mean\_entropy0\.01349sample\_round\_1\_agent\_total\_entropy\_spread0\.009917sample\_round\_1\_max\_agent\_max\_entropy0\.006519sample\_round\_1\_std\_agent\_total\_entropy0\.005921sample\_round\_1\_std\_agent\_variance\_entropy0\.005525sample\_round\_1\_q3\_agent\_total\_entropy0\.005020round\_1\_total\_entropy0\.005812exp\_infer\_average\_entropy0\.008028sample\_round\_1\_q1\_agent\_max\_entropy0\.0047Round 2 & Sample\-Level \(15 features\)2sample\_mean\_answer\_token\_entropy0\.02804answer\_token\_entropy\_change0\.01917sample\_round\_2\_std\_agent\_mean\_entropy0\.01188sample\_round\_2\_agent\_mean\_entropy\_spread0\.010511sample\_round\_2\_std\_agent\_std\_entropy0\.008013round\_2\_infer\_avg\_entropy0\.007614sample\_round\_2\_max\_agent\_max\_entropy0\.007415sample\_round\_2\_std\_agent\_total\_entropy0\.007416sample\_max\_answer\_token\_entropy0\.006718sample\_median\_answer\_token\_entropy0\.006022sample\_round\_2\_agent\_total\_entropy\_spread0\.005423sample\_round\_2\_std\_agent\_max\_entropy0\.005124sample\_round\_2\_max\_agent\_total\_entropy0\.005126exp\_total\_entropy0\.004927sample\_avg\_entropy\_per\_token\_diff\_vs\_base0\.0049
![Refer to caption](https://arxiv.org/html/2602.04234v6/x10.png)Figure 10:Pairwise Spearman correlation matrix of the 28 selected features\. The upper\-triangular heatmap confirms that no remaining pair exceeds the\|ρ\|\>0\.85\|\\rho\|\>0\.85redundancy threshold, validating that the selected subset is non\-redundant while spanning all hierarchical feature levels\.
##### Redundancy Removal and Hierarchical Coverage\.

After ranking, we remove redundant features whose pairwise Spearman correlation exceeds\|ρ\|\>0\.85\|\\rho\|\>0\.85, retaining the higher\-scored feature in each correlated pair\. A coverage constraint then ensures that the final subset contains at least one representative from each hierarchical level defined in Appendix[C](https://arxiv.org/html/2602.04234#A3): agent\-level statistics, round\-level dynamics, sample\-level statistics, system\-level aggregation, base\-model entropy \(ℱbase\-E\\mathcal\{F\}\_\{\\text\{base\-E\}\}\), and computational metrics \(ℱC\\mathcal\{F\}\_\{C\}\)\. The final subset contains28 features; Figure[9](https://arxiv.org/html/2602.04234#A4.F9)visualizes the Borda fusion scores colored by hierarchical tier, Table[3](https://arxiv.org/html/2602.04234#A4.T3)lists all selected features grouped by hierarchical level, and Figure[10](https://arxiv.org/html/2602.04234#A4.F10)shows the pairwise Spearman correlation matrix of the 28 selected features, confirming that no remaining pair exceeds the\|ρ\|\>0\.85\|\\rho\|\>0\.85redundancy threshold\.

### D\.2Causal Structure Discovery

We apply two constraint\-based algorithms to learn the causal graph over the 28 selected features plus the binary outcomeis\_finally\_correct\(29 variables total\)\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/causal_infer/causal_discovery/causal_graph_pc.png)\(a\)PC algorithm \(55 directed \+ 3 undirected edges\)\.
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/causal_infer/causal_discovery/causal_graph_fci.png)\(b\)FCI algorithm \(40 directed and 6 bidirected edges\)\.

Figure 11:Individual causal graphs from constraint\-based discovery\. The PC graph \(a\) assumes causal sufficiency and produces undirected edges where orientation is undetermined\. The FCI graph \(b\) relaxes causal sufficiency, introducing bidirected edges \(↔\\leftrightarrow\) to indicate possible latent confounding between connected variables\.##### Algorithm Configuration\.

Both the PC algorithm and the FCI algorithm use the Fisher\-Z conditional independence test at significance levelα=0\.01\\alpha=0\.01\. We encode temporal background knowledge as tier constraints to enforce temporal consistency:

- •Tier 0\(2 variables\): Base model properties \- exogenous, cannot be caused by downstream features\.
- •Tier 1\(11 variables\): Round 1 entropy features \- cannot cause Tier 0 variables\.
- •Tier 2\(15 variables\): Round 2, cross\-round, and sample\-level aggregation features \- cannot cause Tier 0 or Tier 1 variables\.
- •Tier 3\(1 variable\): Outcome \(is\_finally\_correct\) \- no outgoing edges permitted\.

##### Results\.

The PC algorithm discovers 55 directed and 3 undirected edges; FCI discovers 40 directed and 6 bidirected edges \(the latter indicating possible latent confounding\)\. A total of 80 edges are identified by both algorithms \(consensus edges\), out of 95 unique directed edges in the union\. Figure[11](https://arxiv.org/html/2602.04234#A4.F11)presents the full causal graphs from each algorithm separately, and Figure[6\(a\)](https://arxiv.org/html/2602.04234#S5.F6.sf1)\(in Section[5\.4](https://arxiv.org/html/2602.04234#S5.SS4)\) shows the consensus graph\. Figure[12](https://arxiv.org/html/2602.04234#A4.F12)confirms the stability of the consensus DAG across both algorithms\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/x11.png)Figure 12:PC vs\. FCI edge agreement heatmap over 29 nodes\. Dark blue = both algorithms agree \(80 edges\); light blue = PC only \(15 edges\); red = FCI only \(0 edges\)\. High overlap confirms that the consensus DAG is stable across Markov\-boundary assumptions\.
##### Direct Causes of Correctness\.

The PC algorithm identifies four direct causes of MAS correctness: base\-model average per\-token entropy, round\-1 total entropy, maximum answer\-token entropy, and round\-1 maximum agent entropy dispersion\. FCI, which accounts for latent confounders, identifies three of these: base\-model average per\-token entropy, round\-1 total entropy, and maximum answer\-token entropy\.The consensus direct causes across both algorithms are thus base\-model average per\-token entropy, round\-1 total entropy, and maximum answer\-token entropy, indicating that these three entropy signals exert the most robust direct causal influence on MAS correctness\.Round\-1 maximum agent entropy dispersion is identified as a direct cause by PC only, suggesting it may share an unobserved common cause with correctness that FCI identifies as a latent confounder\.

### D\.3Causal Effect Estimation

We estimate the Average Treatment Effect \(ATE\) of five candidate treatment variables onis\_finally\_correctusing the DoWhy framework with 15 confounders identified from the learned causal graph\.

##### Estimation Methods\.

Three estimators are applied for each treatment: Linear Regression \(LR\), Propensity Score Stratification \(PS\), and Inverse Probability Weighting \(IPW\)\. Robustness is assessed via three refutation tests: \(1\)*Random Common Cause*: adding a random confounder should not alter the estimate; \(2\)*Placebo Treatment*: replacing the treatment with a random variable should eliminate the effect; \(3\)*Data Subset*: re\-estimating on a random 80% subset should yield a consistent estimate\.

Table 4:Causal effect estimates and refutation results\. “Direct” indicates whether the variable is a consensus direct cause from PC/FCI\. \*Direct cause in PC only \(not in FCI\)\. RCC = Random Common Cause; all refutation tests pass\.TreatmentATELRATEPSATEIPWpp\-valueDirectRCCPlaceboSubsetbase\_sample\_avg\_ent\_per\_tok−0\.771\-0\.771−0\.123\-0\.123−0\.146\-0\.1461\.3​e\-​211\.3\\text\{e\-\}21Yes✓✓✓round\_1\_total\_entropy\+0\.000\+0\.000−0\.007\-0\.007\+0\.056\+0\.0561\.3​e\-​191\.3\\text\{e\-\}19Yes✓✓✓sample\_max\_ans\_tok\_ent−0\.350\-0\.350−0\.309\-0\.309−0\.339\-0\.3396\.8​e\-​296\.8\\text\{e\-\}29Yes✓✓✓sample\_r1\_max\_agent\_std\_ent−0\.278\-0\.278−0\.152\-0\.152−0\.127\-0\.1273\.6​e\-​33\.6\\text\{e\-\}3Yes\*✓✓✓ans\_tok\_ent\_change−0\.194\-0\.194−0\.211\-0\.211−0\.238\-0\.2385\.8​e\-​105\.8\\text\{e\-\}10No✓✓✓![Refer to caption](https://arxiv.org/html/2602.04234v6/images/causal_infer/causal_effects/refutation_summary.png)Figure 13:Refutation test results for all five treatment variables\. All treatments pass all three tests \(Random Common Cause, Placebo Treatment, Data Subset\), confirming robustness of the causal effect estimates against unmeasured confounding, spurious associations, and sampling variability\.Figure[6\(b\)](https://arxiv.org/html/2602.04234#S5.F6.sf2)\(in Section[5\.4](https://arxiv.org/html/2602.04234#S5.SS4)\) compares the ATE estimates across treatment variables and estimation methods, and Figure[13](https://arxiv.org/html/2602.04234#A4.F13)summarizes the refutation test results\.

##### Interpretation\.

All five treatment variables exhibit statistically significant effects \(p<0\.01p<0\.01\)\. The strongest and most consistent effect belongs tobase\_sample\_avg\_entropy\_per\_token\(p<10−21p<10^\{\-21\}\): higher base model entropy strongly reduces MAS correctness probability\. The propensity\-based estimators, PS \(−0\.123\-0\.123\) and IPW \(−0\.146\-0\.146\), constitute the primary quantitative evidence for the effect magnitude, as they reweight rather than linearly project and are robust to model misspecification\.round\_1\_total\_entropycarries a near\-zero LR estimate \(0\.0000\.000\) and small PS/IPW effects \(−0\.007\-0\.007and\+0\.056\+0\.056\), reflecting that its direct causal influence is largely mediated through downstream entropy nodes rather than acting as an independent linear driver; the highly significantp<10−19p<10^\{\-19\}reflects its strong structural role in the learned causal graph rather than a large marginal effect size\. All five treatment variables pass all three refutation tests, confirming that the estimated effects are robust to unmeasured confounding, spurious treatment assignment, and sampling variability\.

### D\.4Mediation Analysis

To understand the mechanistic pathways through which entropy features influence MAS correctness, we conduct mediation analysis using the Baron\-Kenny framework with bootstrap confidence intervals \(n=1,000n=1\{,\}000resamples\)\. Of 7 candidate mediation paths, 6 exhibit significant indirect effects \(bootstrap 95% CI excluding zero\)\.

Table 5:Significant mediation paths ordered by\|indirect effect\|\|\\text\{indirect effect\}\|\. CI denotes the bootstrap 95% confidence interval\. Proportion mediated=Indirect/Total=\\text\{Indirect\}/\\text\{Total\}\(Baron\-Kenny\), whereTotal=Direct\+Indirect\\text\{Total\}=\\text\{Direct\}\+\\text\{Indirect\}\.TreatmentMediatorIndirectTotalProp\.95% CIRound 1 Inter\-Agent Dispersion→\\toRound 2 Entropy→\\toCorrectnessr1\_max\_agent\_std\_entr2\_agent\_mean\_ent\_spread\+0\.049\+0\.049−0\.496\-0\.496−9\.8\-9\.8%\[\+0\.033\+0\.033,\+0\.066\+0\.066\]r1\_agent\_mean\_ent\_spreadr2\_agent\_mean\_ent\_spread−0\.048\-0\.048−0\.149\-0\.14932\.5%\[−0\.066\-0\.066,−0\.030\-0\.030\]r1\_std\_agent\_mean\_entr2\_agent\_mean\_ent\_spread−0\.047\-0\.047−0\.152\-0\.15230\.9%\[−0\.066\-0\.066,−0\.028\-0\.028\]r1\_agent\_mean\_ent\_spreadr2\_std\_agent\_mean\_ent−0\.046\-0\.046−0\.149\-0\.14930\.8%\[−0\.064\-0\.064,−0\.028\-0\.028\]r1\_max\_agent\_std\_entr2\_std\_agent\_mean\_ent\+0\.046\+0\.046−0\.496\-0\.496−9\.2\-9\.2%\[\+0\.030\+0\.030,\+0\.062\+0\.062\]r1\_std\_agent\_mean\_entr2\_std\_agent\_mean\_ent−0\.045\-0\.045−0\.152\-0\.15229\.4%\[−0\.061\-0\.061,−0\.027\-0\.027\]Figure[14](https://arxiv.org/html/2602.04234#A4.F14)illustrates the mediation pathway, and Figure[15](https://arxiv.org/html/2602.04234#A4.F15)presents the full mediation decomposition of direct and indirect effects for all 7 candidate paths\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/causal_infer/mediation/path_diagrams.png)Figure 14:Schematic path diagrams for the cross\-round causal mediation chain: Round 1 inter\-agent entropy dispersion→\\toRound 2 entropy→\\tocorrectness\. Arrows represent directed causal links identified from the learned causal graph\.![Refer to caption](https://arxiv.org/html/2602.04234v6/x12.png)Figure 15:Full mediation decomposition for the 7 candidate paths by\|total effect\|\|\\text\{total effect\}\|\. Stacked bars show direct effect \(red,c′c^\{\\prime\}\) and mediated indirect effect \(blue,a×ba\\times b\); faded bars indicate non\-significant indirect effects\. Paths are sorted by total effect magnitude\. The 6 paths with significant indirect effects match those reported in Table[5](https://arxiv.org/html/2602.04234#A4.T5)\.##### Cross\-Round Entropy Propagation\.

Round 1 agent\-level entropy dispersion features transmit 29–33% of their causal effects on correctness through Round 2 entropy, specifically through the spread of agent mean entropy in round 2\. This confirms that initial entropy heterogeneity among agents causally shapes the post\-interaction entropy landscape: early misalignment compounds into the subsequent round rather than self\-correcting\. The two paths originating fromsample\_round\_1\_max\_agent\_std\_entropyshow negative indirect\-to\-total ratios \(−9\.8%\-9\.8\\%and−9\.2%\-9\.2\\%\), reflecting partial suppression: this high\-dispersion signal propagates positively into round\-2 spread features whose direct effect on correctness is itself negative, creating a suppressive chain that partially offsets the strong direct harm of round\-1 maximum dispersion on correctness\.

## Appendix EEntropy Patterns Generalize to Agentic Tasks

This appendix validates that the entropy\-performance findings from reasoning benchmarks extend to agentic settings where agents must orchestrate external tools\. We present full results on the general agentic taskGAIA\(Mialonet al\.,[2024](https://arxiv.org/html/2602.04234#bib.bib55)\)and the financial domain agentic taskFinanceAgent\(Bigeardet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib61)\)\.

### E\.1GAIA: First\-Round Dispersion Mediates Tool Execution Failure

#### E\.1\.1Experimental Setup

##### Benchmark and Models\.

We evaluate on the fullGAIAvalidation split \(165 questions across Levels 1, 2, and 3\), using six base models spanning two families and four parameter scales: Llama\-3\.1\-8B\-Instruct, Llama\-3\.2\-3B\-Instruct, Qwen3\-0\.6B, Qwen3\-4B, Qwen3\-8B, and Qwen3\-14B\. We refer to these as L\-8, L\-3, Q\-0\.6, Q\-4, Q\-8, and Q\-14 in figures\.

##### Architectures and configuration\.

We evaluate the same five architectures \(Single, Sequential, Centralized, Hybrid, Debate\) atR=2R\{=\}2interaction rounds, plus the base\-model baseline\. Each agent operates in a ReAct loop with up to 8192 new tokens per step\. We use the officialGAIAprompt template and require the model to place its final answer after theFINAL ANSWER:marker\.

##### Tool suite\.

The agent has access to five tools: web search via SerperAPI, a sandboxed math\-only calculator, a file reader supporting PDF, Excel, CSV, DOCX, and PPTX, a sub\-process Python executor, and a multimodal viewer\. Since all our base models are text\-only LLMs, we expose the multimodal viewer as a tool so the agent can delegate questions that require visual understanding to Doubao\-Seed\-2\.0\-Lite\. Each tool runs under an asynchronous timeout \(calculator and file reader 30s, web search 90s, multimodal viewer 120s, Python executor 150s\)\. In addition, when the model issues the same tool call with identical arguments three times in a row, we force it to emit a final answer in the next loop iteration to avoid unproductive cycles\.

##### Entropy capture\.

Unlike the single\-pass reasoning benchmarks,GAIArequires multiple rounds of tool invocation and reasoning before a final answer can be produced\. Each agent therefore runs a ReAct loop of up to ten steps, in which the LLM first reasons, decides which tool to invoke, receives the tool’s output, and then reasons again on the next step\. We record token\-level entropy at every such step, which inserts an additional*step*granularity between the existing token and agent levels\. The resulting hierarchy \(token, step, agent, sample, round\) captures the model’s uncertainty at each tool\-calling decision rather than only at the round level used in the main analysis\.

#### E\.1\.2Main Results

![Refer to caption](https://arxiv.org/html/2602.04234v6/x13.png)\(a\)Per\-model accuracy across architectures\. Dashed tick marks show base\-model accuracy\.
![Refer to caption](https://arxiv.org/html/2602.04234v6/x14.png)\(b\)Accuracy vs\. entropy quintile onGAIA\(solid\) and non\-GAIA reasoning benchmarks \(dashed\)\.
![Refer to caption](https://arxiv.org/html/2602.04234v6/x15.png)\(c\)Per\-sample SHAP contribution of the three top entropy features; all slope negatively\.
![Refer to caption](https://arxiv.org/html/2602.04234v6/x16.png)\(d\)Joint distribution of tool effective rate and round\-1 upper\-quartile inter\-agent dispersion\.

Figure 16:GAIAresults\. \(a\) SAS remains competitive: it reaches the highest accuracy on 2 of 6 base models and beats at least one MAS on every model; Debate consistently lags\. \(b\) Tool\-call entropy negatively predicts accuracy across all architectures, mirroring the sample\-total\-entropy pattern on reasoning benchmarks\. \(c\) Round\-1 inter\-agent dispersion, mean tool\-call entropy, and step\-1 mean entropy are the three most predictive features, all with negative SHAP slope\. \(d\) Correct samples cluster at high tool effective rate and low inter\-agent dispersion; wrong samples spread toward the opposite corner\.##### SAS remains competitive on agentic tasks\.

As shown in Figure[16\(a\)](https://arxiv.org/html/2602.04234#A5.F16.sf1)and Table[6](https://arxiv.org/html/2602.04234#A5.T6), SAS achieves the highest accuracy among all architectures on two of the six base models and outperforms at least one MAS variant on every model, whereas Debate consistently lags—a trend also observed onFinanceAgent\(Appendix[E\.2\.2](https://arxiv.org/html/2602.04234#A5.SS2.SSS2)\)\. Notably, on the four Qwen3 models, base model performance equals or exceeds that of SAS, suggesting that whenthe base model is already competent at tool calling, neither SAS self\-deliberation nor MAS interaction yields additional benefits\.Overall, weak models derive little value from MAS, while strong models gain only modestly\.

Table 6:GAIAaccuracy \(%\) by architecture and base model\. The Base column gives base model accuracy\. Bold marks the best MAS architecture per model\.ModelBaseSingleSequentialCentralizedHybridDebateLLaMA\-3\.1\-8B\-Instruct1\.83\.04\.29\.12\.43\.0LLaMA\-3\.2\-3B\-Instruct4\.25\.52\.42\.42\.42\.4Qwen3\-0\.6B2\.41\.21\.22\.42\.41\.2Qwen3\-4B18\.818\.219\.422\.415\.814\.5Qwen3\-8B23\.623\.019\.421\.819\.417\.0Qwen3\-14B22\.422\.421\.820\.624\.27\.9
##### Tool\-call entropy is negatively correlated with performance\.

Figure[16\(b\)](https://arxiv.org/html/2602.04234#A5.F16.sf2)shows accuracy against entropy quintiles for all five architectures onGAIAand non\-GAIA reasoning tasks\. OnGAIA, every architecture shows decreasing accuracy as tool\-call entropy increases, matching the negative correlation on reasoning benchmarks\.Higher entropy of the tokens the task depends on, sample total entropy on reasoning tasks, tool\-call entropy on agentic tasks, predicts lower accuracy in both settings\.

##### Early\-round entropy constrains agentic MAS\.

Figure[16\(c\)](https://arxiv.org/html/2602.04234#A5.F16.sf3)shows the three most important features on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}: round\-1 upper\-quartile inter\-agent dispersion, mean tool\-call entropy, and step\-1 mean entropy, all with negative SHAP slope\.Beyond entropy during tool invocation, MAS performance is limited by first\-round entropy and, more granularly, by uncertainty in each agent’s initial reasoning step\. This aligns with the first\-round dominance finding from reasoning benchmarks\.

##### Round\-1 inter\-agent dispersion lowers tool execution success\.

Figure[16\(d\)](https://arxiv.org/html/2602.04234#A5.F16.sf4)plots tool effective rate against round\-1 upper\-quartile inter\-agent dispersion\. Correct samples cluster at high effective rate and low dispersion; wrong samples spread toward the low\-effective\-rate and high\-dispersion corner\.Higher inter\-agent entropy dispersion lowers tool execution success, which in turn lowers final correctness\.

#### E\.1\.3Additional Feature Analysis

Figure[17\(a\)](https://arxiv.org/html/2602.04234#A5.F17.sf1)plots per\-step tool\-call decision entropy separately for correct and wrong trajectories, computed across all GAIA samples with at least one tool call\. Correct trajectories \(n = 303\) start at step\-0 entropy of about 0\.029 and average 0\.063 across the trajectory; wrong trajectories \(n = 3,389\) start at about 0\.057 and stay elevated through their last active step\. Failure is therefore dynamic rather than visible at step 0 alone:the two classes diverge as the trajectory progresses, hesitation persists longer in wrong trajectories, and wrong trajectories also tend to use up more steps before terminating\. Figure[17\(b\)](https://arxiv.org/html/2602.04234#A5.F17.sf2)compares the five architectures on three normalized axes \(tool effective rate, low tool\-call entropy, low round\-1 max entropy\), showing that no single architecture dominates all three: Centralized leads on tool effectiveness, Hybrid on tool\-call entropy, and Sequential on round\-1 entropy\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/x17.png)\(a\)
![Refer to caption](https://arxiv.org/html/2602.04234v6/x18.png)\(b\)

Figure 17:\(a\) Per\-step tool\-call entropy across all GAIA samples with at least one tool call \(n=3,692n\{=\}3\{,\}692\)\. \(b\) Architecture comparison on three normalized axes; outward is better on every axis\.
#### E\.1\.4Causal Validation

We apply the same causal pipeline used in Appendix[D](https://arxiv.org/html/2602.04234#A4)to the GAIA cohort\. The added step\-level entropy features expand the feature space from 245 to 295 dimensions\. Multi\-method fusion ranking with redundancy removal selects 30 non\-redundant features from this space, PC and FCI with temporal tier constraints recover the causal graph, and DoWhy with three refutation tests estimates causal effects\. Both PC and FCI return a single consensus direct cause of correctness: round\-1 tool success rate \(ATE=0\.068\\text\{ATE\}\{=\}0\.068,p=6\.8×10−4p\{=\}6\.8\\\!\\times\\\!10^\{\-4\}\)\. All three refutation tests pass, with random common cause changing the effect by 0\.1%, placebo treatment by 97\.7%, and data subset by 0\.9%\. This is the agentic\-task analogue of the Base Entropy result:the same upstream model capability that surfaces as base\-model token entropy on reasoning tasks surfaces as tool\-execution success onGAIA, with entropy retained as a mediator\. Mediation analysis identifies a significant indirect path from round\-1 inter\-agent skewness through round\-2 maximum agent dispersion to correctness \(indirect effect−0\.019\-0\.019, bootstrap 95% CI\[−0\.042,−0\.003\]\[\-0\.042,\-0\.003\]\), confirming that the round\-1 to round\-2 dispersion path remains operative on tool\-augmented tasks\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/gaia/causal/causal_graph_consensus.png)\(a\)Consensus causal graph
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/gaia/causal/ate_comparison.png)\(b\)ATE estimates
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/gaia/causal/effect_decomposition.png)\(c\)Mediation effect decomposition

Figure 18:Causal triplet onGAIA\. Round\-1 tool success rate is the unique consensus direct cause of correctness; round\-1 inter\-agent dispersion mediates a significant indirect path through round\-2 dispersion\.
#### E\.1\.5Tool\-Failure Attribution by Base Model

Figure[19](https://arxiv.org/html/2602.04234#A5.F19)attributes the 78,295 tool calls executed during the GAIA experiments to eight failure categories\. The aggregate failure rate is 70\.8%\. The decomposition reveals that failure is model\-shaped, not random\. Weak models \(Llama\-3\.2\-3B at 81\.6% failure rate, Llama\-3\.1\-8B at 76\.6%, Qwen3\-0\.6B at 52\.0%\) issue many calls, and most of those calls fail at the interface layer: malformed invocations that the framework rejects with non\-standard errors \(54\.0% of failures for Q\-0\.6, 41\.9% for L\-3\) or unparseable output \(45\.1% for L\-8\)\. The tool never actually ran\. Strong models \(Qwen3\-8B at 13\.0%, Qwen3\-4B at 15\.7%, Qwen3\-14B at 18\.6%\) call tools much less often, and their failures concentrate on executed\-with\-error and empty results: the tool ran, but with the wrong intent\.Weak models struggle to produce valid formats, so the priority is syntactic correctness, while strong models need to improve the quality of generated content like search queries and code to ensure effective tool execution\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/x19.png)Figure 19:Tool\-call failure attribution per base model\. \(a\) Total tool\-call volume \(log scale, blue\) and overall failure rate \(red\)\. \(b\) Composition of failed calls into eight categories\.

### E\.2FinanceAgent: Architecture Overhead Dominates over Entropy Control

This section complements the GAIA results with a financial QA benchmark to further validate the entropy\-performance findings on tool\-calling agent tasks\.

#### E\.2\.1Experimental Setup

##### Benchmark and Task Characteristics\.

We evaluate onFinanceAgentBenchmark, a financial QA benchmark that fundamentally differs from the mathematical reasoning tasks in the main paper\. Each question requires the agent to retrieve relevant SEC filings, extract financial metrics, perform multi\-step numerical computations, and synthesize information across documents\. This introduces several sources of complexity absent from our primary benchmarks: \(1\) tool selection entropy; \(2\) information retrieval noise from irrelevant documents; and \(3\) multi\-step dependency chains where errors in early tool calls propagate\.

The original benchmark was designed for a single closed\-source API model;Kimet al\.\([2025](https://arxiv.org/html/2602.04234#bib.bib27)\)extended it to a multi\-agent setting but still relied on closed\-source APIs and did not release evaluation code\. We further extend it to open\-source LLM\-based MAS, fully open\-source our evaluation code, and optimize the SEC filing retrieval tool with a local caching mechanism that eliminates rate\-limit failures caused by repeated identical requests\.

##### Model and Configuration\.

We use Qwen3\-4B with thinking mode enabled as the base model, with the same five architectures, temperature 0\.6, top\-pp0\.95, andR=2R=2rounds as in GAIA\. The agent has access to four financial tools: SEC filing retrieval, real\-time stock data queries, financial metric computation, and financial news search\. Each question allows up to 5 tool calls per step and 20 execution turns\. Token\-level entropy is recorded at each ReAct step, providing the same step\-level feature set \(step\_kk\) introduced in Appendix[E\.1\.1](https://arxiv.org/html/2602.04234#A5.SS1.SSS1)\.

#### E\.2\.2Accuracy Results

Table 7:FinanceAgentBenchmark accuracy by architecture\. SAS and Sequential achieve the highest accuracy; architectures with greater coordination overhead exhibit progressively worse performance\.ArchitectureAccuracy \(%\)Δ\\Deltavs\. SASAvg\. Time \(s\)Single \(SAS\)40—1255Sequential40\+04691Centralized22−18\-183804Hybrid12−28\-286387Debate2−38\-385415Table[7](https://arxiv.org/html/2602.04234#A5.T7)reports accuracy across architectures\. Single\-agent \(SAS\) achieves the highest accuracy \(40%\), tied with Sequential, while multi\-agent architectures that introduce coordination overhead substantially underperform: Centralized achieves 22%, Hybrid 12%, and Debate only 2%\. This result is consistent with the main paper finding that SAS outperforms MAS in a substantial fraction of configurations \(Section[4\.4](https://arxiv.org/html/2602.04234#S4.SS4)\)\. On tool\-augmented tasks, the coordination overhead inherent in multi\-agent architectures appears even more detrimental: Debate’s near\-zero accuracy suggests that majority voting is particularly ineffective when tool\-calling decisions must be coordinated across agents\.

#### E\.2\.3Feature Importance Analysis

We apply the same SHAP\-based feature importance pipeline \(Appendix[B\.4](https://arxiv.org/html/2602.04234#A2.SS4)\) to theFinanceAgentBenchmark results\. Figure[20](https://arxiv.org/html/2602.04234#A5.F20)presents three complementary views: \(a\) MAS\-only features excluding base model entropy, \(b\) features including base model entropy, and \(c\) the full feature set including base model correctness\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/agent_benchmark/base_model_all_metrics.png)\(a\)𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}: MAS\-only features
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/agent_benchmark/base_model_wo_entropy.png)\(b\)𝒢base\-H\\mathcal\{G\}\_\{\\text\{base\-H\}\}: Including base model entropy

Figure 20:Top 20 features onFinanceAgentacross two feature groups: \(a\) MAS\-only features \(𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}\), where architecture \(ρ≈0\.83\\rho\\approx 0\.83\) and step\-level entropy dominate; \(b\) including base model entropy \(𝒢base\-H\\mathcal\{G\}\_\{\\text\{base\-H\}\}\), where architecture remains the top predictor \(ρ≈0\.84\\rho\\approx 0\.84\) andstep\_0\_mean\_entropyshows moderate negative correlation \(ρ≈−0\.56\\rho\\approx\-0\.56\)\. On the full feature set \(𝒢base\-full\\mathcal\{G\}\_\{\\text\{base\-full\}\}\),base\_model\_is\_finally\_correct\(ρ≈0\.96\\rho\\approx 0\.96\) dominates all other features, consistent with the pattern observed across other settings\.##### Architecture is the dominant predictor\.

Across all three feature groups, architecture is the top or near\-top predictor withρ≈0\.83\\rho\\approx 0\.83\-0\.880\.88\. The ordinal encoding runs from Centralized \(0\) to Single \(4\), so the positive sign means simpler architectures perform better\. On a tool\-calling task, coordination overhead amplifies tool\-selection entropy rather than averaging it out\.

##### Step\-level entropy refines the “first\-round decisive” finding\.

The step\-level features, which split round 1 into individual ReAct iterations, expose a sub\-round structure invisible to the main analysis\.step\_0\_mean\_entropyenters the top 20 withρ≈−0\.75\\rho\\approx\-0\.75on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}andρ≈−0\.56\\rho\\approx\-0\.56on𝒢base\-H\\mathcal\{G\}\_\{\\text\{base\-H\}\}\(the attenuation reflects partial overlap with base\-model entropy\)\. The interpretation is sharper than “round 1 matters”: the first reasoning step alone carries most of the round\-1 signal\. In parallel,sample\_round\_2\_q3\_agent\_mean\_entropycorrelates positively \(ρ≈\+0\.49\\rho\\approx\+0\.49\), giving the same “decisive initialization, exploratory refinement” profile observed in the RL experiment \(Appendix[G](https://arxiv.org/html/2602.04234#A7)\)\.

##### Base\-model correctness still dominates on𝒢base\-full\\mathcal\{G\}\_\{\\text\{base\-full\}\}\.

base\_model\_is\_finally\_correctreachesρ≈0\.96\\rho\\approx 0\.96, identical to the math\-reasoning result in Appendix[F\.1](https://arxiv.org/html/2602.04234#A6.SS1)\. Tool\-calling adds a layer of complexity that MAS coordination frequently fails to manage, but it does not loosen the base\-model dependency\.

##### Causal validation\.

OnFinanceAgent, both PC and FCI converge on a single consensus direct cause of correctness:sample\_round\_1\_q3\_agent\_max\_entropy\(IPW ATE−0\.197\-0\.197,p=1\.5×10−17p=1\.5\\\!\\times\\\!10^\{\-17\}, all refutation tests pass\)\. The dominant mediation runs from round\-1 Q1 of per\-agent total entropy through round\-2 mean per\-agent total entropy to correctness \(indirect−0\.184\-0\.184, mediating 68\.9%\), the largest cross\-round mediation proportion observed in any setting in this paper\. Together these substantiate the SHAP “first reasoning step / round\-1 decisive” reading with stronger causal evidence: under tool\-calling, where each round bundles multiple ReAct steps, round\-1 dispersion is not merely correlated with failure but the mechanistic source of the round\-2 entropy state that drives the final outcome\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/agent_benchmark/causal_graph_consensus.png)\(a\)Consensus causal graph
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/agent_benchmark/ate_comparison.png)\(b\)ATE estimates
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/agent_benchmark/mediation_effects.png)\(c\)Mediation indirect effects

Figure 21:Causal triplet onFinanceAgent\. Round\-1 Q3 of per\-agent maximum entropy is the unique consensus direct cause; round\-1 dispersion mediates a 68\.9% share of its effect through round\-2 entropy\.

## Appendix FMore Experimental Results

This section extends the main text analysis along two dimensions\. First, it expands the SHAP\-based feature scope from top 2\-5 to top 10 and incorporates the mean SHAP impactS¯\\bar\{S\}, which quantifies each feature’s average contribution to predicted correctness across all samples\. Second, and more importantly, since SHAP analysis only reveals correlations and cannot pinpoint which factor actually drives MAS failure, we supplement every key finding in the main text with the same complete causal analysis pipeline as in Appendix[D](https://arxiv.org/html/2602.04234#A4), making the findings more reliable\. While Appendix[D](https://arxiv.org/html/2602.04234#A4)offers a single global causal view over all data, this section instead applies the pipeline finding\-by\-finding, validating each finding within its own experimental setting\.

### F\.1Base Model Entropy Dominates MAS Prediction

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/base_model_entropy/qwen/exclude_base_model_wo_entropy.png)\(a\)Qwen models\.
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/base_model_entropy/llama/exclude_base_model_wo_entropy.png)\(b\)LLaMA models\.

Figure 22:Top 20 features on𝒢base\-H\\mathcal\{G\}\_\{\\text\{base\-H\}\}for Qwen \(a\) and LLaMA \(b\), ranked by mean normalized importanceI¯\\bar\{I\}\. Each panel is divided into four subplots: top\-left shows feature importance from XGBoost and LightGBM; bottom\-left shows mean SHAP impactS¯\\bar\{S\}, representing the average contribution of each feature to model predictions; right column displays scatter plots of feature values versus SHAP values, with Pearson correlationρ\\rhoannotated in red\. All subsequent figures follow this layout except for the SHAP waterfall plots\.![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/base_model_entropy/qwen/shap_waterfall_sample_LightGBM_classification.png)\(a\)Qwen \(LightGBM\)
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/base_model_entropy/qwen/shap_waterfall_sample_XGBoost_classification.png)\(b\)Qwen \(XGBoost\)
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/base_model_entropy/llama/shap_waterfall_sample_LightGBM_classification.png)\(c\)LLaMA \(LightGBM\)
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/base_model_entropy/llama/shap_waterfall_sample_XGBoost_classification.png)\(d\)LLaMA \(XGBoost\)

Figure 23:SHAP waterfall plots on𝒢base\-H\\mathcal\{G\}\_\{\\text\{base\-H\}\}for representative samples: Qwen and LLaMA, with LightGBM and XGBoost\. Each bar shows the contribution of a feature to the predicted MAS correctness\.Figure[22](https://arxiv.org/html/2602.04234#A6.F22)ranks the top 20 features on𝒢base\-H\\mathcal\{G\}\_\{\\text\{base\-H\}\}for each model family, and Figure[23](https://arxiv.org/html/2602.04234#A6.F23)reports per\-sample SHAP attributions\. Two failure modes separate the families, and they correspond to the two entropy regimes the families operate in\.

##### Qwen \(high\-entropy regime, 100\-1,000\): inter\-agent dispersion\.

For Qwen, the top predictors are dispersion statistics rather than absolute entropy: variance of per\-token entropy and round\-1 Q3 agent variance entropy both show strong negative SHAP correlation \(ρ≈−0\.89\\rho\\approx\-0\.89and−0\.78\-0\.78\)\. Lower\-ranked entropy magnitudes \(per\-token, mean\) point in the same direction\. The picture is internally coherent: when agents in the same round disagree about how confident to be, the system fails, and this signal is robust across LightGBM and XGBoost waterfalls \(Figures[23\(a\)](https://arxiv.org/html/2602.04234#A6.F23.sf1)\-[23\(b\)](https://arxiv.org/html/2602.04234#A6.F23.sf2)\)\.

##### LLaMA \(low\-entropy regime, 0\-100\): output length and base entropy\.

For LLaMA, dispersion is replaced by length\. The dominant predictor is base sample token count \(I¯=1\.0\\bar\{I\}=1\.0,ρ≈−0\.47\\rho\\approx\-0\.47,S¯≈−0\.053\\bar\{S\}\\approx\-0\.053\), reinforced by base\-model entropy \(ρ≈−0\.73\\rho\\approx\-0\.73\) and the base\-to\-MAS answer entropy change \(ρ≈−0\.70\\rho\\approx\-0\.70\)\. Both the local and global SHAP contributions point the same way, so the failure mode is unambiguous: long, uncertain base responses do not get rescued by MAS\. The LLaMA waterfalls \(Figures[23\(c\)](https://arxiv.org/html/2602.04234#A6.F23.sf3)\-[23\(d\)](https://arxiv.org/html/2602.04234#A6.F23.sf4)\) accordingly show many small, partly cancelling contributions, consistent with a model that lacks a single concentrated failure signal\.

Several features \(notablyround\_2\_infer\_avg\_entropyfor LLaMA,ρ=\+0\.65\\rho=\+0\.65butS¯=−0\.002\\bar\{S\}=\-0\.002\) show positive sample\-level correlation but negative global contribution\. This pattern recurs throughout the appendix and is not a contradiction: it indicates a non\-monotone relationship where a moderate value helps, but the upper tail is harmful enough to drag the global mean negative\. We flag it once here and do not repeat the explanation for each subsequent occurrence\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/base_model_entropy/qwen/exclude_default.png)\(a\)Base model correctness: Qwen
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/base_model_entropy/llama/exclude_default.png)\(b\)Base model correctness: LLaMA
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/mas_failure/qwen/exclude_base_model_all_metrics.png)\(c\)MAS failure analysis: Qwen
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/mas_failure/llama/exclude_base_model_all_metrics.png)\(d\)MAS failure analysis: LLaMA

Figure 24:Top:Feature importance and SHAP analysis on𝒢base\-full\\mathcal\{G\}\_\{\\text\{base\-full\}\}for Qwen \(a\) and LLaMA \(b\)\. Both show thatbase\_model\_is\_finally\_correctachievesI¯=1\.0\\bar\{I\}=1\.0andρ≈0\.96\\rho\\approx 0\.96, vastly surpassing all other features with nearly linear correlation to MAS correctness\.Bottom:Top 20 features on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}for MAS failure analysis: Qwen \(c\) and LLaMA \(d\)\. Qwen’s top predictor is entropy variance, while LLaMA is dominated by answer\-level features\.
##### Base model correctness overwhelms all other features\.

On𝒢base\-full\\mathcal\{G\}\_\{\\text\{base\-full\}\}\(Figure[24](https://arxiv.org/html/2602.04234#A6.F24), top row\),base\_model\_is\_finally\_correctdominates withI¯=1\.0\\bar\{I\}=1\.0,ρ≈0\.96\\rho\\approx 0\.96, and a strongly positiveS¯\\bar\{S\}\(\+0\.45\+0\.45for Qwen,\+0\.07\+0\.07for LLaMA\)\. MAS succeeds largely when the base model is already correct; once we condition on this single indicator, all other features become marginal\.

##### Causal validation\.

Running the PC/FCI \+ DoWhy pipeline of Appendix[D](https://arxiv.org/html/2602.04234#A4)per \(model, dataset\) cell on𝒢base\-full\\mathcal\{G\}\_\{\\text\{base\-full\}\}reproduces the SHAP picture: of the 28 cells with successful causal estimation, base\-model entropy features \(base\_sample\_avg\_entropy\_per\_token,base\_sample\_total\_entropy,base\_model\_max/median/std\_answer\_token\_entropy,answer\_token\_entropy\_change,sample\_entropy\_ratio\_vs\_base\_total,sample\_avg\_entropy\_per\_token\_diff\_vs\_base\) appear among the consensus PC∩\\capFCI direct causes in 17/28 cells, and sample\-level answer\-token\-entropy features in 15/28 cells \(the two sets overlap in 9 cells\), with every IPW estimate negative on the dominant cause and every cell passing all three refutation tests\. The causal direct effect of base\-level entropy is therefore not a confound of the base\-correctness indicator: even when conditioning on round\-1/round\-2 entropy, base\-model per\-token entropy retains a direct edge into correctness across both families\. Section[5\.4](https://arxiv.org/html/2602.04234#S5.SS4)of the main text reports the corresponding global𝒢base\-full\\mathcal\{G\}\_\{\\text\{base\-full\}\}result, where the three consensus direct causes arebase\_sample\_avg\_entropy\_per\_token\(ATEPS=−0\.12\\text\{ATE\}\_\{\\text\{PS\}\}=\-0\.12,p<10−21p<10^\{\-21\}\),round\_1\_total\_entropy\(p<10−19p<10^\{\-19\}\), andsample\_max\_answer\_token\_entropy\(ATEPS=−0\.31\\text\{ATE\}\_\{\\text\{PS\}\}=\-0\.31,p<10−28p<10^\{\-28\}\); the per\-cell evidence above shows this finding is not driven by one or two model\-dataset combinations\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/base_model_entropy_default/causal_graph_consensus.png)\(a\)Consensus causal graph
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/base_model_entropy_default/ate_comparison.png)\(b\)ATE estimates
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/base_model_entropy_default/mediation_effects.png)\(c\)Mediation indirect effects

Figure 25:Representative causal triplet on𝒢base\-full\\mathcal\{G\}\_\{\\text\{base\-full\}\}\(Qwen3\-4B,GSM8K\)\. The consensus graph identifies one base\-entropy and one round\-1 dispersion feature as direct causes; both have significantly negative IPW ATE and pass all three refutation tests\.

### F\.2Inter\-Agent Misalignment Causes MAS Failure

Figure[24](https://arxiv.org/html/2602.04234#A6.F24)\(bottom row\) re\-runs the analysis on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}, which removes all base\-model features so the remaining signal is intrinsic to multi\-agent interaction\. The two failure modes from Appendix[F\.1](https://arxiv.org/html/2602.04234#A6.SS1)reappear in essentially the same form, which is the point: even after we strip out base\-model effects, Qwen still fails through dispersion and LLaMA still fails through verbosity\. This is direct evidence that the failure modes are properties of how each family interacts, not artifacts of the base\-model entropy correlations\. Two new findings that are specific to𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}deserve separate notice\.

##### More agents harm Qwen\.

sample\_num\_agentsenters the Qwen ranking withρ≈−0\.44\\rho\\approx\-0\.44andS¯≈−0\.051\\bar\{S\}\\approx\-0\.051, indicating that adding agents amplifies the dispersion failure rather than averaging it out\. This is consistent with theMMLU\-specific result in Section[4\.4](https://arxiv.org/html/2602.04234#S4.SS4)and contrasts with the LLaMA family, where agent count is not a top predictor\.

##### LLaMA shows a non\-monotone round\-2 effect\.

Round\-2 inference entropy correlates positively with sample\-level success \(ρ≈\+0\.65\\rho\\approx\+0\.65\) but contributes negatively at the global level\. Following the caveat in Appendix[F\.1](https://arxiv.org/html/2602.04234#A6.SS1), this is a single non\-monotone feature rather than a contradiction: LLaMA benefits from some round\-2 deliberation but is harmed by excessive late\-round entropy\. SHAP waterfalls in Figure[26](https://arxiv.org/html/2602.04234#A6.F26)confirm that the dominant contributors remain dispersion \(Qwen, panels a\-b\) and answer\-level features \(LLaMA, panels c\-d\)\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/mas_failure/qwen/shap_waterfall_sample_LightGBM_classification.png)\(a\)Qwen \(LightGBM\)
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/mas_failure/qwen/shap_waterfall_sample_XGBoost_classification.png)\(b\)Qwen \(XGBoost\)
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/mas_failure/llama/shap_waterfall_sample_LightGBM_classification.png)\(c\)LLaMA \(LightGBM\)
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/mas_failure/llama/shap_waterfall_sample_XGBoost_classification.png)\(d\)LLaMA \(XGBoost\)

Figure 26:SHAP waterfall plots on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}for representative MAS failure samples\. Qwen \(a\-b\) shows entropy dispersion features \(variance, Q3 agent\) as dominant contributors; LLaMA \(c\-d\) reveals answer\-level features \(token count, answer entropy\) driving failure predictions\.
##### Causal validation\.

Applying the same PC/FCI \+ DoWhy pipeline of Appendix[D](https://arxiv.org/html/2602.04234#A4)to𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}\(Figure[27](https://arxiv.org/html/2602.04234#A6.F27)\) recovers three consensus direct causes of correctness: round\-1 maximum per\-agent total entropy, round\-1 Q3 of per\-agent Q3 entropy, and sample\-level standard deviation of answer\-token entropy\. The conservative IPW estimates yieldATE=−0\.239\\text\{ATE\}=\-0\.239\(p=1\.3×10−15p=1\.3\\\!\\times\\\!10^\{\-15\}\),−0\.160\-0\.160\(p=4\.7×10−8p=4\.7\\\!\\times\\\!10^\{\-8\}\), and−0\.299\-0\.299\(p=1\.8×10−4p=1\.8\\\!\\times\\\!10^\{\-4\}\) respectively, all passing the three refutation tests\. Mediation analysis shows that round\-1 dispersion \(Q3 of per\-agent variance / standard deviation\) transmits 26–28% of its effect through round\-2 per\-token entropy, providing causal evidence for the SHAP reading: inter\-agent dispersion is not just correlated with failure but causally generates the round\-2 entropy state that propagates to the wrong answer\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/mas_failure/causal_graph_consensus.png)\(a\)PC/FCI consensus causal graph
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/mas_failure/ate_comparison.png)\(b\)ATE estimates \(LR / PS / IPW\)
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/mas_failure/mediation_effects.png)\(c\)Bootstrap indirect effects

Figure 27:Causal validation on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}: \(a\) consensus causal graph isolates round\-1 dispersion and answer\-token\-entropy variability as direct causes of correctness; \(b\) all three pass refutation, with conservative IPWATE∈\[−0\.30,−0\.16\]\\text\{ATE\}\\in\[\-0\.30,\-0\.16\]; \(c\) Round 1 dispersion is mediated to correctness through round\-2 per\-token entropy \(∼\\sim27% of total effect\)\.

### F\.3Task Difficulty Determines Optimal Entropy Dynamics

Figures[28](https://arxiv.org/html/2602.04234#A6.F28)and[29](https://arxiv.org/html/2602.04234#A6.F29)present per\-dataset analyses on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/diff_tasks/dataset_gsm8k_exclude_base_model_all_metrics.png)\(a\)GSM8K\(Easy\)
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/diff_tasks/dataset_math500_exclude_base_model_all_metrics.png)\(b\)MATH500\(Medium\)
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/diff_tasks/dataset_aime2024_16384_exclude_base_model_all_metrics.png)\(c\)AIME2024\(Hard\)
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/diff_tasks/dataset_aime2025_16384_exclude_base_model_all_metrics.png)\(d\)AIME2025\(Hard\)

Figure 28:Top 20 features on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}for mathematical reasoning tasks grouped by difficulty: \(a\)GSM8K\(easy,\|ρ\|≤0\.15\|\\rho\|\\leq 0\.15for top features\), \(b\)MATH500\(medium, positiveρ\\rhoandS¯\\bar\{S\}for average entropy\), \(c\-d\)AIME2024/AIME2025\(hard, round\-2 entropy harms performance\)\.![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/diff_tasks/dataset_humaneval_exclude_base_model_all_metrics.png)\(a\)HumanEval\(Code Generation\)
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/diff_tasks/dataset_mmlu_exclude_base_model_all_metrics.png)\(b\)MMLU\(Knowledge Q&A\)

Figure 29:Top 20 features on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}for \(a\) code generation \(HumanEval\) and \(b\) knowledge Q&A \(MMLU\)\.HumanEvalshows negativeρ\\rhoandS¯\\bar\{S\}for answer\-level features;MMLUshows that more agents hurt performance \(ρ<0\\rho<0,S¯<0\\bar\{S\}<0forsample\_num\_agents\)\.##### Easy regime \(GSM8K\): entropy is largely uninformative\.

Top\-feature correlations collapse to\|ρ\|≤0\.15\|\\rho\|\\leq 0\.15for most predictors, indicating that easy arithmetic does not stress the entropy machinery\. The two exceptions, entropy variance and answer\-token entropy, show strong sample\-level correlations but opposite global contributions, which is the same non\-monotone pattern flagged earlier and not a contradiction\. The practical implication is that entropy\-based selection adds little value when the task is easy enough for the base model to solve directly\.

##### Medium regime \(MATH500\): moderate entropy is beneficial\.

Average inference entropy becomes the leading positive predictor \(ρ=\+0\.63\\rho=\+0\.63,S¯=\+0\.12\\bar\{S\}=\+0\.12\), and cumulative round\-1 entropy also helps\. Excessive early entropy, however, still hurts: max agent total entropy and peak answer entropy carry strongly negative correlations\. The regime is not “more entropy = better” but rather “bounded exploration helps when the task has enough structure to converge”\.

##### Hard regime \(AIME2024/2025\): entropy changes dominate\.

On olympiad problems the strongest signal is the round\-1 to round\-2 entropy change \(ρ=−0\.80\\rho=\-0\.80,S¯=−0\.157\\bar\{S\}=\-0\.157\): trajectories whose entropy shifts substantially between rounds tend to fail\. Average inference entropy shows the same non\-monotone signature seen in the easy regime \(ρ=\+0\.83\\rho=\+0\.83,S¯=−0\.22\\bar\{S\}=\-0\.22\), now with much larger magnitudes, indicating a narrower optimal entropy band on hard problems\.

##### Domain\-specific regimes \(HumanEval,MMLU\)\.

Code generation collapses onto answer\-level entropy \(peak and Q3 answer entropy negative,ρ≤−0\.70\\rho\\leq\-0\.70\) while deliberation entropy is essentially uninformative\. Knowledge Q&A is the only setting where agent count itself is harmful \(ρ=−0\.48\\rho=\-0\.48,S¯=−0\.12\\bar\{S\}=\-0\.12\), and where consensus rather than duration drives success\. These two cases illustrate that the difficulty axis alone is insufficient; task type also selects which entropy dimension matters\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/diff_archs/arch_centralized_exclude_base_model_all_metrics.png)\(a\)Centralized
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/diff_archs/arch_debate_exclude_base_model_all_metrics.png)\(b\)Debate
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/diff_archs/arch_hybrid_exclude_base_model_all_metrics.png)\(c\)Hybrid
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/diff_archs/arch_sequential_exclude_base_model_all_metrics.png)\(d\)Sequential

Figure 30:Top 20 features on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}for multi\-agent architectures: \(a\) Centralized \(verbose answers harm performance\), \(b\) Debate \(cumulative entropy benefits once agents align\), \(c\) Hybrid \(extended deliberation helps\), and \(d\) Sequential \(answer\-level entropy is the primary failure mode\)\.
##### Aggregation architectures \(Centralized, Hybrid\)\.

Both architectures route information through a central point and share an answer\-length signal: in Centralized, sample answer token count dominates \(I¯=0\.96\\bar\{I\}=0\.96\) with a strongly negative global contribution; in Hybrid, total experiment time is the leading positive predictor \(ρ=\+0\.76\\rho=\+0\.76\)\. Both benefit from extended deliberation but penalize different sources of length: Centralized penalizes verbose final answers, while Hybrid additionally penalizes round\-2 token expansion \(ρ=−0\.57\\rho=\-0\.57\)\. The orchestrator therefore contributes by constraining later\-round length rather than by extending it\.

##### Decentralized architectures \(Debate, Sequential\)\.

Both rely on agent\-to\-agent transmission rather than aggregation, and both are vulnerable to early divergence\. In Debate, round\-1 peak entropy is the strongest negative predictor \(ρ=−0\.81\\rho=\-0\.81\), but cumulative entropy is positive \(ρ=\+0\.68\\rho=\+0\.68\), giving a “converge early, then explore” profile\. In Sequential, the same early\-round dispersion appears, but cumulative entropy reverses sign \(S¯=−0\.006\\bar\{S\}=\-0\.006globally despiteρ=\+0\.38\\rho=\+0\.38at sample level\), and answer\-level entropy becomes a dominant failure signal\. The contrast cleanly maps onto error\-propagation: Debate’s bidirectional exchanges allow the system to recover from early divergence, while Sequential’s chain compounds it\.

##### SAS as control\.

SAS is dominated by answer\-level features \(Figure[32\(a\)](https://arxiv.org/html/2602.04234#A6.F32.sf1)\)\. Notably, round\-2 inference entropy in SAS shows the same positive\-ρ\\rho, negative\-S¯\\bar\{S\}signature observed in MAS, indicating that the non\-monotone late\-round entropy effect is not a multi\-agent artifact but a property of the model’s own deliberation\. Cumulative entropy is uniformly positive in SAS \(ρ=\+0\.72\\rho=\+0\.72,S¯=\+0\.012\\bar\{S\}=\+0\.012\), the opposite of Sequential, which reinforces the error\-propagation reading above\.

##### Causal validation per task\.

Per\-dataset causal estimation on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}covers all six benchmarks, and each consensus tracks the SHAP\-based difficulty regime above\.GSM8K\(easy\) is driven by dispersion rather than magnitude: round\-1→\\rightarrow2 change, max answer\-token entropy, round\-1 mean per\-agent max, and sample variance \(IPW ATE−0\.090\-0\.090,−0\.353\-0\.353,−0\.183\-0\.183,−0\.183\-0\.183; allp<10−11p<10^\{\-11\}\)\.MATH500\(medium\) is causally controlled by the upper tail of answer\-token entropy \(max/mean/std plus round\-2 max per\-agent total; top IPW ATE−0\.450\-0\.450,p=4\.9×10−18p=4\.9\\\!\\times\\\!10^\{\-18\}\)\. On the hard end,AIME2024’s direct causes are both round\-1 Q3 per\-agent statistics \(IPW−0\.268\-0\.268and−0\.206\-0\.206\), whileAIME2025surfaces the round\-1→\\rightarrow2 change together with round\-1 max per\-agent max \(IPW−0\.075\-0\.075and−0\.139\-0\.139, bothp<10−20p<10^\{\-20\}\); olympiad failure is therefore caused by round\-1 dispersion and the round\-1→\\rightarrow2 shift, not absolute magnitude\. The domain\-specific datasets behave as expected:HumanEvalreduces to the answer\-token \+ round\-1 dispersion pair \(top IPW−0\.374\-0\.374,p=2\.0×10−77p=2\.0\\\!\\times\\\!10^\{\-77\}\);MMLUyields four consensus causes spanning answer\-token entropy and dispersion, the largest being max answer\-token entropy at IPW−0\.447\-0\.447\(p=2\.5×10−14p=2\.5\\\!\\times\\\!10^\{\-14\}\)\. Every consensus direct cause across the six datasets carries a negative IPW ATE and passes all three refutation tests\.

##### Causal validation per architecture\.

Per\-architecture causal estimation reveals the consensus direct cause for each topology \(Figure[31](https://arxiv.org/html/2602.04234#A6.F31)\):sample\_max\_answer\_token\_entropyfor Centralized \(IPW−0\.250\-0\.250,p=1\.8×10−9p=1\.8\\\!\\times\\\!10^\{\-9\}\), Hybrid \(−0\.445\-0\.445,p=2\.8×10−32p=2\.8\\\!\\times\\\!10^\{\-32\}\), Sequential \(−0\.310\-0\.310,p=2\.0×10−8p=2\.0\\\!\\times\\\!10^\{\-8\}\) and SAS \(−0\.467\-0\.467,p=1\.0×10−57p=1\.0\\\!\\times\\\!10^\{\-57\}\); for Debate the consensus direct causes shift to round\-1 per\-agent maximum entropy \(min and Q3, IPW−0\.147\-0\.147and−0\.238\-0\.238\)\. All cases pass the three refutation tests\. Two qualitative observations follow: \(i\) the answer\-level entropy variable is the dominant causal driver in every aggregation/chain architecture, supporting the SHAP claim that verbose answers harm Centralized and that Sequential compounds the same signal; \(ii\) Debate is the only architecture whose direct causes are round\-1 dispersion features rather than answer\-level features, which causally substantiates theconverge early, then exploreSHAP profile, once round\-1 dispersion is removed, cumulative entropy stops harming Debate\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/diff_archs/centralized/ate_comparison.png)\(a\)Centralized\.
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/diff_archs/debate/ate_comparison.png)\(b\)Debate\.
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/diff_archs/hybrid/ate_comparison.png)\(c\)Hybrid\.
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/diff_archs/sequential/ate_comparison.png)\(d\)Sequential\.

Figure 31:ATE estimates \(LR / PS / IPW\) for each MAS architecture on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}\. Answer\-token entropy emerges as the dominant negative causal driver in Centralized, Hybrid and Sequential; Debate is the only topology whose direct causes are round\-1 dispersion features rather than answer\-level features\.
##### Cross\-model consistency of causal findings\.

Table[8](https://arxiv.org/html/2602.04234#A6.T8)reports the consensus PC∩\\capFCI direct cause per \(model, dataset\) cell across the 5×\\times6 grid on𝒢base\-full\\mathcal\{G\}\_\{\\text\{base\-full\}\}and𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}\. Of 28 cells, 28 yield a non\-empty consensus on𝒢base\-full\\mathcal\{G\}\_\{\\text\{base\-full\}\}and 24 on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}; the empty𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}consensuses cluster onAIME2025\(Q\-0\.6/Q\-4/Q\-8\) plus Q\-4/HumanEval, where stripping base\-model signals leaves too few non\-redundant features for the conservative criterion\. Three patterns reproduce: \(i\) sample\-level answer\-token entropy is the most frequent top cause \(15/28 on𝒢base\-full\\mathcal\{G\}\_\{\\text\{base\-full\}\}, 13/24 on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}\); \(ii\) the remaining𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}cells \(11/24\) are dominated by round\-1 dispersion or sample\-level variance features, all with negative IPW ATE; \(iii\) base\-model entropy variables enter the consensus in 19/28𝒢base\-full\\mathcal\{G\}\_\{\\text\{base\-full\}\}cells, so the global result of Section[5\.4](https://arxiv.org/html/2602.04234#S5.SS4)is not driven by a small subset\. The single positive\-signed top cause \(L\-3/AIME2024on𝒢base\-full\\mathcal\{G\}\_\{\\text\{base\-full\}\}\) falls in a cell where SHAP already flagged a non\-monotone signature\. All listed cells pass the three refutation tests\.

Table 8:Cross\-model consistency of consensus PC∩\\capFCI direct causes per \(model, dataset\) cell, on𝒢base\-full\\mathcal\{G\}\_\{\\text\{base\-full\}\}\(left\) and𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}\(right\)\. Model abbreviations: Q\-0\.6/Q\-4/Q\-8 = Qwen3\-0\.6B/4B/8B; L\-3/L\-8 = LLaMA\-3\.2\-3B/3\.1\-8B\. Dataset abbreviations: HE = HumanEval; AIME24/25 = AIME2024/2025\. Feature names useeas shorthand forentropy\. \#DC denotes the number of consensus direct causes; “Top consensus cause” is the consensus direct cause with the largest\|IPW ATE\|\|\\text\{IPW ATE\}\|; the ATE column reports the sign of the IPW estimate of the top cause as↓\\downarrow\(negative, reduces correctness\) or↑\\uparrow\(positive\)\. LLaMA models onAIME25have insufficient samples for stable causal estimation so we skipped those cells\. Cells with \#DC=0=0\(Q\-0\.6/Q\-4/Q\-8 onAIME25, Q\-4 onHE, all on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}\) correspond to runs that completed but yielded an empty PC/FCI intersection under the temporal\-tier constraints\. All listed cells pass all three refutation tests\.𝒢base\-full\\mathcal\{G\}\_\{\\text\{base\-full\}\}𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}ModelDataset\#DCATETop consensus cause\#DCATETop consensus causeQ\-0\.6GSM8K5↓\\downarrowsample\_max\_answer\_token\_e2↓\\downarrowsample\_mean\_answer\_token\_eQ\-0\.6MATH5005↓\\downarrowsample\_max\_answer\_token\_e2↓\\downarrowsample\_max\_answer\_token\_eQ\-0\.6AIME242↓\\downarrowsample\_e\_relative\_iqr\_mean2↓\\downarrowround\_1\_2\_change\_eQ\-0\.6AIME253↓\\downarrowsample\_round\_1\_max\_agent\_std\_e0——Q\-0\.6HE4↓\\downarrowsample\_max\_answer\_token\_e2↓\\downarrowsample\_max\_answer\_token\_eQ\-0\.6MMLU1↓\\downarrowsample\_round\_1\_mean\_agent\_max\_e2↓\\downarrowsample\_round\_1\_q3\_agent\_max\_eQ\-4GSM8K2↓\\downarrowsample\_round\_1\_q3\_agent\_max\_e1↓\\downarrowsample\_round\_1\_q3\_agent\_max\_eQ\-4MATH5004↓\\downarrowanswer\_token\_e\_change2↓\\downarrowsample\_round\_1\_q3\_agent\_max\_eQ\-4AIME243↓\\downarrowsample\_round\_1\_mean\_agent\_variance\_e2↓\\downarrowsample\_round\_1\_max\_agent\_std\_eQ\-4AIME252↓\\downarrowsample\_round\_1\_q3\_agent\_max\_e0——Q\-4HE4↓\\downarrowsample\_mean\_answer\_token\_e0——Q\-4MMLU1↓\\downarrowsample\_round\_1\_median\_agent\_std\_e1↓\\downarrowsample\_round\_1\_median\_agent\_max\_eQ\-8GSM8K1↓\\downarrowsample\_variance\_e2↓\\downarrowsample\_round\_1\_q3\_agent\_max\_eQ\-8MATH5002↓\\downarrowsample\_max\_answer\_token\_e1↓\\downarrowsample\_max\_answer\_token\_eQ\-8AIME242↓\\downarrowsample\_max\_answer\_token\_e1↓\\downarrowsample\_round\_1\_q3\_agent\_variance\_eQ\-8AIME254↓\\downarrowsample\_round\_1\_q3\_agent\_max\_e0——Q\-8HE2↓\\downarrowsample\_mean\_answer\_token\_e3↓\\downarrowsample\_max\_answer\_token\_eQ\-8MMLU3↓\\downarrowsample\_round\_1\_max\_agent\_max\_e2↓\\downarrowsample\_round\_1\_median\_agent\_max\_eL\-3GSM8K3↓\\downarrowsample\_max\_answer\_token\_e2↓\\downarrowsample\_max\_answer\_token\_eL\-3MATH5004↓\\downarrowsample\_max\_answer\_token\_e3↓\\downarrowsample\_max\_answer\_token\_eL\-3AIME241↑\\uparrowsample\_avg\_e\_per\_token\_diff\_vs\_base2↓\\downarrowsample\_round\_1\_q3\_agent\_max\_eL\-3HE3↓\\downarrowsample\_max\_answer\_token\_e3↓\\downarrowsample\_max\_answer\_token\_eL\-3MMLU3↓\\downarrowsample\_max\_answer\_token\_e1↓\\downarrowsample\_max\_answer\_token\_eL\-8GSM8K4↓\\downarrowsample\_max\_answer\_token\_e2↓\\downarrowsample\_max\_answer\_token\_eL\-8MATH5005↓\\downarrowsample\_max\_answer\_token\_e3↓\\downarrowsample\_max\_answer\_token\_eL\-8AIME244↓\\downarrowsample\_round\_1\_max\_agent\_std\_e1↓\\downarrowsample\_variance\_eL\-8HE4↓\\downarrowsample\_std\_answer\_token\_e2↓\\downarrowsample\_max\_answer\_token\_eL\-8MMLU5↓\\downarrowsample\_max\_answer\_token\_e3↓\\downarrowsample\_max\_answer\_token\_e

### F\.4Round\-1 Entropy Dominates Despite Extended Deliberation

Figure[32\(b\)](https://arxiv.org/html/2602.04234#A6.F32.sf2)reports theR=5R=5analysis on the expanded 494\-dimensional feature space\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/diff_archs/arch_single_exclude_base_model_all_metrics.png)\(a\)
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/more_rounds/exclude_base_model_all_metrics.png)\(b\)

Figure 32:Top 20 features for \(a\) SAS and \(b\) MAS \(R=5R=5\) on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}\. SAS is dominated by answer token\-level features; notably, it exhibits a positive \(ρ\\rho\) but negative \(S¯\\bar\{S\}\) for round\-2 entropy\. In contrast, MAS predictions are driven by early\-round dynamics: Round\-1 features occupy the top ranks \(2\-3\), while features from Rounds 3\-5 are absent from the top 10\. Thus, early\-round entropy serves as the primary predictor for both systems\.##### Round\-1 still dominates atR=5R=5\.

Even in the 494\-dimensionalR=5R=5feature space, the top\-ranked entropy features come from round 1:sample\_round\_1\_max\_agent\_total\_entropyranks second \(ρ=−0\.91\\rho=\-0\.91\), and round\-1 Q3 agent variance entropy ranks third\. Answer token count leads overall but with the now\-familiar non\-monotone signature \(ρ=−0\.75\\rho=\-0\.75,S¯=\+0\.064\\bar\{S\}=\+0\.064\)\.

##### Rounds 3\-4 carry essentially no signal\.

The most informative observation in this section is what is*absent*from the top 20: no feature from rounds 3 or 4 appears, and only one round\-5 feature does \(I¯=0\.24\\bar\{I\}=0\.24\)\. Adding rounds beyond two does not surface new predictive structure; it adds dimensions that the classifier learns to ignore\.

##### Causal validation\.

Causal estimation on theR=5R\{=\}5data confirms what the SHAP ranking suggests: the consensus PC∩\\capFCI direct cause is the answer\-levelsample\_max\_answer\_token\_entropy\(IPW ATE−0\.344\-0\.344,p=2\.9×10−53p=2\.9\\\!\\times\\\!10^\{\-53\}\), with round\-2 Q3 of per\-agent maximum entropy as the second candidate but not robust across estimators\. The strongest mediation pathway runs from round\-1 Q3 of per\-agent variance entropy through its round\-2 counterpart to correctness \(indirect−0\.090\-0\.090, mediating 18%\); no round\-3, round\-4, or round\-5 feature appears as either a direct cause or a mediator above the bootstrap significance threshold\. The causal evidence therefore supports the SHAP reading that rounds beyond two add no predictive structure, they also add no causal structure\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/more_rounds/causal_graph_consensus.png)\(a\)Consensus causal graph
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/more_rounds/ate_comparison.png)\(b\)ATE estimates
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/more_rounds/mediation_effects.png)\(c\)Mediation indirect effects

Figure 33:Causal triplet atR=5R\{=\}5\. Direct causes and mediators are confined to round\-1 and round\-2 entropy features; rounds 3–5 contribute no surviving causal edges\.

### F\.5Temperature Variation Preserves Relative Entropy Patterns

Temperatureτ\\taudirectly modulates the sharpness of the softmax distribution: higherτ\\tauproduces flatter distributions with higher absolute entropy, while lowerτ\\tauconcentrates probability mass and reduces entropy\. So we conduct temperature ablation experiments to assess the robustness of our findings to decoding hyperparameters\.

##### Experimental Setup\.

We evaluate Qwen3\-4B onMATH500across all five MAS architectures at three temperature settingsτ∈\{0\.4,0\.6,0\.8\}\\tau\\in\\\{0\.4,0\.6,0\.8\\\}, with top\-p=0\.95p=0\.95held constant\. For each configuration, we use 100 paired samples and apply McNemar’s test for statistical significance between all temperature pairs\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/temperature_ablation/accuracy_trend_plot.png)Figure 34:MAS accuracy across temperaturesτ∈\{0\.4,0\.6,0\.8\}\\tau\\in\\\{0\.4,0\.6,0\.8\\\}for all five architectures onMATH500\. Accuracy remains remarkably stable: the maximum variation within any architecture is 3\.2% \(Single\), and the multi\-agent average varies by only 0\.5%\. McNemar’s test yieldsp\>0\.37p\>0\.37for all 15 pairwise comparisons, confirming statistical invariance\.
##### Accuracy Is Statistically Invariant to Temperature\.

Figure[34](https://arxiv.org/html/2602.04234#A6.F34)presents MAS accuracy across temperatures\. The results demonstrate remarkable stability: the multi\-agent average accuracy varies by only 0\.5% across the temperature range \(73\.7%73\.7\\%atτ=0\.4\\tau=0\.4,74\.0%74\.0\\%atτ=0\.6\\tau=0\.6,73\.5%73\.5\\%atτ=0\.8\\tau=0\.8\)\. Among individual architectures, Sequential and Centralized achieve identical accuracy \(74\.0%74\.0\\%\) at bothτ=0\.4\\tau=0\.4andτ=0\.8\\tau=0\.8, while the maximum variation occurs in Single \(73\.7%→75\.2%→72\.0%73\.7\\%\\to 75\.2\\%\\to 72\.0\\%,Δ=3\.2%\\Delta=3\.2\\%\)\. McNemar’s test across all 15 pairwise comparisons \(5 architectures×\\times3 temperature pairs\) yieldsp\>0\.37p\>0\.37, with most comparisons showingp≥0\.48p\\geq 0\.48orp=1\.0p=1\.0, confirming that temperature does not significantly alter MAS performance outcomes within this range\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/temperature_ablation/entropy_distribution_histogram.png)Figure 35:Entropy distribution statistics across temperatures for all architectures\. Absolute entropy values scale approximately2×2\\timesfromτ=0\.4\\tau=0\.4toτ=0\.8\\tau=0\.8\(mean entropy: Centralized0\.048→0\.075→0\.1010\.048\\to 0\.075\\to 0\.101\), but the relative ordering of architectures is preserved: Centralized and Single consistently exhibit higher entropy than Sequential, Debate, and Hybrid across all temperatures\.
##### Entropy Scales Predictably While Preserving Relative Patterns\.

Figure[35](https://arxiv.org/html/2602.04234#A6.F35)shows that absolute entropy values increase monotonically with temperature, as expected from the mathematical properties of softmax scaling\. Mean entropy approximately doubles fromτ=0\.4\\tau=0\.4toτ=0\.8\\tau=0\.8: Centralized \(0\.048→0\.1010\.048\\to 0\.101,2\.1×2\.1\\times\), Single \(0\.044→0\.0980\.044\\to 0\.098,2\.2×2\.2\\times\), Debate \(0\.034→0\.0790\.034\\to 0\.079,2\.3×2\.3\\times\)\. Critically, therelative orderingof architectures by entropy is fully preserved across all temperatures: Centralized and Single consistently occupy the high\-entropy regime, while Sequential, Debate, and Hybrid cluster in the low\-entropy regime\. This confirms that while absolute entropy magnitudes are temperature\-dependent, the comparative patterns central to our analysis remain valid\.

##### Summary\.

The temperature ablation confirms that the entropy\-performance relationship documented in this paper is not an artifact of specific decoding settings\. Within the commonly\-used rangeτ∈\[0\.4,0\.8\]\\tau\\in\[0\.4,0\.8\]: \(1\) MAS accuracy is statistically invariant \(p\>0\.37p\>0\.37for all comparisons\); \(2\) relative entropy patterns across architectures are fully preserved; and \(3\) core feature importance rankings remain stable\. Our findings rely on relative comparisons rather than absolute entropy thresholds, making them robust to temperature variation\.

### F\.6Model Scaling Preserves Entropy Feature Dominance

To check whether stronger base models change the picture, we evaluate Qwen3\-14B onGSM8K,AIME2024,AIME2025, andHumanEvalacross all five architectures, using the same three feature groups and SHAP pipeline as in the main analysis\.

##### SAS competitiveness persists at 14B\.

SAS achieves the highest accuracy on 2 of 4 benchmarks and beats at least one MAS on all 4\. The advantage concentrates on hard math:\+2\.8\+2\.8pp onAIME2024\(82\.8% SAS vs\. 80\.0% Hybrid\) and\+3\.4\+3\.4pp onAIME2025\(76\.7% SAS vs\. 73\.3% Centralized\)\. MAS only catches up on near\-ceiling tasks \(GSM8K: 98\.0% Centralized vs\. 97\.0% SAS\) and the domain\-specificHumanEval\(78\.0% Centralized vs\. 75\.6% SAS\)\. The SAS\-MAS trade\-off is therefore task\-driven, not scale\-driven\.

##### Feature hierarchy reproduces at 14B\.

Across𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\},𝒢base\-H\\mathcal\{G\}\_\{\\text\{base\-H\}\}, and𝒢base\-full\\mathcal\{G\}\_\{\\text\{base\-full\}\}, accuracy climbs83\.2%→88\.0%→93\.2%83\.2\\%\\to 88\.0\\%\\to 93\.2\\%, the same monotone progression observed at smaller scales\. On𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}, round\-1 entropy features fill roughly 70% of the top 20 withρ∈\[−0\.84,−0\.55\]\\rho\\in\[\-0\.84,\-0\.55\]\. On𝒢base\-H\\mathcal\{G\}\_\{\\text\{base\-H\}\}, the leading signal becomes base\-model answer length, andanswer\_token\_entropy\_change\(ρ≈−0\.84\\rho\\approx\-0\.84\) once again indicates that entropy growth from base to MAS predicts failure\. On𝒢base\-full\\mathcal\{G\}\_\{\\text\{base\-full\}\},base\_model\_is\_finally\_correctdominates withρ≈0\.98\\rho\\approx 0\.98, exactly as in Appendix[F\.1](https://arxiv.org/html/2602.04234#A6.SS1)\. The cross\-scale stability of this𝒢\\mathcal\{G\}\-progression is the relevant evidence: the predictive structure is not an artifact of small models\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/model_size/base_model_all_metrics.png)\(a\)𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}: MAS\-only features
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/model_size/base_model_wo_entropy.png)\(b\)𝒢base\-H\\mathcal\{G\}\_\{\\text\{base\-H\}\}: Including base model entropy

Figure 36:Feature importance and SHAP analysis for Qwen3\-14B across two feature groups\. \(a\) On𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}, round\-1 entropy dominates with approximately 70% of top\-20 features being entropy\-related \(LightGBM accuracy: 83\.2%, F1: 90\.3%\)\. \(b\) On𝒢base\-H\\mathcal\{G\}\_\{\\text\{base\-H\}\}, base model answer length emerges as the top predictor, whileanswer\_token\_entropy\_change\(ρ≈−0\.84\\rho\\approx\-0\.84\) signals that entropy increase from base to MAS predicts failure \(accuracy: 88\.0%, F1: 93\.0%\)\. On𝒢base\-full\\mathcal\{G\}\_\{\\text\{base\-full\}\},base\_model\_is\_finally\_correctachieves importance≈1\.0\\approx 1\.0withρ≈0\.98\\rho\\approx 0\.98, monopolizing prediction \(accuracy: 93\.2%, F1: 95\.9%\), consistent with the pattern observed across other settings\.
##### Causal validation at 14B\.

At the 14B scale, the causal pipeline reduces the structure to a single consensus PC∩\\capFCI direct cause:round\_1\_total\_entropy\(IPW ATE\+0\.166\+0\.166,p=6\.0×10−92p=6\.0\\\!\\times\\\!10^\{\-92\}, all refutation tests pass\)\. The positive sign reflects the Qwen3\-14B regime in which moderate first\-round entropy is needed to trigger inter\-agent refinement, and is consistent with the SHAP observation that round\-1 features dominate the top 20 with negative correlations only*above*a threshold\.sample\_max\_entropyappears as a secondary direct cause with negative ATE \(−0\.151\-0\.151,p=1\.2×10−6p=1\.2\\\!\\times\\\!10^\{\-6\}\)\. Cross\-round mediation is exceptionally strong: round\-1 maximum per\-agent total entropy transmits 45\.1% of its effect on correctness through its round\-2 counterpart, andround\_1\_total\_entropymediates 55\.8% through round\-2 median agent maximum entropy\. The causal evidence therefore confirms two of the SHAP findings \(round\-1 dominance, and the round\-1→\\rightarrowround\-2 propagation\) at a scale where round\-2 deliberation already shows productive value\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/model_size/causal_graph_consensus.png)\(a\)Consensus causal graph
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/model_size/ate_comparison.png)\(b\)ATE estimates
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/model_size/mediation_effects.png)\(c\)Mediation indirect effects

Figure 37:Causal triplet for Qwen3\-14B\. Round\-1 total entropy is the consensus direct cause and propagates 45–56% of its effect to correctness through round\-2 entropy\.

## Appendix GRL Training Inverts the Role of Entropy

Few studies have investigated whether using a specialized, fine\-tuned model as the base model can improve MAS performance on reasoning tasks\. We explore this using Qwen2\.5\-7B\-SimpleRL\-Zoo\(Zenget al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib70)\), denotedMRL\-baseM\_\{\\text\{RL\-base\}\}, which is obtained by applying zero\-shot RL to Qwen2\.5\-7B\(Teamet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib73)\)on 8K MATH problems without prior supervised fine\-tuning\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/x20.png)Figure 38:The role of entropy is reshaped in MAS built on Qwen2\.5\-7B\-SimpleRL\-Zoo\. \(a\) The performance of different MAS architectures across datasets\. \(b\) Relationship between base\-model entropy and MAS accuracy\. \(c\) Most predictive features in𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}\.Figure[38](https://arxiv.org/html/2602.04234#A7.F38)highlights three findings: \(1\)MRL\-baseM\_\{\\text\{RL\-base\}\}consistently enables MAS to outperform SAS, whereas with the standardMbaseM\_\{\\text\{base\}\}, SAS surpasses MAS in 43\.3% of cases, but never withMRL\-baseM\_\{\\text\{RL\-base\}\}; \(2\)The entropy\-accuracy relationship is reshaped: forMbaseM\_\{\\text\{base\}\}, higher entropy monotonically reduces accuracy; forMRL\-baseM\_\{\\text\{RL\-base\}\}, accuracy peaks at near\-zero entropy, declines to a minimum as entropy increases, then recovers to a secondary plateau before fluctuating, as shown in Figure[38](https://arxiv.org/html/2602.04234#A7.F38)b\. The initial peak likely corresponds to easy problems that require little reasoning\. For harder problems, however, the subsequent recovery reflects the benefit of longer trajectories: since Qwen2\.5\-7B already possesses strong inherent reasoning capabilities, RL training amplifies these by extending deliberation\(Zenget al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib70)\), allowing agents to explore structured solution paths under controlled entropy, where moderate entropy signifies productive exploration rather than degeneration; \(3\) On𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}, the top predictors are round\-1 median entropy \(ρ≈−0\.758\\rho\\approx\-0\.758\) and round\-2 entropy \(ρ≈0\.267\\rho\\approx 0\.267\), as shown in Figure[38](https://arxiv.org/html/2602.04234#A7.F38)c, indicating thatearly consensus, reflected in low initial entropy, combined with calibrated later\-round exploration, supports effective multi\-agent collaboration\.

Further comparison across Figures[2](https://arxiv.org/html/2602.04234#S4.F2)b,d and[38](https://arxiv.org/html/2602.04234#A7.F38)b shows thatMRL\-baseM\_\{\\text\{RL\-base\}\}achieves both lower average entropy and higher correctness thanMbaseM\_\{\\text\{base\}\}, indicating that RL training produces more reliable entropy estimates where entropy better reflects solution diversity rather than noise\. In contrast, forMbaseM\_\{\\text\{base\}\}, higher entropy often signals hallucination or incoherent reasoning\. Consequently, MAS can leverage this improved signal for more effective coordination\.

##### Detailed SHAP analysis\.

WithMRL\-baseM\_\{\\text\{RL\-base\}\}\(Qwen2\.5\-7B\-SimpleRL\-Zoo\), one feature changes sign in a way that is consistent across both𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}and𝒢base\-H\\mathcal\{G\}\_\{\\text\{base\-H\}\}:round\_2\_total\_entropynow correlates positively with sample\-level success \(ρ=\+0\.27\\rho=\+0\.27\), reversing the predominantly negative sign observed for the standard base models\. Round\-1 features remain negative \(ρ=−0\.77\\rho=\-0\.77for round\-1 median agent entropy\), so the inversion is specifically a late\-round effect: RL\-trained models use round\-2 entropy as productive refinement rather than noise\. The global\-levelS¯\\bar\{S\}for round\-2 entropy stays negative \(−0\.011\-0\.011on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\},−0\.085\-0\.085on𝒢base\-H\\mathcal\{G\}\_\{\\text\{base\-H\}\}\), the same non\-monotone pattern flagged earlier, indicating that even RL\-trained agents are harmed by excessive late\-round entropy\. Base\-model entropy remains a strong negative predictor on𝒢base\-H\\mathcal\{G\}\_\{\\text\{base\-H\}\}\(ρ=−0\.74\\rho=\-0\.74\), and the answer\-entropy change from base to MAS is still harmful \(ρ=−0\.70\\rho=\-0\.70\), showing that RL training shifts the role of late\-round entropy without overturning the base\-model dependency\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/rl_finetuned_base/exclude_base_model_all_metrics.png)\(a\)
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/rl_finetuned_base/exclude_base_model_wo_entropy.png)\(b\)

Figure 39:Top 20 features for MAS usingMRL\-baseM\_\{\\text\{RL\-base\}\}on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}\(a\) and𝒢base\-H\\mathcal\{G\}\_\{\\text\{base\-H\}\}\(b\)\.𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}\(a\): MAS\-only features\. Round\-2 entropy shows positiveρ\\rhobut negativeS¯\\bar\{S\}, suggesting moderate later\-round entropy is optimal\.𝒢base\-H\\mathcal\{G\}\_\{\\text\{base\-H\}\}\(b\): Including base model entropy\. Increased entropy from base to MAS still harms performance\. Early\-round entropy dominates prediction in both cases\.
##### Causal validation\.

Causal estimation underMRL\-baseM\_\{\\text\{RL\-base\}\}surfaces*two*consensus PC∩\\capFCI direct causes with opposite signs\.sample\_avg\_entropy\_per\_tokencarries a positive ATE \(LR\+1\.98\+1\.98,p=3\.7×10−17p=3\.7\\\!\\times\\\!10^\{\-17\}, all refutation tests pass\), whilesample\_max\_answer\_token\_entropyretains the familiar negative effect \(LR−0\.31\-0\.31,p=4\.1×10−23p=4\.1\\\!\\times\\\!10^\{\-23\}\)\. The mediation pathway runs from round\-1 Q3 of per\-agent total entropy through round\-2 total entropy to correctness with a positive indirect effect \(\+0\.175\+0\.175\)\. This is the causal signature of the SHAP “RL inverts entropy’s role” finding: under RL fine\-tuning, the per\-token entropy distribution becomes a productive exploration signal at the same time as answer\-token entropy remains a failure marker\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/rl_finetuned/causal_graph_consensus.png)\(a\)Consensus causal graph
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/rl_finetuned/ate_comparison.png)\(b\)ATE estimates
![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/rl_finetuned/mediation_effects.png)\(c\)Mediation indirect effects

Figure 40:Causal triplet underMRL\-baseM\_\{\\text\{RL\-base\}\}\. Two opposite\-signed direct causes coexist: per\-token entropy is productively positive while peak answer\-token entropy stays negative\.

## Appendix HDivergent Reasoning Styles of Qwen and LLaMA

We illustrate the divergent reasoning styles between Qwen3\-4B and LLaMA\-3\.2\-3B\-Instruct using a representative example fromAIME 2025\. As discussed in Section[4\.4](https://arxiv.org/html/2602.04234#S4.SS4), Qwen employs aself\-correctingstrategy, verifying and refining answers before producing final output, yielding more reliable results that suppress error propagation in MAS, albeit with higher entropy\. In contrast, LLaMA tends toaccept and reuseanswers from other agents without verification, leading to uncontrolled error propagation\.

##### Question\.

Find the sum of all integer basesb\>9b\>9for which17b17\_\{b\}is a divisor of97b97\_\{b\}\.

Ground Truth:70\.

### H\.1Qwen3\-4B \(Sequential Architecture\)

Planner Agent\.

System PromptYou are the planner agent\. Generate plans that are the general instructions only\. Do not execute the plan, do not perform any calculations, and do not produce any answers or intermediate numerical results\. Output a structured, numbered plans\.

User PromptFor the question: Find the sum of all integer basesb\>9b\>9for which17b17\_\{b\}is a divisor of97b97\_\{b\}\. Please only generate plans that are guidances required for the subsequent reasoning for the problem\-solving\. Do not include any specific calculation or numerical results\. Your input may include previous round outputs content\. You can consider the given contents as the initial state of the problem\-solving\.

Response \(excerpt from<think\>block\)“…if I subtract 9\*\(b \+ 7\) from 9b \+ 7, that would be 9b \+ 7 \- 9b \- 63 = \-56\. Therefore, \(b \+ 7\) divides \-56\. Since divisors can be positive or negative, but since b ¿ 9, b \+ 7 is positive, so \(b \+ 7\) must be a positive divisor of 56… the possible divisors of 56 that are greater than 16 are 28 and 56\. So, b \+ 7 = 28→\\tob = 21, and b \+ 7 = 56→\\tob = 49\. Therefore, the sum would be 21 \+ 49 = 70\.”Final Output:1\.Convert the base\-b numbers17b17\_\{b\}and97b97\_\{b\}to decimal form, resulting in1⋅b\+71\\cdot b\+7and9⋅b\+79\\cdot b\+7, respectively\.2\.Establish the divisibility condition:9​b\+79b\+7must be divisible byb\+7b\+7, i\.e\.,9​b\+7b\+7\\frac\{9b\+7\}\{b\+7\}is an integer\.3\.Perform algebraic manipulation to simplify the divisibility condition, leading to the conclusion thatb\+7b\+7divides−56\-56\(equivalently,b\+7b\+7divides5656\)\.4\.Identify all positive divisors of5656that satisfyb\+7\>16b\+7\>16\(sinceb\>9b\>9\)\.5\.For each valid divisord=b\+7d=b\+7, computeb=d−7b=d\-7and ensureb\>9b\>9\.6\.Sum all valid integer values ofbbobtained from step 5\.7\.Computeb=21,49b=21,49; sum = 70

Analysis:Qwen performs deep reasoningwithinthe<think\>block, independently deriving the complete solution before outputting a structured plan\.

Solver Agent\.

System PromptYou are the solver agent\. Solve strictly according to the provided plans\. Execute each step precisely and produce the final result\. Output the final result into \\boxed\{\{\}\}\.

User PromptQuestion: Find the sum of all integer bases $b\>9$ for which $17\_\{b\}$ is a divisor of $97\_\{b\}$\. \#\#\# Plans \#\#\# \[Planner’s output from previous step\] \#\#\# Plans \#\#\# Follow the plans to solve the question step by step and place the final answer in \\boxed\{\}\.

Response \(verification step\)“Forb=21b=21:1721=21\+7=2817\_\{21\}=21\+7=28,9721=9×21\+7=19697\_\{21\}=9\\times 21\+7=196, and196÷28=7196\\div 28=7, which is an integer\. Forb=49b=49:1749=49\+7=5617\_\{49\}=49\+7=56,9749=9×49\+7=44897\_\{49\}=9\\times 49\+7=448, and448÷56=8448\\div 56=8, which is also an integer\. Both values are valid\.”Final Answer:70\\boxed\{70\}

Analysis:Solver independently verifies each candidate value before accepting, catching potential errors\.

Critic Agent\.

System PromptYou are the critic agent\. Review the solver’s solution in detail, re\-derive independently, and correct any mistakes\. Keep the review terse\.

User PromptReview the solution for: Find the sum of all integer bases $b\>9$ for which $17\_\{b\}$ is a divisor of $97\_\{b\}$\. \#\#\# Solution \#\#\# \[Solver’s output from previous step\] \#\#\# Solution \#\#\# If corrections are needed, output the mistaken steps and the analysis, otherwise output ‘Correct’\.

Response“The remainder is indeed−56\-56, so the fraction is an integer ifb\+7b\+7divides 56\. The divisors of 56 are: 1, 2, 4, 7, 8, 14, 28, 56\. From these, the divisors greater than 16 are 28 and 56\. This is correct\.”Verdict:The solution iscorrect\. All steps are logically sound and verified\. The final answer is70\\boxed\{70\}\.

Analysis:Critic independently re\-derives the solution and confirms correctness\.

Judger Agent\.

System PromptYou are the final judge\. Audit only the final candidate and ensure it is correct\.

User Prompt \(instructs to outputonlythe final answer\)Final check for: Find the sum of all integer bases $b\>9$ for which $17\_\{b\}$ is a divisor of $97\_\{b\}$\. \#\#\# Solution \#\#\# \[Critic’s output from previous step\] \#\#\# Solution \#\#\# If correct, only output the final answer without words, labels, and steps, and wrapped in \\boxed\{\}\.

Response \(self\-verification in<think\>block despite instruction\)<think\> “Okay, let’s go through the problem again to make sure everything checks out… First, converting those numbers to decimal makes sense\. So 17 in base b is1⋅b\+7=b\+71\\cdot b\+7=b\+7\. Similarly, 97 in base b is9​b\+79b\+7… The next step is to set up the division:\(9​b\+7\)/\(b\+7\)\(9b\+7\)/\(b\+7\)\. The user did polynomial division here, which gives 9 with a remainder of \-56… Therefore, for the division to be an integer,b\+7b\+7must divide 56\. That’s correct… The divisors of 56 greater than 16 are 28 and 56\. Therefore,b\+7=28b\+7=28givesb=21b=21, andb\+7=56b\+7=56givesb=49b=49… Checking those values: Forb=21b=21,1721=2817\_\{21\}=28,9721=19697\_\{21\}=196, and196÷28=7196\\div 28=7\(integer\)\. Forb=49b=49,1749=5617\_\{49\}=56,9749=44897\_\{49\}=448, and448÷56=8448\\div 56=8\(integer\)\. So both bases are valid, and their sum is21\+49=7021\+49=70\. The solution seems correct\. All steps are logically sound and verified\.” </think\>Final Answer:70\\boxed\{70\}✓

Analysis:Despite the explicit instruction to “only output the final answer without words, labels, and steps”, Qwenvoluntarily re\-derives the entire solutionwithin its<think\>block before outputting the answer\. This self\-verification behavior, even when not required, exemplifies Qwen’s tendency to validate correctness independently, which suppresses error propagation but incurs higher entropy\.

### H\.2LLaMA\-3\.2\-3B\-Instruct \(Sequential Architecture\)

Note: LLaMA uses identical system and user prompts as Qwen; only responses are shown below\.

Planner Agent\.

Response \(flawed reasoning\)“…\(9​b\+7\)/\(b\+7\)\(9b\+7\)/\(b\+7\)can be rewritten as9−2​b\+7/\(b\+7\)9\-2b\+7/\(b\+7\), which simplifies to\(16−2​b\)/\(b\+7\)\(16\-2b\)/\(b\+7\)…”✗Error at Step 6:Incorrect algebraic simplification\. The correct form is:9​b\+7b\+7=9\+−56b\+7\\frac\{9b\+7\}\{b\+7\}=9\+\\frac\{\-56\}\{b\+7\}not\(16−2​b\)/\(b\+7\)\(16\-2b\)/\(b\+7\)\.“…Given thatb\+7b\+7must divide2​\(b\+7\)2\(b\+7\),b\+7b\+7must divide 14\. The factors of 14 are 1, 2, 7, and 14\. Thereforebbcan be \-6, \-5, 0, or 7\. Onlyb=7b=7is a valid solution\.”✗Error at Step 21:Filters forb\>9b\>9but incorrectly concludesb=7b=7\(which violatesb\>9b\>9\)\.

Analysis:LLaMA makes an algebraic error early in the reasoning chain, which propagates through all subsequent steps\.

Solver Agent\.

Response“\#\# Step 1: Identify the problem requirements… \#\# Step 5: Simplify the expression\(9​b\+7\)/\(b\+7\)\(9b\+7\)/\(b\+7\)can be rewritten as9−2​b\+7/\(b\+7\)9\-2b\+7/\(b\+7\), which simplifies to16−2​b/\(b\+7\)16\-2b/\(b\+7\)… \#\# Step 21: Determine the final possible values ofbb: Onlyb=7b=7is a valid solution\.”Final Answer:7\\boxed\{7\}✗

Analysis:Solver follows the flawed plan exactlywithout independent verification, directly accepting the erroneous conclusion\.

Critic Agent\.

Response“Correct\. The provided solution was mostly correct\. However, the analysis was overly complicated… Here’s a simplified version: Since2​\(b\+7\)2\(b\+7\)is always divisible byb\+7b\+7, allb\>9b\>9are valid\. The sum of all validbbvalues is10\+11\+12\+…\+16=20410\+11\+12\+\.\.\.\+16=204\.”✗New Error:Critic introduces an entirelynewerror while attempting to “simplify”\.

Analysis:Critic fails to catch the original error and introduces additional errors\.

Judger Agent\.

ResponseFinal Answer:204\\boxed\{204\}✗

Analysis:In stark contrast to Qwen, LLaMA’s judgerdirectly accepts the flawed analysis without any independent verification, even though its role is to “audit and ensure correctness”\. This lack of self\-verification behavior, while producing lower entropy, allows errors to propagate unchecked through the entire reasoning chain, ultimately yielding an incorrect final answer\.

## Appendix IEntropy Judger

This section provides implementation details for theEntropy Judger, including data preprocessing, hyperparameter configuration, and cross\-validation protocols\. We note that the Entropy Judger is not an uncertainty quantification \(UQ\) method: it does not quantify or calibrate model uncertainty, but rather serves as a practical selection tool that leverages entropy\-derived features via a trained classifier to identify the most likely correct output from pass@kkcandidates\.

### I\.1Training Data Construction

##### Data Aggregation Strategy\.

We conduct data mining at two granularities\. First, for each model series \(LLaMA or Qwen\), we aggregate samples from all experimental configurations, spanningGSM8K,MATH500,AIME 2024/2025,MMLU, andHumanEvaldatasets acrossCentralized,Debate,Hybrid,Sequential, andSinglearchitectures, yielding a diverse training corpus that ensures the classifier generalizes across task types and interaction patterns\. Second, to enable fine\-grained analysis, we train separate classifiers for each individual configuration: 5 models×\\times6 datasets×\\times3 feature groups, totaling 180 configuration\-specific classifiers\. Results from these fine\-grained analyses are partially reported in Appendix[F](https://arxiv.org/html/2602.04234#A6)\.

##### Feature Preprocessing\.

All features are standardized to zero mean and unit variance viax~i​j=\(xi​j−μj\)/σj\\tilde\{x\}\_\{ij\}=\(x\_\{ij\}\-\\mu\_\{j\}\)/\\sigma\_\{j\}, whereμj\\mu\_\{j\}andσj\\sigma\_\{j\}are computed from the training fold\. Missing values \(e\.g\., when an architecture has fewer agents\) are imputed with zero after standardization\.

### I\.2Model Hyperparameters

##### XGBoost Configuration\.

We set max\_depth = 6, learning\_rate = 0\.1, n\_estimators = 100, subsample = 0\.8, colsample\_bytree = 0\.8, with L1 regularization \(reg\_alpha = 0\.1\) and L2 regularization \(reg\_lambda = 1\.0\)\. To handle class imbalance, scale\_pos\_weight is computed asNneg/NposN\_\{\\text\{neg\}\}/N\_\{\\text\{pos\}\}\.

##### LightGBM Configuration\.

We use num\_leaves = 31, learning\_rate = 0\.1, n\_estimators = 100, subsample = 0\.8, colsample\_bytree = 0\.8, reg\_alpha = 0\.1, and reg\_lambda = 1\.0, with class\_weight =‘balanced’for automatic class imbalance adjustment\.

### I\.3Cross\-Validation Protocol

##### Stratified 5\-Fold Splitting\.

To ensure robust evaluation, we employ stratified 5\-fold cross\-validation: the full dataset is split into 5 folds while preserving the class distribution\(Npos:Nneg\)\(N\_\{\\text\{pos\}\}:N\_\{\\text\{neg\}\}\); for each foldi∈\{1,…,5\}i\\in\\\{1,\\ldots,5\\\}, we train on folds\{1,…,5\}∖\{i\}\\\{1,\\ldots,5\\\}\\setminus\\\{i\\\}and validate on foldii; the reported accuracy is the mean across all 5 folds\.

##### Early Stopping\.

During training, we monitor validation loss and stop if no improvement is observed for 10 consecutive boosting rounds, preventing overfitting\.

### I\.4Per\-Dataset Classification Performance

To understand how prediction difficulty varies across tasks, we train classifiers on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}for each dataset independently, aggregating samples across all models and architectures\. Table[9](https://arxiv.org/html/2602.04234#A9.T9)reports XGBoost and LightGBM accuracy\.

Table 9:Per\-dataset classification accuracy on𝒢MAS\\mathcal\{G\}\_\{\\text\{MAS\}\}\. Results are averaged over 5\-fold cross\-validation\.DatasetXGBoostLightGBMGSM8K0\.8760\.862AIME20240\.8600\.873AIME20250\.8330\.827MATH5000\.7870\.787HumanEval0\.7480\.732MMLU0\.7390\.742All \(Aggregated\)0\.7710\.769##### Task Difficulty Influences Predictability\.

Classification accuracy varies substantially across datasets \(0\.732\-0\.876\), revealing that entropy\-based prediction is easier for some tasks than others\. Notably, the easiest task \(GSM8K, 82% MAS accuracy\) and the hardest tasks \(AIME24/25, 25\-31% MAS accuracy\) both achieve high classification accuracy \(\>\>0\.82\), while medium\-difficulty tasks \(MATH500,HumanEval,MMLU\) prove more challenging to classify \(0\.73\-0\.79\)\. This suggests that extreme cases, where MAS either succeeds reliably or fails predictably, exhibit more distinctive entropy signatures, whereas intermediate performance involves subtler uncertainty patterns\.

##### Mathematical Reasoning Shows Clearest Entropy Signals\.

Among the six datasets, mathematical reasoning tasks \(GSM8K,AIME24/25\) consistently achieve the highest classification accuracy\. This aligns with our main finding that entropy dynamics are most informative for tasks requiring structured deliberation\. In contrast,MMLU\(knowledge Q&A\) shows the lowest accuracy, consistent with our observation that MMLU performance depends primarily on inter\-agent agreement rather than entropy magnitude\.

##### Aggregated Training Slightly Reduces Accuracy\.

The aggregated classifier \(0\.771\) underperforms the best per\-dataset classifiers \(0\.876 forGSM8K\), indicating that task\-specific entropy patterns exist\. However, the aggregated model still achieves competitive accuracy across all tasks, demonstrating that the Entropy Judger generalizes reasonably well without task\-specific tuning\.

### I\.5Pass@kkSelection

Beyond binary classification, the Entropy Judger enables label\-free selection from multiple MAS candidates\. Givenkkcandidate solutions\{𝐱1,…,𝐱k\}\\\{\\mathbf\{x\}\_\{1\},\\ldots,\\mathbf\{x\}\_\{k\}\\\}generated bykkindependent MAS runs \(same architecture, same problem, different random seeds\), the Entropy Judger selects the candidate with the highest predicted probability of correctness:

ℓ^=arg⁡maxℓ∈\[k\]⁡f​\(𝐱ℓ\),\\hat\{\\ell\}=\\arg\\max\_\{\\ell\\in\[k\]\}f\(\\mathbf\{x\}\_\{\\ell\}\),wheref:ℝd→\[0,1\]f:\\mathbb\{R\}^\{d\}\\to\[0,1\]is the trained ensemble classifier\. This formulation requires no ground\-truth labels at inference time, making it applicable to real\-world deployment\.

##### Experimental Setup\.

Due to computational constraints, this evaluation uses a focused subset of four models \(Qwen3\-4B, Qwen3\-8B, LLaMA\-3\.1\-8B\-Instruct, LLaMA\-3\.2\-3B\-Instruct\) and the first 50 samples from each of the six benchmarks\. We setK=3K=3repeated independent runs per \(model, dataset, architecture\) combination, withR=2R=2rounds per run and all other hyperparameters matching Appendix[B](https://arxiv.org/html/2602.04234#A2)\. The Entropy Judger is trained once on the full existing single\-run data \(all five models, all six datasets, all architectures\) from Section[I](https://arxiv.org/html/2602.04234#A9)and then*frozen*; it is never retrained on the repeated\-run data, eliminating any risk of data leakage\.

##### Selection Strategies and Baselines\.

We compare the Entropy Judger against three baselines that span the space of compute\-aware selection methods:

- •Random@kk: Uniformly select one of thekkruns at random\. Averaged over random draws, this equals the single\-run accuracy\.
- •MajVote@kk: Majority vote over thekkruns, the standard self\-consistency baseline\.
- •Judger Best\-of\-kk: Select the run with the highest predicted correctness probabilityf​\(𝐱ℓ\)f\(\\mathbf\{x\}\_\{\\ell\}\)among thekkcandidates\.
- •Pass@kk: Select correctly if any of thekkruns is correct\. This is the theoretical upper bound\.

In addition to Best\-of\-kk, we evaluate anEarly\-Stopvariant: scan runs sequentially and commit to the first run whose predicted probability exceeds a thresholdθ\\theta; if no run clears the threshold after allKKruns, fall back to the highest\-scored run\. Early\-Stop trades selection quality for inference efficiency: a stricterθ\\thetayields higher accuracy when it triggers but consumes more runs on average before stopping\.

Table 10:Best\-of\-kkselection accuracy atk∈\{1,2,3\}k\\in\\\{1,2,3\\\}, averaged over four models \(Qwen3\-4B, Qwen3\-8B, LLaMA\-3\.1\-8B\-Instruct, LLaMA\-3\.2\-3B\-Instruct\) and all architectures\.Boldmarks the best non\-pass@kkresult at eachkk\.DatasetStrategykk123GSM8KRandom0\.8950\.8870\.886MajVote0\.8950\.2930\.315Judger0\.8950\.9070\.911Pass@kk0\.8950\.9380\.945MATH500Random0\.5940\.5970\.597MajVote0\.5940\.1540\.161Judger0\.5940\.6110\.607Pass@kk0\.5940\.6380\.642AIME2024Random0\.3110\.3110\.311MajVote0\.3110\.0670\.077Judger0\.3110\.3250\.325Pass@kk0\.3110\.3450\.351AIME2025Random0\.2300\.2220\.222MajVote0\.2300\.0470\.047Judger0\.2300\.2270\.223Pass@kk0\.2300\.2300\.233HumanEvalRandom0\.2950\.3000\.302MajVote0\.2950\.0710\.075Judger0\.2950\.3090\.319Pass@kk0\.2950\.3270\.339MMLURandom0\.5930\.5900\.587MajVote0\.5930\.2730\.289Judger0\.5930\.6030\.600Pass@kk0\.5930\.6360\.646![Refer to caption](https://arxiv.org/html/2602.04234v6/images/judger_best_of_k.png)Figure 41:Best\-of\-kkselection accuracy across datasets and strategies atk∈\{1,3\}k\\in\\\{1,3\\\}\. Results are averaged over four models and all architectures\. The Entropy Judger consistently outperforms Random and MajVote atk=3k=3, approaching the Pass@kkupper bound\.
##### Analysis\.

Table[10](https://arxiv.org/html/2602.04234#A9.T10)and Figure[41](https://arxiv.org/html/2602.04234#A9.F41)show the Best\-of\-kkresults\. The experiment isolates the Entropy Judger’s selection quality from raw compute scaling: at each fixedk∈\{1,2,3\}k\\in\\\{1,2,3\\\}, Random@kk, MajVote@kk, and Judger@kkall consume exactlykkruns, so any accuracy gap reflects selection quality rather than additional computation\. Comparing Judger against MajVote specifically isolates the contribution of entropy\-based ranking over majority agreement, since both have access to the samekkoutputs\. Atk=1k=1all strategies are equivalent by definition, as no selection is possible\. Atk=2k=2andk=3k=3, the Judger consistently outperforms Random while MajVote degrades substantially, as MajVote accuracy collapses atk=2k=2because a tie requires a fallback and binary disagreement carries no majority signal, making entropy\-based ranking far more reliable in this low\-kkregime\. The one exception isAIME2025, where Judger marginally underperforms Random atk≥2k\\geq 2\(e\.g\., 0\.223 vs\. 0\.222 atk=3k=3\); this near\-zero gap reflects thatAIME2025is at the frontier of model capability and entropy features carry little discriminative signal when almost all runs fail\.

##### Early\-Stop Analysis\.

In the Early\-Stop variant, the judger scans runs sequentially and commits to the first run whose predicted correctness probability exceeds a thresholdθ\\theta, falling back to the highest\-scored run if none clears the threshold\. WithK=3K=3, the stopping behavior concentrates in two regimes\. For most multi\-agent architectures, the judger tends to exhaust all runs before committing \(average runs≈2\.1\\approx 2\.1–2\.22\.2atθ=0\.7\\theta=0\.7\), reflecting that multi\-agent entropy trajectories are more variable and the judger needs more evidence\. For single\-agent runs, early stopping is more frequent: average runs consumed drop to 1\.16–1\.86 across datasets whenθ=0\.7\\theta=0\.7, as SAS entropy profiles are more consistent and the judger reaches confidence after the first run\. The Qwen family, whose higher base accuracy produces stronger confidence signals, shows the most pronounced early stopping, with Qwen3\-8B reaching average runs=1\.0=1\.0onGSM8Kacross all architectures, meaning the judger always commits after a single run\. These patterns suggest that Early\-Stop efficiency gains are most pronounced for stronger models and well\-structured tasks, where entropy signals are decisive enough to trigger early commitment without sacrificing accuracy\.

##### Practical Utility\.

Together, these experiments address a key deployment challenge: when running MAS in production, practitioners lack ground\-truth labels to evaluate candidate outputs\. The Entropy Judger provides a principled, label\-free mechanism that \(i\) improves over majority voting at the same compute budget and \(ii\) enables adaptive stopping that reduces average inference cost while maintaining accuracy\. The strong cross\-model generalization, where the judger is trained on five models yet applied to a held\-out subset of four, further demonstrates that entropy\-based selection is not tied to specific model characteristics\.

## Appendix JEntropy Calibration Analysis

Throughout this paper, we treatentropy primarily as a predictive feature for MAS correctness rather than a universal measure of uncertainty: while empirical correlations between entropy and accuracy hold consistently across model families and tasks, entropy\-derived confidence is not uniformly well\-calibrated\. A concrete manifestation is that LLMs can produce confidently wrong outputs, low entropy paired with incorrect answers, indicating that high confidence does not guarantee correctness\. This section provides a full calibration analysis across all five base models and six datasets, quantifying how predictive reliability depends on model family and task difficulty, and confirming thatentropy remains a useful signal for MAS evaluation even when its absolute calibration is imperfect\.

### J\.1Calibration Methodology

We assess entropy\-accuracy alignment via a surrogate calibration error, computed using the standard ECE formula applied to an entropy\-derived confidence proxy\. Formally, givenBBequal\-width bins partitioning the confidence range\[0,1\]\[0,1\], ECE is defined as

ECE=∑b=1BnbN​\|acc​\(b\)−conf​\(b\)\|,\\text\{ECE\}=\\sum\_\{b=1\}^\{B\}\\frac\{n\_\{b\}\}\{N\}\\left\|\\text\{acc\}\(b\)\-\\text\{conf\}\(b\)\\right\|,\(1\)wherenbn\_\{b\}is the number of samples in binbb,NNis the total sample count,acc​\(b\)\\text\{acc\}\(b\)is the empirical accuracy within binbb, andconf​\(b\)\\text\{conf\}\(b\)is the mean predicted confidence in that bin\. We derive confidence from entropy viaconf=1/\(1\+H\)\\text\{conf\}=1/\(1\+H\), whereHHdenotes the mean trajectory entropy \(sample\_mean\_entropy\)\.

We note that1/\(1\+H\)1/\(1\+H\)is a monotone surrogate mapping, not a true posterior probability of correctness: Shannon entropy scales with vocabulary size and sequence length, so the resulting confidence values are not comparable across models with different tokenizers or across tasks of different lengths\. Accordingly, the ECE values reported here quantify*entropy\-accuracy alignment*, the degree to which low\-entropy predictions coincide with correct answers, rather than classical probabilistic calibration in the standard sense\. This distinction does not affect the qualitative conclusions of this section, which concern relative patterns across models and tasks rather than absolute calibration guarantees\.

Reliability diagrams \(calibration curves\) visualize calibration by plotting observed accuracy against predicted confidence for each bin; deviation from the diagonaly=xy=xindicates miscalibration\. We evaluate all 30 model–dataset combinations across five models \(Qwen3\-8B, Qwen3\-4B, Qwen3\-0\.6B, LLaMA\-3\.1\-8B\-Instruct, LLaMA\-3\.2\-3B\-Instruct\) and six datasets \(GSM8K,MMLU,MATH500,HumanEval,AIME 2024,AIME 2025\)\.

### J\.2Calibration Results

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/calibration/calibration_curves.png)Figure 42:Reliability diagrams for all five models across six datasets\. Each subplot shows observed accuracy \(blue bars\) versus entropy\-derived confidence \(x\-axis\), with the red dashed diagonal indicating perfect calibration\. Bars above \(below\) the diagonal indicate under\-confidence \(over\-confidence\)\. Bin sample counts are annotated above each bar\. Qwen3\-4B and Qwen3\-8B achieve near\-perfect calibration onGSM8K\(ECE≤0\.03\\leq 0\.03\), while LLaMA models on competition\-level tasks \(AIME\) exhibit severe over\-confidence\.Figure[42](https://arxiv.org/html/2602.04234#A10.F42)presents per\-model reliability diagrams\. The global average ECE across all 30 configurations is 0\.391, revealing substantial heterogeneity in calibration quality\. Three key patterns emerge\.

##### Model Family Determines Calibration Quality\.

Qwen models \(average ECE = 0\.275\) are substantially better calibrated than LLaMA models \(average ECE = 0\.565\)\. Within the Qwen family, Qwen3\-4B achieves the best overall calibration \(average ECE = 0\.163\), with ECE as low as 0\.018 onGSM8Kand 0\.099 onMMLU\. Qwen3\-8B follows closely \(average ECE = 0\.191\), reaching ECE = 0\.030 onGSM8Kand 0\.058 onMMLU\. By contrast, LLaMA\-3\.1\-8B\-Instruct \(average ECE = 0\.548\) and LLaMA\-3\.2\-3B\-Instruct \(average ECE = 0\.583\) show poor calibration across most datasets\. Notably, the correlation between parameter count and ECE is weak \(r=−0\.25r=\-0\.25\), indicating that training paradigm matters more than model scale\.

##### Calibration Degrades Systematically with Task Difficulty\.

Average ECE increases monotonically with task difficulty:GSM8K\(0\.110\),MMLU\(0\.215\),MATH500\(0\.359\),HumanEval\(0\.459\),AIME 2024\(0\.571\), andAIME 2025\(0\.632\)\. The best\-calibrated configuration \(Qwen3\-4B onGSM8K, ECE = 0\.018\) and worst \(LLaMA\-3\.1\-8B onAIME 2025, ECE = 0\.899\) span nearly two orders of magnitude\. This systematic degradation reflects a fundamental interaction between model capability and task demands: when models lack the reasoning capacity for a task, their entropy\-derived confidence becomes uninformative\.

##### Well\-Calibrated Regime Exists for Strong Models on Tractable Tasks\.

Despite the global average suggesting non\-trivial miscalibration, 10 of 30 configurations achieve ECE<0\.20<0\.20, all involving either Qwen3\-4B or Qwen3\-8B\. These well\-calibrated regimes span routine reasoning \(GSM8K\), knowledge tasks \(MMLU\), code generation \(HumanEval, ECE≈0\.18\\approx 0\.18–0\.200\.20\), and even moderate mathematics \(MATH500, ECE≈0\.20\\approx 0\.20–0\.220\.22\)\. Within these regimes, the reliability diagrams in Figure[42](https://arxiv.org/html/2602.04234#A10.F42)show close alignment with the perfect calibration diagonal, particularly in the 0\.4–0\.8 confidence range\.

### J\.3Overconfidence and Confidently Wrong Analysis

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/calibration/ece_comparison.png)Figure 43:Heatmap of the*confidently wrong*proportion across all model–dataset combinations\. Each cell reports the fraction of samples where the model exhibits low entropy \(high confidence\) yet answers incorrectly\. Darker red indicates higher overconfident error rates\. Qwen3\-4B and Qwen3\-8B maintain confidently wrong rates below 10% on most datasets, while LLaMA models and competition\-level tasks exhibit rates exceeding 25%\.A critical concern for entropy\-based uncertainty estimation is whether models can be*confidently wrong*, exhibiting low entropy while producing incorrect answers\. We conduct quadrant analysis by splitting samples at the median entropy threshold into four categories: confidently correct \(low entropy, correct\), confidently wrong \(low entropy, incorrect\), uncertain correct \(high entropy, correct\), and uncertain wrong \(high entropy, incorrect\)\.

Figure[43](https://arxiv.org/html/2602.04234#A10.F43)presents the confidently wrong proportion across all configurations\. The global confidently wrong rate is 14\.95%, with strong model\-dependent variation\. Qwen3\-8B and Qwen3\-4B exhibit rates of only 7\.37% and 7\.60%, respectively, while LLaMA\-3\.1\-8B\-Instruct reaches 25\.56% and LLaMA\-3\.2\-3B\-Instruct 29\.71%\. Dataset difficulty amplifies overconfidence:GSM8Kelicits only 2\.96% confidently wrong predictions, compared to 31\.47% forAIME 2025\.

The Pearson correlation between the confidently wrong proportion and ECE isr=0\.989r=0\.989, confirming that these two metrics are nearly interchangeable as calibration diagnostics\. This tight coupling validates ECE as a reliable summary statistic for overconfidence risk\.

These calibration results directly connect to the main text findings: the model\-family gap corroborates the distinct entropy and performance patterns for Qwen versus LLaMA documented in Section[4\.4](https://arxiv.org/html/2602.04234#S4.SS4), while the difficulty\-dependent degradation aligns with the findings in Section[5](https://arxiv.org/html/2602.04234#S5), which demonstrate that task difficulty fundamentally modulates the entropy\-performance relationship\. We accordingly position entropy not as a direct measure of epistemic uncertainty, but as an empirically informative signal for understanding MAS dynamics\. For configurations with poorer single\-model calibration, inter\-agent entropy features such as entropy variance and cross\-round dynamics remain predictive of MAS effectiveness, as they capture*relative*uncertainty patterns across agents rather than absolute confidence levels\.

### J\.4Calibration on Tool\-Augmented Tasks

We repeat the same calibration analysis on theGAIAcohort introduced in Section[E\.1](https://arxiv.org/html/2602.04234#A5.SS1)and Appendix[E\.1](https://arxiv.org/html/2602.04234#A5.SS1)\. The pattern is qualitatively different from the reasoning benchmarks\. The global average ECE across the six base models onGAIAis 0\.790, far higher than any reasoning dataset reported above, and varies only modestly across model scale \(Qwen3\-14B 0\.745, Qwen3\-8B 0\.751, Qwen3\-4B 0\.775, LLaMA\-3\.1\-8B 0\.811, Qwen3\-0\.6B 0\.820, LLaMA\-3\.2\-3B 0\.836\)\. The confidently wrong proportion reaches 40\.77% onGAIA, compared to 14\.95% averaged over the reasoning datasets, with weak models exceeding 47% and even the best\-calibrated Qwen3\-8B at 36\.36%\. Two implications follow\. First, the well\-calibrated regime that exists for strong Qwen models on tractable reasoning tasks does not transfer to tool\-augmented settings: even Qwen3\-14B is severely overconfident onGAIA\. Second, entropy\-based confidence filtering remains a useful but insufficient deployment safeguard on agentic tasks, because nearly half of low\-entropy predictions are wrong\. This is consistent with the GAIA causal result in Appendix[E\.1\.4](https://arxiv.org/html/2602.04234#A5.SS1.SSS4), where round\-1 tool success rate, not entropy alone, is the consensus direct cause of correctness: when the proximate driver of failure is tool execution rather than token\-level uncertainty, the entropy signal weakens as a calibration anchor\.

## Appendix KControlled SAS vs\. MAS Comparison

This appendix complements the observational causal pipeline in Appendix[D](https://arxiv.org/html/2602.04234#A4)\. Whereas Appendix[D](https://arxiv.org/html/2602.04234#A4)estimates the causal effect of entropy features on MAS correctness via PC/FCI structure learning and the DoWhy framework, the analysis here addresses the upstream half of the causal account: a controlled three\-way comparison that isolates how MAS itself reshapes entropy\. The empirical findings in the main text primarily characterize correlational relationships between entropy features and MAS performance; to provide preliminary causal insights into where MAS\-induced entropy comes from, we compare token entropy distributions across three experimental conditions: a single\-agent system \(SAS\), MAS Round 1 \(prior to interaction\), and MAS Round 2 \(following interaction\)\. This analysis encompasses 35,660 paired samples drawn from five base models and six benchmark datasets\.

### K\.1Controlled Experiment Design

To disentangle MAS effects from base LLM capability, we construct a three\-way comparison that isolates two potential causal mechanisms:

##### Condition A: Single\-Agent System \(SAS\)\.

The model answers each question independently, without multi\-agent role prompts or inter\-agent communication\. This serves as the baseline reflecting pure base model capability\.

##### Condition B: MAS Round 1 \(Pre\-Interaction\)\.

Agents are assigned MAS\-specific roles and context prompts but have not yet exchanged messages\. Any entropy difference between Conditions A and B is attributable solely to the*role assignment intervention*, not to inter\-agent interaction\.

##### Condition C: MAS Round 2 \(Post\-Interaction\)\.

Agents have completed one round of inter\-agent discussion\. Entropy differences between Conditions B and C reflect the causal effect of*multi\-agent interaction*on model entropy\.

This design yields the additive accounting identity

HR2−HSAS⏟total shift=\(HR1−HSAS\)⏟role\-assignment shift\+\(HR2−HR1\)⏟interaction shift,\\underbrace\{H\_\{\\text\{R2\}\}\-H\_\{\\text\{SAS\}\}\}\_\{\\text\{total shift\}\}=\\underbrace\{\(H\_\{\\text\{R1\}\}\-H\_\{\\text\{SAS\}\}\)\}\_\{\\text\{role\-assignment shift\}\}\+\\underbrace\{\(H\_\{\\text\{R2\}\}\-H\_\{\\text\{R1\}\}\)\}\_\{\\text\{interaction shift\}\},\(2\)whereHSASH\_\{\\text\{SAS\}\},HR1H\_\{\\text\{R1\}\}, andHR2H\_\{\\text\{R2\}\}denote the per\-token entropy under each condition\. We treat this as an attributional decomposition rather than a causal\-identification result: the role\-assignment intervention itself alters the state from which the interaction shift is measured, so the two terms are not independently identifiable causal effects\. We pair 35,660 SAS\-MAS sample pairs across 5 models \(LLaMA\-3\.1\-8B\-Instruct, LLaMA\-3\.2\-3B\-Instruct, Qwen3\-0\.6B, Qwen3\-4B, Qwen3\-8B\) and 6 datasets \(AIME 2024,AIME 2025,GSM8K,HumanEval,MATH500,MMLU\), using the Wilcoxon signed\-rank test for statistical inference and Cohen’sddfor effect size estimation\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/three_way_comparison.png)Figure 44:Three\-way entropy comparison across all 30 model\-dataset combinations\. Each subplot shows violin plots of per\-token entropy distributions for SAS \(teal\), MAS Round 1 \(red\), and MAS Round 2 \(blue\), with meanμ\\muannotated above each violin and the Wilcoxon signed\-rankpp\-value for SAS vs\. MAS R1 in the subplot title\. The systematic shift from SAS to MAS R1 demonstrates that role assignment alone constitutes a statistically significant intervention on model behavior \(23/30 combinations withp<0\.05p<0\.05\), while the further shift from R1 to R2 reflects inter\-agent interaction effects\.

### K\.2Role Assignment as a Causal Intervention

Figure[44](https://arxiv.org/html/2602.04234#A11.F44)reveals statistically significant differences between SAS and MAS Round 1 in 23 out of 30 model\-dataset combinations \(Wilcoxon signed\-rank test,p<0\.05p<0\.05\)\.

##### Aggregate Statistics\.

The average entropy difference \(MAS R1−\-SAS\) across all 30 combinations is\+0\.010\+0\.010on the per\-token entropy scale, indicating that MAS role prompts generally produce slightly higher initial entropy\. However, this aggregate masks substantial heterogeneity:

- •LLaMA\-3\.1\-8B\-Instructshows consistently increased entropy under MAS role assignment, with the largest effect onMMLU\(d=0\.72d=0\.72,p<10−200p<10^\{\-200\}\) andHumanEval\(d=1\.06d=1\.06,p<10−64p<10^\{\-64\}\)\.
- •LLaMA\-3\.2\-3B\-Instructexhibits mixed patterns: entropy increases onHumanEval\(d=1\.34d=1\.34\) andMMLU\(d=0\.54d=0\.54\) but decreases onAIME 2025\(d=−0\.26d=\-0\.26\)\.
- •Qwen3\-8Bshows entropy increases on mathematical reasoning tasks \(AIME 2024:d=0\.51d=0\.51;GSM8K:d=0\.46d=0\.46;MATH500:d=0\.44d=0\.44\) but a large decrease onMMLU\(d=−1\.06d=\-1\.06,p<10−200p<10^\{\-200\}\)\.
- •Qwen3\-4Bmirrors this pattern withMMLUshowing a strong negative shift \(d=−0\.79d=\-0\.79\) whileAIMEtasks show modest positive shifts \(d≈0\.26d\\approx 0\.26\)\.
- •Qwen3\-0\.6Bshows the weakest effects overall, with mostAIMEcombinations failing to reach significance\.

##### Interpretation\.

This finding is itself causally significant: it demonstrates that multi\-agent role assignment constitutes a*meaningful intervention*on model behavior, altering the entropy distribution even before any inter\-agent communication occurs\. The model\- and task\-dependent directionality suggests that role prompts interact with the model’s internal representations in non\-trivial ways\. Larger models may become more exploratory \(higher entropy\) on knowledge tasks when assigned collaborative roles, while smaller models show more variable responses\. Rather than treating this SAS≠\\neqMAS R1 difference as a confound, we interpret it as evidence that the MAS context fundamentally shapes the initial uncertainty landscape from which subsequent interaction dynamics emerge\.

### K\.3Inter\-Agent Interaction Dynamics

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/entropy_change_direction.png)Figure 45:Left:Mean entropy change from MAS Round 1 to Round 2 \(HR2−HR1H\_\{\\text\{R2\}\}\-H\_\{\\text\{R1\}\}\) across all model\-architecture\-dataset combinations\. Blue cells indicate entropy decrease \(consensus formation\); red cells indicate entropy increase\.Right:Mean accuracy change \(MAS−\-SAS\) for the same combinations\. Green cells indicate accuracy improvement; red cells indicate degradation\. Cells that are blue on the left but red on the right suggest anchoring behavior \(entropy decreases without accuracy gains\)\.Having established that role assignment alone shifts entropy, we now examine the causal effect of inter\-agent interaction by analyzing the Round 1→\\toRound 2 transition\. Figure[45](https://arxiv.org/html/2602.04234#A11.F45)presents entropy and accuracy changes across all model\-architecture\-dataset combinations\.

##### Predominant Entropy Reduction\.

Multi\-agent interaction predominantly drives entropy reduction: 89\.5% of samples show decreased entropy from Round 1 to Round 2, consistent with consensus formation\. The rate varies systematically across architectures:

- •Hybrid: 96\.8% entropy decrease, meanΔ​accuracy=−0\.004\\Delta\\text\{accuracy\}=\-0\.004
- •Centralized: 93\.4% entropy decrease, meanΔ​accuracy=−0\.018\\Delta\\text\{accuracy\}=\-0\.018
- •Debate: 88\.9% entropy decrease, meanΔ​accuracy=−0\.020\\Delta\\text\{accuracy\}=\-0\.020
- •Sequential: 78\.8% entropy decrease, meanΔ​accuracy=−0\.058\\Delta\\text\{accuracy\}=\-0\.058

##### Architecture\-Dependent Causal Mechanisms\.

The results reveal architecture\-dependent dynamics that suggest distinct causal mechanisms:

- •Hybridachieves the highest entropy\-decrease rate \(96\.8%\) with the smallest accuracy degradation \(−0\.004\-0\.004\), suggesting that the dual feedback from peers and an orchestrator enables productive consensus rather than mere copying\.
- •Sequentialshows the lowest entropy\-decrease rate \(78\.8%\) coupled with the largest accuracy degradation \(−0\.058\-0\.058\)\. The 21\.2% of samples with entropy*increases*indicates that information propagation through the chain destabilizes rather than consolidates entropy, consistent with compounding errors along the sequence\.
- •CentralizedandDebateoccupy intermediate positions, with centralized aggregation achieving higher entropy reduction but moderate accuracy loss, while debate’s lower entropy\-decrease rate reflects the adversarial dynamics inherent in its design\.

These architecture\-dependent patterns provide evidence against a pure anchoring explanation: if agents simply copied the most confident response, the entropy reduction mechanism should be architecture\-invariant\.

### K\.4Genuine Improvement and Anchoring

![Refer to caption](https://arxiv.org/html/2602.04234v6/images/appendix/causal/paired_entropy_scatter.png)Figure 46:Paired entropy scatter plots \(MAS Round 1 vs\. Round 2\) across datasets\. Each point represents a single sample; green circles \(∘\\circ\) denote correct answers and red crosses \(×\\times\) denote incorrect answers\. The dashed diagonal \(y=xy=x\) separates entropy decrease \(below\) from entropy increase \(above\)\. The predominance of points below the diagonal confirms systematic entropy reduction, while the similar spatial distribution of correct and incorrect samples suggests that entropy reduction alone does not reliably distinguish genuine improvement from anchoring\.One concern with the accuracy degradation reported in the main text is that it could be a pure anchoring artifact: agents may simply copy or align with peers’ Round\-1 responses, lowering entropy without actually reasoning, in which case the apparent MAS effect would not reflect a real interaction mechanism\. To separate genuine reasoning improvement from anchoring/copying, we decompose Round 1→\\toRound 2 changes by the co\-occurrence of entropy decrease and accuracy improvement \(Figure[46](https://arxiv.org/html/2602.04234#A11.F46)\)\.

##### Decomposition\.

We classify each sample into four categories based on the joint outcome of entropy change \(Δ​H\\Delta H\) and accuracy change \(Δ​acc\\Delta\\text\{acc\}\):

- •Genuine improvement\(Δ​H​<0∧Δ​acc\>​0\\Delta H<0\\wedge\\Delta\\text\{acc\}\>0\): 6\.2% of samples
- •Possible anchoring\(Δ​H<0∧Δ​acc≤0\\Delta H<0\\wedge\\Delta\\text\{acc\}\\leq 0\): 83\.4% of samples
- •Productive exploration\(Δ​H\>0∧Δ​acc\>0\\Delta H\>0\\wedge\\Delta\\text\{acc\}\>0\): rare
- •Deterioration\(Δ​H\>0∧Δ​acc≤0\\Delta H\>0\\wedge\\Delta\\text\{acc\}\\leq 0\): remaining samples

These results quantitatively validate the central finding of the main text: inter\-agent interaction rarely yields genuine accuracy improvements, confirming that MAS outcomes are predominantly fixed by round\-1 dynamics rather than subsequent deliberation\.The overwhelming prevalence of “possible anchoring” \(83\.4%\) versus “genuine improvement” \(6\.2%\) indicates that while agents often converge to lower entropy states, this convergence seldom translates into corrected reasoning\. Instead, it largely reflects an anchoring effect where the system reinforces its initial consensus, which may already be correct \(i\.e\.,1→11\\to 1samples withΔ​acc=0\\Delta\\text\{acc\}=0\) or incorrect \(i\.e\.,0→00\\to 0samples\), though the latter case represents the predominant failure mode\.

##### Model\-Dependent Genuine Improvement Rates\.

The genuine improvement rate varies substantially across models:

- •LLaMA\-3\.2\-3B\-Instruct: 10\.1% genuine, 81\.1% possible anchoring \(ratio = 0\.12\)
- •Qwen3\-0\.6B: 7\.3% genuine, 75\.4% possible anchoring \(ratio = 0\.10\)
- •LLaMA\-3\.1\-8B\-Instruct: 6\.9% genuine, 82\.6% possible anchoring \(ratio = 0\.08\)
- •Qwen3\-4B: 3\.6% genuine, 90\.8% possible anchoring \(ratio = 0\.04\)
- •Qwen3\-8B: 2\.9% genuine, 87\.0% possible anchoring \(ratio = 0\.03\)

This ordering does not track parameter count monotonically: Qwen3\-0\.6B sits between the two LLaMAs rather than at the top, and the two largest Qwen models occupy the bottom\. The pattern is more consistent with model family than scale, with both LLaMA models showing higher genuine\-improvement rates than the Qwen models of comparable or smaller size\. This is the same family\-over\-scale effect documented for calibration in Appendix[J](https://arxiv.org/html/2602.04234#A10)\.

##### Architecture\-Dependent Patterns\.

The genuine\-vs\-anchoring decomposition by architecture mirrors the entropy\-decrease pattern in Section[K\.3](https://arxiv.org/html/2602.04234#A11.SS3): Hybrid and Centralized achieve slightly higher genuine improvement \(7\.2% and 7\.0%\), while Sequential’s combination of a low entropy\-decrease rate \(78\.8%\) and the largest accuracy degradation \(−0\.058\-0\.058\) reflects the same chain\-propagation failure already documented in that section, rather than an “anchoring\-light” regime\.

## Appendix LCase Study: Token\-Level Entropy Dynamics

For each dataset, we select the first sample and visualize the token\-level entropy trajectory across all agents and rounds\. In each figure, white backgrounds denote round 1 and gray backgrounds denote round 2\. Black dashed lines separate different agents’ outputs within each round\. The correctness indicator \(checkmark or cross\) in the upper\-right corner shows the final MAS prediction outcome\. This visualization captures the finest granularity of entropy dynamics, enabling direct observation of how entropy patterns relate to MAS success or failure\.

##### Qwen3\-0\.6B: High Entropy Persistence\.

Figure[47](https://arxiv.org/html/2602.04234#A12.F47)shows that the smallest Qwen model exhibits persistently high entropy across both rounds, with frequent spikes throughout the reasoning trajectory\. On harder tasks \(AIME,MATH500\), the entropy rarely stabilizes, and most predictions fail\. This aligns with our finding that uncontrolled entropy harms MAS performance\. Notably, even when round\-2 entropy decreases, it often collapses to near\-zero rather than converging to a stable moderate level, indicating premature termination rather than confident resolution\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/x21.png)Figure 47:Token\-level entropy dynamics for Qwen3\-0\.6B across six datasets\. High entropy persistence and frequent spikes characterize this smaller model, with entropy either remaining elevated or collapsing abruptly to zero in round 2\.
##### Qwen3\-4B: Improved Stability with Task\-Dependent Patterns\.

Figure[48](https://arxiv.org/html/2602.04234#A12.F48)reveals that scaling to 4B parameters improves entropy stability\. On simpler tasks \(GSM8K,MMLU\), agents converge to low, stable entropy in round 2, yielding correct predictions\. On medium\-difficulty tasks \(MATH500,HumanEval\), moderate entropy is maintained, supporting productive exploration\. However, onAIME, entropy dynamics remain erratic, suggesting that even larger models struggle with olympiad\-level problems\. This pattern supports ourTask Awarenessprinciple: optimal entropy profiles vary by task difficulty\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/x22.png)Figure 48:Token\-level entropy dynamics for Qwen3\-4B\. Increased model capacity yields more stable entropy on easier tasks, while harder tasks still induce erratic entropy patterns\.
##### Qwen3\-8B: Structured Deliberation Emerges\.

Figure[49](https://arxiv.org/html/2602.04234#A12.F49)demonstrates that the largest Qwen model exhibits the most structured entropy dynamics\. Round\-1 entropy shows controlled exploration with clear peaks at decision points, followed by gradual stabilization\. In round 2, entropy either maintains a productive moderate level \(on hard tasks where deliberation helps\) or converges smoothly to low values \(on simple tasks\)\. This structured pattern correlates with higher accuracy, confirming thatCertainty Preferencebenefits MAS when achieved through genuine convergence rather than premature collapse\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/x23.png)Figure 49:Token\-level entropy dynamics for Qwen3\-8B\. The largest Qwen model shows structured deliberation with controlled exploration in round 1 and smooth convergence in round 2\.
##### LLaMA\-3\.2\-3B\-Instruct: Distinct Reasoning Style\.

Figure[50](https://arxiv.org/html/2602.04234#A12.F50)shows that LLaMA models exhibit fundamentally different entropy dynamics compared to Qwen\. Round\-2 entropy frequently drops to near\-zero across all agents, indicating a more decisive \(but potentially overconfident\) reasoning style\. While this yields correct predictions on simpler tasks, it often leads to failure on harder problems where sustained exploration is beneficial\. This contrast highlights how different model families develop distinct entropy profiles, with implications for MAS architecture selection\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/x24.png)Figure 50:Token\-level entropy dynamics for LLaMA\-3\.2\-3B\-Instruct\. LLaMA exhibits lower round\-2 entropy compared to Qwen, often collapsing to near\-zero, reflecting a more decisive but potentially overconfident reasoning style\.
##### LLaMA\-3\.1\-8B\-Instruct: Scale Improves but Style Persists\.

Figure[51](https://arxiv.org/html/2602.04234#A12.F51)reveals that scaling LLaMA to 8B parameters improves overall accuracy but preserves the characteristic low\-entropy style in round 2\. The model shows better calibration, with entropy remaining non\-zero on harder tasks where exploration helps\. However, the tendency toward rapid entropy reduction remains more pronounced than in Qwen models of comparable size\. This suggests that model family \(not just scale\) shapes entropy dynamics, reinforcing ourBase Entropyfinding that base model characteristics directly influence MAS effectiveness\.

![Refer to caption](https://arxiv.org/html/2602.04234v6/x25.png)Figure 51:Token\-level entropy dynamics for LLaMA\-3\.1\-8B\-Instruct\. Scaling improves calibration, but the characteristic rapid entropy reduction in round 2 persists compared to Qwen models\.
##### Summary\.

These case studies provide visual evidence for our main findings: \(1\)Round\-1 dynamics are critical: entropy patterns established in the first round largely persist or determine the trajectory in round 2; \(2\)Moderate, stable entropy correlates with success: both excessively high entropy \(erratic reasoning\) and near\-zero entropy \(premature collapse\) predict failure; \(3\)Model family shapes entropy style: Qwen and LLaMA exhibit distinct entropy profiles that influence MAS effectiveness across different tasks\. These observations complement our quantitative analysis by revealing the fine\-grained mechanisms underlying entropy\-performance relationships\.

## Appendix MLimitations, Broader Impacts, and Future Work

While our study provides comprehensive insights into entropy dynamics of LLM\-based MAS, several limitations suggest directions for future research\.

##### Model Scale and Benchmark Coverage\.

Due to computational constraints, our experiments are limited to open\-source LLMs with at most 14B parameters; models at the 27B\-70B scale may exhibit qualitatively different entropy dynamics as emergent capabilities grow with parameter count\. Furthermore, our evaluation focuses on six benchmarks spanning mathematical reasoning, code generation, and knowledge Q&A\. While we have extended coverage to tool\-calling scenarios throughGAIA\(Mialonet al\.,[2024](https://arxiv.org/html/2602.04234#bib.bib55)\)andFinanceAgent\(Bigeardet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib61)\), evaluating on broader agentic benchmarks involving web browsing and multi\-step environment interactions\(Liuet al\.,[2024](https://arxiv.org/html/2602.04234#bib.bib58); Chenet al\.,[2025b](https://arxiv.org/html/2602.04234#bib.bib56); Zhuet al\.,[2025a](https://arxiv.org/html/2602.04234#bib.bib57)\)remains as future work, where entropy dynamics may play a more pronounced role due to the complexity and interactivity of the tasks\.

##### Homogeneous Model Assumption\.

Our experimental design constructs MAS using homogeneous agents, where all agents share the same base modelMbaseM\_\{\\text\{base\}\}\. However, recent work\(Yeet al\.,[2025](https://arxiv.org/html/2602.04234#bib.bib69)\)demonstrates that heterogeneous MAS, composed of diverse LLMs with complementary strengths, can achieve superior performance compared to homogeneous configurations\. The interplay between model heterogeneity and entropy dynamics remains unexplored: whether agents with different entropy profiles can compensate for each other’s weaknesses, and how to optimally compose heterogeneous teams based on entropy characteristics, are promising research directions\.

##### Causal Identification\.

While our causal discovery analysis \(Section[5\.4](https://arxiv.org/html/2602.04234#S5.SS4)\) establishes that base\-model entropy causally drives MAS correctness and that early inter\-agent entropy dispersion mediates downstream performance, the causal identification relies on observational data and algorithmic discovery rather than controlled interventions\. In particular, the predominant entropy\-reduction\-without\-accuracy\-gain rate \(83\.4%\) indicates that entropy reduction from inter\-agent interaction does not automatically translate to performance gains, and the underlying mechanisms remain incompletely understood\. Stronger causal identification requires complementary strategies: message ablation studies that isolate information content from social influence, and counterfactual interventions that manipulate entropy directly through temperature control while holding other factors fixed\.

##### Broader Impacts\.

This work provides a principled understanding of entropy dynamics in LLM\-based multi\-agent systems, offering practical value for both researchers and practitioners\. For the research community, our entropy\-based analysis framework establishes a new perspective for diagnosing MAS failures and guiding architectural design decisions\. For practitioners, our findings that single agents outperform MAS in 43\.3% of cases, combined with the insight that first\-round entropy dynamics largely determine outcomes, can inform more resource\-efficient deployment strategies\. TheEntropy Judgerfurther enables quality\-aware output selection without requiring ground\-truth labels, reducing annotation costs in real\-world applications\. We do not foresee specific negative societal consequences beyond those generally associated with advancing LLM capabilities\.