When Planning Fails Despite Correct Execution: On Epistemic Calibration for LLM-Based Multi-Agent Systems

# When Planning Fails Despite Correct Execution: On Epistemic Calibration for LLM-Based Multi-Agent Systems
Source: [https://arxiv.org/html/2605.23414](https://arxiv.org/html/2605.23414)
###### Abstract

LLM\-based multi\-agent systems can fail even when planned actions are executed correctly because agents may misjudge their knowledge when evaluating plan feasibility, a phenomenon we term epistemic miscalibration in planning\. Unlike execution errors, epistemic miscalibration is latent during planning, as generated plans can remain self\-consistent and executable without observable errors; the miscalibration is also dynamic, as new information can alter feasibility assessments, potentially obscuring past miscalibration signals and causing them to recur over time\. To address this, we propose the Epistemic Planning Calibration Agentic Workflow \(EPC\-AW\), which assesses whether plans remain supported under varying information conditions rather than directly verifying feasibility\. EPC\-AW employs Information\-consistency\-based Plan Selection, selecting plans whose evaluations are stable across agents, together with Consistency\-guided Epistemic State Refinement to adapt calibration over time by leveraging past discrepancies to guide future planning\. Experiments show that EPC\-AW improves system\-level success by an average of 9\.75%\. Code is available in[the public repository \(https://github\.com/wzhSteve/EPC\-AW\)](https://github.com/wzhSteve/EPC-AW)\.

Machine Learning, ICML

## 1Introduction

Large language model\-based multi\-agent systems have become a dominant paradigm for complex decision making, tool use, and long\-horizon task execution\(Becattiniet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib118); Sunet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib132); Heet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib88)\)\. By decomposing tasks across interacting agents, such systems enable scalable problem solving beyond the capability of single\-agent approaches\(Becattiniet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib118); Ronanki,[2025](https://arxiv.org/html/2605.23414#bib.bib115)\)\. However, despite their rapid development, these multi\-agent systems remain fragile in practice, with failures frequently observed in real deployments\(Cemriet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib141); Hammondet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib128)\)\. Diagnosing and correcting such failures has therefore become an emerging requirement for reliable agentic systems\(Lianget al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib43); Eppersonet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib48)\)\.

Recently, a growing body of work has sought to repair failures in LLM\-based multi\-agent systems by intervening at different stages of system operation, including post\-hoc and online methods\(Lianget al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib43); Maet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib49)\)\. Post\-hoc approaches\(Eppersonet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib48); Maet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib49)\)analyze interaction traces after task failure and apply retrospective corrections, while online methods\(Lianget al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib43); Shenet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib113)\)monitor error accumulation during task progression and intervene to prevent errors from escalating into task failure\. Across these approaches, failures are predominantly attributed to execution\-level faults, such as logically incorrect tool outputs or invalid tool returns\(Zhanget al\.,[2025b](https://arxiv.org/html/2605.23414#bib.bib3); Luet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib4); Reidet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib19)\)\.

However, practical deployments reveal a distinct class of failures that cannot be explained by execution faults alone\. Even when all actions are correctly executed following the prescribed plan, the task may remain unsolved because the plan itself is infeasible with respect to the intended objective\. Such failures arise at the planning stage, where the plan encodes a mismatch between intended goals and the actions required to achieve them\. For example, a plan may invoke external tools and receive valid responses, yet fail to acquire verifiable evidence for the objective\. This occurs when the plan is based on a miscalibrated feasibility assessment, causing the system to repeatedly execute valid actions without ever satisfying the goal\.

We characterize this failure mode as*epistemic miscalibration in planning*: a planning agent assigns unwarranted confidence to feasibility assessments, failing to recognize the limits of its knowledge in tasks requiring iterative information acquisition\. This phenomenon poses two key challenges\. First, epistemic miscalibration is latent\. Unlike execution errors, which manifest as observable errors in action realization or inconsistencies in reasoning traces\(Zhanget al\.,[2025c](https://arxiv.org/html/2605.23414#bib.bib139); Cemriet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib141)\), epistemic miscalibration arises from erroneous feasibility assessments that leave no explicit error signals\. Consequently, plans affected by epistemic miscalibration can appear coherent and executable, making such failures particularly difficult to detect at the planning stage\. Second, epistemic miscalibration is dynamic\. As the planning agent continuously acquires new information, its feasibility assessments evolve, which can gradually obscure past miscalibration signals and lead to repeated miscalibration over time\. While prior methods leverage dynamic feedback to correct execution\-level errors\(Maet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib49); Eppersonet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib48)\), they rely on observable errors as supervision and therefore do not address plan\-level epistemic miscalibration, limiting their ability to sustain calibration under evolving information\.

To mitigate epistemic miscalibration, we propose the*Epistemic Planning Calibration Agentic Workflow*\(EPC\-AW\), a planning\-centered workflow for multi\-agent collaboration\. EPC\-AW does not attempt to directly verify plan feasibility\. Instead, it focuses on whether the plan continues to be supported by the agents under different information conditions\. EPC\-AW comprises two complementary components\.*Information\-consistency\-based Plan Selection*\(IPS\) operates within each round and serves as a planning\-time diagnostic\. Rather than checking whether the plan is judged as feasible, IPS examines whether a plan’s evaluation remains stable across agents that possess different information\. Plans whose evaluation varies substantially across heterogeneous information conditions are treated as epistemically fragile, while plans with stable evaluations are selected\.*Consistency\-guided Epistemic State Refinement*\(CESR\) operates across rounds by recording discrepancies between the planning agent’s locally selected plans and the IPS\-selected plans, interpreting these discrepancies as signals for epistemic miscalibration\. These signals are integrated into persistent memory, which constrains subsequent planning and prevents previously observed miscalibration patterns from reoccurring as information accumulates\. Together, EPC\-AW shifts failure mitigation from execution\-time correction to planning\-time epistemic calibration\. In summary, this paper makes the following contributions:

- •We formalize*epistemic miscalibration in planning*as a repair target in LLM\-based multi\-agent systems, revealing failures arising from miscalibrated planning assessments even under correct execution\.
- •To address epistemic miscalibration in planning assessments under the absence of observable errors, we propose IPS, which selects plans whose evaluations remain stable across agents operating under heterogeneous information\.
- •To adapt epistemic calibration under evolving information, we introduce CESR, a memory\-driven mechanism that leverages past calibration errors to constrain future planning and suppress persistent misjudgments\.
- •Extensive experiments on six LLM\-based multi\-agent benchmarks demonstrate that EPC\-AW significantly improves task success, achieving an average 9\.75% increase in system\-level success rate\.

## 2Related Works

### 2\.1Failure and Repair in LLM\-based Multi\-Agent Systems

LLM\-based multi\-agent systems have been increasingly adopted for complex reasoning, tool use, and long\-horizon decision making\(Becattiniet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib118); Ronanki,[2025](https://arxiv.org/html/2605.23414#bib.bib115); Liet al\.,[2024](https://arxiv.org/html/2605.23414#bib.bib129)\), but they remain vulnerable to failures caused by reasoning errors, coordination breakdowns, and unreliable information use\(Dobrovskyet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib17); Hammondet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib128); Zhanget al\.,[2025a](https://arxiv.org/html/2605.23414#bib.bib134)\)\. To improve system reliability, prior work has proposed mechanisms for diagnosing and repairing failures by analyzing execution traces and agent interactions during or after runtime\(Cemriet al\.,[2026](https://arxiv.org/html/2605.23414#bib.bib18); Shenet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib113); Eppersonet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib48); Maet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib49)\)\.

Existing approaches differ in when and how corrections are applied\. Some methods perform post\-hoc or cross\-run debugging by inspecting interaction traces or validating failure hypotheses through repeated executions\(Eppersonet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib48); Maet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib49)\)\. Others introduce online monitoring and correction, such as rollback and reflection during execution\(Lianget al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib43)\)or history\-conditioned anomaly detection with local repairs\(Shenet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib113)\)\. Despite these differences, most methods focus on correcting incorrect actions or local reasoning errors during execution\.

In contrast, this work formulates*epistemic miscalibration in planning*as a distinct repair target in LLM\-based multi\-agent systems\. It reveals that system\-level failures can arise from miscalibrated feasibility assessments at planning time, even when all subsequent executions are locally correct\.

### 2\.2Model\- and Agent\-Level Epistemic Calibration

Epistemic calibration in large language models has been widely studied in the context of overconfidence, and uncertainty estimation\(Abbasi Yadkoriet al\.,[2024](https://arxiv.org/html/2605.23414#bib.bib29); Leeet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib30); Chhikara,[2025](https://arxiv.org/html/2605.23414#bib.bib24)\)\. This line of work treats miscalibration as a property of an individual model or agent, aiming to align expressed confidence with actual knowledge\.

Several methods leverage multi\-agent structures to enhance calibration through consensus, voting, or verifier agents that critique intermediate outputs\(Clarket al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib28); Wenet al\.,[2024](https://arxiv.org/html/2605.23414#bib.bib25); Pitreet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib31)\)\. These approaches rely on LLMs acting as judges, but such first\-order judgments are themselves subject to epistemic miscalibration, limiting their ability to diagnose epistemic failures\. Another related line of work draws from peer prediction and Bayesian truth inference\(Witkowski and Parkes,[2012](https://arxiv.org/html/2605.23414#bib.bib37); Chenet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib39)\), which calibrate reports using incentive signals and repeated feedback under static game\-theoretic assumptions\.

In contrast, we study epistemic miscalibration as a system\-level failure mode in running LLM\-based multi\-agent systems\. Our goal is to repair task failures by intervening during system operation, where there is no additional payoff signals or external supervision\. Motivated by peer\-based calibration methods, we exploit the stability of agents’ evaluations across heterogeneous information and refine epistemic states over time based on persistent cross\-agent inconsistencies\. Further details are provided in Appendix\.[G](https://arxiv.org/html/2605.23414#A7)\.

## 3Problem Formulation

### 3\.1Operation of LLM\-based Multi\-Agent Systems

An LLM\-based multi\-agent system is denoted byℳ\\mathcal\{M\}and consists of multiple interacting agents that coordinate through structured messages, shared memory, and external tools\(Liet al\.,[2026](https://arxiv.org/html/2605.23414#bib.bib1); Wuet al\.,[2024](https://arxiv.org/html/2605.23414#bib.bib35)\)\. Given a user queryQQ, the system aims to produce an answer by iteratively planning, executing tool\-mediated actions, and aggregating newly acquired information\.

At iterationtt, the system maintains an information contextℐ\(t\)\\mathcal\{I\}^\{\(t\)\}, which aggregates the user query, previously acquired evidence, and interaction history\. Conditioned onℐ\(t\)\\mathcal\{I\}^\{\(t\)\}, a planning agent generates a planπ\(t\)=\(g\(t\),a\(t\)\)\\pi^\{\(t\)\}=\(g^\{\(t\)\},a^\{\(t\)\}\), whereg\(t\)g^\{\(t\)\}denotes an intermediate goal anda\(t\)a^\{\(t\)\}specifies an action intended to acquire information relevant tog\(t\)g^\{\(t\)\}through available tools, such as web search or code execution\. The selected action is executed, yielding an observation or piece of evidencee\(t\)e^\{\(t\)\}, which is incorporated into the information context via an aggregation operator

ℐ\(t\+1\)=D​\(ℐ\(t\),π\(t\),e\(t\)\),\\mathcal\{I\}^\{\(t\+1\)\}=\\mathrm\{D\}\(\\mathcal\{I\}^\{\(t\)\},\\pi^\{\(t\)\},e^\{\(t\)\}\),\(1\)whereD​\(⋅\)\\mathrm\{D\}\(\\cdot\)integrates new evidence into the existing context\.

This planning\-execution loop continues until a stopping criterion is met\. The overall behavior of the system defines a mapping from the user query to a final answer,

Y^=ℳ​\(Q\),\\hat\{Y\}=\\mathcal\{M\}\(Q\),\(2\)whereY^\\hat\{Y\}denotes the generated answer to the queryQQ\.

### 3\.2Epistemic Miscalibration in Planning

*Epistemic miscalibration in planning*refers to a failure of feasibility assessment, in which the agent assigns unwarranted confidence to its judgments and fails to recognize the limits of its knowledge in tasks that require iterative information acquisition\.

Letϕ\\phidenote the latent factual situation of the task, which specifies, in principle, the complete information required to justify a plan\. This information is not directly observable and must be acquired through iterative interactions\.

Given the information available at steptt, denoted byℐ\(t\)\\mathcal\{I\}^\{\(t\)\}, the planning agent implicitly affirms the feasibility of a planπ\(t\)\\pi^\{\(t\)\}\. The subjective feasibility assessment is formulated as

J​\(π\(t\)∣ℐ\(t\)\)\.J\(\\pi^\{\(t\)\}\\mid\\mathcal\{I\}^\{\(t\)\}\)\.\(3\)whereJ​\(⋅\)J\(\\cdot\)denotes an abstract judgment function and this assessment reflects the agent’s confidence that executing the actiona\(t\)a^\{\(t\)\}in the plan can satisfy the current goalg\(t\)g^\{\(t\)\}through external tool invocations under the available information contextℐ\(t\)\\mathcal\{I\}^\{\(t\)\}\.

In contrast, the objective feasibility of a planπ\(t\)\\pi^\{\(t\)\}under the latent factual situationϕ\\phiis characterized by

E​\(π\(t\)∣ϕ\),E\(\\pi^\{\(t\)\}\\mid\\phi\),\(4\)which indicates whether the plan can, in principle, be justified given the complete underlying information\.

Epistemic miscalibration in planning arises when the agent’s feasibility assessment formed under partial information,J​\(π\(t\)∣ℐ\(t\)\)J\(\\pi^\{\(t\)\}\\mid\\mathcal\{I\}^\{\(t\)\}\), is misaligned with the objective feasibility conditionE​\(π\(t\)∣ϕ\)E\(\\pi^\{\(t\)\}\\mid\\phi\)defined under complete information\.

### 3\.3Problem Statement

Given a user queryQQ, an LLM\-based multi\-agent system produces a final answerY^\\hat\{Y\}, whose correctness is evaluated against the ground\-truth answerY⋆Y^\{\\star\}\. In practice, even all executions are correct, failures often arise from epistemic miscalibration during the planning phase\.

The problem addressed in this work is to design an agentic workflow that mitigates epistemic miscalibration in planning by guiding the system toward plans whose feasibility assessments remain well\-aligned with the information available as the system acquires new evidence\. By improving epistemic calibration at the planning stage, the workflow aims to reduce downstream errors and increase the likelihood that the final output satisfiesY^=Y⋆\\hat\{Y\}=Y^\{\\star\}\.

![Refer to caption](https://arxiv.org/html/2605.23414v1/x1.png)Figure 1:Overview of EPC\-AW\. EPC\-AW consists of three agents, the Planner, Executor, and Diagnoser, each with heterogeneous information in memory\. At each round, Information\-consistency\-based Plan Selection evaluates candidate plans across agents and selects those with stable evaluations, providing a planning\-time calibration signal\. Across rounds, Consistency\-guided Epistemic State Refinement aggregates consistency feedback to guide planning under evolving information\. The process terminates upon satisfying a stopping condition, after which the final answer is produced\.

## 4EPC\-AW: Epistemic Planning Calibration Agentic Workflow

Epistemic miscalibration originates at planning and thus persists even under correct execution and valid tool outputs\. Moreover, continual information updates shift feasibility assessments, allowing miscalibration to recur in the subsequent planning\. To address these challenges, we propose the*Epistemic Planning Calibration Agentic Workflow*\(EPC\-AW\), which intervenes directly at planning time by separating planning, execution, and diagnosis under heterogeneous information\. The overview is shown in Fig\.[1](https://arxiv.org/html/2605.23414#S3.F1), with architectural details in Sec\.[4\.1](https://arxiv.org/html/2605.23414#S4.SS1)\. EPC\-AW operates both within rounds and across rounds\. Within each round,*Information\-consistency\-based Plan Selection*\(IPS\) favors plans whose evaluation remain stable across agents with heterogeneous information, mitigating epistemic miscalibration in planning assessments in the absence of observable error signals \(Sec\.[4\.2](https://arxiv.org/html/2605.23414#S4.SS2)\)\. Across rounds,*Consistency\-guided Epistemic State Refinement*\(CESR\) aggregates consistency feedback into persistent constraints that guide subsequent planning as information evolves \(Sec\.[4\.3](https://arxiv.org/html/2605.23414#S4.SS3)\)\.

### 4\.1System Architecture and Workflow

EPC\-AW is instantiated as a role\-specialized LLM\-based multi\-agent system consisting of three agents with fixed roles: a*Planner*\(APA\_\{P\}\), an*Executor*\(AEA\_\{E\}\), and a*Diagnoser*\(ADA\_\{D\}\)\. The system operates over discrete interaction roundst=1,2,…,Tt=1,2,\\ldots,T\. Within each round, agent roles and communication interfaces remain unchanged, forming a stabilized interaction protocol\.

##### Workflow\.

At roundtt, the PlannerAPA\_\{P\}generates an intermediate planπ\(t\)=\(g\(t\),a\(t\)\)\\pi^\{\(t\)\}\\\!=\\\!\(g^\{\(t\)\},a^\{\(t\)\}\), whereg\(t\)g^\{\(t\)\}denotes the goal anda\(t\)a^\{\(t\)\}denotes the action intended to acquire evidence relevant tog\(t\)g^\{\(t\)\}through external tools\. The ExecutorAEA\_\{E\}instantiatesa\(t\)a^\{\(t\)\}by invoking the corresponding tool with concrete inputs and parameters, producing an execution outcomeo\(t\)o^\{\(t\)\}\. The DiagnoserADA\_\{D\}evaluates whether the outcomeo\(t\)o^\{\(t\)\}meets the planπ\(t\)\\pi^\{\(t\)\}, producing diagnostic feedbackd\(t\)d^\{\(t\)\}that is communicated back to the Planner\.

##### Memory Structure\.

EPC\-AW maintains a shared*system\-level memory*and*agent\-level memories*that are private to individual agents\. The agent\-level memories induce heterogeneous epistemic states across agent roles\.

At roundtt, the complete interaction history is denoted asℋ\(t\)=\{\(π\(k\),o\(k\),d\(k\)\)\}k=1t−1\\mathcal\{H\}^\{\(t\)\}=\\\{\(\\pi^\{\(k\)\},o^\{\(k\)\},d^\{\(k\)\}\)\\\}\_\{k=1\}^\{t\-1\}, whereπ\(k\)\\pi^\{\(k\)\}is the planned action,o\(k\)o^\{\(k\)\}the execution outcome, andd\(k\)d^\{\(k\)\}the diagnostic evaluation\. This complete history is not directly accessible to any single agent\. To formalize the execution analysis withind\(k\)d^\{\(k\)\}, we define a diagnostic indicator as

𝟏fail​\(d\(k\)\)=\{1,if​d\(k\)=Unsupported,0,otherwise\.\\mathbf\{1\}\_\{\\mathrm\{fail\}\}\(d^\{\(k\)\}\)=\\begin\{cases\}1,&\\text\{if \}d^\{\(k\)\}=\\textsc\{Unsupported\},\\\\ 0,&\\text\{otherwise\}\.\\end\{cases\}\(5\)whered\(k\)=Unsupportedd^\{\(k\)\}=\\textsc\{Unsupported\}indicates that the execution outcomeo\(k\)o^\{\(k\)\}does not meet the planπ\(k\)\\pi^\{\(k\)\}\.

##### System\-Level Memory\.

The system\-level memory at roundttis defined as

ℳsys\(t\)=⟨Q,Ψ\(t\),𝒰⟩,\\mathcal\{M\}\_\{\\mathrm\{sys\}\}^\{\(t\)\}=\\langle Q,\\;\\Psi^\{\(t\)\},\\mathcal\{U\}\\rangle,\(6\)whereQQdenotes the user query,Ψ\(t\)\\Psi^\{\(t\)\}is a set of verifiable evidence accumulated over rounds, and𝒰=\{UP,UE,UD\}\\mathcal\{U\}=\\\{U\_\{P\},U\_\{E\},U\_\{D\}\\\}denotes the abstract description of the role for all agents\.

At roundkk, an evidence itemψ\(k\)\\psi^\{\(k\)\}is extracted from the execution outcomeo\(k\)o^\{\(k\)\}only if the execution outcome meets the plan\. Accordingly, the system\-level memory is updated in a set\-augmentation manner:

Ψ\(k\)=Ψ\(k−1\)∪\{ψ\(k\)\}if​𝟏fail​\(d\(k\)\)=0,\\Psi^\{\(k\)\}=\\Psi^\{\(k\-1\)\}\\cup\\\{\\psi^\{\(k\)\}\\\}\\quad\\text\{if \}\\mathbf\{1\}\_\{\\mathrm\{fail\}\}\(d^\{\(k\)\}\)=0,\(7\)

##### Agent\-level Memory\.

Each agent𝒜i∈\{AP,AE,AD\}\\mathcal\{A\}^\{i\}\\in\\\{A\_\{P\},A\_\{E\},A\_\{D\}\\\}maintains a private agent\-level memoryℳi\(t\)\\mathcal\{M\}\_\{i\}^\{\(t\)\}, defined as a role\-specific projection ofℋ\(t\)\\mathcal\{H\}^\{\(t\)\}\. The Executor maintains a complete execution traceℳE\(t\)=ℋ\(t\)\\mathcal\{M\}\_\{E\}^\{\(t\)\}=\\mathcal\{H\}^\{\(t\)\}\.

In contrast, the Planner and Diagnoser apply complementary selection criteria\. The Planner records plans that are not satisfied by the executions:

ℳP\(t\)=\{\(π\(k\),o\(k\),d\(k\)\)∈ℋ\(t\)∣𝟏fail​\(d\(k\)\)=1\}\.\\mathcal\{M\}\_\{P\}^\{\(t\)\}=\\\{\(\\pi^\{\(k\)\},o^\{\(k\)\},d^\{\(k\)\}\)\\in\\mathcal\{H\}^\{\(t\)\}\\mid\\mathbf\{1\}\_\{\\mathrm\{fail\}\}\(d^\{\(k\)\}\)=1\\\}\.\(8\)Conversely, the Diagnoser selectively records executions whose outcomes satisfy the corresponding plan:

ℳD\(t\)=\{\(π\(k\),o\(k\),d\(k\)\)∈ℋ\(t\)∣𝟏fail​\(d\(k\)\)=0\}\.\\mathcal\{M\}\_\{D\}^\{\(t\)\}=\\\{\(\\pi^\{\(k\)\},o^\{\(k\)\},d^\{\(k\)\}\)\\in\\mathcal\{H\}^\{\(t\)\}\\mid\\mathbf\{1\}\_\{\\mathrm\{fail\}\}\(d^\{\(k\)\}\)=0\\\}\.\(9\)

##### Information State\.

Each agentAiA\_\{i\}operates under a private information state

γi\(t\)≜\(Ui,ℳi\(t\)\),\\gamma\_\{i\}^\{\(t\)\}\\triangleq\(U\_\{i\},\\;\\mathcal\{M\}\_\{i\}^\{\(t\)\}\),\(10\)which induces heterogeneous information encodings even under a shared system\-level memoryℳsys\(t\)\\mathcal\{M\}\_\{\\mathrm\{sys\}\}^\{\(t\)\}\.

### 4\.2Information\-consistency\-based Plan Selection

In realistic multi\-agent planning scenarios, the feasibility of a plan is typically not verifiable at the planning phase\. Moreover, epistemic miscalibration does not manifest as explicit execution errors or observable reasoning failures\. To address this challenge, we introduce*Information\-consistency\-based Plan Selection*\(IPS\)\. Rather than attempting to only assess feasibility, IPS uses cross\-agent evaluation consistency as a planning\-time signal of epistemic support\. It prioritizes plans whose evaluations remain stable across agents operating under heterogeneous information states\.

#### 4\.2\.1Planner Candidate Plan Generation

At roundtt, the PlannerAPA\_\{P\}generates a finite set of candidate plans conditioned on the shared system\-level memoryℳsys\(t\)\\mathcal\{M\}\_\{\\mathrm\{sys\}\}^\{\(t\)\}and its private information stateγP\(t\)\\gamma\_\{P\}^\{\(t\)\}:

Π\(t\)=AP​\(ℳsys\(t\),γP\(t\)\),\\Pi^\{\(t\)\}=A\_\{P\}\\\!\\left\(\\mathcal\{M\}\_\{\\mathrm\{sys\}\}^\{\(t\)\},\\gamma\_\{P\}^\{\(t\)\}\\right\),\(11\)whereΠ\(t\)=\{π1\(t\),…,πK\(t\)\}\\Pi^\{\(t\)\}=\\\{\\pi^\{\(t\)\}\_\{1\},\\dots,\\pi^\{\(t\)\}\_\{K\}\\\}, and eachπk\(t\)∈Π\(t\)\\pi\_\{k\}^\{\(t\)\}\\in\\Pi^\{\(t\)\}represents a structurally distinct strategy\.

#### 4\.2\.2Information\-Conditioned Plan Evaluation

A common evaluation interface maps information states and candidate plans to real\-valued scores:

ℰ:\(γi\(t\),ℳsys\(t\),Π\(t\)\)⟶ℝK\.\\mathcal\{E\}:\\;\(\\gamma\_\{i\}^\{\(t\)\},\\mathcal\{M\}\_\{\\mathrm\{sys\}\}^\{\(t\)\},\\Pi^\{\(t\)\}\)\\longrightarrow\\mathbb\{R\}^\{K\}\.\(12\)Each agent computes its evaluation vector

𝐞i\(t\)=ℰ​\(γi\(t\),ℳsys\(t\),Π\(t\)\),\\mathbf\{e\}\_\{i\}^\{\(t\)\}=\\mathcal\{E\}\(\\gamma\_\{i\}^\{\(t\)\},\\mathcal\{M\}\_\{\\mathrm\{sys\}\}^\{\(t\)\},\\Pi^\{\(t\)\}\),\(13\)whereei\(t\)​\(k\)∈𝐞i\(t\)e\_\{i\}^\{\(t\)\}\(k\)\\in\\mathbf\{e\}\_\{i\}^\{\(t\)\}reflects agentii’s assessment of planπk\(t\)\\pi\_\{k\}^\{\(t\)\}under its information state\.

#### 4\.2\.3Cross\-Agent Evaluation Prediction

Under epistemic miscalibration, an agent’s direct evaluation is biased by its private information, making self\-assigned scores an unreliable indicator of feasibility\. We therefore leverage cross\-agent evaluations under heterogeneous information as signals\. Each agent predicts how candidate plans would be evaluated under alternative information states, and plans whose predicted evaluations remain stable across these perspectives are less sensitive to epistemic variation and thus less prone to epistemic miscalibration\.

For each agentj≠ij\\neq i, agentiiconstructs an approximation ofjj’s information state

γ^i→j\(t\)≜\(Uj,ℳi\(t\)\)\.\\hat\{\\gamma\}\_\{i\\rightarrow j\}^\{\(t\)\}\\triangleq\(U\_\{j\},\\mathcal\{M\}\_\{i\}^\{\(t\)\}\)\.\(14\)
Then, agentiiapplies the evaluation interface to obtain prediction scores

e^i→j\(t\)​\(k\)=ℰ​\(γ^i→j\(t\),ℳsys\(t\),Π\(t\)\)k\.\\hat\{e\}\_\{i\\rightarrow j\}^\{\(t\)\}\(k\)=\\mathcal\{E\}\\big\(\\hat\{\\gamma\}\_\{i\\rightarrow j\}^\{\(t\)\},\\mathcal\{M\}\_\{\\mathrm\{sys\}\}^\{\(t\)\},\\Pi^\{\(t\)\}\\big\)\_\{k\}\.\(15\)Collecting these predictions defines the cross\-agent evaluation vector𝐞^i→−i\(t\)=\{e^i→j\(t\)​\(k\)\}j≠i,k∈\[K\]\\widehat\{\\mathbf\{e\}\}\_\{i\\rightarrow\-i\}^\{\(t\)\}=\\\{\\hat\{e\}\_\{i\\rightarrow j\}^\{\(t\)\}\(k\)\\\}\_\{j\\neq i,\\,k\\in\[K\]\}\.

Agentiisummarizes the predicted evaluations by averaging over peers:

e¯−i\(t\)​\(k\)=1\|𝒜\|−1​∑j≠ie^i→j\(t\)​\(k\),\\bar\{e\}\_\{\-i\}^\{\(t\)\}\(k\)=\\frac\{1\}\{\|\\mathcal\{A\}\|\-1\}\\sum\_\{j\\neq i\}\\hat\{e\}\_\{i\\rightarrow j\}^\{\(t\)\}\(k\),\(16\)which represents agentii’s estimate of the plan’s expected evaluation under heterogeneous information\.

#### 4\.2\.4Information\-Consistency Aggregation

To quantify cross\-agent information consistency, each agent compares its own plan evaluation with the simulated aggregate evaluation from other agents\. The agent\-level information\-consistency score is defined as

si\(t\)​\(k\)=log⁡\(ei\(t\)​\(k\)\)−log⁡\(e¯−i\(t\)​\(k\)\)\.s\_\{i\}^\{\(t\)\}\(k\)=\\log\\big\(e\_\{i\}^\{\(t\)\}\(k\)\\big\)\-\\log\\big\(\\bar\{e\}\_\{\-i\}^\{\(t\)\}\(k\)\\big\)\.\(17\)
Aggregating across all agents yields the plan\-level information\-consistency score:

CIPS​\(πk\(t\)\)=1\|𝒜\|​∑i∈𝒜si\(t\)​\(k\)\.C\_\{\\mathrm\{IPS\}\}\(\\pi\_\{k\}^\{\(t\)\}\)=\\frac\{1\}\{\|\\mathcal\{A\}\|\}\\sum\_\{i\\in\\mathcal\{A\}\}s\_\{i\}^\{\(t\)\}\(k\)\.\(18\)
Plans with higherCIPS​\(πk\(t\)\)C\_\{\\mathrm\{IPS\}\}\(\\pi\_\{k\}^\{\(t\)\}\)indicate that the plan remains more consistently favored across heterogeneous information\-conditioned evaluations, serving as a heuristic measure of cross\-agent information consistency under heterogeneous information states\. Then, the IPS\-selected planπ∗\(t\)\\pi^\{\(t\)\}\_\{\\ast\}at roundttis formulated as

π∗\(t\)=arg⁡maxπk\(t\)∈Π\(t\)⁡CIPS​\(πk\(t\)\)\.\\pi^\{\(t\)\}\_\{\\ast\}=\\arg\\max\_\{\\pi\_\{k\}^\{\(t\)\}\\in\\Pi^\{\(t\)\}\}C\_\{\\mathrm\{IPS\}\}\(\\pi\_\{k\}^\{\(t\)\}\)\.\(19\)

### 4\.3Consistency\-guided Epistemic State Refinement

While Information\-consistency\-based Plan Selection \(IPS\) calibrates planning decisions within a single round, it cannot prevent epistemic miscalibration from recurring over long\-horizon interactions\. As planning proceeds, the Planner’s internal memory is continually updated, inducing non\-stationary plan generation behavior that per\-round selection alone cannot correct\. We therefore introduce*Consistency\-guided Epistemic State Refinement*\(CESR\), a memory\-driven mechanism that accumulates cross\-round consistency signals and integrates them into persistent epistemic state constraints, shaping subsequent planning and preventing the reintroduction of previously identified miscalibration\.

#### 4\.3\.1Cross\-Round Plan Divergence

Within roundtt, two plans are distinguished\. The Planner selects a plan based on its self\-conditioned evaluation,

πP\(t\)=arg⁡maxπk\(t\)∈Π\(t\)⁡eP\(t\)​\(k\),\\pi^\{\(t\)\}\_\{P\}=\\arg\\max\_\{\\pi\_\{k\}^\{\(t\)\}\\in\\Pi^\{\(t\)\}\}e\_\{P\}^\{\(t\)\}\(k\),\(20\)while IPS selects the information\-consistent planπ∗\(t\)\\pi^\{\(t\)\}\_\{\\ast\}\. A divergenceπP\(t\)≠π∗\(t\)\\pi^\{\(t\)\}\_\{P\}\\neq\\pi^\{\(t\)\}\_\{\\ast\}serves as a diagnostic signal that the Planner’s selected plan is more susceptible to epistemic miscalibration than the plan whose evaluations remain stable under heterogeneous information\.

#### 4\.3\.2Epistemic Constraint Induction

Upon detecting such a discrepancy, the Diagnoser generates a lightweight epistemic constraint

c\(t\)=𝒢​\(πP\(t\),π∗\(t\),ℳsys\(t\)\),c^\{\(t\)\}=\\mathcal\{G\}\\\!\\left\(\\pi^\{\(t\)\}\_\{P\},\\;\\pi^\{\(t\)\}\_\{\\ast\},\\;\\mathcal\{M\}\_\{\\mathrm\{sys\}\}^\{\(t\)\}\\right\),\(21\)which abstracts the salient information and structural features associated with the potential epistemic miscalibration revealed by the discrepancy\.

The constraint is accumulated in the Planner’s memory via

ℳP\(t\+1\)=ℳP\(t\)∪\{c\(t\)\}\.\\mathcal\{M\}\_\{P\}^\{\(t\+1\)\}=\\mathcal\{M\}\_\{P\}^\{\(t\)\}\\cup\\\{c^\{\(t\)\}\\\}\.\(22\)

### 4\.4Answer Generation and Termination Control

At each round, the Diagnoser determines whether the accumulated system\-level information suffices to answer the user queryQQ\. This decision is captured by a binary indicator and we formalize it as follows:

𝟏stop\(t\)​\(Q,Ψ\(t\)\)=\{1,if sufficient evidence,0,otherwise\.\\mathbf\{1\}\_\{\\mathrm\{stop\}\}^\{\(t\)\}\\left\(Q,\\;\\Psi^\{\(t\)\}\\right\)=\\begin\{cases\}1,&\\text\{if sufficient evidence\},\\\\ 0,&\\text\{otherwise\}\.\\end\{cases\}\(23\)
If𝟏\\mathbf\{1\}, it transitions to answer generation; otherwise, the system proceeds to the next round\.

Upon termination, the final answer is produced by

Y^=𝒢​\(Q,Ψ\(t\),ℋ\(t\)\),\\hat\{Y\}=\\mathcal\{G\}\\\!\\left\(Q,\\;\\Psi^\{\(t\)\},\\;\\mathcal\{H\}^\{\(t\)\}\\right\),\(24\)whereΨ\(t\)\\Psi^\{\(t\)\}contains accumulated verified evidence andℋ\(t\)\\mathcal\{H\}^\{\(t\)\}summarizes the interaction history\.

Table 1:Accuracy \(%\) comparison of different repair methods\.Δ\\Deltadenotes the absolute improvement of EPC\-AW over the corresponding baseline method on each benchmark\. Darker gray indicates larger improvement\.

## 5Experiment

### 5\.1Experimental Setup

#### 5\.1\.1Dataset

We evaluate on six benchmarks covering diverse reasoning and search demands\.Bamboogle\(Presset al\.,[2023](https://arxiv.org/html/2605.23414#bib.bib47)\)targets compositional multi\-step reasoning with minimal external search\.2Wiki\(Hoet al\.,[2020](https://arxiv.org/html/2605.23414#bib.bib46)\)emphasizes multi\-hop question answering with strong dependence on information retrieval across heterogeneous sources\.HotpotQA\(Yanget al\.,[2018](https://arxiv.org/html/2605.23414#bib.bib45)\)requires multi\-hop reasoning over Wikipedia with moderate search complexity\.Musique\(Trivediet al\.,[2022](https://arxiv.org/html/2605.23414#bib.bib44)\)stresses sequential reasoning with tightly coupled intermediate inferences\.GAIA\(Mialonet al\.,[2023](https://arxiv.org/html/2605.23414#bib.bib137)\)evaluates agentic planning in open\-world settings, heavily relying on search and tool use\.MedQA\(Yanget al\.,[2024](https://arxiv.org/html/2605.23414#bib.bib40)\)focuses on clinical reasoning without external retrieval\.

Overall, these datasets span a spectrum from reasoning\-intensive inference to search\-driven open\-world problem solving, enabling systematic evaluation of epistemic miscalibration across diverse planning and reasoning settings\. More details are provided in Appendix[A](https://arxiv.org/html/2605.23414#A1)\.

#### 5\.1\.2Baselines

We compare EPC\-AW with three baselines implemented under the AgentFlow framework\(Liet al\.,[2026](https://arxiv.org/html/2605.23414#bib.bib1)\), which differ in how epistemic miscalibration is handled during the planning phase\.*No\-Repair*follows the original AgentFlow setting, generating and executing a single forward plan without any diagnosis or recovery\.*Retry*, inspired by metacognitive retry strategies in MAST\(Shenet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib113)\), detects epistemic miscalibration at the current planning step and re\-generates the plan locally using diagnostic feedback\.*Rollback*, inspired by COCO\(Lianget al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib43)\), performs system\-level recovery by rolling back the entire system state to a selected historical step before re\-planning\. All baselines share the same agent architecture, system memory, and execution history, and restrict miscalibration diagnosis to the planning phase for fair comparison\. Additional details are provided in Appendix\.[B](https://arxiv.org/html/2605.23414#A2)\.

#### 5\.1\.3Implementation

All agents in EPC\-AW are instantiated using Qwen3\-Coder\-30B\(Yanget al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib26)\)\. The model is deployed on a server equipped with four NVIDIA RTX 4090 GPUs and served viavLLMfor efficient inference\. During plan generation, the Planner samplesn=9n=9candidate next\-step plans with temperature0\.90\.9to encourage exploration, while all other generations operate at temperature0for deterministic execution\. Candidate plans are evaluated using a predefined feasibility metric, yielding scores from 1 to 5\. The maximum number of iterations is set to1010\. In IPS, plan evaluations and predictions are performed independently for each agent, with outputs obtained via separate LLM calls\.

The system interacts with five tools following AgentFlow\(Liet al\.,[2026](https://arxiv.org/html/2605.23414#bib.bib1)\): a base generator, a Python coder, a Google Search, a Wikipedia Search, and a Web Search\. To reduce external dependencies, we replace AgentFlow’s official Google Search API calls with a lightweight local pipeline that retrieves the top\-10 search results via a Chrome\-based interface\. To further mitigate information leakage due to publicly available datasets, we apply keyword\-based filtering during the search stage to exclude HuggingFace repositories\. Retrieved pages are then processed by the Web Search tool, followed by LLM\-based summarization\.

For evaluation, GPT\-4o is employed as an automatic judge to determine whether model predictions match the corresponding ground\-truth answers, following standard practice in tool\-augmented reasoning benchmarks\(Liet al\.,[2026](https://arxiv.org/html/2605.23414#bib.bib1); Arifet al\.,[2024](https://arxiv.org/html/2605.23414#bib.bib20)\)\. To reduce the impact of stochasticity, all experiments are repeated three times, and we report the average accuracy across runs\. All evaluation settings are aligned with those used in AgentFlow to facilitate direct comparison\. Additional details are provided in Appendix\.[C](https://arxiv.org/html/2605.23414#A3)\.

### 5\.2Performance Analysis

Table[1](https://arxiv.org/html/2605.23414#S4.T1)summarizes the accuracy of all methods across six benchmarks\. All repair\-based methods consistently outperform the No\-Repair baseline, indicating that addressing epistemic miscalibration mitigates task failures\. On average, Retry improves accuracy from 36\.52% to 39\.41%, while Rollback further increases performance to 41\.63% \(\+5\.11% over No\-Repair\)\. EPC\-AW achieves the best results on all benchmarks, yielding a 9\.75% absolute improvement over No\-Repair and a 4\.64% gain over Rollback\.

Comparing the two baselines, Retry performs local re\-planning at the current step, whereas Rollback enables recovery from earlier planning decisions by reverting the system state\. This broader intervention scope allows Rollback to consistently outperform Retry across all benchmarks, with especially notable improvements on HotpotQA \(\+5\.00%\) and Musique \(\+3\.33%\)\.

However, both Retry and Rollback rely on agent\-generated diagnosis and repair signals\. Since the agents themselves remain epistemically miscalibrated, the resulting failure analysis and recovery strategies can still be unreliable\. Consequently, although these methods partially alleviate execution errors, they do not directly reduce the epistemic uncertainty underlying erroneous plan selection\.

By contrast, EPC\-AW intervenes directly during planning\. Through posterior aggregation in IPS and persistent corrective constraints in CESR, EPC\-AW reduces epistemic risk during plan selection and prevents recurrent miscalibration across rounds, resulting in substantially stronger and more consistent performance gains\.

### 5\.3Ablation Study

We conduct ablation studies on four benchmarks spanning distinct reasoning and search characteristics\. GAIA and 2Wiki represent search\-dominant settings, with GAIA emphasizing open\-world exploration and tool use, and 2Wiki focusing on retrieval\-intensive multi\-hop QA, whereas Bamboogle targets compositional reasoning with minimal search and HotpotQA occupies an intermediate reasoning and retrieval regime\. The remaining benchmarks are omitted due to substantial overlap with the selected set\.

To isolate component contributions, we evaluate two controlled variants\. EPC\-AW†removes both IPS and CESR, approximating planning without epistemic repair with agents under heterogeneous information, while EPC\-AW‡retains IPS but disables CESR, corresponding to plan selection without constraint feedback\.

Table 2:Ablation study on four representative benchmarks, evaluating the individual and combined effects of IPS and CESR\.As shown in Table[2](https://arxiv.org/html/2605.23414#S5.T2), IPS alone exhibits regime\-dependent effects\. On the reasoning\-centric Bamboogle benchmark, EPC\-AW‡yields modest gains, indicating that IPS can regularize planning by favoring epistemically supported reasoning paths\. However, on search\-intensive benchmarks such as 2Wiki, the same variant degrades performance, suggesting that IPS in isolation induces overly conservative planning that suppresses necessary exploration\.

CESR plays a critical corrective role by transforming this conservatism into structured planning guidance\. By identifying sources of epistemic miscalibration through plan preference discrepancies and enforcing memory\-based constraints across rounds, CESR enables the Planner to operate within safer feasibility regions without suppressing exploration\. As a result, the full EPC\-AW model consistently outperforms both ablations across all datasets, achieving gains of up to \+15\.00% on 2Wiki and \+12\.00% on HotpotQA, indicating that intra\-round plan selection and cross\-round constraints are jointly essential for robust agentic planning\.

### 5\.4Mean\-Score Aggregation vs\. Cross\-Agent Consistency

We compare Information\-consistency\-based Plan Selection \(IPS\) with a score aggregation baseline, denoted as Mean\-Score Aggregation \(MSA\), which selects plans by averaging feasibility scores from three agents operating under heterogeneous information\. This comparison isolates cross\-agent consistency from naive score aggregation, highlighting their distinct effects in agentic planning\.

Table 3:Comparison between EPC\-AW with Mean\-Score Aggregation \(MSA\) and IPS on four representative benchmarks\.Results in Table[3](https://arxiv.org/html/2605.23414#S5.T3)reveal a clear task\-dependent pattern\. On 2Wiki, a predominantly search\-oriented benchmark with limited reasoning depth, MSA slightly outperforms IPS \(52\.33% vs\. 51\.83%\)\. In this setting, performance is largely driven by information retrieval, and averaging feasibility estimates provides a reasonable approximation\.

In contrast, IPS consistently outperforms MSA on tasks requiring multi\-step reasoning and exploration\. On HotpotQA and GAIA, IPS achieves gains of 10\.33% and 3\.67%, respectively\. These tasks exhibit stronger agent\-level disagreement, where naive score averaging masks inconsistencies across reasoning paths and leads to overconfident yet weakly grounded plan selection\. By favoring plans that remain consistent across agent assessments, IPS enables EPC\-AW to better mitigate miscalibration in complex planning regimes\.

![Refer to caption](https://arxiv.org/html/2605.23414v1/x2.png)Figure 2:Sensitivity of EPC\-AW to the number of sampled candidate plans\.Table 4:Generalization results across different LLM backbones\.
### 5\.5Hyperparameter Sensitivity

We analyze the sensitivity of EPC\-AW to the number of sampled plansnnin IPS, varyingn∈\{1,3,5,7,9\}n\\in\\\{1,3,5,7,9\\\}across all datasets\. Whenn=1n=1, IPS degenerates to generating a single plan under heterogeneous information, leaving no alternative plans under different knowledge states for comparison\. As a result, EPC\-AW cannot assess cross\-agent consistency and yields the weakest performance\.

Oncen\>1n\>1, performance improves markedly, with the largest gain observed fromn=1n=1ton=3n=3, indicating that the main benefit comes from introducing epistemic diversity rather than exhaustive sampling\.

Asnnfurther increases, performance continues to improve monotonically with diminishing returns, as additional samples provide denser coverage of the epistemic space and enable more reliable epistemic calibration\. This consistent trend also suggests that EPC\-AW is robust to the choice ofnnand does not rely on delicate hyperparameter tuning\.

Overall, EPC\-AW exhibits a stable and monotonic performance trend with respect tonn, demonstrating that its gains stem from improved epistemic calibration rather than sensitive hyperparameter tuning\.

### 5\.6Generalization Across LLM Backbones

To evaluate whether the effectiveness of EPC\-AW depends on a specific backbone, we further conduct experiments on two additional LLMs: Qwen3\-14B and DeepSeek\-R1\-32B\. As shown in Table[4](https://arxiv.org/html/2605.23414#S5.T4), EPC\-AW consistently improves performance across all datasets and model architectures\. Compared with No\-Repair, EPC\-AW yields substantial gains on both backbones, with average improvements of 11\.18% on Qwen3\-14B and 11\.13% on DeepSeek\-R1\-32B\.

These results suggest that the effectiveness of EPC\-AW does not rely on backbone\-specific behaviors\. Instead, EPC\-AW consistently improves the epistemic calibration process during multi\-agent planning and execution, enabling agents to better identify unreliable plans, revise intermediate reasoning trajectories, and recover from latent planning failures\. Overall, the results demonstrate that EPC\-AW generalizes across heterogeneous LLM architectures\.

### 5\.7Time and Token Cost Analysis

We analyze the computational overhead of EPC\-AW from both time and token perspectives\. In agentic workflows, the dominant time cost mainly comes from external tool execution and LLM inference, while the dominant token cost comes from accumulated memory and retrieved evidence\.

##### Base Workflow\.

Suppose the system runs forTTinteraction rounds\. Letτllm\\tau\_\{\\text\{llm\}\}denote the average latency of one LLM call andτtool\\tau\_\{\\text\{tool\}\}denote the latency of external tool execution\. Each interaction round consists of one planning call, one execution step, and diagnosis\-related LLM feedback\. The overall time complexity is therefore𝒪​\(T⋅\(τllm\+τtool\)\)\\mathcal\{O\}\\big\(T\\cdot\(\\tau\_\{\\text\{llm\}\}\+\\tau\_\{\\text\{tool\}\}\)\\big\)\.

For token complexity, letmi\(t\)m\_\{i\}^\{\(t\)\}denote the memory token length of agenti∈\{p,e,d\}i\\in\\\{p,e,d\\\}at roundtt, wherepp,ee, andddcorrespond to the Planner, Executor, and Diagnoser, respectively\. Letqiq\_\{i\}denote the corresponding prompt token length andnpn\_\{p\}denote the average plan length\. The token complexity of the base workflow is

𝒪​\(∑t=1T\(∑i∈\{p,e,d\}mi\(t\)\+∑i∈\{p,e,d\}qi\+np\)\)\.\\mathcal\{O\}\\Bigg\(\\sum\_\{t=1\}^\{T\}\\Big\(\\sum\_\{i\\in\\\{p,e,d\\\}\}m\_\{i\}^\{\(t\)\}\+\\sum\_\{i\\in\\\{p,e,d\\\}\}q\_\{i\}\+n\_\{p\}\\Big\)\\Bigg\)\.\(25\)

##### EPC\-AW\.

The additional cost in EPC\-AW is from IPS and CESR\. IPS introduces two additional components: \(i\) generation ofKKcandidate plans \(single LLM call producing𝒪​\(K​np\)\\mathcal\{O\}\(Kn\_\{p\}\)output tokens\), and \(ii\) six evaluation calls \(self\- and cross\-agent\), each processing allKKplans\. Since these evaluation calls can be executed in parallel, the asymptotic time complexity remains𝒪​\(T⋅\(τllm\+τtool\)\)\\mathcal\{O\}\\big\(T\\cdot\(\\tau\_\{\\text\{llm\}\}\+\\tau\_\{\\text\{tool\}\}\)\\big\)\.The token complexity becomes

𝒪​\(∑t=1T\(∑i∈\{p,e,d\}mi\(t\)\+∑i∈\{p,e,d\}qi\+K​np\)\),\\mathcal\{O\}\\Bigg\(\\sum\_\{t=1\}^\{T\}\\Big\(\\sum\_\{i\\in\\\{p,e,d\\\}\}m\_\{i\}^\{\(t\)\}\+\\sum\_\{i\\in\\\{p,e,d\\\}\}q\_\{i\}\+Kn\_\{p\}\\Big\)\\Bigg\),\(26\)where the additionalK​npKn\_\{p\}term reflects multi\-plan reasoning and evaluation\. CESR does not introduce additional LLM calls and only adds a constant number of constraint tokens to the Diagnoser input\. Therefore, it does not change the asymptotic time or token complexity\.

Overall, EPC\-AW preserves asymptotic complexity while improving success rate \(\+9\.75% avg\.\), offering a favorable cost–performance trade\-off\. Notably, the additional cost primarily scales the accumulated memory and retrieved context, which already dominate token consumption in the base system, rather than introducing a new computational bottleneck\. Further details and statistics on tool usage and inference are provided in Appendix[E](https://arxiv.org/html/2605.23414#A5)\.

## 6Conclusion

We identify epistemic miscalibration during planning as a distinct and previously underexplored failure mode in LLM\-based multi\-agent systems\. Unlike execution errors, such miscalibration remains latent and dynamic during planning, as plans can appear executable while feasibility assessments evolve with newly acquired information\. By shifting mitigation from execution\-time correction to planning\-time calibration, our approach improves overall system reliability\. EPC\-AW achieves the best results on all benchmarks, yielding a 9\.75% absolute improvement on the system\-level accuracy\. These results underscore the importance of treating epistemic calibration as a first\-class, system\-level consideration in the design of robust multi\-agent workflows\. Looking forward, epistemic calibration provides a promising perspective for improving LLM\-based multi\-agent systems\.

## Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grant 62572346\. This work was also supported by the Joint Laboratory of AI for Smart Education, Gaotu\-RUC, and the Shanghai Key Laboratory of Data Science\.

## Impact Statement

This paper presents work whose goal is to improve the correctness and reliability of LLM\-based multi\-agent systems by mitigating epistemic miscalibration during planning\. By reducing unsupported feasibility assessments, the proposed approach improves task success in settings with incomplete and evolving information, and may contribute to more reliable deployment of LLM\-based systems in applications where planning quality directly affects outcomes\. The work advances system\-level planning reliability without introducing new capabilities or ethical concerns beyond those already well studied in large language model deployment\.

## References

- Y\. Abbasi Yadkori, I\. Kuzborskij, A\. György, and C\. Szepesvari \(2024\)To believe or not to believe your llm: iterative prompting for estimating epistemic uncertainty\.Advances in Neural Information Processing Systems37,pp\. 58077–58117\.Cited by:[§2\.2](https://arxiv.org/html/2605.23414#S2.SS2.p1.1)\.
- S\. Arif, S\. Farid, A\. H\. Azeemi, A\. Athar, and A\. A\. Raza \(2024\)The fellowship of the llms: multi\-agent workflows for synthetic preference optimization dataset generation\.arXiv preprint arXiv:2408\.08688\.Cited by:[Appendix C](https://arxiv.org/html/2605.23414#A3.SS0.SSS0.Px4.p1.1),[§5\.1\.3](https://arxiv.org/html/2605.23414#S5.SS1.SSS3.p3.1)\.
- M\. Becattini, R\. Verdecchia, and E\. Vicario \(2025\)SALLMA: a software architecture for llm\-based multi\-agent systems\.In2025 IEEE/ACM International Workshop New Trends in Software Architecture \(SATrends\),pp\. 5–8\.Cited by:[§1](https://arxiv.org/html/2605.23414#S1.p1.1),[§2\.1](https://arxiv.org/html/2605.23414#S2.SS1.p1.1)\.
- M\. Cemri, M\. Z\. Pan, S\. Yang, L\. A\. Agrawal, B\. Chopra, R\. Tiwari, K\. Keutzer, A\. Parameswaran, D\. Klein, K\. Ramchandran,et al\.\(2025\)Why do multi\-agent llm systems fail?\.arXiv preprint arXiv:2503\.13657\.Cited by:[§G\.1](https://arxiv.org/html/2605.23414#A7.SS1.p1.1),[§1](https://arxiv.org/html/2605.23414#S1.p1.1),[§1](https://arxiv.org/html/2605.23414#S1.p4.1)\.
- M\. Cemri, M\. Z\. Pan, S\. Yang, L\. A\. Agrawal, B\. Chopra, R\. Tiwari, K\. Keutzer, A\. Parameswaran, D\. Klein, K\. Ramchandran,et al\.\(2026\)Why do multi\-agent llm systems fail?\.Advances in Neural Information Processing Systems38\.Cited by:[§2\.1](https://arxiv.org/html/2605.23414#S2.SS1.p1.1)\.
- B\. Chen, T\. Zhu, J\. Han, L\. Li, G\. Li, and X\. Dai \(2025\)Incentivizing truthful language models via peer elicitation games\.arXiv preprint arXiv:2505\.13636\.Cited by:[§G\.2](https://arxiv.org/html/2605.23414#A7.SS2.p3.1),[§2\.2](https://arxiv.org/html/2605.23414#S2.SS2.p2.1)\.
- P\. Chhikara \(2025\)Mind the confidence gap: overconfidence, calibration, and distractor effects in large language models\.arXiv preprint arXiv:2502\.11028\.Cited by:[§2\.2](https://arxiv.org/html/2605.23414#S2.SS2.p1.1)\.
- Y\. Cho, S\. C\. Guntuku, and L\. Ungar \(2025\)Herd behavior: investigating peer influence in llm\-based multi\-agent systems\.arXiv preprint arXiv:2505\.21588\.Cited by:[§G\.2](https://arxiv.org/html/2605.23414#A7.SS2.p3.1)\.
- N\. Clark, H\. Shen, B\. Howe, and T\. Mitra \(2025\)Epistemic alignment: a mediating framework for user\-llm knowledge delivery\.arXiv preprint arXiv:2504\.01205\.Cited by:[§G\.2](https://arxiv.org/html/2605.23414#A7.SS2.p2.1),[§2\.2](https://arxiv.org/html/2605.23414#S2.SS2.p2.1)\.
- A\. Dobrovsky, K\. Schekotihin, and C\. Burmer \(2025\)Managing complex failure analysis workflows with llm\-based reasoning and acting agents\.arXiv e\-prints,pp\. arXiv–2506\.Cited by:[§2\.1](https://arxiv.org/html/2605.23414#S2.SS1.p1.1)\.
- W\. Epperson, G\. Bansal, V\. C\. Dibia, A\. Fourney, J\. Gerrits, E\. Zhu, and S\. Amershi \(2025\)Interactive debugging and steering of multi\-agent ai systems\.InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems,pp\. 1–15\.Cited by:[§G\.1](https://arxiv.org/html/2605.23414#A7.SS1.p1.1),[§G\.1](https://arxiv.org/html/2605.23414#A7.SS1.p2.1),[§1](https://arxiv.org/html/2605.23414#S1.p1.1),[§1](https://arxiv.org/html/2605.23414#S1.p2.1),[§1](https://arxiv.org/html/2605.23414#S1.p4.1),[§2\.1](https://arxiv.org/html/2605.23414#S2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2605.23414#S2.SS1.p2.1)\.
- W\. Gong, S\. Qiao, C\. Cao, S\. Tan, J\. Ye, H\. Liu, S\. Chen, and X\. Wang \(2025\)Harnessing collective intelligence of multi\-agent llm systems for sensor failure reasoning in smart manufacturing\.Journal of Industrial Information Integration,pp\. 101012\.Cited by:[§G\.1](https://arxiv.org/html/2605.23414#A7.SS1.p1.1)\.
- L\. Gu, Y\. Zhu, H\. Sang, Z\. Wang, D\. Sui, W\. Tang, E\. Harrison, J\. Gao, L\. Yu, and L\. Ma \(2025\)MedAgentAudit: diagnosing and quantifying collaborative failure modes in medical multi\-agent systems\.arXiv preprint arXiv:2510\.10185\.Cited by:[§G\.1](https://arxiv.org/html/2605.23414#A7.SS1.p1.1)\.
- A\. Gupta \(2025\)Verifiability\-first agents: provable observability and lightweight audit agents for controlling autonomous llm systems\.arXiv preprint arXiv:2512\.17259\.Cited by:[§G\.2](https://arxiv.org/html/2605.23414#A7.SS2.p2.1)\.
- N\. Hagar, W\. Agustianto, and N\. Diakopoulos \(2025\)Not wrong, but untrue: llm overconfidence in document\-based queries\.arXiv preprint arXiv:2509\.25498\.Cited by:[§G\.2](https://arxiv.org/html/2605.23414#A7.SS2.p1.1)\.
- L\. Hammond, A\. Chan, J\. Clifton, J\. Hoelscher\-Obermaier, A\. Khan, E\. McLean, C\. Smith, W\. Barfuss, J\. Foerster, T\. Gavenčiak,et al\.\(2025\)Multi\-agent risks from advanced ai\.arXiv preprint arXiv:2502\.14143\.Cited by:[§G\.1](https://arxiv.org/html/2605.23414#A7.SS1.p1.1),[§1](https://arxiv.org/html/2605.23414#S1.p1.1),[§2\.1](https://arxiv.org/html/2605.23414#S2.SS1.p1.1)\.
- J\. He, C\. Treude, and D\. Lo \(2025\)LLM\-based multi\-agent systems for software engineering: literature review, vision, and the road ahead\.ACM Transactions on Software Engineering and Methodology34\(5\),pp\. 1–30\.Cited by:[§1](https://arxiv.org/html/2605.23414#S1.p1.1)\.
- X\. Ho, A\. D\. Nguyen, S\. Sugawara, and A\. Aizawa \(2020\)Constructing a multi\-hop qa dataset for comprehensive evaluation of reasoning steps\.InProceedings of the 28th International Conference on Computational Linguistics,pp\. 6609–6625\.Cited by:[Appendix A](https://arxiv.org/html/2605.23414#A1.p3.1),[§5\.1\.1](https://arxiv.org/html/2605.23414#S5.SS1.SSS1.p1.1)\.
- M\. Kim, S\. Kim, and J\. Thorne \(2025\)From evidence to belief: a bayesian epistemology approach to language models\.arXiv preprint arXiv:2504\.19622\.Cited by:[§G\.2](https://arxiv.org/html/2605.23414#A7.SS2.p3.1)\.
- D\. Lee, Y\. Hwang, Y\. Kim, J\. Park, and K\. Jung \(2025\)Are llm\-judges robust to expressions of uncertainty? investigating the effect of epistemic markers on llm\-based evaluation\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 8962–8984\.Cited by:[§G\.2](https://arxiv.org/html/2605.23414#A7.SS2.p1.1),[§2\.2](https://arxiv.org/html/2605.23414#S2.SS2.p1.1)\.
- X\. Li, S\. Wang, S\. Zeng, Y\. Wu, and Y\. Yang \(2024\)A survey on llm\-based multi\-agent systems: workflow, infrastructure, and challenges\.Vicinagearth1\(1\),pp\. 9\.Cited by:[§2\.1](https://arxiv.org/html/2605.23414#S2.SS1.p1.1)\.
- Z\. Li, H\. Zhang, S\. Han, S\. Liu, J\. Xie, Y\. Zhang, Y\. Choi, J\. Zou, and P\. Lu \(2026\)In\-the\-flow agentic system optimization for effective planning and tool use\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[Appendix A](https://arxiv.org/html/2605.23414#A1.p9.1),[Appendix B](https://arxiv.org/html/2605.23414#A2.p1.1),[Appendix C](https://arxiv.org/html/2605.23414#A3.SS0.SSS0.Px3.p1.1),[Appendix C](https://arxiv.org/html/2605.23414#A3.SS0.SSS0.Px4.p1.1),[§3\.1](https://arxiv.org/html/2605.23414#S3.SS1.p1.2),[§5\.1\.2](https://arxiv.org/html/2605.23414#S5.SS1.SSS2.p1.1),[§5\.1\.3](https://arxiv.org/html/2605.23414#S5.SS1.SSS3.p2.1),[§5\.1\.3](https://arxiv.org/html/2605.23414#S5.SS1.SSS3.p3.1)\.
- C\. Liang, J\. Gan, K\. Hong, Q\. Tian, Z\. Wu, and R\. Li \(2025\)COCO: cognitive operating system with continuous oversight for multi\-agent workflow reliability\.arXiv preprint arXiv:2508\.13815\.Cited by:[Appendix B](https://arxiv.org/html/2605.23414#A2.SS0.SSS0.Px3.p1.1),[Appendix B](https://arxiv.org/html/2605.23414#A2.p1.1),[§G\.1](https://arxiv.org/html/2605.23414#A7.SS1.p1.1),[§G\.1](https://arxiv.org/html/2605.23414#A7.SS1.p3.1),[§1](https://arxiv.org/html/2605.23414#S1.p1.1),[§1](https://arxiv.org/html/2605.23414#S1.p2.1),[§2\.1](https://arxiv.org/html/2605.23414#S2.SS1.p2.1),[§5\.1\.2](https://arxiv.org/html/2605.23414#S5.SS1.SSS2.p1.1)\.
- Y\. Lin, K\. Chen, Z\. Li, T\. Wu, T\. Wu, K\. Chen, H\. Lee, and Y\. Chen \(2025\)Creativity in llm\-based multi\-agent systems: a survey\.arXiv preprint arXiv:2505\.21116\.Cited by:[§G\.1](https://arxiv.org/html/2605.23414#A7.SS1.p1.1)\.
- X\. Liu, T\. Chen, L\. Da, C\. Chen, Z\. Lin, and H\. Wei \(2025\)Uncertainty quantification and confidence calibration in large language models: a survey\.InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V\. 2,pp\. 6107–6117\.Cited by:[§G\.2](https://arxiv.org/html/2605.23414#A7.SS2.p1.1)\.
- R\. Lu, Y\. Li, and Y\. Huo \(2025\)Exploring autonomous agents: a closer look at why they fail when completing tasks\.arXiv preprint arXiv:2508\.13143\.Cited by:[§1](https://arxiv.org/html/2605.23414#S1.p2.1)\.
- M\. Ma, J\. Zhang, F\. Yang, Y\. Kang, Q\. Lin, S\. Rajmohan, and D\. Zhang \(2025\)DoVer: intervention\-driven auto debugging for llm multi\-agent systems\.arXiv preprint arXiv:2512\.06749\.Cited by:[§G\.1](https://arxiv.org/html/2605.23414#A7.SS1.p1.1),[§G\.1](https://arxiv.org/html/2605.23414#A7.SS1.p2.1),[§1](https://arxiv.org/html/2605.23414#S1.p2.1),[§1](https://arxiv.org/html/2605.23414#S1.p4.1),[§2\.1](https://arxiv.org/html/2605.23414#S2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2605.23414#S2.SS1.p2.1)\.
- G\. Mialon, C\. Fourrier, T\. Wolf, Y\. LeCun, and T\. Scialom \(2023\)Gaia: a benchmark for general ai assistants\.InThe Twelfth International Conference on Learning Representations,Cited by:[Appendix A](https://arxiv.org/html/2605.23414#A1.p6.1),[§5\.1\.1](https://arxiv.org/html/2605.23414#S5.SS1.SSS1.p1.1)\.
- T\. D\. Nguyen\-Hien, D\. R\. Ivanova, Y\. W\. Teh, and W\. S\. Lee \(2025\)Extending epistemic uncertainty beyond parameters would assist in designing reliable llms\.arXiv preprint arXiv:2506\.07448\.Cited by:[§G\.2](https://arxiv.org/html/2605.23414#A7.SS2.p1.1)\.
- P\. Pitre, N\. Ramakrishnan, and X\. Wang \(2025\)CONSENSAGENT: towards efficient and effective consensus in multi\-agent llm interactions through sycophancy mitigation\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 22112–22133\.Cited by:[§G\.2](https://arxiv.org/html/2605.23414#A7.SS2.p2.1),[§2\.2](https://arxiv.org/html/2605.23414#S2.SS2.p2.1)\.
- O\. Press, M\. Zhang, S\. Min, L\. Schmidt, N\. A\. Smith, and M\. Lewis \(2023\)Measuring and narrowing the compositionality gap in language models\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 5687–5711\.Cited by:[Appendix A](https://arxiv.org/html/2605.23414#A1.p2.1),[§5\.1\.1](https://arxiv.org/html/2605.23414#S5.SS1.SSS1.p1.1)\.
- A\. Reid, S\. O’Callaghan, L\. Carroll, and T\. Caetano \(2025\)Risk analysis techniques for governed llm\-based multi\-agent systems\.arXiv preprint arXiv:2508\.05687\.Cited by:[§1](https://arxiv.org/html/2605.23414#S1.p2.1)\.
- K\. Ronanki \(2025\)Facilitating trustworthy human\-agent collaboration in llm\-based multi\-agent system oriented software engineering\.InProceedings of the 33rd ACM International Conference on the Foundations of Software Engineering,pp\. 1333–1337\.Cited by:[§G\.1](https://arxiv.org/html/2605.23414#A7.SS1.p1.1),[§1](https://arxiv.org/html/2605.23414#S1.p1.1),[§2\.1](https://arxiv.org/html/2605.23414#S2.SS1.p1.1)\.
- X\. Shen, Q\. Zhang, S\. Wang, Z\. Tan, X\. Zhao, L\. Yao, V\. Tadiparthi, H\. N\. Mahjoub, E\. M\. Pari, K\. Lee,et al\.\(2025\)Metacognitive self\-correction for multi\-agent system via prototype\-guided next\-execution reconstruction\.arXiv preprint arXiv:2510\.14319\.Cited by:[Appendix B](https://arxiv.org/html/2605.23414#A2.SS0.SSS0.Px2.p1.1),[Appendix B](https://arxiv.org/html/2605.23414#A2.p1.1),[§G\.1](https://arxiv.org/html/2605.23414#A7.SS1.p1.1),[§G\.1](https://arxiv.org/html/2605.23414#A7.SS1.p3.1),[§1](https://arxiv.org/html/2605.23414#S1.p2.1),[§2\.1](https://arxiv.org/html/2605.23414#S2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2605.23414#S2.SS1.p2.1),[§5\.1\.2](https://arxiv.org/html/2605.23414#S5.SS1.SSS2.p1.1)\.
- C\. Sun, S\. Huang, and D\. Pompili \(2025\)LLM\-based multi\-agent decision\-making: challenges and future directions\.IEEE Robotics and Automation Letters\.Cited by:[§1](https://arxiv.org/html/2605.23414#S1.p1.1)\.
- Y\. Y\. Sung, H\. Kim, and D\. Zhang \(2025\)Verila: a human\-centered evaluation framework for interpretable verification of llm agent failures\.arXiv preprint arXiv:2503\.12651\.Cited by:[§G\.2](https://arxiv.org/html/2605.23414#A7.SS2.p2.1)\.
- Y\. Tripathi, V\. Kohli, and S\. Misra \(2025\)Confident but incorrect: mitigating hallucination and overconfidence in agentic ai coders\.In2025 Eighth International Conference on Image Information Processing \(ICIIP\),pp\. 723–728\.Cited by:[§G\.2](https://arxiv.org/html/2605.23414#A7.SS2.p1.1)\.
- H\. Trivedi, N\. Balasubramanian, T\. Khot, and A\. Sabharwal \(2022\)MuSiQue: multihop questions via single\-hop question composition\.Transactions of the Association for Computational Linguistics10,pp\. 539–554\.Cited by:[Appendix A](https://arxiv.org/html/2605.23414#A1.p5.1),[§5\.1\.1](https://arxiv.org/html/2605.23414#S5.SS1.SSS1.p1.1)\.
- G\. Wan, L\. Sun, L\. Dou, Z\. Shi, F\. Wu, E\. H\. Jiang, W\. Huang, G\. Zhang, H\. Geng, X\. Tang,et al\.\(2025\)Diagnose, localize, align: a full\-stack framework for reliable llm multi\-agent systems under instruction conflicts\.arXiv preprint arXiv:2509\.23188\.Cited by:[§G\.1](https://arxiv.org/html/2605.23414#A7.SS1.p1.1)\.
- R\. Wang, Q\. Lin, J\. Liu, Q\. Zong, T\. Zheng, W\. Wang, and Y\. Song \(2025\)Prospect theory fails for llms: revealing instability of decision\-making under epistemic uncertainty\.arXiv preprint arXiv:2508\.08992\.Cited by:[§G\.2](https://arxiv.org/html/2605.23414#A7.SS2.p1.1)\.
- R\. Weaver and D\. Prelec \(2013\)Creating truth\-telling incentives with the bayesian truth serum\.Journal of Marketing Research50\(3\),pp\. 289–302\.Cited by:[§G\.2](https://arxiv.org/html/2605.23414#A7.SS2.p3.1)\.
- B\. Wen, C\. Xu, R\. Wolfe, L\. L\. Wang, B\. Howe,et al\.\(2024\)Mitigating overconfidence in large language models: a behavioral lens on confidence estimation and calibration\.InNeurIPS 2024 Workshop on Behavioral Machine Learning,Cited by:[§G\.2](https://arxiv.org/html/2605.23414#A7.SS2.p1.1),[§G\.2](https://arxiv.org/html/2605.23414#A7.SS2.p2.1),[§2\.2](https://arxiv.org/html/2605.23414#S2.SS2.p2.1)\.
- C\. Wen, J\. Liu, and L\. Du \(2026\)Consensus planning boosts llm code generation\.Information and Software Technology,pp\. 108030\.Cited by:[§G\.2](https://arxiv.org/html/2605.23414#A7.SS2.p2.1)\.
- J\. Witkowski and D\. Parkes \(2012\)A robust bayesian truth serum for small populations\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.26,pp\. 1492–1498\.Cited by:[§G\.2](https://arxiv.org/html/2605.23414#A7.SS2.p3.1),[§2\.2](https://arxiv.org/html/2605.23414#S2.SS2.p2.1)\.
- Q\. Wu, G\. Bansal, J\. Zhang, Y\. Wu, B\. Li, E\. Zhu, L\. Jiang, X\. Zhang, S\. Zhang, J\. Liu,et al\.\(2024\)Autogen: enabling next\-gen llm applications via multi\-agent conversations\.InFirst Conference on Language Modeling,Cited by:[§3\.1](https://arxiv.org/html/2605.23414#S3.SS1.p1.2)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[Appendix C](https://arxiv.org/html/2605.23414#A3.SS0.SSS0.Px1.p1.1),[§5\.1\.3](https://arxiv.org/html/2605.23414#S5.SS1.SSS3.p1.4)\.
- H\. Yang, H\. Chen, H\. Guo, Y\. Chen, C\. Lin, S\. Hu, J\. Hu, X\. Wu, and X\. Wang \(2024\)Llm\-medqa: enhancing medical question answering through case studies in large language models\.arXiv preprint arXiv:2501\.05464\.Cited by:[Appendix A](https://arxiv.org/html/2605.23414#A1.p7.1),[§5\.1\.1](https://arxiv.org/html/2605.23414#S5.SS1.SSS1.p1.1)\.
- Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. Cohen, R\. Salakhutdinov, and C\. D\. Manning \(2018\)HotpotQA: a dataset for diverse, explainable multi\-hop question answering\.InProceedings of the 2018 conference on empirical methods in natural language processing,pp\. 2369–2380\.Cited by:[Appendix A](https://arxiv.org/html/2605.23414#A1.p4.1),[§5\.1\.1](https://arxiv.org/html/2605.23414#S5.SS1.SSS1.p1.1)\.
- G\. Zhang, J\. Wang, J\. Chen, W\. Zhou, K\. Wang, and S\. Yan \(2025a\)AgenTracer: who is inducing failure in the llm agentic systems?\.External Links:2509\.03312,[Link](https://arxiv.org/abs/2509.03312)Cited by:[§G\.1](https://arxiv.org/html/2605.23414#A7.SS1.p1.1),[§2\.1](https://arxiv.org/html/2605.23414#S2.SS1.p1.1)\.
- L\. Zhang, Y\. Zhai, T\. Jia, X\. Huang, C\. Duan, and Y\. Li \(2025b\)Agentfm: role\-aware failure management for distributed databases with llm\-driven multi\-agents\.InProceedings of the 33rd ACM International Conference on the Foundations of Software Engineering,pp\. 525–529\.Cited by:[§1](https://arxiv.org/html/2605.23414#S1.p2.1)\.
- S\. Zhang, M\. Yin, J\. Zhang, J\. Liu, Z\. Han, J\. Zhang, B\. Li, C\. Wang, H\. Wang, Y\. Chen,et al\.\(2025c\)Which agent causes task failures and when? on automated failure attribution of llm multi\-agent systems\.arXiv preprint arXiv:2505\.00212\.Cited by:[§1](https://arxiv.org/html/2605.23414#S1.p4.1)\.
- C\. Zhou, Y\. Tang, K\. Chen, X\. Bai, S\. Qi, L\. Shen, and M\. Zhang \(2026\)From fragmentation to systematic design: architecting llm\-based multi\-agent systems\.Authorea Preprints\.Cited by:[§G\.1](https://arxiv.org/html/2605.23414#A7.SS1.p1.1)\.

## Appendix ADataset Details

We evaluate our approach on a diverse set of benchmarks that span a broad spectrum of dependence on internal inference versus external information retrieval\.

Bamboogle\(Presset al\.,[2023](https://arxiv.org/html/2605.23414#bib.bib47)\)is a manually constructed multi\-step reasoning dataset designed to probe compositional inference over interconnected facts\. Each question typically requires up to four explicit inferential steps\.

2Wiki\(Hoet al\.,[2020](https://arxiv.org/html/2605.23414#bib.bib46)\)combines structured knowledge from Wikidata with unstructured evidence from Wikipedia, requiring models to retrieve and integrate information across heterogeneous sources\. While annotated reasoning chains facilitate interpretable multi\-hop inference, successful reasoning is contingent on accurate intermediate retrieval, making the dataset jointly dependent on search quality and sequential reasoning\.

HotpotQA\(Yanget al\.,[2018](https://arxiv.org/html/2605.23414#bib.bib45)\)is a widely used multi\-hop question answering benchmark constructed from Wikipedia articles\. Questions require integrating information from multiple documents within a relatively homogeneous textual space\.

Musique\(Trivediet al\.,[2022](https://arxiv.org/html/2605.23414#bib.bib44)\)is a challenging multi\-step reasoning dataset characterized by strong sequential dependencies, where each inference stage relies critically on conclusions derived from prior steps\.

GAIA\(Mialonet al\.,[2023](https://arxiv.org/html/2605.23414#bib.bib137)\)is a benchmark designed to assess general AI agents in open\-world settings, requiring capabilities such as multi\-step reasoning, web navigation, and tool use\. Unlike conventional QA datasets, GAIA emphasizes planning and information acquisition under partial observability, with reasoning tightly coupled to search and tool utilization\.

MedQA\(Yanget al\.,[2024](https://arxiv.org/html/2605.23414#bib.bib40)\)consists of multiple\-choice questions derived from professional medical licensing examinations\. The dataset evaluates clinical reasoning grounded in domain knowledge, requiring models to infer diagnoses or treatments through structured medical reasoning\.

Overall, these datasets collectively cover a continuum from reasoning\-dominant inference to search\-driven open\-world problem solving\. This diversity allows us to systematically examine how epistemic miscalibration manifests across different planning and reasoning conditions in LLM\-based multi\-agent systems\.

We follow AgentFlow\(Liet al\.,[2026](https://arxiv.org/html/2605.23414#bib.bib1)\)and directly adopt its evaluation subsets for all datasets to ensure comparability with prior agent\-based reasoning systems\. Details are shown in Table\.[5](https://arxiv.org/html/2605.23414#A1.T5)\.

Table 5:Evaluation subsets adopted from AgentFlow\. Each dataset occupies a single row, and all samples are uniformly drawn from the official splits and fixed across all methods\.
## Appendix BBaseline Details

This section provides detailed descriptions of the baselines used in our experiments\. Notably, neither MAST\(Shenet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib113)\)nor COCO\(Lianget al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib43)\)has released official implementations at the time of writing\. Accordingly, all baselines are our re\-implementations inspired by the methodological descriptions in the original papers, instantiated within the AgentFlow framework\(Liet al\.,[2026](https://arxiv.org/html/2605.23414#bib.bib1)\)\. All methods share the same agent architecture, system memory design, and execution protocol, ensuring a controlled and fair comparison\.

Across all baselines, agents maintain access to a shared system memory that records the complete execution history, including generated plans \(goals and actions\) and corresponding execution results\. For baselines involving epistemic assessment, diagnosis is performed exclusively during the planning phase\.

##### No\-Repair \(AgentFlow\)\.

The No\-Repair baseline directly follows the original AgentFlow design\. The agent generates a complete plan in a single forward pass and executes it without any explicit feedback, diagnosis, or recovery mechanism\. This baseline serves as a reference for unmitigated epistemic miscalibration\.

##### Retry \(Planning\-Level Metacognitive Re\-Generation\)\.

The Retry baseline adapts metacognitive retry strategies originally proposed in MAST\(Shenet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib113)\)to the planning stage\. At each planning step, the agent evaluates whether the newly generated plan is epistemically miscalibrated based on the execution history up to the current step\. If miscalibration is detected, the agent re\-generates the current planning step while preserving all prior history\. The diagnostic analysis is explicitly provided as feedback to guide the re\-generation\. Importantly, Retry does not modify the global system state or historical memory; it only replaces the current planning output\.

##### Rollback \(System\-Level State Recovery\)\.

The Rollback baseline is inspired by the contextual rollback mechanism in COCO\(Lianget al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib43)\)\. When epistemic miscalibration is detected at the current planning step, the agent first determines an appropriate historical step to revert to\. The entire system state, including shared memory and recorded execution history, is rolled back to the selected step\. Planning is then re\-executed from that point onward, with the diagnostic analysis provided as feedback to guide re\-planning\. Compared to Retry, Rollback performs a global state recovery rather than a local re\-generation, allowing the system to correct errors that may have propagated across multiple planning steps\.

Notably, Retry and Rollback differ fundamentally in their scope of intervention\. Retry performs a local correction at the current planning step without altering historical state, while Rollback restores the system to a previous state and re\-optimizes subsequent planning decisions\. Both baselines operate under identical detection criteria and planning\-phase constraints, enabling controlled evaluation of their effectiveness in mitigating epistemic miscalibration\.

## Appendix CImplementation Details

This section provides additional implementation details to support reproducibility and clarify key design choices\.

##### Backbone Model\.

All agents in EPC\-AW, including the Planner, Executor, and Diagnoser, are instantiated using Qwen3\-Coder\-30B\(Yanget al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib26)\)\. The model is deployed on a server equipped with four NVIDIA RTX 4090 GPUs and served viavLLMfor efficient inference\. This backbone is selected to accommodate the long\-context requirements of multi\-step agentic planning and execution, where intermediate plans, tool interactions, and execution traces must be preserved across extended reasoning horizons\. To ensure a fair comparison, the same backbone model and deployment configuration are used consistently across EPC\-AW and all baseline methods\.

##### Planning and Generation Settings\.

During the planning phase, the Planner generates a fixed number ofn=9n=9candidate next\-step plans\. To encourage exploration of alternative planning hypotheses, candidate generation is performed with temperature0\.90\.9\. All subsequent generations, including execution, diagnosis, and tool\-based reasoning, are performed with temperature0to ensure deterministic behavior and reduce variance across runs\. The maximum number of iterations is set to1010for all experiments\.

In IPS, multi\-agent plan evaluation and belief prediction are performed independently and in parallel\. Candidate plans are evaluated using a predefined feasibility metric, where agents assign integer scores from 1 to 5 reflecting their epistemic assessment of plan feasibility\.

Feasibility MetricScore 5 — Exceptional FeasibilityThe plan is internally coherent, precise, and strongly justified\.•Tool selection and parameters are fully specified and sufficient*in principle*to support the stated sub\-goal under the given context\.•Reasoning is complete, logically tight, and optimally grounded in the available information\.•No implicit assumptions or missing steps are required to interpret the plan\.Score 4 — Near\-Perfect FeasibilityThe plan is coherent and well\-aligned with the stated sub\-goal\.•Tool selection is correct; parameters are appropriate but may allow minor refinement\.•Reasoning is sound, though some details could be made more explicit\.•The plan is interpretable without major inference\.Score 3 — Strong FeasibilityThe plan is plausible and directly addresses the sub\-goal\.•Tool selection is mostly correct; some parameters or steps require mild inference\.•Reasoning is generally sound but partially underspecified\.•The plan remains interpretable, though not maximally precise\.Score 2 — Mostly FeasibleThe plan is relevant but exhibits notable epistemic gaps\.•Tool selection is reasonable, but parameters are under\-specified or ambiguous\.•Reasoning relies on implicit assumptions or missing details\.•Additional clarification would be required for confident interpretation\.Score 1 — Weak FeasibilityThe plan shows limited coherence or weak alignment with the sub\-goal\.•Tool selection or parameter specification is incomplete or mismatched\.•Reasoning is vague, fragmented, or poorly grounded in the given context\.•The intended effect of the plan is epistemically unclear\.

##### Tool Configuration\.

Following AgentFlow\(Liet al\.,[2026](https://arxiv.org/html/2605.23414#bib.bib1)\), the system interacts with five tools: 1\) a base generator for default reasoning, 2\) a Python coder that generates and executes Python code and returns execution results, 3\) Google Search for retrieving relevant web pages, 4\) Wikipedia Search for structured knowledge retrieval, and 5\) Web Search for page\-level information extraction and summarization\.

Compared to the original AgentFlow implementation, we replace direct Google Search API calls with a lightweight local retrieval pipeline that collects the top\-10 relevant web pages via a local Chrome\-based Google Search interface\. This modification avoids potential interference from backend LLMs \(e\.g\., Gemini\-based models\) implicitly involved in official API responses, ensuring that information retrieval does not benefit from stronger external language models\.

To further mitigate information leakage due to publicly available datasets, we apply keyword\-based filtering during the search stage to exclude HuggingFace repositories and dataset\-specific pages\. Retrieved pages are then processed by the Web Search tool, followed by LLM\-based summarization\.

##### Evaluation Protocol\.

For evaluation, GPT\-4o is employed as an automatic judge to determine whether model predictions match the corresponding ground\-truth answers, following standard practice in tool\-augmented reasoning benchmarks\(Liet al\.,[2026](https://arxiv.org/html/2605.23414#bib.bib1); Arifet al\.,[2024](https://arxiv.org/html/2605.23414#bib.bib20)\)\. To reduce the impact of stochasticity, all experiments are repeated three times, and we report the average accuracy across runs\. All evaluation settings are aligned with those used in AgentFlow to facilitate direct comparison\.

## Appendix DEPC\-AW Pseudocode and Explanation

Algorithm 1EPC\-AW: Epistemic Planning Calibration Agentic WorkflowInput:query

QQ, maximum rounds

TT
Output:final answer

YY
Initialize system memory

ℳsys←⟨Q,∅,𝒰⟩\\mathcal\{M\}\_\{\\mathrm\{sys\}\}\\leftarrow\\langle Q,\\emptyset,\\mathcal\{U\}\\rangle
Initialize agent memories

ℳP←∅\\mathcal\{M\}\_\{P\}\\leftarrow\\emptyset,

ℳE←∅\\mathcal\{M\}\_\{E\}\\leftarrow\\emptyset,

ℳD←∅\\mathcal\{M\}\_\{D\}\\leftarrow\\emptyset
Initialize evidence set

Ψ←∅\\Psi\\leftarrow\\emptyset
for

t=1t=1to

TTdo

Generate candidate plans

Π←AP​\(ℳsys,γP\)\\Pi\\leftarrow A\_\{P\}\(\\mathcal\{M\}\_\{\\mathrm\{sys\}\},\\gamma\_\{P\}\)
Select plan

π∗←IPS​\(Π,ℳsys,\{γi\}\)\\pi^\{\\ast\}\\leftarrow\\textsc\{IPS\}\(\\Pi,\\mathcal\{M\}\_\{\\mathrm\{sys\}\},\\\{\\gamma\_\{i\}\\\}\)
Execute plan and observe outcome

o←AE​\(π∗\)o\\leftarrow A\_\{E\}\(\\pi^\{\\ast\}\)
Diagnose execution

d←AD​\(π∗,o\)d\\leftarrow A\_\{D\}\(\\pi^\{\\ast\},o\)
if

𝟏fail​\(d\)=0\\mathbf\{1\}\_\{\\mathrm\{fail\}\}\(d\)=0then

Extract evidence

ψ←ℱ​\(d\)\\psi\\leftarrow\\mathcal\{F\}\(d\)
Ψ←Ψ∪\{ψ\}\\Psi\\leftarrow\\Psi\\cup\\\{\\psi\\\}

endif

Compute planner\-selected plan

πP←arg⁡maxπk∈Π⁡eP​\(k\)\\pi\_\{P\}\\leftarrow\\arg\\max\_\{\\pi\_\{k\}\\in\\Pi\}e\_\{P\}\(k\)
Update planner memory

ℳP←CESR​\(πP,π∗,ℳsys,ℳP\)\\mathcal\{M\}\_\{P\}\\leftarrow\\textsc\{CESR\}\(\\pi\_\{P\},\\pi^\{\\ast\},\\mathcal\{M\}\_\{\\mathrm\{sys\}\},\\mathcal\{M\}\_\{P\}\)
if

𝟏stop​\(Q,Ψ\)=1\\mathbf\{1\}\_\{\\mathrm\{stop\}\}\(Q,\\Psi\)=1then

return

𝒢​\(Q,Ψ,ℋ\)\\mathcal\{G\}\(Q,\\Psi,\\mathcal\{H\}\)
endif

endfor

return

𝒢​\(Q,Ψ,ℋ\)\\mathcal\{G\}\(Q,\\Psi,\\mathcal\{H\}\)

The EPC\-AW algorithm iteratively performs planning, execution, and diagnosis in a multi\-agent workflow to improve epistemic calibration\. The workflow can be described as follows:

- •Initialization:System memoryℳsys\\mathcal\{M\}\_\{\\mathrm\{sys\}\}stores the queryQQ, execution history, and the abstract description of the role for all agents𝒰\\mathcal\{U\}\. Each agent \(PlannerPP, ExecutorEE, DiagnoserDD\) maintains a private memoryℳi\\mathcal\{M\}\_\{i\}\. The evidence setΨ\\Psiis initialized empty\.
- •Candidate Plan Generation:The Planner generates a set of candidate plansΠ\\Pibased on the system memory and its policyγP\\gamma\_\{P\}\.
- •Plan Selection:IPS chooses the most epistemically promising planπ∗\\pi^\{\\ast\}fromΠ\\Pi, taking into account cross\-agent alignment\.
- •Plan Execution and Diagnosis:The Executor executesπ∗\\pi^\{\\ast\}and observes outcomeoo\. The Diagnoser evaluates the execution outcomeoowhether it satisfies the planπ∗\\pi^\{\\ast\}\.
- •Evidence Extraction:If the execution satisfies the planπ∗\\pi^\{\\ast\}\(𝟏fail​\(d\)=0\\mathbf\{1\}\_\{\\mathrm\{fail\}\}\(d\)=0\), evidenceψ\\psiis extracted and added toΨ\\Psi\.
- •Planner Memory Update:The Planner identifies its selected planπP\\pi\_\{P\}and updates its memoryℳP\\mathcal\{M\}\_\{P\}via CESR, aligning plan selection with accumulated feedback\.
- •Termination Condition:The algorithm stops if the accumulated evidenceΨ\\Psisuffices to answer queryQQ\(𝟏stop​\(Q,Ψ\)=1\\mathbf\{1\}\_\{\\mathrm\{stop\}\}\(Q,\\Psi\)=1\), returning the final answer𝒢​\(Q,Ψ,ℋ\)\\mathcal\{G\}\(Q,\\Psi,\\mathcal\{H\}\)\.
- •Iteration:If termination criteria are not met, the loop continues until maximum roundsTTare reached\.

This workflow ensures that the multi\-agent system iteratively refines plans and updates epistemic states, leading to more reliable final answers under uncertainty\.

## Appendix ETime and Token Cost Analysis

In this section, we provide a detailed analysis of the computational overhead introduced by EPC\-AW\. From the system perspective, the overall time cost is mainly dominated by external tool execution and LLM inference latency, while the token cost is primarily determined by accumulated memory, retrieved evidence, and long\-context reasoning\. We therefore separately analyze how IPS and CESR affect these two aspects\.

### E\.1Base Workflow Complexity

Suppose the system runs forTTinteraction rounds\. At each round, the Planner generates a plan, the Executor invokes external tools, and the Diagnoser performs failure analysis and feedback generation\. Letmi\(t\)m\_\{i\}^\{\(t\)\}denote the memory token length of agenti∈\{p,e,d\}i\\in\\\{p,e,d\\\}at roundtt, wherepp,ee, andddcorrespond to the Planner, Executor, and Diagnoser, respectively\. Letqiq\_\{i\}denote the fixed prompt token length associated with each agent, and letnpn\_\{p\}denote the average token length of a generated plan\. We further denote byτllm\\tau\_\{\\text\{llm\}\}the average latency of a single LLM call and byτtool\\tau\_\{\\text\{tool\}\}the latency of external tool execution\.

Under this formulation, the dominant runtime cost at each interaction round comes from LLM inference and external tool usage\. Therefore, the overall time complexity can be written as𝒪​\(T​\(τllm\+τtool\)\)\\mathcal\{O\}\\big\(T\(\\tau\_\{\\text\{llm\}\}\+\\tau\_\{\\text\{tool\}\}\)\\big\)\. The corresponding token complexity is

𝒪​\(∑t=1T\(∑i∈\{p,e,d\}mi\(t\)\+∑i∈\{p,e,d\}qi\+np\)\),\\mathcal\{O\}\\Bigg\(\\sum\_\{t=1\}^\{T\}\\Big\(\\sum\_\{i\\in\\\{p,e,d\\\}\}m\_\{i\}^\{\(t\)\}\+\\sum\_\{i\\in\\\{p,e,d\\\}\}q\_\{i\}\+n\_\{p\}\\Big\)\\Bigg\),\(27\)
where the first term corresponds to accumulated memory and retrieved evidence, the second term represents fixed prompt instructions, and the final term corresponds to the generated planning content\. In practice, the dominant token source mainly comes from growing memory and retrieved external evidence rather than the plan itself\.

### E\.2EPC\-AW Complexity

##### IPS Overhead\.

IPS introduces two additional operations: multi\-plan generation and epistemic evaluation\. Specifically, the Planner generatesKKcandidate plans, producing approximately𝒪​\(K​np\)\\mathcal\{O\}\(Kn\_\{p\}\)output tokens, followed by six self\-/cross\-agent evaluation calls that jointly assess feasibility consistency across agents\.

Although IPS increases the number of LLM calls, these evaluation calls are mutually independent and can therefore be executed fully in parallel\. As a result, the asymptotic time complexity remains𝒪​\(T​\(τllm\+τtool\)\)\\mathcal\{O\}\\big\(T\(\\tau\_\{\\text\{llm\}\}\+\\tau\_\{\\text\{tool\}\}\)\\big\), with only a constant\-factor increase in practical latency\.

The token complexity becomes

𝒪​\(∑t=1T\(∑i∈\{p,e,d\}mi\(t\)\+∑i∈\{p,e,d\}qi\+K​np\)\)\.\\mathcal\{O\}\\Bigg\(\\sum\_\{t=1\}^\{T\}\\Big\(\\sum\_\{i\\in\\\{p,e,d\\\}\}m\_\{i\}^\{\(t\)\}\+\\sum\_\{i\\in\\\{p,e,d\\\}\}q\_\{i\}\+Kn\_\{p\}\\Big\)\\Bigg\)\.\(28\)
Compared with the base workflow, the only additional term is the multiplicative factorKKapplied to the planning output\. SinceKKis treated as a small constant in practice, IPS does not change the asymptotic scaling behavior of the system\.

Importantly, IPS mainly amplifies an already dominant token source rather than introducing a fundamentally new cost component\. In realistic multi\-hop workflows, retrieved evidence already contributes substantial context length \(approximately800800–10001000tokens per step\), while accumulated memory continuously grows across interaction rounds\. Therefore, the additional overhead introduced by IPS primarily scales existing context processing\.

##### CESR Overhead\.

Compared with IPS, CESR introduces negligible computational overhead\. CESR does not require additional planning branches or extra LLM calls, and only appends a constant number of consistency\-related constraint tokens to the Diagnoser input\. Therefore, CESR does not change either the asymptotic time complexity or the asymptotic token complexity of the workflow\.

### E\.3System\-Level Runtime Analysis

From a practical systems perspective, overall latency is mostly dominated by external tool usage\. Table[6](https://arxiv.org/html/2605.23414#A5.T6)summarizes the average latency of different workflow components in our implementation\.

Among all operations, Google Search incurs the highest latency, requiring approximately8080s on average due to multi\-hop webpage retrieval and downstream browsing\. Wikipedia retrieval requires approximately4040s depending on the number of retrieved pages\. In contrast, a single 30B\-scale LLM inference call only requires approximately33s\.

Therefore, although IPS introduces additional evaluation calls, its contribution to end\-to\-end latency remains relatively small, especially under parallel execution\. In practice, IPS mainly introduces a constant\-factor increase in token usage \(approximately2×2\\times\) due to multi\-plan generation and evaluation, while preserving the original asymptotic scaling behavior of the system\.

Overall, EPC\-AW preserves asymptotic complexity while improving success rate \(\+9\.75% avg\.\), offering a favorable cost–performance trade\-off\. Notably, the additional cost primarily scales the accumulated memory and retrieved context, which already dominate token consumption in the base system, rather than introducing a new computational bottleneck\.

Table 6:Time Cost Breakdown of EPC\-AW Components

## Appendix FCESR Case Studies

To further illustrate the lightweight epistemic constraints generated in CESR, we provide two representative examples from real execution traces\.

##### Case 1: Tool capability mismatch\.

Inbamboogle 8\.json, the Planner assumes that a specific ScienceDirect URL is accessible and can provide primary scientific evidence\. However, the Executor returns no usable content, revealing a mismatch between the assumed and actual capability of the retrieval tool\. CESR diagnoses this failure as tool information mismatch and redirects planning toward feasible and verifiable alternatives\.

Generated ConstraintWhen a specific tool or URL fails to retrieve required evidence, do not assume alternative paths are equally viable without validation; instead, revise the plan toward achievable and verifiable sources\.

##### Case 2: Infeasible verification under information limits\.

Inbamboogle 11\.json, the Planner repeatedly issues Google searches assuming that a complete list of Theranos whistleblowers and their connections to senior U\.S\. government officials can be exhaustively verified\. However, the Executor results show that such authoritative and comprehensive evidence is unavailable, exposing a mismatch between assumed and actual verifiability\. CESR identifies this as an unsupported sufficiency assumption and revises the objective to an epistemically reachable scope, focusing instead on confirmed whistleblowers \(e\.g\., Tyler Shultz and Erika Cheung\) using reliable sources such as Wikipedia\. This revision enables successful task completion\.

Generated ConstraintWhen a task requires verifying relationships or affiliations dependent on authoritative or inaccessible sources, avoid assuming such information is fully obtainable; instead, revise the goal to align with epistemically reachable and verifiable evidence\.

## Appendix GRelated Work

### G\.1Failure and Repair in LLM\-based Multi\-agent Systems

LLM\-based multi\-agent systems have been widely adopted for complex decision making, tool use, and long\-horizon task execution\(Zhouet al\.,[2026](https://arxiv.org/html/2605.23414#bib.bib10); Ronanki,[2025](https://arxiv.org/html/2605.23414#bib.bib115); Linet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib16)\)\. Despite their capabilities, these systems remain prone to failures arising from reasoning errors, coordination breakdowns, and unreliable or inconsistent use of information\(Cemriet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib141); Hammondet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib128); Zhanget al\.,[2025a](https://arxiv.org/html/2605.23414#bib.bib134)\)\. Such failures can propagate across agents and rounds, often compounding errors in long\-horizon tasks and multi\-step planning scenarios\. As a result, a growing body of work\(Wanet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib6); Guet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib8); Gonget al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib7)\)has studied how to diagnose, localize, and repair failures in these systems\. Most existing approaches focus on improving reliability by analyzing execution traces, monitoring agent interactions, or applying interventions at different stages of system runtime\(Lianget al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib43); Shenet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib113); Eppersonet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib48); Maet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib49)\), yet challenges remain in achieving robust cross\-agent consistency and epistemic calibration, particularly when agents hold heterogeneous or partial information\.

Several methods have been proposed to address failures in LLM\-based multi\-agent systems, broadly falling into post\-hoc or cross\-run debugging, and online correction during execution\. In the post\-hoc category, AGDebugger\(Eppersonet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib48)\)allows developers to inspect and edit multi\-agent interaction traces after execution, enabling identification and manual correction of failures\. DoVer\(Maet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib49)\)further formalizes this approach by testing failure hypotheses through targeted interventions and validating their effects across repeated executions, effectively providing a controlled mechanism for debugging and improving system robustness over multiple runs\.

In contrast, online correction methods aim to detect and mitigate failures as workflows unfold\. COCO\(Lianget al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib43)\)applies continuous monitoring, rollback, and reflective reasoning to correct execution drift and resolve inter\-agent inconsistencies in real time\. Similarly, MASC\(Shenet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib113)\)leverages history\-conditioned anomaly signals to detect problematic execution steps and applies local corrections to prevent cascading errors that could compromise long\-horizon tasks\. While these methods improve reliability during runtime, they typically operate on observed traces or local signals without explicitly modeling cross\-agent epistemic miscalibration, leaving challenges in ensuring robust coordination and consistent belief updating across heterogeneous agents\.

Despite differences in how and when repair is applied, these methods mainly target incorrect actions or local reasoning errors but neglect the plan’s commitments, allowing unreliable feasibility assessments to persist\. In contrast, this work formulates*epistemic miscalibration in planning*as a distinct repair target in LLM\-based multi\-agent systems\. It reveals that system\-level failures can arise from miscalibrated feasibility assessments at planning time, even when all subsequent executions are locally correct\.

### G\.2Model\- and Agent\-Level Epistemic Calibration

A growing body of work studies epistemic calibration in large language models \(LLMs\), focusing on overconfidence\(Wenet al\.,[2024](https://arxiv.org/html/2605.23414#bib.bib25); Hagaret al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib5); Tripathiet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib12)\)and uncertainty estimation\(Liuet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib13); Wanget al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib15); Leeet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib30); Nguyen\-Hienet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib14)\)\. This line of research primarily treats epistemic miscalibration as a property of an individual model or agent, aiming to align a model’s expressed confidence with its actual knowledge\. Methods include post\-hoc confidence adjustment, uncertainty\-aware decoding, and calibration\-aware fine\-tuning, which have been shown to reduce overconfident errors and improve the reliability of single\-agent predictions\. Even when multiple agents are considered, the focus largely remains on the calibration of individual LLM judgments rather than on the interactions or coordination of a deployed multi\-agent system\.

Several approaches leverage multi\-agent structures as a means to improve agent\-level calibration\(Clarket al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib28); Wenet al\.,[2024](https://arxiv.org/html/2605.23414#bib.bib25)\)\. Consensus or voting mechanisms aggregate agents’ assessments to stabilize judgments\(Pitreet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib31); Wenet al\.,[2026](https://arxiv.org/html/2605.23414#bib.bib33)\), while verifier agents critique intermediate outputs based on detectable errors or invalid responses\(Sunget al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib53); Gupta,[2025](https://arxiv.org/html/2605.23414#bib.bib34)\)\. These methods rely on LLMs acting as judges to evaluate content quality, but such first\-order judgments are themselves subject to epistemic miscalibration, which can propagate errors and limit their ability to reliably diagnose failures\. Additionally, aggregation\-based approaches may smooth out but not eliminate systemic biases shared across agents, leaving residual miscalibration unaddressed\.

Another related line of work draws from peer prediction and game\-theoretic truth inference\(Choet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib11); Kimet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib21)\)\. Bayesian Truth Serum\(Witkowski and Parkes,[2012](https://arxiv.org/html/2605.23414#bib.bib37); Weaver and Prelec,[2013](https://arxiv.org/html/2605.23414#bib.bib36)\)and its extensions elicit truthful reporting by rewarding agreement with peers’ predictions\. Recent adaptations implement explicit scoring rules or discrimination games for LLM\-based agents\(Chenet al\.,[2025](https://arxiv.org/html/2605.23414#bib.bib39)\), aiming to incentivize accurate and self\-consistent outputs\. While effective under static Bayesian game assumptions, these approaches rely on repeated interactions, payoff signals, or structured incentives, and thus do not naturally extend to dynamic multi\-agent systems with heterogeneous knowledge, limited interventions, and long\-horizon task dependencies\.

In contrast, we study epistemic miscalibration as a source of failure in LLM\-based multi\-agent systems\. Our goal is to repair task failures by intervening during system operation, where there is no additional payoff signals or external supervision\. Inspired by peer\-based calibration methods, we instead exploit the stability of agents’ evaluations across heterogeneous information and refine epistemic states over time based on persistent cross\-agent inconsistencies\.