SMADE-IE: Sparse Multi-Agent Framework with Evidence-Driven Debate for Zero-Shot Information Extraction

# SMADE-IE: Sparse Multi-Agent Framework with Evidence-Driven Debate for Zero-Shot Information Extraction
Source: [https://arxiv.org/html/2606.04691](https://arxiv.org/html/2606.04691)
Kenfeng Huang1, Yi Cai1, Xin Wu1, Zikun Deng1, Li Yuan1† 1School of Software Engineering, South China University of Technology, Guangzhou, China kenfenghuang@gmail\.com, seyuanli@mail\.scut\.edu\.cn \{ycai, xinwu, zkdeng\}@scut\.edu\.cn

###### Abstract

Zero\-shot information extraction \(IE\) with large language models \(LLMs\) has attracted increasing attention due to its flexibility in adapting to new schemas and domains without task\-specific training\. Existing approaches mainly rely on monolithic prompting, each\-type prompting, or multi\-agent debate\. However, monolithic prompting often suffers from boundary and type errors, while each\-type prompting and multi\-agent debate introduce cross\-type conflicts, redundant agent interactions, and substantial token overhead\. To address these challenges, we proposeSMADE\-IE, a sparse and evidence\-driven multi\-agent framework for zero\-shot IE\.SMADE\-IEfirst employs an Adaptive Mode Selector to dynamically route inputs into either a lightweight Global Extraction Mode or a Type\-Centric Extraction Mode, reducing unnecessary type selection and reasoning noise\. For conflicting predictions, we further introduce an Evidence\-Driven Debate mechanism that structures arguments into Toulmin\-style components and performs confidence aggregation through external evidence scoring and Bayesian updates\. Experimental results on 9 benchmark datasets across NER, RE, and JERE tasks show thatSMADE\-IEconsistently outperforms existing zero\-shot IE baselines while also improving token efficiency through sparse agent selection and early\-stopping debate\.

SMADE\-IE: Sparse Multi\-Agent Framework with Evidence\-Driven Debate for Zero\-Shot Information Extraction

††footnotetext:†Corresponding author\.## 1Introduction

Information Extraction \(IE\) aims to extract structured facts from unstructured text\(Li et al\.,[2023b](https://arxiv.org/html/2606.04691#bib.bib17)\), supporting applications such as knowledge graph construction and question answeringLiu et al\. \([2020](https://arxiv.org/html/2606.04691#bib.bib21)\); Jain \([2020](https://arxiv.org/html/2606.04691#bib.bib13)\)\. Representative tasks include named entity recognition \(NER\), relation extraction \(RE\), and joint entity and relation extraction \(JERE\)Zeng et al\. \([2014](https://arxiv.org/html/2606.04691#bib.bib46)\); Zheng et al\. \([2017](https://arxiv.org/html/2606.04691#bib.bib50)\)\. Although supervised and instruction\-tuned IE methods achieve strong performance\(Lu et al\.,[2022](https://arxiv.org/html/2606.04691#bib.bib25); Lou et al\.,[2023](https://arxiv.org/html/2606.04691#bib.bib23)\), they often rely on extensive annotations, task\-specific tuning, or fixed schemas, limiting adaptability to evolving entity types, relation patterns, and domains\(Sainz et al\.,[2024](https://arxiv.org/html/2606.04691#bib.bib34); Fuente et al\.,[2025](https://arxiv.org/html/2606.04691#bib.bib5)\)\.

Some studies leverage the extensive background knowledge of LLMs for zero\-shot IE, enabling flexible adaptation to new schemas and domainsXu et al\. \([2024](https://arxiv.org/html/2606.04691#bib.bib43)\)\. Early LLM\-based zero\-shot IE methods typically use a*monolithic\-prompt*strategy, extracting all elements in a single stage\(Xie et al\.,[2023](https://arxiv.org/html/2606.04691#bib.bib42); Li et al\.,[2024](https://arxiv.org/html/2606.04691#bib.bib18)\)\. Despite its efficiency, this strategy requires jointly predicting entity spans and types, which often leads to missed mentions, boundary errors, and type confusion\. As shown in Figure[1](https://arxiv.org/html/2606.04691#S1.F1)\(a\), the model misses “John Smith”, expands “California” to “in California”, and misclassifies “Apple” asProduct\. To alleviate these issues, recent work has introduced*each\-type prompting*\(Li et al\.,[2023a](https://arxiv.org/html/2606.04691#bib.bib16),[2024](https://arxiv.org/html/2606.04691#bib.bib18)\), which extracts each entity or relation type independently using separate prompts\. However, independently generated type\-specific predictions can overlap or conflict with one another\. For example, in Figure[1](https://arxiv.org/html/2606.04691#S1.F1)\(b\), “Apple” is simultaneously assigned the typesOrganizationandProduct\.

To mitigate such conflicts, recent*multi\-agent debate*methods employ specialized agents for different entity or relation types and use inter\-agent discussions to resolve conflicting predictions\(Guan et al\.,[2025](https://arxiv.org/html/2606.04691#bib.bib7); Lu et al\.,[2025](https://arxiv.org/html/2606.04691#bib.bib24)\)\. However, exhaustive agent interactions still limit their efficiency and reliability\. First, since each sample typically contains only a small subset of all entity or relation types, as shown in Table[1](https://arxiv.org/html/2606.04691#S1.T1), selecting all types for every sample introduces unnecessary token overhead and computational cost\. Second, incorrect candidate types inject irrelevant context into the debate, distracting agents from meaningful evidence and leading to unreliable decisions\(Fan et al\.,[2026](https://arxiv.org/html/2606.04691#bib.bib4); Shi et al\.,[2023](https://arxiv.org/html/2606.04691#bib.bib35); Liu et al\.,[2024](https://arxiv.org/html/2606.04691#bib.bib22); Yang et al\.,[2025](https://arxiv.org/html/2606.04691#bib.bib44); Tian et al\.,[2026](https://arxiv.org/html/2606.04691#bib.bib37)\)\. For instance, as shown in Figure[1](https://arxiv.org/html/2606.04691#S1.F1)\(c\), three agents \(*PER*,*ORG*, and*LOC*\) label the same entity with different types, causing redundant discussions that obscure useful evidence and increase the risk of incorrect adjudication\. Finally, free\-form debates lack explicit structures for modeling claims, points of contention, and supporting evidence, making it difficult to systematically aggregate confidence scores and reach stable decisions\(Choi et al\.,[2026](https://arxiv.org/html/2606.04691#bib.bib3)\)\. As illustrated in Figure[1](https://arxiv.org/html/2606.04691#S1.F1)\(c\), theProductagent argues that “Apple could also be a product,” while theOrganizationagent claims that “Apple is a technology company\.” Without a structured mechanism to distinguish reliable evidence from noisy or irrelevant arguments, such debates may fail to converge and incur substantial token overhead\(Choi et al\.,[2026](https://arxiv.org/html/2606.04691#bib.bib3)\)\.

![Refer to caption](https://arxiv.org/html/2606.04691v1/x1.png)Figure 1:Existing LLM\-based zero\-shot IE methods\.TaskDatasetOntology SizeAvg\. Types / SampleNEROntoNotes5181\.31CrossRE384\.72REDocRED322\.46REDFM322\.77SciERC81\.88Table 1:Statistics of some zero\-shot IE benchmarks\.To address these challenges, we proposeSMADE\-IE, a sparse and evidence\-driven multi\-agent framework for zero\-shot IE\. As shown in Figure[2](https://arxiv.org/html/2606.04691#S1.F2),SMADE\-IEbalances extraction reliability and computational cost through two components\. First, the Adaptive Mode Selector estimates sample complexity and selects only relevant types: simple inputs follow the lightweight Global Extraction Mode, while complex inputs enter the Type\-Centric Extraction Mode, reducing unnecessary token overhead and irrelevant reasoning noise\. Second, for type conflicts in the Type\-Centric Extraction Mode,SMADE\-IEintroduces a structured debate mechanism\. Unlike free\-form debate, it decomposes candidate predictions into Toulmin\-style argumentative components\(Gupta et al\.,[2024](https://arxiv.org/html/2606.04691#bib.bib8)\), allowing agents to attack the evidence and reasoning behind each claim\. An external evidence scorer and Bayesian Beta updates then aggregate support and refutation signals into evidence\-centered confidence estimates for final adjudication\. The main contributions are as follows:

- •We presentSMADE\-IE, a sparse multi\-agent framework for zero\-shot information extraction\. With the Adaptive Mode Selector, it automatically dispatches samples to either the Global Extraction Mode or the Type\-Centric Extraction Mode based on their complexity, balancing efficiency and performance\.
- •We introduce the Evidence\-Driven Debate mechanism that structures each conflict into Toulmin\-style components, and combines external evidence scoring with Bayesian Beta updates for more reliable multi\-agent conflict adjudication\.
- •We evaluateSMADE\-IEon 9 zero\-shot IE datasets and show its advantages in both extraction performance and token efficiency\.

![Refer to caption](https://arxiv.org/html/2606.04691v1/x2.png)Figure 2:Overview ofSMADE\-IE\. The Router Agent selects extraction modes\. For conflicted predictions, Evidence\-Driven Debate performs Toulmin\-based ranking, Bayesian debate, and conflict resolution\.
## 2Methodology

In this section, we first introduce the definition of zero\-shot IE tasks and then describeSMADE\-IEin detail\. As shown in Figure[2](https://arxiv.org/html/2606.04691#S1.F2), the Adaptive Mode Selector dispatches each input to either the lightweight Global Extraction Mode or the finer\-grained Type\-Centric Extraction Mode\. For cross\-type conflicts in the Type\-Centric Extraction Mode, we introduce the Evidence\-Driven Debate module for structured conflict adjudication\. For JERE, an Iterative Entity–Relation Alignment step further enforces ontology consistency between entity and relation predictions\.

### 2\.1Problem Formulation

Given a sentencessand predefined sets of entity types𝒯e\\mathcal\{T\}\_\{e\}and relation types𝒯r\\mathcal\{T\}\_\{r\}, NER aims to identify all entity mentions\{e1,…,en\}\\\{e\_\{1\},\\dots,e\_\{n\}\\\}inssand assign a typetei∈𝒯et\_\{e\}^\{i\}\\in\\mathcal\{T\}\_\{e\}to each, producing a set of typed entitiesE=\{\(ei,tei\)\}E=\\\{\(e\_\{i\},t\_\{e\}^\{i\}\)\\\}\. RE requires first identifying the entity mentions\{e1,…,en\}\\\{e\_\{1\},\\dots,e\_\{n\}\\\}and then predicting, for each ordered pair of entities\(ei,ej\)\(e\_\{i\},e\_\{j\}\), the relationri​j∈𝒯rr\_\{ij\}\\in\\mathcal\{T\}\_\{r\}that holds between them, producing a set of relationsR=\{\(ei,ri​j,ej\)∣ri​j∈𝒯r\}R=\\\{\(e\_\{i\},r\_\{ij\},e\_\{j\}\)\\mid r\_\{ij\}\\in\\mathcal\{T\}\_\{r\}\\\}\. Finally, JERE simultaneously recognizes all entities and relations, yielding a set of fully typed quintuplesR∗=\{\(ei,tei,ri​j,ej,tej\)\}R^\{\*\}=\\\{\(e\_\{i\},t\_\{e\}^\{i\},r\_\{ij\},e\_\{j\},t\_\{e\}^\{j\}\)\\\}, where each quintuple specifies a head entity, a tail entity, their types, and the relation connecting them\. Since zero\-shot NER and RE are handled by the same LLM framework inSMADE\-IEand differ only in the task\-definition prompt, we presentSMADE\-IEfrom the zero\-shot NER perspective\.

### 2\.2Adaptive Mode Selector

SMADE\-IEdynamically selects the inference extraction mode for each input, reducing redundant calls while preserving type\-level scrutiny in ambiguous cases\. Inspired by the fast\-and\-slow thinking paradigm in LLM reasoning\(Zhang et al\.,[2025a](https://arxiv.org/html/2606.04691#bib.bib48)\), we introduce a Router Agent to estimate both the relevant type subset and the sample complexity\. Given a sentencessand the entity type set𝒯e\\mathcal\{T\}\_\{e\}, the Router Agent produces,

\(𝒜,c\)=LLM​\(s,P𝒯es​e​l\),\(\\mathcal\{A\},c\)=\\mathrm\{LLM\}\(s,P^\{sel\}\_\{\\mathcal\{T\}\_\{e\}\}\),\(1\)whereLLM​\(⋅\)\\mathrm\{LLM\}\(\\cdot\)denotes the large language model,P𝒯es​e​lP^\{sel\}\_\{\\mathcal\{T\}\_\{e\}\}is the selector prompt,𝒜⊆𝒯e\\mathcal\{A\}\\subseteq\\mathcal\{T\}\_\{e\}is the candidate type subset, andc∈\{low,med,high\}c\\in\\\{\\mathrm\{low\},\\mathrm\{med\},\\mathrm\{high\}\\\}denotes the estimated sample complexity\. We adopt three complexity levels instead of binary labels to better capture borderline uncertainty\(Mahajan et al\.,[2026](https://arxiv.org/html/2606.04691#bib.bib28)\)\. The estimates are then mapped into two execution modes: low\-complexity inputs use the lightweight Global Extraction Mode, while medium\- and high\-complexity inputs use the Type\-Centric Extraction Mode\.

### 2\.3Global Extraction Mode

To effectively balance extraction performance and computational cost, particularly in terms of token usage,SMADE\-IEadopts the Global Extraction Mode for low\-complexity inputs \(c=lowc=\\mathrm\{low\}\)\. It employs two sequential LLM agents: a Universal Agent that first generates a one\-shot candidate set, followed by a Verification Agent that performs targeted verification and refinement\.

#### 2\.3\.1Universal Agent

For simple samples, we first apply the monolithic\-prompt methodXie et al\. \([2023](https://arxiv.org/html/2606.04691#bib.bib42)\)to extract all candidate entities with types in𝒯e\\mathcal\{T\}\_\{e\}\. Formally, given the global extraction promptP𝒯eglobalP\_\{\\mathcal\{T\}\_\{e\}\}^\{\\mathrm\{global\}\}, the output of the Universal Agent is defined as,

EU=LLM​\(s,P𝒯eglobal\),E\_\{U\}=\\mathrm\{LLM\}\(s,P\_\{\\mathcal\{T\}\_\{e\}\}^\{\\mathrm\{global\}\}\),\(2\)where every element\(ei,tei\)∈EU\(e\_\{i\},t^\{i\}\_\{e\}\)\\in E\_\{U\}pairs a candidate entity with a type\.

#### 2\.3\.2Verification Agent

Although the Universal Agent extracts all elements in a single pass, its coarse\-grained generation strategy may still miss valid entities or introduce redundant candidates\. To improve extraction reliability, the Verification Agent, inspired by the clean\-up module in G&O\(Li et al\.,[2024](https://arxiv.org/html/2606.04691#bib.bib18)\), performs bidirectional refinement over the candidate setEUE\_\{U\}\. Given the verification promptP𝒯everP\_\{\\mathcal\{T\}\_\{e\}\}^\{\\mathrm\{ver\}\}andEUE\_\{U\}, the Verification Agent inserts missing entities and removes unsupported candidates:

\(E\+,E−\)=LLM​\(s,EU,P𝒯ever\),\(E^\{\+\},E^\{\-\}\)=\\mathrm\{LLM\}\(s,E\_\{U\},P\_\{\\mathcal\{T\}\_\{e\}\}^\{\\mathrm\{ver\}\}\),\(3\)whereE\+E^\{\+\}andE−E^\{\-\}denote the sets of entities to be inserted and deleted, respectively\. Accordingly, the final output of the Global Extraction Mode is:

E=\(EU∖E−\)∪E\+\.E=\(E\_\{U\}\\setminus E^\{\-\}\)\\cup E^\{\+\}\.\(4\)

### 2\.4Type\-Centric Extraction Mode

For medium\- and high\-complexity inputs,SMADE\-IEinstantiates Type\-Specific Agents for candidate types𝒜\\mathcal\{A\}selected by the Adaptive Mode Selector in Eq\. \([1](https://arxiv.org/html/2606.04691#S2.E1)\) and adds a Review Agent to recover residual\-type candidates\. To resolve conflicts among agents, we introduce the Evidence\-Driven Debate module that limits unconstrained discussion and weak\-candidate interference, framing conflict resolution as evidence\-based confidence comparison\.

#### 2\.4\.1Type\-Specific Agent

For each candidate typet𝒜i∈𝒜t\_\{\\mathcal\{A\}\}^\{i\}\\in\\mathcal\{A\}, we design a dedicated Type\-Specific Agent with promptP𝒜iP\_\{\\mathcal\{A\}\}^\{i\}\. Given the input sentencess, the agent extracts candidate entitiese𝒜ie^\{i\}\_\{\\mathcal\{A\}\}associated with typet𝒜it\_\{\\mathcal\{A\}\}^\{i\}:

e𝒜i\\displaystyle e^\{i\}\_\{\\mathcal\{A\}\}=\\displaystyle=LLM​\(s,P𝒜i\),\\displaystyle\\mathrm\{LLM\}\(s,P\_\{\\mathcal\{A\}\}^\{i\}\),\(5\)where each elemente𝒜i,j∈e𝒜ie\_\{\\mathcal\{A\}\}^\{i,j\}\\in e\_\{\\mathcal\{A\}\}^\{i\}denotes a candidate entity extracted for typet𝒜it\_\{\\mathcal\{A\}\}^\{i\}\. As illustrated in Figure[2](https://arxiv.org/html/2606.04691#S1.F2), thePersonagent extracts bothAmazonandJordan Lee\. We then assign each extracted entitye𝒜i,je\_\{\\mathcal\{A\}\}^\{i,j\}to its corresponding typet𝒜it\_\{\\mathcal\{A\}\}^\{i\}and aggregate all predictions to obtain the final output of the Type\-Specific Agents, denoted asE𝒜E\_\{\\mathcal\{A\}\}\.

#### 2\.4\.2Review Agent

The Adaptive Mode Selector estimates𝒜\\mathcal\{A\}in a single recall\-oriented pass, but may miss peripheral types in long or context\-rich sentences\. To address this issue, we introduce the Review Agent over the residual type set𝒜∗=𝒯e∖𝒜\\mathcal\{A\}^\{\*\}=\\mathcal\{T\}\_\{e\}\\setminus\\mathcal\{A\}\. LetP𝒜∗revP\_\{\\mathcal\{A\}^\{\*\}\}^\{\\mathrm\{rev\}\}denote the Review Agent prompt restricted to𝒜∗\\mathcal\{A\}^\{\*\}\. The Review Agent produces a residual candidate set:

E𝒜∗=LLM​\(s,P𝒜∗rev\),E\_\{\\mathcal\{A\}\}^\{\*\}=\\mathrm\{LLM\}\(s,P\_\{\\mathcal\{A\}^\{\*\}\}^\{\\mathrm\{rev\}\}\),\(6\)The final output is defined as the union of the initial and residual sets:

E~𝒜=E𝒜∪E𝒜∗\.\\tilde\{E\}\_\{\\mathcal\{A\}\}=E\_\{\\mathcal\{A\}\}\\cup E\_\{\\mathcal\{A\}\}^\{\*\}\.\(7\)where each element inE~𝒜\\tilde\{E\}\_\{\\mathcal\{A\}\}is represented as a tuple\(e~i,t~i\)\(\\tilde\{e\}^\{i\},\\tilde\{t\}^\{i\}\)containing a candidate entity and its assigned type\. Since different Type\-Specific Agents may assign different types to the same entity, we perform conflict detection before downstream processing\. Specifically, if an entitye~i\\tilde\{e\}^\{i\}is assigned multiple types, it is placed into the conflict setE~𝒜conf\\tilde\{E\}\_\{\\mathcal\{A\}\}^\{\\mathrm\{conf\}\}; otherwise, it is placed into the clean setE~𝒜clean\\tilde\{E\}\_\{\\mathcal\{A\}\}^\{\\mathrm\{clean\}\}:

E~𝒜clean\\displaystyle\\tilde\{E\}^\{\\mathrm\{clean\}\}\_\{\\mathcal\{A\}\}=\{\(e~,t~\)∈E~𝒜∣\|\{t~′∣\(e~,t~′\)∈E~𝒜\}\|=1\},\\displaystyle=\\\{\(\\tilde\{e\},\\tilde\{t\}\)\\in\\tilde\{E\}\_\{\\mathcal\{A\}\}\\;\\mid\\;\|\\\{\\tilde\{t\}^\{\\prime\}\\mid\(\\tilde\{e\},\\tilde\{t\}^\{\\prime\}\)\\in\\tilde\{E\}\_\{\\mathcal\{A\}\}\\\}\|=1\\\},\(8\)E~𝒜conf\\displaystyle\\tilde\{E\}^\{\\mathrm\{conf\}\}\_\{\\mathcal\{A\}\}=\{\(e~,𝒴e~\)∣𝒴e~=\{t~∣\(e~,t~\)∈E~𝒜\},\|𝒴e~\|\>1\}\.\\displaystyle=\\left\\\{\(\\tilde\{e\},\\mathcal\{Y\}\_\{\\tilde\{e\}\}\)\\mid\\mathcal\{Y\}\_\{\\tilde\{e\}\}=\\\{\\tilde\{t\}\\mid\(\\tilde\{e\},\\tilde\{t\}\)\\in\\tilde\{E\}\_\{\\mathcal\{A\}\}\\\},\\ \|\\mathcal\{Y\}\_\{\\tilde\{e\}\}\|\>1\\right\\\}\.
The clean setE~𝒜clean\\tilde\{E\}^\{\\mathrm\{clean\}\}\_\{\\mathcal\{A\}\}is directly output, while the conflict setE~𝒜conf\\tilde\{E\}\_\{\\mathcal\{A\}\}^\{\\mathrm\{conf\}\}is forwarded to the Evidence\-Driven Debate module for conflict resolution and final type confirmation\. Each element inE~𝒜conf\\tilde\{E\}\_\{\\mathcal\{A\}\}^\{\\mathrm\{conf\}\}is represented as\(e~i,\{t~1i,…,t~Ni\}\)\(\\tilde\{e\}^\{i\},\\\{\\tilde\{t\}^\{i\}\_\{1\},\\dots,\\tilde\{t\}^\{i\}\_\{N\}\\\}\), wheree~i\\tilde\{e\}^\{i\}denotes a candidate entity and\{t~1i,…,t~Ni\}\\\{\\tilde\{t\}^\{i\}\_\{1\},\\dots,\\tilde\{t\}^\{i\}\_\{N\}\\\}are its conflicting type assignments, andNNis the number of conflicting types assigned to the entity\.

#### 2\.4\.3Evidence\-Driven Debate

This module enhances the robustness and evidence\-groundedness of conflict adjudication by mitigating unsupported reasoning and low\-confidence interference in free\-form multi\-agent debates\. Specifically, each conflicting type assignment is recast as a Toulmin\-style argument to enable structured evidence inspection\. An external evidence scorer estimates candidate confidence from sentence\-level support, ranks candidates for pre\-debate filtering, and computes evidence\-based attack scores during the debate\. Support and refutation signals are then aggregated through Beta updates, and the debate terminates according to posterior stability and leading\-candidate dominance\.

##### Toulmin\-Guided Argument Construction

![Refer to caption](https://arxiv.org/html/2606.04691v1/x3.png)Figure 3:Example of a Toulmin\-guided argument for the contested entity “Amazon”, generated by theOrganizationagent\.Recent work uses Toulmin theory to make implicit arguments explicit and improve the logical validity of LLM reasoning\(Gupta et al\.,[2024](https://arxiv.org/html/2606.04691#bib.bib8); Xiao et al\.,[2024](https://arxiv.org/html/2606.04691#bib.bib41)\)\. Following this paradigm, we construct a structured argument for each conflicting candidate entitye~i∈E~𝒜conf\\tilde\{e\}^\{i\}\\in\\tilde\{E\}\_\{\\mathcal\{A\}\}^\{\\mathrm\{conf\}\}and its associated types\{t~1i,…,t~Ni\}\\\{\\tilde\{t\}^\{i\}\_\{1\},\\dots,\\tilde\{t\}^\{i\}\_\{N\}\\\}to constrain LLM discourse\. Each agent organizes its argument into five components: Claim \(CC\), Ground \(GG\), Warrant \(WW\), Backing \(BB\), and Rebuttal \(RR\)\. For a conflicting entitye~i\\tilde\{e\}^\{i\}, the Toulmin\-guided argument for conflict typet~ni\\tilde\{t\}^\{i\}\_\{n\}is defined as:

\(Cn,Gn,Wn,Bn,Rn\)=LLM​\(s,e~i,t~ni,PToulmin\),\(C\_\{n\},G\_\{n\},W\_\{n\},B\_\{n\},R\_\{n\}\)=\\mathrm\{LLM\}\(s,\\tilde\{e\}^\{i\},\\tilde\{t\}^\{i\}\_\{n\},P\_\{\\mathrm\{Toulmin\}\}\),\(9\)whereCnC\_\{n\}is the claim fort~ni\\tilde\{t\}^\{i\}\_\{n\},GnG\_\{n\}provides evidence from sentencess,WnW\_\{n\}explains howGnG\_\{n\}supportsCnC\_\{n\},BnB\_\{n\}gives auxiliary support, andRnR\_\{n\}states a possible rebuttal\. To measure how wellsssupports each argument, we compute a confidence scoreQnQ\_\{n\}using an evidence scorerϕ\\phiZha et al\. \([2023](https://arxiv.org/html/2606.04691#bib.bib47)\)\. Specifically, we concatenate\{Cn,Gn,Wn,Bn\}\\\{C\_\{n\},G\_\{n\},W\_\{n\},B\_\{n\}\\\}intoParan\\mathrm\{Para\}\_\{n\}and construct a shared contexts^\\hat\{s\}from sentencessand all conflicting type definitions\{t~1i,…,t~Ni\}\\\{\\tilde\{t\}^\{i\}\_\{1\},\\dots,\\tilde\{t\}^\{i\}\_\{N\}\\\},

Qn=ϕ​\(s^,Paran\)∈\[0,1\],Q\_\{n\}=\\phi\(\\hat\{s\},\\mathrm\{Para\}\_\{n\}\)\\in\[0,1\],\(10\)whereϕ\\phidenotes an external evidence scorerZha et al\. \([2023](https://arxiv.org/html/2606.04691#bib.bib47)\)that measures how strongly the textual evidence\(Gn,Wn,Bn\)\(G\_\{n\},W\_\{n\},B\_\{n\}\)supports the claimCnC\_\{n\}\. Each agent is represented as the tuple𝒟n=\{Cn,Gn,Wn,Bn,Rn,Qn\}\\mathcal\{D\}\_\{n\}=\\\{C\_\{n\},G\_\{n\},W\_\{n\},B\_\{n\},R\_\{n\},Q\_\{n\}\\\}, which forms a structured, evidence\-grounded argument\. As shown in Figure[3](https://arxiv.org/html/2606.04691#S2.F3), anOrganizationagent may use this format to justify the contested entityAmazon\. Considering all candidate types for a contested entitye~i\\tilde\{e\}^\{i\}incurs𝒪​\(N2\)\\mathcal\{O\}\(N^\{2\}\)pairwise attacks per debate round\. To reduce computational complexity and mitigate noise from weak candidates, we rank theNNToulmin\-guided arguments\{𝒟n\}n=1N\\\{\\mathcal\{D\}\_\{n\}\\\}\_\{n=1\}^\{N\}by their qualifiersQnQ\_\{n\}and retain only the top two:

𝒟L=\(𝒟top1,𝒟top2\)=Top2𝒟n⁡\(\{𝒟n\}n=1N;Qn\)\.\\mathcal\{D\}\_\{L\}=\(\\mathcal\{D\}\_\{\\mathrm\{top1\}\},\\mathcal\{D\}\_\{\\mathrm\{top2\}\}\)=\\operatorname\{Top2\}\_\{\\mathcal\{D\}\_\{n\}\}\\left\(\\\{\\mathcal\{D\}\_\{n\}\\\}\_\{n=1\}^\{N\};\\ Q\_\{n\}\\right\)\.\(11\)

##### Bayesian Confidence Modeling

We model candidate confidence with a Beta posteriorBeta​\(αl,βl\)\\mathrm\{Beta\}\(\\alpha\_\{l\},\\beta\_\{l\}\)for each agent𝒟l\\mathcal\{D\}\_\{l\}\(Hu et al\.,[2025](https://arxiv.org/html/2606.04691#bib.bib11)\)\. The posterior accumulates supporting \(αl\\alpha\_\{l\}\) and refuting \(βl\\beta\_\{l\}\) evidence across debate iterations, producing calibrated scores comparable across candidates\. For candidatell, we compute an alignment\-based scaling factor:

ρl\\displaystyle\\rho\_\{l\}=1−ϕ​\(xl,Rl\),\\displaystyle=1\-\\phi\(x\_\{l\},R\_\{l\}\),κl\\displaystyle\\kappa\_\{l\}=κmin\+ρl​\(κmax−κmin\),\\displaystyle=\\kappa\_\{\\min\}\+\\rho\_\{l\}\(\\kappa\_\{\\max\}\-\\kappa\_\{\\min\}\),\(12\)wherexlx\_\{l\}concatenatessswith the candidate type definitiont~li\\tilde\{t\}^\{i\}\_\{l\}, andκl\\kappa\_\{l\}controls prior strength\. Higherϕ​\(xl,Rl\)\\phi\(x\_\{l\},R\_\{l\}\)implies weaker self\-criticism and a smallerκl\\kappa\_\{l\}\. The posterior is initialized withQlQ\_\{l\}:

αl\(0\)=Ql​κl,βl\(0\)=\(1−Ql\)​κl,\\alpha\_\{l\}^\{\(0\)\}=Q\_\{l\}\\kappa\_\{l\},\\qquad\\beta\_\{l\}^\{\(0\)\}=\(1\-Q\_\{l\}\)\\kappa\_\{l\},\(13\)The initialized posterior parameters are then updated through the joint evidence and confidence updating process described below\.

##### Evidence\-Grounded Adversarial Refutation

To ensure evidence\-grounded refutations, we restrict the attack surface to the two Toulmin components that directly support candidatell: Ground \(GlG\_\{l\}\) and Warrant \(WlW\_\{l\}\)\. Ground provides supporting evidence, while Warrant links the evidence to the claimClC\_\{l\}; other components offer contextual or auxiliary justification\. Letm∈\{G,W\}m\\in\\\{G,W\\\}denote an attackable component, and letzl,mz\_\{l,m\}be its textual content for candidatell\. Each component has a validity scorevl,m\(t\)∈\[0,1\]v\_\{l,m\}^\{\(t\)\}\\in\[0,1\], initialized asvl,m\(0\)=1v\_\{l,m\}^\{\(0\)\}=1, indicating its remaining credibility at iterationtt\. A component remains attackable whilevl,m\(t\)\>0v\_\{l,m\}^\{\(t\)\}\>0and is removed once its validity reaches zero:

𝒱l\(t\)=\{m∈\{G,W\}\|vl,m\(t\)\>0\}\.\\mathcal\{V\}\_\{l\}^\{\(t\)\}=\\left\\\{m\\in\\\{G,W\\\}\\;\\middle\|\\;v\_\{l,m\}^\{\(t\)\}\>0\\right\\\}\.\(14\)
![Refer to caption](https://arxiv.org/html/2606.04691v1/x4.png)Figure 4:Iterative Entity–Relation Alignment under ontology constraints\.At iterationtt, the opposing agent𝒟l¯\\mathcal\{D\}\_\{\\bar\{l\}\}attacks the defended agent𝒟l\\mathcal\{D\}\_\{l\}using sentencess, the active component set𝒱l\(t\)\\mathcal\{V\}\_\{l\}^\{\(t\)\}, and the directed attack promptPl¯atkP\_\{\\bar\{l\}\}^\{\\mathrm\{atk\}\}\. For each defended componentm∈𝒱l\(t\)m\\in\\mathcal\{V\}\_\{l\}^\{\(t\)\}, it generates a refutationul¯,mu\_\{\\bar\{l\},m\}:

ul¯,m=LLM​\(s,𝒱l\(t\),Pl¯,matk\)\.u\_\{\\bar\{l\},m\}=\\mathrm\{LLM\}\\left\(s,\\,\\mathcal\{V\}\_\{l\}^\{\(t\)\},\\,P\_\{\\bar\{l\},m\}^\{\\mathrm\{atk\}\}\\right\)\.\(15\)We measure attack effectiveness by how much the refutation weakens the defended componentzl,mz\_\{l,m\}under the type\-specific contextxlx\_\{l\}\. The attack strength at iterationttis

al,m\(t\)=σ​\(γ​\[ϕ​\(xl,ul¯,m\)−ϕ​\(xl,zl,m\)\]\),a\_\{l,m\}^\{\(t\)\}=\\sigma\\left\(\\gamma\\left\[\\phi\(x\_\{l\},\\,u\_\{\\bar\{l\},m\}\)\-\\phi\(x\_\{l\},\\,z\_\{l,m\}\)\\right\]\\right\),\(16\)whereγ\\gammais a temperature parameter andσ​\(⋅\)\\sigma\(\\cdot\)is the sigmoid function\. Largeral,m\(t\)a\_\{l,m\}^\{\(t\)\}indicates stronger support for the refutation relative to the defended content\.

Table 2:NER results on five benchmark datasets\. Parentheses indicate the number of entity types \(e\.g\., CoNLL03 has 3 entity types\)\.Boldandunderlinedvalues denote the best and second\-best results, respectively\.♢\\diamondsuit,♣\\clubsuit, and♠\\spadesuitindicate monolithic prompting, each\-type prompting, and multi\-agent debate methods, respectively\.Table 3:RE results\.SMADE\-IEobtains the strongest macro average across all relation\-extraction benchmarksTable 4:JERE results \(%\);F1P\\text\{F1\}\_\{\\text\{P\}\}/F1S\\text\{F1\}\_\{\\text\{S\}\}as in Table[2](https://arxiv.org/html/2606.04691#S2.T2)\. Both types and the relation of head and tail type must match\.
##### Joint Evidence and Confidence Updating

To prevent posterior recovery through reformulated invalid components, we couple component validity with the Beta posterior via the shared attack scoreal,m\(t\)a\_\{l,m\}^\{\(t\)\}\. This ties posterior updates to evidence credibility and keeps both update processes consistent\. For each component,vl,m\(t\)v\_\{l,m\}^\{\(t\)\}is updated in two stages\. First, the attack induces exponential decay:

v~l,m\(t\+1\)=vl,m\(t\)​exp⁡\(−η​al,m\(t\)\),\\tilde\{v\}\_\{l,m\}^\{\(t\+1\)\}=v\_\{l,m\}^\{\(t\)\}\\exp\\\!\\left\(\-\\eta\\,a\_\{l,m\}^\{\(t\)\}\\right\),\(17\)whereη\>0\\eta\>0is the decay rate\. Given a validity activity thresholdθv\\theta\_\{v\}, the realized validity is

vl,m\(t\+1\)=\{v~l,m\(t\+1\),v~l,m\(t\+1\)\>θv,θv\+12​ωkml,0<v~l,m\(t\+1\)≤θv,and​θv\+12​ωkml≥θv,0,otherwise\.v\_\{l,m\}^\{\(t\+1\)\}=\\begin\{cases\}\\tilde\{v\}\_\{l,m\}^\{\(t\+1\)\},&\\tilde\{v\}\_\{l,m\}^\{\(t\+1\)\}\>\\theta\_\{v\},\\\\\[6\.0pt\] \\dfrac\{\\theta\_\{v\}\+1\}\{2\}\\,\\omega^\{k^\{l\}\_\{m\}\},&\\begin\{aligned\} 0<\\tilde\{v\}\_\{l,m\}^\{\(t\+1\)\}&\\leq\\theta\_\{v\},\\\\ \\text\{and \}\\dfrac\{\\theta\_\{v\}\+1\}\{2\}\\,\\omega^\{k^\{l\}\_\{m\}\}&\\geq\\theta\_\{v\},\\end\{aligned\}\\\\\[10\.0pt\] 0,&\\text\{otherwise\}\.\\end\{cases\}\(18\)
wherev~l,m\(t\+1\)\\tilde\{v\}\_\{l,m\}^\{\(t\+1\)\}is retained if it remains aboveθv\\theta\_\{v\}\. Otherwise, the component enters revision and is reset to the discounted ceilingθv\+12​ωkml\\frac\{\\theta\_\{v\}\+1\}\{2\}\\omega^\{k^\{l\}\_\{m\}\}, wherekmlk^\{l\}\_\{m\}is the number of prior revisions andω∈\(0,1\)\\omega\\in\(0,1\)is the revision discount factor\. Repeated revisions therefore reduce credibility\. If the revised validity still exceedsθv\\theta\_\{v\}, the component remains in𝒱l\\mathcal\{V\}\_\{l\}; otherwise, it is permanently removed from the attack surface\. The same attack score updates the Beta posterior of candidatellby adding refutation mass toβl\\beta\_\{l\}and residual support mass toαl\\alpha\_\{l\}:

βl\(t\+1\)\\displaystyle\\beta\_\{l\}^\{\(t\+1\)\}=βl\(t\)\+v¯l¯\(t\)\|𝒱l\(t\)\|​∑m∈𝒱l\(t\)al,m\(t\)⋅vl,m\(t\)\\displaystyle=\\beta\_\{l\}^\{\(t\)\}\+\\frac\{\\bar\{v\}\_\{\\bar\{l\}\}^\{\(t\)\}\}\{\\left\|\\mathcal\{V\}\_\{l\}^\{\(t\)\}\\right\|\}\\sum\_\{m\\in\\mathcal\{V\}\_\{l\}^\{\(t\)\}\}\{a\}\_\{l,m\}^\{\(t\)\}\\cdot\{v\}\_\{l,m\}^\{\(t\)\}\(19\)αl\(t\+1\)\\displaystyle\\alpha\_\{l\}^\{\(t\+1\)\}=αl\(t\)\+1\|𝒱l\(t\)\|​∑m∈𝒱l\(t\)\(1−al,m\(t\)\)⋅vl,m\(t\)\\displaystyle=\\alpha\_\{l\}^\{\(t\)\}\+\\frac\{1\}\{\\left\|\\mathcal\{V\}\_\{l\}^\{\(t\)\}\\right\|\}\\sum\_\{m\\in\\mathcal\{V\}\_\{l\}^\{\(t\)\}\}\\big\(1\-a\_\{l,m\}^\{\(t\)\}\\big\)\\cdot v\_\{l,m\}^\{\(t\)\}wherev¯l¯\(t\)\\bar\{v\}\_\{\\bar\{l\}\}^\{\(t\)\}denotes the average validity of the opponent’s active evidence components:

v¯l¯\(t\)=1\|𝒱l¯\(t\)\|​∑m∈𝒱l¯\(t\)vl¯,m\(t\)\.\\bar\{v\}\_\{\\bar\{l\}\}^\{\(t\)\}=\\frac\{1\}\{\|\\mathcal\{V\}\_\{\\bar\{l\}\}^\{\(t\)\}\|\}\\sum\_\{m\\in\\mathcal\{V\}\_\{\\bar\{l\}\}^\{\(t\)\}\}v\_\{\\bar\{l\},m\}^\{\(t\)\}\.\(20\)The weights have distinct roles:v¯l¯\(t\)\\bar\{v\}\_\{\\bar\{l\}\}^\{\(t\)\}downweights attacks from opponents with weakened evidence, whilevl,m\(t\)v\_\{l,m\}^\{\(t\)\}reduces the influence of components weakened in earlier rounds\.

##### Dual\-Track Early Stopping

To avoid redundant debate rounds, we terminate the process when eitherposterior stabilityorleading candidate confidenceis satisfied\. Both criteria operate on the Beta posteriors induced by the joint update rule:

πl\(t\)∼Beta​\(αl\(t\),βl\(t\)\),\\pi\_\{l\}^\{\(t\)\}\\sim\\mathrm\{Beta\}\\\!\\left\(\\alpha\_\{l\}^\{\(t\)\},\\beta\_\{l\}^\{\(t\)\}\\right\),\(21\)for each candidatel∈Ll\\in Lat iterationtt\.

Theposterior stabilitycriterion measures convergence between consecutive posterior distributions using the average squared Hellinger distance:

Δconf\(t\)=12​∑l∈LH2​\(πl\(t\),πl\(t−1\)\)<ε,\\displaystyle\\Delta\_\{\\mathrm\{conf\}\}^\{\(t\)\}=\\frac\{1\}\{2\}\\sum\_\{l\\in L\}H^\{2\}\\\!\\Big\(\\pi\_\{l\}^\{\(t\)\},\\pi\_\{l\}^\{\(t\-1\)\}\\Big\)<\\varepsilon,\(22\)whereH2​\(⋅\)H^\{2\}\(\\cdot\)denotes squared Hellinger distance\. The debate terminates onceΔconf\(t\)<ε\\Delta\_\{\\mathrm\{conf\}\}^\{\(t\)\}<\\varepsilon, indicating that the posteriors have effectively converged\.

Theleading candidate confidencecriterion stops the debate when the leading candidate is sufficiently likely to outperform its opponent\. For scalar comparison, we use the posterior mean

p^l\(t\)=αl\(t\)αl\(t\)\+βl\(t\),\\hat\{p\}\_\{l\}^\{\(t\)\}=\\frac\{\\alpha\_\{l\}^\{\(t\)\}\}\{\\alpha\_\{l\}^\{\(t\)\}\+\\beta\_\{l\}^\{\(t\)\}\},\(23\)which integrates all supporting and refuting evidence accumulated up to iterationtt\. Let

l∗=arg⁡maxl∈L⁡p^l\(t\),l^\{\\ast\}=\\arg\\max\_\{l\\in L\}\\hat\{p\}\_\{l\}^\{\(t\)\},\(24\)wherel∗l^\{\\ast\}denotes the leading candidate at iterationtt, andl¯∗\\bar\{l\}^\{\\ast\}its opponent\. The debate terminates if

𝒫​\(πl∗\(t\)\>πl¯∗\(t\)\)\>δstop\.\\mathcal\{P\}\\\!\\left\(\\pi\_\{l^\{\\ast\}\}^\{\(t\)\}\>\\pi\_\{\\bar\{l\}^\{\\ast\}\}^\{\(t\)\}\\right\)\>\\delta\_\{\\mathrm\{stop\}\}\.\(25\)If neither criterion is satisfied withinTmaxT\_\{\\max\}rounds, the final winner is selected using the scorerϕ\\phi:

lwinner=arg⁡maxl∈L⁡ϕ​\(s^,Paral\)\.l^\{\\mathrm\{winner\}\}=\\arg\\max\_\{l\\in L\}\\phi\(\\hat\{s\},\\mathrm\{Para\}\_\{l\}\)\.\(26\)Appendix[D](https://arxiv.org/html/2606.04691#A4)provides a Chebyshev\-style bound motivating the dual\-track stopping rule\.

### 2\.5Iterative Entity–Relation Alignment

For JERE, we enforce ontology\-consistent alignment betweenEEandRR, as shown in Figure[4](https://arxiv.org/html/2606.04691#S2.F4)\. At iterationtt, the procedure proceeds as follows\.

##### Entity completion

Given the candidate relation setR¯\(t\)=R​E​\(E\(t\)\)\\bar\{R\}^\{\(t\)\}=RE\(E^\{\(t\)\}\), we add entities that appear inR¯\(t\)\\bar\{R\}^\{\(t\)\}but are missing fromE\(t\)E^\{\(t\)\}after inferring their types:

E¯\(t\+1\)=E\(t\)∪\{\(e,t\)∉E\(t\)∣e∈ξ​\(R¯\(t\)\)\},\\bar\{E\}^\{\(t\+1\)\}=E^\{\(t\)\}\\cup\\\{\\,\(e,t\)\\notin E^\{\(t\)\}\\mid e\\in\\xi\(\\bar\{R\}^\{\(t\)\}\)\\,\\\},\(27\)whereξ​\(⋅\)\\xi\(\\cdot\)denotes the entities involved in the candidate relations\. This step completes missing entities under relational evidence\.

##### Consistency correction

We further enforce ontology consistency onR¯\(t\)\\bar\{R\}^\{\(t\)\}with respect toE¯\(t\+1\)\\bar\{E\}^\{\(t\+1\)\}\. For each violation, the Consistency Agent either revises the conflicting entity type if the relation type is trusted, or blacklists the relation otherwise\. The aligned relations and entities are updated as:

R\(t\)=R¯\(t\)∖B\(t\)\.R^\{\(t\)\}=\\bar\{R\}^\{\(t\)\}\\setminus B^\{\(t\)\}\.\(28\)Entities not involved in any remaining relation are removed:

E\(t\+1\)=\{\(e,t\)∈E¯\(t\+1\)∣e∈ξ​\(R\(t\)\)\}\.E^\{\(t\+1\)\}=\\\{\\,\(e,t\)\\in\\bar\{E\}^\{\(t\+1\)\}\\mid e\\in\\xi\(R^\{\(t\)\}\)\\,\\\}\.\(29\)We regenerate candidate relations from the updated entity set and filter them using the updated blacklist:

R\(t\+1\)=R¯\(t\+1\)∖B\(t\+1\)\.R^\{\(t\+1\)\}=\\bar\{R\}^\{\(t\+1\)\}\\setminus B^\{\(t\+1\)\}\.\(30\)The procedure terminates once all violations are resolved orKJEREK\_\{\\mathrm\{JERE\}\}is reached, yielding the aligned JERE output\.

## 3Experiment and Analysis

This section presents the main experimental results and analyses\. Additional details on datasets, evaluation metrics, baselines, implementation settings, hyperparameters, and prompts are provided in Appendices B–F\.

### 3\.1Main Results

Tables[2](https://arxiv.org/html/2606.04691#S2.T2)and[3](https://arxiv.org/html/2606.04691#S2.T3)report NER and RE results on five benchmarks using GPT\-3\.5\-Turbo\-0125 as the backbone\.SMADE\-IEachieves the best performance across both tasks, demonstrating the effectiveness of multi\-agent deliberation for information extraction\. For NER,SMADE\-IEoutperforms all baselines, with averageF1P\\mathrm\{F1\}\_\{\\mathrm\{P\}\}gains of 14\.67 over AEiO, 19\.68 over One\-Step, 18\.54 over G&O, and 11\.37 overCrossAgentIE\. The gains are larger on datasets with richer type inventories, surpassingCrossAgentIEby 20\.78F1P\\mathrm\{F1\}\_\{\\mathrm\{P\}\}on OntoNotes5, 19\.35 on CrossRE, and 10\.60 on SciERC\. For RE,SMADE\-IEobtains the highestF1P\\mathrm\{F1\}\_\{\\mathrm\{P\}\}andF1S\\mathrm\{F1\}\_\{\\mathrm\{S\}\}scores on all benchmarks, improving averageF1P\\mathrm\{F1\}\_\{\\mathrm\{P\}\}by 3\.83, 7\.18, 10\.48, and 10\.92 over AEiO, One\-Step, G&O, andCrossAgentIE, respectively\. Its largest gain overCrossAgentIEoccurs on SemEval2010 \(\+17\.77F1P\\mathrm\{F1\}\_\{\\mathrm\{P\}\}\), while consistent improvements on DocRED, SciERC, CrossRE, and REDFM indicate strong generalizability across relation extraction settings\.

For the JERE task, which requires joint entity\-relation extraction and cross\-task completion, we primarily compare withCrossAgentIE\. As shown in Table[4](https://arxiv.org/html/2606.04691#S2.T4),SMADE\-IEimproves overCrossAgentIEby 14\.46F1P\\mathrm\{F1\}\_\{\\mathrm\{P\}\}and 14\.07F1S\\mathrm\{F1\}\_\{\\mathrm\{S\}\}on average\. The gains are consistent on both datasets, withF1P\\mathrm\{F1\}\_\{\\mathrm\{P\}\}increasing from 42\.50 to 58\.44 on CoNLL04 and from 17\.23 to 30\.22 on NYT\.

### 3\.2Ablation Studies

Table 5:Ablation results on JERE\. The first block evaluates*IERA*, while the remaining blocks analyze Adaptive Mode Selection, Verification Agent, Review Agent, and Evidence\-Driven Debate\.Table[5](https://arxiv.org/html/2606.04691#S3.T5)presents ablation results on CoNLL04 and NYT, grouped by module functionality\. Removing*Iterative Entity–Relation Alignment*\(*IERA*\) substantially degrades performance on NYT \(F1P\\mathrm\{F1\}\_\{\\mathrm\{P\}\}30\.22→\\rightarrow24\.94,F1S\\mathrm\{F1\}\_\{\\mathrm\{S\}\}29\.22→\\rightarrow22\.47\), with a smaller effect on CoNLL04, indicating that IERA is beneficial when entity–relation inconsistencies are frequent\. Comparisons with the monolithic\-prompt baseline further validate the effectiveness of*Adaptive Mode Selection*\. It consistently improvesF1S\\mathrm\{F1\}\_\{\\mathrm\{S\}\}across datasets, outperforming*Global Extraction Mode*by 4\.54 on CoNLL04 and 14\.95 on NYT, and*Type\-Centric Only*by 4\.41 and 0\.75, respectively\.

Within each mode, ablations highlight the importance of internal components: in*Global Extraction Mode*, the*Verification Agent*boosts precision \(F1P\\mathrm\{F1\}\_\{\\mathrm\{P\}\}drops from 52\.47 → 46\.92 on CoNLL04 when removed\); in*Type\-Centric Mode*,*Relevant Type Selection*,*Review Agent*, and*Evidence\-Driven Debate*\(*Debate*\) all contribute, with the latter particularly critical on NYT \(F1P\\mathrm\{F1\}\_\{\\mathrm\{P\}\}29\.65 → 24\.24,F1S\\mathrm\{F1\}\_\{\\mathrm\{S\}\}28\.47 → 23\.38\) and the former on CoNLL04 \(F1P\\mathrm\{F1\}\_\{\\mathrm\{P\}\}59\.43 → 48\.66\)\. These results confirm that each module plays a complementary role in improving precision and structured F1 across datasets\.

Table 6:Average total token cost per sample across different datasets\.
### 3\.3Token Efficiency

Table[6](https://arxiv.org/html/2606.04691#S3.T6)reports the average total token cost per sample on selected NER and RE datasets\. On CoNLL03,SMADE\-IEuses 1,094 tokens, close to AEiO \(1,078\) and lower than One\-Step \(1,407\), G&O \(2,304\), andCrossAgentIE\(1,508\)\. On REDFM NER,SMADE\-IEalso reduces cost compared with G&O andCrossAgentIE, using 11,514 tokens versus 14,202 and 13,480\. The savings are more pronounced on RE:SMADE\-IEuses 3,240 tokens on DocRED and 3,765 on CrossRE, far belowCrossAgentIE’s 21,784 and 18,923 tokens\. It is also more efficient than One\-Step and G&O on both RE datasets\. These results show that sparse type selection avoids unnecessary full\-schema agent calls, keeping the cost close to lightweight prompting on simple cases while substantially reducing multi\-agent debate overhead on relation\-heavy datasets\.

## 4Conclusion

This paper proposesSMADE\-IE, a sparse and evidence\-driven multi\-agent framework for zero\-shot information extraction\.SMADE\-IEemploys the Adaptive Mode Selector to dynamically route inputs into the lightweight Global Extraction Mode or the Type\-Centric Extraction Mode, reducing redundant agent interactions and reasoning noise\. For conflicting predictions, we further introduce the Evidence\-Driven Debate mechanism with Toulmin\-style argument modeling and Bayesian confidence updates\. In addition, the Iterative Entity–Relation Alignment strategy is designed for joint extraction tasks\. Experimental results on 9 benchmark datasets across NER, RE, and JERE tasks demonstrate thatSMADE\-IEconsistently outperforms existing zero\-shot IE baselines while improving token efficiency\. Future work will explore more efficient debate scheduling and broader structured extraction settings\.

## Limitations

SMADE\-IEhas three main limitations\. First, the Evidence\-Driven Debate relies on a frozen external evidence scorer\. Its NLI calibration upper\-bounds the reliability of Bayesian confidence updates\. Second, although the Adaptive Mode Selector keeps token cost close to monolithic\-prompt baselines on simple inputs, the Type\-Centric Extraction Mode still issues multiple LLM calls per cross\-type conflict, and its efficiency advantage diminishes on long\-document, type\-dense datasets such as REDFM, where invoked debates are only halved rather than reduced by an order of magnitude\. Third, our main experiments are conducted with GPT\-3\.5\-Turbo\-0125 and cross\-backbone evidence is restricted to gemini\-3\-flash\-preview on a subset of datasets, leaving smaller open\-source models, longer documents, and multilingual schemas as natural directions for future work\.

## Ethics Statement

In this work, we proposeSMADE\-IE, a sparse multi\-agent framework designed to improve large language model performance on fundamental information extraction tasks, including named entity recognition and relation extraction\. All our analyses and evaluations are based on standard, publicly available academic datasets\. We do not anticipate any specific ethical issues or negative societal impacts regarding the methodology or the topics of this research\.

## References

- Bassignana and Plank \(2022\)Elisa Bassignana and Barbara Plank\. 2022\.CrossRE: A cross\-domain dataset for relation extraction\.In*Findings of the EMNLP 2022*, pages 3592–3604\.
- Chen et al\. \(2023\)Zhijun Chen, Hailong Sun, Wanhao Zhang, Chunyi Xu, Qianren Mao, and Pengpeng Chen\. 2023\.Neural\-hidden\-crf: A robust weakly\-supervised sequence labeler\.In*Proceedings of the ACM SIGKDD 2023*, page 274–285\. ACM\.
- Choi et al\. \(2026\)Hyeong Kyu Choi, Jerry Zhu, and Sharon Li\. 2026\.Debate or vote: Which yields better decisions in multi\-agent large language models?In*Proceedings of the NIPS 2026*\.
- Fan et al\. \(2026\)Wei Fan, JinYi Yoon, and Bo Ji\. 2026\.imad: Intelligent multi\-agent debate for efficient and accurate llm inference\.*Proceedings of the AAAI 2026*, 40\(35\):29403–29411\.
- Fuente et al\. \(2025\)Neil De La Fuente, Oscar Sainz, Iker García\-Ferrero, and Eneko Agirre\. 2025\.GUIDEX: Guided synthetic data generation for zero\-shot information extraction\.In*Findings of the ACL 2025*, pages 24248–24262\.
- Gao et al\. \(2024\)Chufan Gao, Xulin Fan, Jimeng Sun, and Xuan Wang\. 2024\.PromptRE: Weakly\-supervised document\-level relation extraction via prompting\-based data programming\.In*Proceedings of the 1st Workshop on Towards Knowledgeable Language Models \(KnowLLM 2024\)*, pages 132–145\.
- Guan et al\. \(2025\)Yong Guan, Hao Peng, Lei Hou, and Juanzi Li\. 2025\.MMD\-ERE: Multi\-agent multi\-sided debate for event relation extraction\.In*Proceedings of the COLING 2025*, pages 6889–6896\.
- Gupta et al\. \(2024\)Ankita Gupta, Ethan Zuckerman, and Brendan O’Connor\. 2024\.Harnessing toulmin’s theory for zero\-shot argument explication\.In*Proceedings of the ACL 2024*, pages 10259–10276\.
- Han et al\. \(2024\)Ridong Han, Chaohao Yang, Tao Peng, Prayag Tiwari, Xiang Wan, Lu Liu, and Benyou Wang\. 2024\.[An empirical study on information extraction using large language models](https://arxiv.org/abs/2409.00369)\.*Preprint*, arXiv:2409\.00369\.
- Hendrickx et al\. \(2010\)Iris Hendrickx, Su Nam Kim, Zornitsa Kozareva, Preslav Nakov, Diarmuid Ó Séaghdha, Sebastian Padó, Marco Pennacchiotti, Lorenza Romano, and Stan Szpakowicz\. 2010\.SemEval\-2010 task 8: Multi\-way classification of semantic relations between pairs of nominals\.In*Proceedings of the 5th International Workshop on Semantic Evaluation*, pages 33–38\.
- Hu et al\. \(2025\)Tianyu Hu, Zhen Tan, Song Wang, Huaizhi Qu, and Tianlong Chen\. 2025\.Multi\-agent debate for llm judges with adaptive stability detection\.In*Advances in Neural Information Processing Systems*, pages 46504–46540\.
- Huguet Cabot et al\. \(2023\)Pere\-Lluís Huguet Cabot, Simone Tedeschi, Axel\-Cyrille Ngonga Ngomo, and Roberto Navigli\. 2023\.REDfm\{\}^\{\\textrm\{fm\}\}: A filtered and multilingual relation extraction dataset\.In*Proceedings of the ACL 2023*, pages 4326–4343\.
- Jain \(2020\)Nitisha Jain\. 2020\.Domain\-specific knowledge graph construction for semantic analysis\.In*Proceedings of the Semantic Web: ESWC 2020*, pages 250–260\.
- Kambhatla \(2004\)Nanda Kambhatla\. 2004\.Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations\.In*Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics \(ACL\) Companion Volume*, pages 22–25\.
- Lafferty et al\. \(2001\)John D\. Lafferty, Andrew McCallum, and Fernando C\. N\. Pereira\. 2001\.Conditional random fields: Probabilistic models for segmenting and labeling sequence data\.In*Proceedings of the Eighteenth International Conference on Machine Learning \(ICML\)*, pages 282–289\.
- Li et al\. \(2023a\)Guozheng Li, Peng Wang, and Wenjun Ke\. 2023a\.Revisiting large language models as zero\-shot relation extractors\.In*Findings of the EMNLP 2023*, pages 6877–6892\.
- Li et al\. \(2023b\)Yinghao Li, Colin Lockard, Prashant Shiralkar, and Chao Zhang\. 2023b\.Extracting shopping interest\-related product types from the web\.In*Findings of the ACL 2023*, pages 7509–7525\.
- Li et al\. \(2024\)Yinghao Li, Rampi Ramprasad, and Chao Zhang\. 2024\.A simple but effective approach to improve structured language model output for information extraction\.In*Findings of the EMNLP 2024*, pages 5133–5148\.
- Li et al\. \(2022\)Yinghao Li, Le Song, and Chao Zhang\. 2022\.Sparse conditional hidden markov model for weakly supervised named entity recognition\.In*Proceedings of the ACM SIGKDD 2022*, page 978–988\.
- Lison et al\. \(2021\)Pierre Lison, Jeremy Barnes, and Aliaksandr Hubin\. 2021\.skweak: Weak supervision made easy for nlp\.In*Proceedings of the ACL 2021*, page 337–346\.
- Liu et al\. \(2020\)Jian Liu, Yubo Chen, Kang Liu, Wei Bi, and Xiaojiang Liu\. 2020\.Event extraction as machine reading comprehension\.In*Proceedings of the EMNLP 2020*, pages 1641–1651\.
- Liu et al\. \(2024\)Nelson F\. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang\. 2024\.Lost in the middle: How language models use long contexts\.*Transactions of the Association for Computational Linguistics*, 12:157–173\.
- Lou et al\. \(2023\)Jie Lou, Yaojie Lu, Dai Dai, Wei Jia, Hongyu Lin, Xianpei Han, Le Sun, and Hua Wu\. 2023\.Universal information extraction as unified semantic matching\.In*Proceedings of the AAAI 2023*, pages 13318–13326\.
- Lu et al\. \(2025\)Meng Lu, Yuzhang Xie, Zhenyu Bi, Shuxiang Cao, and Xuan Wang\. 2025\.CrossAgentIE: Cross\-type and cross\-task multi\-agent LLM collaboration for zero\-shot information extraction\.In*Findings of the ACL 2025*, pages 13953–13977\.
- Lu et al\. \(2022\)Yaojie Lu, Qing Liu, Dai Dai, Xinyan Xiao, Hongyu Lin, Xianpei Han, Le Sun, and Hua Wu\. 2022\.Unified structure generation for universal information extraction\.In*Proceedings of the ACL 2022*, pages 5755–5772\.
- Luan et al\. \(2018\)Yi Luan, Luheng He, Mari Ostendorf, and Hannaneh Hajishirzi\. 2018\.Multi\-task identification of entities, relations, and coreference for scientific knowledge graph construction\.In*Proceedings of the EMNLP 2018*, pages 3219–3232\.
- Luo et al\. \(2025\)Yujie Luo, Xiangyuan Ru, Kangwei Liu, Lin Yuan, Mengshu Sun, Ningyu Zhang, Lei Liang, Zhiqiang Zhang, Jun Zhou, Lanning Wei, and 1 others\. 2025\.Oneke: A dockerized schema\-guided llm agent\-based knowledge extraction system\.In*Companion Proceedings of the ACM WWW 2025*, pages 2871–2874\.
- Mahajan et al\. \(2026\)Pranav Mahajan, Ihor Kendiukhov, Syed Hussain, and Lydia Nottingham\. 2026\.[Mind the gap: How elicitation protocols shape the stated\-revealed preference gap in language models](https://arxiv.org/abs/2601.21975)\.*Preprint*, arXiv:2601\.21975\.
- McCallum and Li \(2003\)Andrew McCallum and Wei Li\. 2003\.Early results for named entity recognition with conditional random fields, feature induction and web\-enhanced lexicons\.In*Proceedings of the Seventh Conference on Natural Language Learning \(CoNLL\)*, pages 188–191\.
- Pradhan et al\. \(2013\)Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Hwee Tou Ng, Anders Björkelund, Olga Uryupina, Yuchen Zhang, and Zhi Zhong\. 2013\.Towards robust linguistic analysis using OntoNotes\.In*Proceedings of the CoNLL 2013*, pages 143–152\.
- Riedel et al\. \(2010\)Sebastian Riedel, Limin Yao, and Andrew McCallum\. 2010\.Modeling relations and their mentions without labeled text\.In*Proceedings of the ECML\-PKDD 2010*, pages 148–163\.
- Rink and Harabagiu \(2010\)Bryan Rink and Sanda Harabagiu\. 2010\.UTD: Classifying semantic relations by combining machine learning and rich linguistic features\.In*Proceedings of the 5th International Workshop on Semantic Evaluation \(SemEval\)*, pages 256–259\.
- Roth and Yih \(2004\)Dan Roth and Wen\-tau Yih\. 2004\.A linear programming formulation for global inference in natural language tasks\.In*Proceedings of the NAACL 2004*, pages 1–8\.
- Sainz et al\. \(2024\)Oscar Sainz, Iker García\-Ferrero, Rodrigo Agerri, Oier Lacalle, German Rigau, and Eneko Agirre\. 2024\.Gollie: Annotation guidelines improve zero\-shot information\-extraction\.In*Proceedings of the ICLR 2024*, pages 47083–47107\.
- Shi et al\. \(2023\)Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H\. Chi, Nathanael Schärli, and Denny Zhou\. 2023\.Large language models can be easily distracted by irrelevant context\.In*Proceedings of the ICML 2023*, pages 31210–31227\.
- Shi et al\. \(2024\)Yuchen Shi, Guochao Jiang, Tian Qiu, and Deqing Yang\. 2024\.Agentre: An agent\-based framework for navigating complex information landscapes in relation extraction\.In*Proceedings of the ACM CIKM 2024*, page 2045–2055\.
- Tian et al\. \(2026\)Hongduan Tian, Xiao Feng, Rolan Yan, Bo Han, and 1 others\. 2026\.Multi\-agent debate with memory masking\.In*The Fourteenth International Conference on Learning Representations*\.
- Tjong Kim Sang and De Meulder \(2003\)Erik F\. Tjong Kim Sang and Fien De Meulder\. 2003\.Introduction to the CoNLL\-2003 shared task: Language\-independent named entity recognition\.In*Proceedings of the NAACL 2003*, pages 142–147\.
- Wang and Huang \(2024\)Sijia Wang and Lifu Huang\. 2024\.Debate as optimization: Adaptive conformal prediction and diverse retrieval for event extraction\.In*Findings of the EMNLP 2024*, pages 16422–16435\.
- Wang et al\. \(2025\)Zihan Wang, Ziqi Zhao, Yougang Lyu, Zhumin Chen, Maarten de Rijke, and Zhaochun Ren\. 2025\.A cooperative multi\-agent framework for zero\-shot named entity recognition\.In*Proceedings of the WWW 2025*, pages 4183–4195\.
- Xiao et al\. \(2024\)Ruiyu Xiao, Lei Wu, Yuhang Gou, Weinan Zhang, and Ting Liu\. 2024\.Prove your point\!: Bringing proof\-enhancement principles to argumentative essay generation\.In*EMNLP 2024*, pages 18995–19008\.
- Xie et al\. \(2023\)Tingyu Xie, Qi Li, Jian Zhang, Yan Zhang, Zuozhu Liu, and Hongwei Wang\. 2023\.Empirical study of zero\-shot NER with ChatGPT\.In*Proceedings of the EMNLP 2023*, pages 7935–7956\.
- Xu et al\. \(2024\)Derong Xu, Wei Chen, Wenjun Peng, Chao Zhang, Tong Xu, Xiangyu Zhao, Xian Wu, Yefeng Zheng, Yang Wang, and Enhong Chen\. 2024\.Large language models for generative information extraction: A survey\.*Frontiers of Computer Science*, 18\(6\):186357\.
- Yang et al\. \(2025\)Minglai Yang, Ethan Huang, Liang Zhang, Mihai Surdeanu, William Yang Wang, and Liangming Pan\. 2025\.How is llm reasoning distracted by irrelevant context? an analysis using a controlled benchmark\.In*Proceedings of the EMNLP 2025*, pages 13340–13358\.
- Yao et al\. \(2019\)Yuan Yao, Deming Ye, Peng Li, Xu Han, Yankai Lin, Zhenghao Liu, Zhiyuan Liu, Lixin Huang, Jie Zhou, and Maosong Sun\. 2019\.DocRED: A large\-scale document\-level relation extraction dataset\.In*Proceedings of the ACL 2019*, pages 764–777\.
- Zeng et al\. \(2014\)Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, and Jun Zhao\. 2014\.Relation classification via convolutional deep neural network\.In*Proceedings of the COLING 2014*, pages 2335–2344\.
- Zha et al\. \(2023\)Yuheng Zha, Yichi Yang, Ruichen Li, and Zhiting Hu\. 2023\.AlignScore: Evaluating factual consistency with a unified alignment function\.In*Proceedings of the ACL 2023*, pages 11328–11348\.
- Zhang et al\. \(2025a\)Duzhen Zhang, Zhong\-Zhi Li, Ming\-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Xiuyi Chen, Yingying Zhang, and 1 others\. 2025a\.From system 1 to system 2: A survey of reasoning large language models\.*IEEE Transactions on Pattern Analysis and Machine Intelligence*\.
- Zhang et al\. \(2025b\)Zikang Zhang, Wangjie You, Tianci Wu, Xinrui Wang, Juntao Li, and Min Zhang\. 2025b\.A survey of generative information extraction\.In*Proceedings of the COLING 2025*, pages 4840–4870\.
- Zheng et al\. \(2017\)Suncong Zheng, Feng Wang, Hongyun Bao, Yuexing Hao, Peng Zhou, and Bo Xu\. 2017\.Joint extraction of entities and relations based on a novel tagging scheme\.In*Proceedings of the ACL 2017*, pages 1227–1236\.

## Appendix ARelated Works

### A\.1Supervision\-based Information Extraction

Early IE methods mainly rely on supervised learning, typically modeling NER as sequence labeling and RE as classification or structured prediction\(Lafferty et al\.,[2001](https://arxiv.org/html/2606.04691#bib.bib15); McCallum and Li,[2003](https://arxiv.org/html/2606.04691#bib.bib29); Kambhatla,[2004](https://arxiv.org/html/2606.04691#bib.bib14); Rink and Harabagiu,[2010](https://arxiv.org/html/2606.04691#bib.bib32)\)\. Neural IE methods reduce reliance on manual feature engineering by learning distributed representations and contextual encoders for sequence labeling, relation classification, and joint extraction\(Zeng et al\.,[2014](https://arxiv.org/html/2606.04691#bib.bib46); Zheng et al\.,[2017](https://arxiv.org/html/2606.04691#bib.bib50)\)\. More recently, pretrained\-language\-model fine\-tuning and instruction\-tuning methods have further improved IE performance under supervised and transfer settings\(Lu et al\.,[2022](https://arxiv.org/html/2606.04691#bib.bib25); Lou et al\.,[2023](https://arxiv.org/html/2606.04691#bib.bib23)\)\. However, these methods usually assume sufficient annotated data, stable domains, and fixed entity and relation schemas, making them difficult to adapt to annotation\-scarce settings or dynamically changing schemas\. Weakly and distantly supervised methods reduce annotation cost by using unlabeled corpora, external knowledge bases, or heuristic labeling functions\(Li et al\.,[2022](https://arxiv.org/html/2606.04691#bib.bib19); Chen et al\.,[2023](https://arxiv.org/html/2606.04691#bib.bib2); Gao et al\.,[2024](https://arxiv.org/html/2606.04691#bib.bib6)\), but they still require additional supervision signals or manual design efforts and may suffer from noisy pseudo\-labels\(Lison et al\.,[2021](https://arxiv.org/html/2606.04691#bib.bib20)\)\. In contrast, this work focuses on zero\-shot IE, where models perform extraction using only input text, type definitions, and output constraints\. While zero\-shot IE poses greater challenges than fully or semi\-supervised methods, it offers superior adaptability to a dynamically changing world without requiring any additional fine\-tuning\.

### A\.2LLM\-based Zero\-shot Information Extraction

Recent studies have explored the use of LLMs for zero\-shot IE by prompting models with task instructions, type definitions, or output schemas\(Xu et al\.,[2024](https://arxiv.org/html/2606.04691#bib.bib43); Zhang et al\.,[2025b](https://arxiv.org/html/2606.04691#bib.bib49)\)\.*Monolithic\-prompt*methods extract all elements in a single pass, offering high efficiency but often suffering from type confusion, boundary errors, and missed mentions as the schema size increases\(Xie et al\.,[2023](https://arxiv.org/html/2606.04691#bib.bib42); Li et al\.,[2024](https://arxiv.org/html/2606.04691#bib.bib18); Han et al\.,[2024](https://arxiv.org/html/2606.04691#bib.bib9)\)\. In contrast,*each\-type prompt*methods query entity or relation types separately, enabling more focused extraction but frequently producing overlapping or conflicting outputs across prompts\(Li et al\.,[2023a](https://arxiv.org/html/2606.04691#bib.bib16),[2024](https://arxiv.org/html/2606.04691#bib.bib18)\)\.

To address such cross\-type inconsistencies,*multi\-agent debate*methods instantiate type\-specific agents and resolve conflicting predictions through inter\-agent deliberation\. Related research has further explored multi\-agent collaboration, tool use, knowledge augmentation, and workflow orchestration for IE\(Wang and Huang,[2024](https://arxiv.org/html/2606.04691#bib.bib39); Shi et al\.,[2024](https://arxiv.org/html/2606.04691#bib.bib36); Wang et al\.,[2025](https://arxiv.org/html/2606.04691#bib.bib40); Luo et al\.,[2025](https://arxiv.org/html/2606.04691#bib.bib27)\)\. For example,CrossAgentIE\(Lu et al\.,[2025](https://arxiv.org/html/2606.04691#bib.bib24)\)assigns dedicated agents to entity and relation types and resolves structured prediction conflicts through cross\-type and cross\-task debate\. However, these approaches select the full type schema for every input, causing token cost and contextual noise to scale with schema size\. Moreover, their free\-form debate mechanisms lack explicit modeling of disputed claims, evidence constraints, and confidence updates, which can lead to unreliable adjudication\. In contrast,SMADE\-IEselects only Router\-selected types and replaces free\-form deliberation with a Toulmin\-structured, evidence\-scored Bayesian debate framework\.

StatisticCoNLL03OntoNotes5CoNLL04NYTCrossREREDFMSciERC\# Selected Instances300200288369200446269Avg\. Sequence Length13182936377623Avg\. Entities per Instance1\.501\.312\.382\.004\.723\.572\.19Entity Types3183338105Table 7:Statistics of the NER benchmarks used in our evaluation\. “\# Selected Instances” denotes the number of evaluation samples used in our experiments, “Avg\. Sequence Length” indicates the average number of tokens per instance, “Avg\. Entities per Instance” refers to the average number of entity mentions in each sample and “Entity Types” refers to the number of entity categories evaluated in each dataset\.StatisticDocREDSemEval2010CoNLL04NYTCrossREREDFMSciERC\# Selected Instances200200288369200446269Avg\. Sequence Length30172936377623Avg\. Relations per Instance2\.460\.841\.410\.725\.052\.771\.88Relation Types3295717328Table 8:Statistics of the relation extraction \(RE\) benchmarks used in our evaluation\. “\# Selected Instances” denotes the number of evaluation samples used in our experiments, “Avg\. Sequence Length” indicates the average number of tokens per instance, “Avg\. Relations per Instance” refers to the average number of relation mentions in each sample, and “Relation Types” is reported as selected relation types / total ontology relation types when only a subset of relations is evaluated\. All evaluated relations are binary\.

## Appendix BExperimental Setup

### B\.1Datasets and Evaluation Metrics

We evaluateSMADE\-IEon 12 zero\-shot IE task settings over 9 benchmark datasets, spanning NER, RE, and JERE across general\-domain, scientific, and document\-level text\. For NER, we use CoNLL03\(Tjong Kim Sang and De Meulder,[2003](https://arxiv.org/html/2606.04691#bib.bib38)\), OntoNotes5\(Pradhan et al\.,[2013](https://arxiv.org/html/2606.04691#bib.bib30)\), SciERC\(Luan et al\.,[2018](https://arxiv.org/html/2606.04691#bib.bib26)\), CrossRE\(Bassignana and Plank,[2022](https://arxiv.org/html/2606.04691#bib.bib1)\), and REDFM\(Huguet Cabot et al\.,[2023](https://arxiv.org/html/2606.04691#bib.bib12)\)\. For RE, we evaluate on DocRED\(Yao et al\.,[2019](https://arxiv.org/html/2606.04691#bib.bib45)\), SemEval\-2010 Task 8\(Hendrickx et al\.,[2010](https://arxiv.org/html/2606.04691#bib.bib10)\), and reuse SciERC, CrossRE, and REDFM\. For JERE, we use CoNLL04\(Roth and Yih,[2004](https://arxiv.org/html/2606.04691#bib.bib33)\)and NYT\(Riedel et al\.,[2010](https://arxiv.org/html/2606.04691#bib.bib31)\)\. During inference, the model is provided only with natural\-language ontology definitions as external knowledge, following the zero\-shot evaluation protocol ofLi et al\. \([2024](https://arxiv.org/html/2606.04691#bib.bib18)\)andLu et al\. \([2025](https://arxiv.org/html/2606.04691#bib.bib24)\)\.

For datasets whose ontologies contain rare, ambiguous, or under\-specified labels, we follow the evaluation settings adopted by prior work\(Li et al\.,[2024](https://arxiv.org/html/2606.04691#bib.bib18)\)\. Specifically, CoNLL03 and CoNLL04 retain only the three concrete entity types\. CrossRE retains 38 out of 39 entity types after removing theOthercategory and uses all 17 semantic relation types\. REDFM and SciERC exclude only reserved generic labels\. DocRED is restricted to the 32 most frequent relation types, while SemEval is limited to the 9 forward relation types\. Detailed dataset entity\-level statistics shown in Table[7](https://arxiv.org/html/2606.04691#A1.T7)and relation\-level statistics in Table[8](https://arxiv.org/html/2606.04691#A1.T8)\. For datasets containing more than a few hundred test instances, we randomly sample a fixed evaluation subset shared across all baselines and ablation settings to ensure fair and directly comparable evaluations of token cost and F1 performance\.

We report micro Partial F1 \(F1P\\mathrm\{F1\}\_\{\\mathrm\{P\}\}\) and Strict F1 \(F1S\\mathrm\{F1\}\_\{\\mathrm\{S\}\}\), together with averages across datasets\. Partial F1 allows boundary overlap between predicted and gold spans, whereas Strict F1 requires exact span matching\.

### B\.2Baselines

We compareSMADE\-IEagainst four representative zero\-shot LLM\-based IE methods spanning three paradigms\. For*monolithic prompting*,All\-Entity\-in\-One \(AEiO\)\(Li et al\.,[2024](https://arxiv.org/html/2606.04691#bib.bib18)\)extracts all candidate types in a single prompt\. For*each\-type prompting*,One\-Step\(Li et al\.,[2024](https://arxiv.org/html/2606.04691#bib.bib18)\)issues a dedicated prompt for each entity or relation type, whileGeneration and Organization \(G&O\)\(Li et al\.,[2024](https://arxiv.org/html/2606.04691#bib.bib18)\)decouples content generation from output organization\. For*multi\-agent debate*,CrossAgentIE\(Lu et al\.,[2025](https://arxiv.org/html/2606.04691#bib.bib24)\)instantiates type\-specific agents and resolves conflicts through cross\-type and cross\-task debate\.

### B\.3Implementation Details

All LLM\-based methods use GPT\-3\.5\-Turbo\-0125 in a zero\-shot setting, receiving only the input text, task schema, and type definitions\. Agent interactions are implemented with AutoGen††[https://microsoft\.github\.io/autogen/](https://microsoft.github.io/autogen/), and the external evidence scorerϕ\\phiuses the AlignScore\-base checkpoint\(Zha et al\.,[2023](https://arxiv.org/html/2606.04691#bib.bib47)\)††[https://huggingface\.co/yzha/AlignScore](https://huggingface.co/yzha/AlignScore)deployed locally\. ForSMADE\-IE, the maximum number of debate rounds is three, withδstop=0\.75\\delta\_\{\\mathrm\{stop\}\}=0\.75,ε=0\.02\\varepsilon=0\.02, andγ=8\\gamma=8\. All hyperparameters are fixed across the twelve evaluation settings; the complete configuration is reported in Table[9](https://arxiv.org/html/2606.04691#A2.T9)\.

For baselines, we use the official implementations of AEiO, One\-Step, and G&O\. We extend AEiO to RE by retaining its all\-in\-one chat structure and replacing only the output schema: a single GPT\-3\.5\-Turbo\-0125 session jointly extracts relation mentions, prunes ungrounded ones, and emits per\-type Markdown tables with head, tail, and validity columns\. Outputs are parsed by normalizing relation\-type headers, discarding rows whose validity token is not “yes” or whose head and tail coincide, and mapping the rest to token spans via the spaCy and locate\-phrase routines from the G&O codebase\.

CrossAgentIE\(Lu et al\.,[2025](https://arxiv.org/html/2606.04691#bib.bib24)\)has no public code and is re\-implemented on AutoGen with the same backbone\. Each type is bound to a Type\-Specific Agent that emits per\-mention extractions with confidence scores via the original\#\#\# head \#\#\#/@@@ tail @@@markers\. When a head–tail pair is claimed by more than one type, the top\-two claimants by confidence enter aGroupChatdebate of up to two rounds, settled once the leader\-runner\-up gap exceeds0\.10\.1; a Judge Agent breaks intra\-group ties and a Consistency Agent reconciles cross\-task conflicts\. For joint extraction, the Consistency Agent runs one additional reassessment that drops or retypes flagged entities when its confidence falls below0\.750\.75\. All agents use temperature0\.90\.9and frequency penalty0\.10\.1\.

Table 9:Hyperparameters ofSMADE\-IE, held fixed across the twelve evaluation settings\. Symbol names follow §[2\.4\.3](https://arxiv.org/html/2606.04691#S2.SS4.SSS3)and §[2\.5](https://arxiv.org/html/2606.04691#S2.SS5)\.Table 10:Backbone generalization on gemini\-3\-flash\-preview\. “–” indicates that the corresponding method is not supported by this task\.

## Appendix CAnalysis and Discussion

### C\.1Backbone Generalization

Table[10](https://arxiv.org/html/2606.04691#A2.T10)evaluates whether the gains transfer to gemini\-3\-flash\-preview\.SMADE\-IEachieves the bestF1P\\mathrm\{F1\}\_\{\\mathrm\{P\}\}andF1S\\mathrm\{F1\}\_\{\\mathrm\{S\}\}on OntoNotes5, the bestF1S\\mathrm\{F1\}\_\{\\mathrm\{S\}\}on SemEval2010, and the best JERE results on CoNLL04, outperformingCrossAgentIEon CoNLL04 by 7\.48F1P\\mathrm\{F1\}\_\{\\mathrm\{P\}\}and 4\.84F1S\\mathrm\{F1\}\_\{\\mathrm\{S\}\}\. These results indicate that the framework generalizes across LLM backbones\.

![Refer to caption](https://arxiv.org/html/2606.04691v1/x5.png)

![Refer to caption](https://arxiv.org/html/2606.04691v1/x6.png)

Figure 5:Early\-stopping reason distributions ofSMADE\-IEon NER and RE datasets\.![Refer to caption](https://arxiv.org/html/2606.04691v1/x7.png)Figure 6:Average number of debate rounds under a maximum budget of 3\.
### C\.2Effectiveness of Early Stopping

Figures[5](https://arxiv.org/html/2606.04691#A3.F5)and[6](https://arxiv.org/html/2606.04691#A3.F6)report the early\-stopping behavior ofSMADE\-IE\. Confidence convergence is the dominant stopping reason, accounting for 60\.0%–88\.2% of early stops on NER and 74\.5%–88\.9% on most RE datasets, whereas decisive stopping is more prominent on CoNLL03 \(40\.0%\) and SemEval2010 \(50\.0%\)\. The*Other*category corresponds to cases in which the Ground and Warrant components of one or both debate agents are terminated\. Consistent with these distributions,SMADE\-IErequires only 1\.75–2\.76 rounds on average under a maximum budget of three\. Simpler datasets such as CoNLL03 and SemEval2010 stop earliest at 1\.80 and 1\.75 rounds, while the more ambiguous CrossRE requires 2\.76 rounds for NER and 2\.43 for RE\. These results show that early stopping suppresses redundant debate while preserving additional rounds for harder cases\.

![Refer to caption](https://arxiv.org/html/2606.04691v1/x8.png)Figure 7:Debate and redundant\-debate counts ofCrossAgentIEandSMADE\-IEon NER and RE datasets\. Redundant debates refer to debates where the gold type does not exist among the proposed candidate types\.
### C\.3Debate Efficiency

Figure[7](https://arxiv.org/html/2606.04691#A3.F7)reports a count\-level analysis of the source of token savings\. Across the ten NER and RE settings,SMADE\-IEreduces the total number of debates from 4,766 to 660\. For NER, substantial reductions are observed on every dataset: from 197 to 10 on CoNLL03, from 322 to 24 on OntoNotes5, from 410 to 51 on CrossRE, and from 519 to 22 on SciERC\. REDFM remains the most challenging setting, yet the debate count is still nearly halved, from 860 to 431\. A similar trend holds for RE, where the number of debates decreases from 394 to 12 on DocRED, from 398 to 4 on SemEval2010, from 384 to 42 on CrossRE, from 887 to 55 on REDFM, and from 395 to 9 on SciERC\. Redundant debates are likewise reduced from 3,100 to 412 overall, indicating that Router\-based type selection eliminates most unnecessary comparisons rather than merely shortening informative ones\.

![Refer to caption](https://arxiv.org/html/2606.04691v1/x9.png)Figure 8:Hyperparameter sensitivity on OntoNotes5\. We varyδstop\\delta\_\{\\mathrm\{stop\}\},ε\\varepsilon, andγ\\gammaaround their default values, where the default configuration \(δstop=0\.75\\delta\_\{\\mathrm\{stop\}\}=0\.75,ε=0\.02\\varepsilon=0\.02,γ=8\\gamma=8\) achieves the best overall F1\.
### C\.4Hyperparameter Sensitivity

Figure[8](https://arxiv.org/html/2606.04691#A3.F8)reports hyperparameter sensitivity on OntoNotes5\. The default configuration achieves the best overall result with 66\.08 partial\-match F1 and 52\.05 full\-match F1\. Varyingδstop\\delta\_\{\\mathrm\{stop\}\}produces only moderate fluctuation: partial\-match F1 stays within 63\.85–66\.08 and full\-match F1 within 49\.07–52\.85, indicating that the posterior\-superiority threshold is robust\. The convergence thresholdε\\varepsilonis more influential: the default value 0\.02 outperforms both stricter and looser settings, with partial\-match F1 dropping to 62\.38 at 0\.01 and 61\.82 at 0\.1\. The parameterγ\\gammaexhibits the clearest trade\-off\. A small value of 3 under\-allocates debate and reduces F1 to 60\.30/45\.79, while an overly large value of 12 also hurts performance, reaching only 62\.20/45\.03\. These results suggest that the default configuration balances evidence accumulation and early stopping, and thatSMADE\-IEremains robust under moderate perturbations\.

## Appendix DPosterior Superiority Bound for Dual\-Track Termination

In this section, we derive the Chebyshev\-type upper bound used in §[2\.4\.3](https://arxiv.org/html/2606.04691#S2.SS4.SSS3)and show how it motivates two complementary stopping criteria, posterior superiority and confidence convergence, which jointly govern the dual\-track termination mechanism\.

### D\.1Setup and Notation

Consider two competing candidates with independent Beta posteriorspi∼Beta​\(αi,βi\)p\_\{i\}\\sim\\mathrm\{Beta\}\(\\alpha\_\{i\},\\beta\_\{i\}\)fori∈\{1,2\}i\\in\\\{1,2\\\}\. Their posterior mean and evidence mass are

p^i≜αiαi\+βi,Ni≜αi\+βi,\\hat\{p\}\_\{i\}\\triangleq\\frac\{\\alpha\_\{i\}\}\{\\alpha\_\{i\}\+\\beta\_\{i\}\},\\qquad N\_\{i\}\\triangleq\\alpha\_\{i\}\+\\beta\_\{i\},and the variance ofpip\_\{i\}is

σi2≜Var​\(pi\)=p^i​\(1−p^i\)Ni\+1\.\\sigma\_\{i\}^\{2\}\\triangleq\\mathrm\{Var\}\(p\_\{i\}\)=\\frac\{\\hat\{p\}\_\{i\}\(1\-\\hat\{p\}\_\{i\}\)\}\{N\_\{i\}\+1\}\.Assume candidate11currently has the larger posterior mean, i\.e\.,

Δ≜p^1−p^2\>0\.\\Delta\\triangleq\\hat\{p\}\_\{1\}\-\\hat\{p\}\_\{2\}\>0\.\(31\)
Our goal is to bound the probability of incorrect selection,ℙ​\(p2\>p1\)\\mathbb\{P\}\(p\_\{2\}\>p\_\{1\}\), namely the probability that the current runner\-up is in fact better than the leader\. This quantity is the complement of the posterior\-superiority probability𝒫​\(p1\>p2\)\\mathcal\{P\}\(p\_\{1\}\>p\_\{2\}\)used in §[2\.4\.3](https://arxiv.org/html/2606.04691#S2.SS4.SSS3)\.

### D\.2Chebyshev Bound on the Confidence Gap

LetD≜p1−p2D\\triangleq p\_\{1\}\-p\_\{2\}\. Independence ofp1p\_\{1\}andp2p\_\{2\}gives

𝔼​\[D\]=Δ,Var​\(D\)=σ12\+σ22\.\\mathbb\{E\}\[D\]=\\Delta,\\qquad\\mathrm\{Var\}\(D\)=\\sigma\_\{1\}^\{2\}\+\\sigma\_\{2\}^\{2\}\.The event\{p2\>p1\}\\\{p\_\{2\}\>p\_\{1\}\\\}implies\{\|D−Δ\|≥Δ\}\\\{\|D\-\\Delta\|\\geq\\Delta\\\}\. Applying Chebyshev’s inequality yields

ℙ​\(p2\>p1\)\\displaystyle\\mathbb\{P\}\(p\_\{2\}\>p\_\{1\}\)≤Var​\(D\)Δ2\\displaystyle\\leq\\frac\{\\mathrm\{Var\}\(D\)\}\{\\Delta^\{2\}\}\(32\)=1Δ2\[p^1​\(1−p^1\)N1\+1\\displaystyle=\\frac\{1\}\{\\Delta^\{2\}\}\\bigg\[\\frac\{\\hat\{p\}\_\{1\}\(1\-\\hat\{p\}\_\{1\}\)\}\{N\_\{1\}\+1\}\+p^2​\(1−p^2\)N2\+1\]\.\\displaystyle\\quad\+\\frac\{\\hat\{p\}\_\{2\}\(1\-\\hat\{p\}\_\{2\}\)\}\{N\_\{2\}\+1\}\\bigg\]\.The bound decreases as either the posterior gap increases or more evidence is accumulated\.

### D\.3Balanced\-Evidence Special Case

When the two candidates have matched evidence mass,N1=N2≜NN\_\{1\}=N\_\{2\}\\triangleq N, define the average posterior mean asμ≜\(p^1\+p^2\)/2\\mu\\triangleq\(\\hat\{p\}\_\{1\}\+\\hat\{p\}\_\{2\}\)/2, so thatp^i=μ±Δ/2\\hat\{p\}\_\{i\}=\\mu\\pm\\Delta/2\. The numerator in Eq\. \([32](https://arxiv.org/html/2606.04691#A4.E32)\) then becomes

f​\(μ\)\\displaystyle f\(\\mu\)=p^1​\(1−p^1\)\+p^2​\(1−p^2\)\\displaystyle=\\hat\{p\}\_\{1\}\(1\-\\hat\{p\}\_\{1\}\)\+\\hat\{p\}\_\{2\}\(1\-\\hat\{p\}\_\{2\}\)\(33\)=−2​μ2\+2​μ−Δ22\.\\displaystyle=\-2\\mu^\{2\}\+2\\mu\-\\tfrac\{\\Delta^\{2\}\}\{2\}\.This concave quadratic attains its maximum atμ=1/2\\mu=1/2, yielding the worst\-case valuef​\(1/2\)=1−Δ22f\(1/2\)=\\frac\{1\-\\Delta^\{2\}\}\{2\}\. Substituting this result into Eq\. \([32](https://arxiv.org/html/2606.04691#A4.E32)\) gives the symmetric bound

ℙ​\(p2\>p1\)≤12​\(N\+1\)​\(1Δ2−1\)\.\\mathbb\{P\}\(p\_\{2\}\>p\_\{1\}\)\\leq\\frac\{1\}\{2\(N\+1\)\}\\\!\\left\(\\frac\{1\}\{\\Delta^\{2\}\}\-1\\right\)\.\(34\)Under balanced evidence, the probability of incorrect selection is jointly controlled by two observable quantities: the posterior mean gapΔ\\Deltaand the evidence massNN\.

### D\.4Implications for Dual\-Track Stopping

Eq\. \([34](https://arxiv.org/html/2606.04691#A4.E34)\) reveals two complementary regimes under which the upper bound becomes small\. Each regime corresponds to one stopping signal used in §[2\.4\.3](https://arxiv.org/html/2606.04691#S2.SS4.SSS3)\.

Posterior superiority\.When the leading candidate is well separated from the alternatives,Δ\\Deltabecomes large and the bound in Eq\. \([34](https://arxiv.org/html/2606.04691#A4.E34)\) decreases accordingly\. The posterior\-superiority criterion of Eq\. \([25](https://arxiv.org/html/2606.04691#S2.E25)\) fires exactly in this regime: once𝒫​\(πl∗\(t\)\>πl¯∗\(t\)\)\>δstop\\mathcal\{P\}\(\\pi\_\{l^\{\\ast\}\}^\{\(t\)\}\>\\pi\_\{\\bar\{l\}^\{\\ast\}\}^\{\(t\)\}\)\>\\delta\_\{\\mathrm\{stop\}\}, the probability of incorrect selection is sufficiently suppressed, making additional debate unlikely to overturn the current decision\.

Confidence convergence\.When candidate types are semantically similar,Δ\\Deltamay remain small even after multiple rounds\. In this regime, the bound can only be further reduced through additional informative evidence\. The validity\-decay rule in Eq\. \([18](https://arxiv.org/html/2606.04691#S2.E18)\) progressively downweights repeatedly weakened components, causing the posterior updates accumulated by Eq\. \([19](https://arxiv.org/html/2606.04691#S2.E19)\) to diminish over time\. Consequently, once the posteriors become nearly unchanged across consecutive rounds, further debate contributes little additional information\. This stabilization is exactly what the squared\-Hellinger criterion of Eq\. \([22](https://arxiv.org/html/2606.04691#S2.E22)\) measures: terminating onceΔconf\(t\)<ε\\Delta\_\{\\mathrm\{conf\}\}^\{\(t\)\}<\\varepsilonmatches the regime in which Eq\. \([34](https://arxiv.org/html/2606.04691#A4.E34)\) cannot be tightened further without new evidence\.

Together, posterior superiority addresses the regime of a dominant candidate, while confidence convergence captures the regime of diminishing posterior updates\. The two signals jointly control the probability of incorrect selection without requiring a fixed debate budget\.

## Appendix ECase Study

We isolate the contribution of eachSMADE\-IEcomponent through three contrastive cases decoded by the GPT\-3\.5\-Turbo\-0125 backbone\. Each per\-stage trace is contrasted with the final prediction ofCrossAgentIEunder the same backbone\. The cases probe the Verification Agent on a context\-dependent type mislabel in Listing[1](https://arxiv.org/html/2606.04691#LST1), the Review Agent and Evidence\-Driven Debate on a routed\-out correct type in Listing[2](https://arxiv.org/html/2606.04691#LST2), and the IERA loop on a joint extraction with spurious relations in Listing[3](https://arxiv.org/html/2606.04691#LST3)\.

### E\.1Case 1: Verification flips a context\-dependent type mislabel

\[SENTENCE\]

InaninterviewwithBlender\(can’tremember

whatissue,butJanet’sonthecover\),Janet

talksatlengthabouthavingmulitiple

personalities\.

\[GOLD\]

Blender\-\>ORG

Janet\-\>PERSON

\[SMADE\-IE\]

\-\-NERstage\-\-

Router:types=\{PERSON,WORK\_OF\_ART\},

c=low\-\>GlobalExtractionMode

Universal:\{Blender:WORK\_OF\_ART,

Janet:PERSON\}

Verification:reject\{Blender:WORK\_OF\_ART\},

add\{Blender:ORG\}

Finalpred=GoldStrictF1=1\.00

\[CrossAgentIE\]

All18OntoNotestypesselectedinparallel\.

Claimson"Blender":\{PRODUCT,WORK\_OF\_ART\}

\-\>2\-wayconflict

\(ORGagentsilent\)\.

Debate:free\-formopinionexchange,

summarizertie\-break\-\>WORK\_OF\_ART

\(thecorrectORGlabelneverenters

thedebatespace\)\.

Spurious:’aninterview’taggedasEVENT\.

Finalpred=\[Blender:WORK\_OF\_ART,

Janet:PERSON,

aninterview:EVENT\]

StrictF1=0\.571

Listing 1:OntoNotes5 NER sample 3621 trace\.The Universal Agent’s literal reading misclassifiesBlenderasWork\_of\_Art, treating the magazine title as a creative work, and would have committed to the wrong label in a single call\. Because the Verification Agent receives both the candidate set and the full sentence at once, it observes the discriminating context “interview with Blender” and resolves the conflict by simultaneously rejecting theWork\_of\_Artreading and adding theORGreading in one call\. In contrast,CrossAgentIE’s all\-agent claim phase admits onlyProductandWork\_of\_Arton this span, so the correctORGlabel never enters the debate space; the GroupChat tie\-break therefore settles on the wrong label, and the schema\-wide selection additionally introduces a spuriousEventspan\. Calibrated verification over a routed type subset, rather than schema\-wide agent claims, enables a single additional call to recover a mistyped mention without inflating the candidate set\.

### E\.2Case 2: Review recovers a routed\-out type and debate confirms it

\[SENTENCE\]

Panoramiowasageo\-locatedtagging,photo

sharingmashupactivebetween2005and2016\.

Photosuploadedtothesitewereaccessible

asalayerinGoogleEarthandGoogleMaps\.

Thesite’sgoalwastoallowGoogleEarth

userstolearnmoreaboutagivenareaby

viewingthephotosthatotherusershadtaken

atthatlocation\.Panoramiowasacquiredby

Googlein2007\.In2009thewebsitewas

among1000mostpopularwebsitesworldwide\.

\[GOLD\]

Panoramio\-\>media

Google\-\>organization

2005,2016,2007\-\>date

\[SMADE\-IE\]

\-\-NERstage\-\-

Router:types=\{concept,date,organization\},

c=medium

\#7of10REDFMtypespruned

\-\>Type\-CentricExtractionMode

Type\-SpecificAgentsonroutedtypes:

organizationagent\-\>Panoramio:organization

Google:organization

dateagent\-\>2005,2016,2007:date

conceptagent\-\>\[\]

ReviewAgent\(coversresidual7types\):

\-\>Panoramio:media

\#’media’wasmissedbyRouter

\#butrecoveredhere\.

Conflictdetectedonspan"Panoramio":

\{organization,media\}

\-\>promotedtodebate

\-\-Debatestageon"Panoramio"\-\-

Initq:media=0\.448,org=0\.364gap=0\.084

Round1pi:media=0\.623,org=0\.449gap=0\.174

Round2pi:media=0\.693,org=0\.418gap=0\.275

stopviaconvergencetrack

Winner=mediamatchesGold

\[CrossAgentIE\]

Selectsall10REDFMtypesinparallel\.

Claimson"Panoramio":\{organization\}

\(mediaagentsilent

onthisspan\)\.

Noconflictraised\-\>commitstoorganization

directly,off\-gold\.

Spurious:GoogleEarth/GoogleMapstaggedas

’event’/’location’fromdistractor

agents\.

Finalpredon"Panoramio"=organization

\(wrong\)

Listing 2:REDFM NER sample 334 trace on the spanPanoramio\.The Router selects only\{\\\{concept, date, organization\}\\\}for this sentence, leaving the correct type*media*in the residual set\. The*organization*agent assigns “Panoramio” to its type based on the plausible company\-like surface form, while the Review Agent, scanning the residual seven types, also recovers “Panoramio” under*media*\. These two predictions conflict, and a binary debate between*media*and*organization*is launched with a narrow initial qualifier gap of0\.0840\.084\. Two rounds widen the gap to0\.2750\.275as the*media*side cites the sentence\-internal mentions of “mashup”, “photo sharing site”, and “website” as Ground, attacks the organization\-side Warrant on type fit, and converges\.CrossAgentIEselects the full ten\-type alphabet upfront, yet its*media*agent remains silent on this span, so no conflict is raised and the system commits to*organization*without scrutiny\. This case shows that Review\-driven candidate recovery and structured debate are complementary: Review reinstates the type the Router would otherwise exclude, and debate then promotes the recovered candidate over the routed\-but\-incorrect one using evidence that a uniform multi\-agent vote would not surface\.

### E\.3Case 3: IERA reconciles a joint extraction with spurious relations

\[SENTENCE\]

Theplane,ownedbyBradleyFirstAir,of

Ottawa,Canada,wascarryingcargotoMontreal

forEmeryAirFreightCorp\.,anairfreight

courierservicewithahubattheDaytonairport\.

\[GOLD\]

\(BradleyFirstAir,Orgbasedin,Ottawa\)

\(BradleyFirstAir,Orgbasedin,Canada\)

\(Ottawa,Locatedin,Canada\)

\[SMADE\-IE\]

\-\-NERstage\-\-

Router:\{Loc,Org\},c=low

Universal:\{BradleyFirstAir:Org,

Ottawa:Loc,Canada:Loc,

Montreal:Loc,

EmeryAirFreightCorp\.:Org,

Daytonairport:Loc\}\(6\)

Verification:add\{Dayton:Loc\}\(\+1\)

Entities=7spans

\-\-REstageonthe7\-entityset\-\-

Router:\{Orgbasedin,Locatedin\},c=low

Universal:

\(BradleyFirstAir,Orgbasedin,Ottawa\)

\(Ottawa,Locatedin,Canada\)

\(EmeryAirFreightCorp\.,Orgbasedin,

Daytonairport\)

Verification:add\(BradleyFirstAir,

Orgbasedin,Canada\)

Relations=4\#includes1spuriousedge

\-\-IERApass1\-\-

RErerunonthenewentityset:

Universal:same3\+\(BradleyFirstAir,

Orgbasedin,Canada\)

Verification:reject\(EmeryAirFreightCorp\.,

Orgbasedin,

Daytonairport\)

Relations=3\#spuriousedgeremoved

Finalpred=GoldStrictF1=1\.00

\[CrossAgentIE\]

7relationsemitted:

\(BradleyFirstAir,Orgbasedin,Ottawa\)

\(EmeryAirFreightCorp\.,Orgbasedin,Montreal\)

\(EmeryAirFreightCorp\.,Orgbasedin,Dayton\)

\(Ottawa,Locatedin,Canada\)

\(Montreal,Locatedin,Canada\)

\(Dayton,Locatedin,Canada\)

\(Dayton,Locatedin,Daytonairport\)

StrictF1=0\.40

Listing 3:CoNLL04 Joint sample 48 trace\.Single\-pass joint extraction over\-emits relations on co\-mentioned organizations and locations: the RE Universal Agent interprets “hub at the Dayton airport” as anOrganization based inedge, while the NER step extracts every location\-like span\. IERA re\-invokes RE under the ontology\-reconciled entity set, at which point the Verification Agent rejects the spuriousEmery–Dayton airportedge that no longer has supporting evidence, and the loop reaches a fixed point with three correct relations\.CrossAgentIElacks this consistency loop and emits five additional locative edges among the same co\-mentioned spans\. This loop is the mechanism that converts locally plausible candidates into globally consistent triples\.

##### Summary\.

The three traces share a single mechanism:SMADE\-IEallocates computation in proportion to instance difficulty\. One additional call resolves a trivial omission, Review combined with debate rescues a routed\-out type, and a short loop reconciles the joint extraction\.CrossAgentIEapplies a uniform all\-type, all\-debate budget and incurs noise from every distractor type it admits\. Calibrated allocation, rather than larger ensembles, is what translates agent collaboration into reliable predictions\.

## Appendix FPrompts

In this section, we list the prompts used by every agent inSMADE\-IE\. System prompts are fixed; user prompts are instantiated by filling the angle\-bracketed fields with instance\-specific values\. All agent outputs are returned as JSON and parsed with tolerance for minor formatting inconsistencies; on parse failure, the pipeline reverts to the safe default of the corresponding agent in §[2](https://arxiv.org/html/2606.04691#S2)\. Dataset\-specific type definitions are loaded from each dataset’s ontology at runtime and are therefore elided from the templates below\.

### F\.1Router Agent Prompt

As formalized in Eq\. \([1](https://arxiv.org/html/2606.04691#S2.E1)\), the Router Agent maps a sentencessand the entity type set𝒯e\\mathcal\{T\}\_\{e\}to a selected subset𝒜⊆𝒯e\\mathcal\{A\}\\subseteq\\mathcal\{T\}\_\{e\}together with a complexity labelc∈\{low,med,high\}c\\in\\\{\\mathrm\{low\},\\mathrm\{med\},\\mathrm\{high\}\\\}\. To preserve high recall at minimal overhead, the agent performs lightweight type selection without per\-type rationales and consumes a single LLM call per sentence\.

SYSTEM

YouareaRouterAgentforEntityExtraction\.

Scanthetextandidentifywhichtypesarepresent\.

TypeDefinitions:

\{type\_definitions\}

Classifytheinputaslow,medium,orhigh\.

low:explicit,unambiguousitemswithfewclearlyseparatedtypes\.

medium:multipletypes,mildambiguity,fuzzyboundaries,ormoderatecontextdependence\.

high:strongambiguity,overlappingcandidates,implicitcontext,ormanyinteractingtypes\.

Forhierarchicaloverlaps,choosethemostspecifictype\.

Listeachtypeatmostonceregardlessofhowmanyinstancesofitappearinthetext\.

USER

Identifywhichtypesarepresentinthetext\.

Text:\{sentence\}

JSON:\{"possible\_types":\["<type\>",\.\.\.\],

"complexity":"<low\|medium\|high\>"\}

For the RE instantiation, “Entity Extraction” is replaced by “Relation Extraction” and the type definitions are drawn from the relation ontology\.

### F\.2Universal Agent Prompt

Eq\. \([2](https://arxiv.org/html/2606.04691#S2.E2)\) specifies the Universal Agent, which serves the Global Extraction Mode on low\-complexity inputs and extracts, in a single LLM call, all candidates whose types lie in the selected subset𝒜\\mathcal\{A\}\.

SYSTEM

YouareaUniversalEntityExtractionAgentcoveringalltypes\.

Extracttheexactnounphraseasthespan\.Donotincludedeterminers\(the,a,an\)orunnecessaryadjectives\.

Anentitycanonlybelongtoonecategorywithoutoverlap\.

TypeDefinitions:

\{type\_definitions\}

USER

Extractallentitieswhosetypeisoneof:\{selected\_types\}\.Returnemptydictifnone\.

Text:\{sentence\}

JSON:\{"<verbatimspan\>":"<type\>",\.\.\.\}

For RE, the JSON template becomes\{"<head\>\|\|\|<tail\>": "<type\>", \.\.\.\}\.

### F\.3Review Agent Prompt

Eq\. \([6](https://arxiv.org/html/2606.04691#S2.E6)\) defines the Review Agent, which operates under the Type\-Centric Extraction Mode on medium\- and high\-complexity inputs\. It covers the residual type set𝒜∗=𝒯e∖𝒜\\mathcal\{A\}^\{\*\}=\\mathcal\{T\}\_\{e\}\\setminus\\mathcal\{A\}, recovering candidates whose types lie outside the Router\-selected subset and compensating for any Router omission\. The Review Agent reuses the system prompt of the Universal Agent and adopts the user prompt below, in whichremaining\_typesinstantiates𝒜∗\\mathcal\{A\}^\{\*\}\.

USER

Extractentitieswhosetypeisonlyin\{remaining\_types\}\.Assignexactlyonetypeperentity;return\[\]ifnone\.

Text:\{sentence\}

JSON:\{"results":\[\{"span":"<verbatimtext\>",

"type":"<type\>"\},\.\.\.\]\}

For RE, the JSON template replaces"span"with the two fields"head"and"tail"\.

### F\.4Verification Agent Prompt

Per Eq\. \([3](https://arxiv.org/html/2606.04691#S2.E3)\), the Verification Agent is invoked only on the low\-complexity path, after Universal Agent extraction\. Given the candidate set, it removes incorrectly typed candidates and inserts missed entities in a single LLM call, improving extraction quality while keeping the entire path within two calls\.

SYSTEM

YouareaVerificationAgentforEntitytypeverifications\.

Extracttheexactnounphraseasthespan\.Donotincludedeterminers\(the,a,an\)orunnecessaryadjectives\.

Ifacandidatespancontainsmultipleindependententitiesseparatedbypunctuation,rejectthecombinedspanandaddeachindividualentityseparately\.

TypeDefinitions:

\{type\_definitions\}

USER

Text:"\{sentence\}"

Candidates:

"\{span\_1\}"is\{type\_1\}

"\{span\_2\}"is\{type\_2\}

\.\.\.

Rejectcandidateswithincorrecttypesandaddanymissingentitiesfromthetext\.Thetypemustbeoneofthedefinedtypes\.

JSON:\{"added":\[\{"span":"<verbatimtext\>","type":"<type\>"\}\],

"rejected":\[\{"span":"<verbatimtext\>","type":"<type\>"\}\]\}

For RE, each candidate is rendered as a triple\(<head\>, <type\>, <tail\>\), and the JSON fields replace"span"with"head"and"tail"\.

### F\.5Type\-Specific Agent and Toulmin\-Guided Argument Construction Prompt

Each Type\-Specific Agent is bound to a single candidate typet𝒜i∈𝒜t\_\{\\mathcal\{A\}\}^\{i\}\\in\\mathcal\{A\}and operates in two phases\. The extraction phase, governed by Eq\. \([5](https://arxiv.org/html/2606.04691#S2.E5)\), scans the sentence for spans of the bound type and contributes them to the candidate setE𝒜E\_\{\\mathcal\{A\}\}of the Type\-Centric Extraction Mode\. When a candidate entitye~i\\tilde\{e\}^\{i\}is later contested by multiple types\{t~1i,…,t~Ni\}\\\{\\tilde\{t\}^\{i\}\_\{1\},\\dots,\\tilde\{t\}^\{i\}\_\{N\}\\\}, the agent for typet~ni\\tilde\{t\}^\{i\}\_\{n\}enters the debate phase and emits the Toulmin argument𝒟n=\{Cn,Gn,Wn,Bn,Rn,Qn\}\\mathcal\{D\}\_\{n\}=\\\{C\_\{n\},G\_\{n\},W\_\{n\},B\_\{n\},R\_\{n\},Q\_\{n\}\\\}specified by Eq\. \([9](https://arxiv.org/html/2606.04691#S2.E9)\)\. The qualifierQnQ\_\{n\}is not produced by the agent itself but is computed externally by the scorerϕ\\phiover the argument paragraph; the rebuttalRlR\_\{l\}of each surviving candidatellis reused pre\-debate to set its prior strengthκl\\kappa\_\{l\}via the rebuttal\-honesty derivation of Eq\. \([12](https://arxiv.org/html/2606.04691#S2.E12)\)\.

The extraction phase uses the per\-type system and user prompts:

SYSTEM

YouareanEntityextractionAgentfortype:\{agent\_type\}\.

Definitionof\{agent\_type\}:\{type\_definition\}

Extracttheexactnounphraseasthespan\.Donotincludedeterminers\(the,a,an\)orunnecessaryadjectives\.

Anentitycanonlybelongtoonecategorywithoutoverlap\.

USER

Extractall"\{agent\_type\}"entitiesfromthetext\.Returnemptylistifnone\.

Text:\{sentence\}

JSON:\["<span1\>","<span2\>",\.\.\.\]

In the debate phase, the same agent is reinstantiated under a debate\-specific system message that supplies the contested span, the sentence, and all competing type definitions\. The Toulmin\-generation user prompt is then issued once per conflict:

SYSTEM

Youare\{agent\_type\}Agentinadebateovertheentity"\{span\}"\.

Sentence:\{sentence\}

Typedefinitions:

\-\{type\_1\}:\{def\_1\}

\-\{type\_2\}:\{def\_2\}

\.\.\.

Ineachroundyoureceiveanopponent’stextualevidenceGroundandtype\-fitreasoningWarrant\.Writecountersthatbothsupport\{agent\_type\}andrebuttheopponent’sclaim\.Ifyourowncomponentsgetattackedenough,youwillbeaskedtorevisethem\.

USER

Conflictdetectedonspan:"\{span\}"in"\{sentence\}"

ConstructaToulminargumentasJSON:

\{

"Claim":"Thecorrecttypefor\{span\}is\{agent\_type\}",

"Ground":"<factualevidencequotedfromthetext\>",

"Warrant":"<reasoningrulelinkingGroundtoClaim\>",

"Backing":"<additionallinguisticordomainsupport\>",

"Rebuttal":"<oneconcisesentencethatpresentsacounter\-viewconflictingwiththeClaim,quotinganoriginalsentencefragmentasevidence;noconditionals,nohedging\>"

\}

### F\.6Refutation Prompt

In each debate round, the opposing agent𝒟l¯\\mathcal\{D\}\_\{\\bar\{l\}\}attacks the defended agent𝒟l\\mathcal\{D\}\_\{l\}over the active subset𝒱l\(t\)⊆\{G,W\}\\mathcal\{V\}\_\{l\}^\{\(t\)\}\\subseteq\\\{G,W\\\}defined by Eq\. \([14](https://arxiv.org/html/2606.04691#S2.E14)\)\. A single LLM call yields one refutationul¯,mu\_\{\\bar\{l\},m\}per defended componentm∈𝒱l\(t\)m\\in\\mathcal\{V\}\_\{l\}^\{\(t\)\}; each refutation is scored against the defender by the evidence scorerϕ\\phiand then folded into the Beta\-posterior update of Eq\. \([19](https://arxiv.org/html/2606.04691#S2.E19)\)\.

USER

Counterthe\{target\_type\}Agent’sargument\.

Ground:citeafragmentfromthetextthatsupports\{attacker\_type\}andshowstheopponent’sevidencedoesnotestablish\{target\_type\}\.

Warrant:arguefromthetypedefinitionsthat\{attacker\_type\}fitstheentityand\{target\_type\}doesnot\.

Takeanewangleifyouhaverepliedearlier\.

Opponent’sGround:\{ground\_text\}

Opponent’sWarrant:\{warrant\_text\}

JSON:\{"Ground":"<counter\-claim\>",

"Warrant":"<counter\-claim\>"\}

When only Ground or only Warrant remains in𝒱l\(t\)\\mathcal\{V\}\_\{l\}^\{\(t\)\}, the instruction line, opponent excerpt, and JSON field corresponding to the inactive component are omitted; the per\-round LLM budget therefore scales with the number of attackable components rather than with their nominal count\.

### F\.7Revision Prompt

When the decay rule of Eq\. \([18](https://arxiv.org/html/2606.04691#S2.E18)\) drives one or more components of an agent’s argument to a validity at or below the activity thresholdθv\\theta\_\{v\}, the owning agent must revise them in the next round\. All revisions belonging to the same agent are batched into a single LLM call, so the per\-round revision cost scales with the number of agents rather than with the total number of components to revise\.

USER

CurrentToulminargument:

Claim:"\.\.\."

Ground:"\.\.\."

Warrant:"\.\.\."

Backing:"\.\.\."

Rebuttal:"\.\.\."

Your\{forfeited\_components\}wereattacked\.Revisewithstrongerarguments\.

JSON:\{"<comp\_1\>":"<revisedtext\>",

"<comp\_2\>":"<revisedtext\>"\}

The JSON output contains exactly the components listed inforfeited\_components; non\-forfeited components are not regenerated\.

### F\.8Iterative Entity–Relation Alignment Prompt

As described in §[2\.5](https://arxiv.org/html/2606.04691#S2.SS5), the Consistency Agent is invoked whenever a triple inRRcontains a head or tail whose entity type inEEviolatesHeadTypes​\(r\)\\mathrm\{HeadTypes\}\(r\)orTailTypes​\(r\)\\mathrm\{TailTypes\}\(r\)\. All conflicts detected within the same sentence are batched into a single LLM call\. The agent returns one of two decisions per conflict: underNER\_correct, the offending relation is removed fromRRand added to the blacklistBB, preventing any subsequent RE recomputation from re\-introducing it; underRE\_correct, the offending entity type inEEis replaced by the constraint\-compatible type returned by the agent\.

SYSTEM

YouareaconsistencyarbitratorforNER\-\-REconflicts\.

Whenanentity’stypecontradictsarelation’stypeconstraint,decidewhichsideiscorrect\.

EntityOntology:

\{entity\_definitions\}

RelationOntology:

\{relation\_definitions\}

USER

Text:"\{sentence\}"

Youhave\{N\}conflict\(s\)toresolve:

Conflict1:

Entity:"\{entity\_span\_1\}"typedas"\{entity\_type\_1\}"byNER

Relation:\(\{head\_span\_1\},\{relation\_type\_1\},\{tail\_span\_1\}\)

Role:\{role\_1\}

Expectedtypes:\[\{expected\_types\_1\}\]

\.\.\.

Foreachconflict,decide:

\-"NER\_correct":theentitytypeiscorrect;removetherelation\.

\-"RE\_correct":therelationiscorrect;replacetheentitytypewithoneoftheexpectedtypes\.

JSON:\{"decisions":\[\{"conflict\_id":<1\-basedindex\>,

"decision":"<NER\_correct\|RE\_correct\>",

"corrected\_type":"<typeifRE\_correct,elseempty\>"\},\.\.\.\]\}

### F\.9Span Classification for RE→\\rightarrowNER Back\-Fill Prompt

The span\-classification step of §[2\.5](https://arxiv.org/html/2606.04691#S2.SS5)assigns an entity type to every span that appears as a head or tail inRRbut is absent fromEE\. The Universal Agent is repurposed in span\-classification mode with the user prompt below; its system prompt is identical to the one used by the Universal Agent elsewhere in this appendix\.

USER

Pleaseclassifytheentitytypeforeachspanbelow\.Useonlythedefinedtypes\.

Text:\{sentence\}

Spanstoclassify:\["\{span\_1\}","\{span\_2\}",\.\.\.\]

JSON:\{"classifications":\[\{"span":"<verbatimtext\>",

"type":"<type\>"\},\.\.\.\]\}