MS-Eval：MCP 密集型工作场景中的子代理委派与编排基准评测

# MS-Eval: Benchmarking Sub-Agent Delegation and Orchestration in MCP-Intensive Working Scenarios
Source: [https://arxiv.org/html/2610.22869](https://arxiv.org/html/2610.22869)
\[1\]for each\#1do

Bowen Ding1,2,\*, Qixin Wang2,\*,†, Lexing Yuan2,†, Weiyi Lu2,†, Qiwei Ge4, Ruishan Fang1,2, Tao Lin2,3,‡ 1Zhejiang University2School of Engineering, Westlake University 3Institute of Advanced Technology, Westlake Institute for Advanced Study 4Independent Researcher 2\{dingbowen, lintao\}@westlake\.edu\.cn

###### Abstract

LLM agents are increasingly integrated into real\-world works, yet they must handle diverse domain\-specific tools and complex multi\-sub\-task problems that sequential tool invocation resolves inefficiently and with severe context bloat\. TheMain\-Agent Delegation and Sub\-Agent Execution\(MAD\-SAE\) paradigm addresses these issues by delegating sub\-tasks to specialized sub\-agents for parallel execution in isolated contexts, but its delegation and orchestration quality remains unquantified\. We introduce MS\-Eval, a unified evaluation kit comprising an MCP\-intensive benchmark, a controlled execution harness, tailored delegation and orchestration metrics, and the MSQ index for comprehensive assessment\. Evaluating four MAD\-SAE variants and eight frontier models reveals that \(1\) sub\-agent specialization \(Spec\) is the best practical practice; \(2\) open\-source models are largely competitive with few exceptions; and \(3\) MAD\-SAE generally raises cost with contingent payoff, benefiting most open\-source models in both performance and efficiency while leaving closed\-source models with higher cost and degraded performance\. Our case study further traces these gains to better single\-agent solutions, effective context offloading, and genuine sub\-task parallelism, whereas unnecessary or redundant delegation turns the same cost into pure overhead\. Codes:[https://github\.com/LINs\-lab/MS\-Eval](https://github.com/LINs-lab/MS-Eval)

## 1Introduction

![Refer to caption](https://arxiv.org/html/2610.22869v1/msq_ranking_MSE_Kit_2.png)Figure 1:The leaderboard of the MSQ Index\. The index measures the comprehensive quality ofMain\-Agent Delegation and Sub\-Agent Executionparadigm \(detailed in §[4\.3](https://arxiv.org/html/2610.22869#S4.SS3)\), which is dominated by open\-source models\. Oracle, highlighted in yellow, probes the upper bound of orchestration utilities\.Driven by the scaling of parameters and data\([Kaplan et al\., 2020](https://arxiv.org/html/2610.22869#bib.bib1);[Ding et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib2)\), LLMs have evolved into tool\-empowered autonomous agents\([GLM\-5\-Team et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib3);[DeepSeek\-AI, 2026b](https://arxiv.org/html/2610.22869#bib.bib28)\)that are increasingly integrated into daily works to augment human productivity\([Xu et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib12)\)\.

However, acting as a useful working assistant poses immense challenges\. Real\-world working tasks demand diverse and domain\-specific tools inherently\. For instance, researchers necessitates scholarly retrieval tools for literature reviews, educational administrators relies on Canvas for course management\([Li et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib6)\)\. Additionally, problems in working scenarios are often complicated, involving multiple sub\-tasks\([Zhang et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib13)\), as the example illustrated in Figure[2](https://arxiv.org/html/2610.22869#S4.F2)\(a\)\. Traditionally, agents resolve these sub\-tasks by rigidly invoking tools in a sequential manner\([Bandi et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib36);[Li et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib6)\)\. This lack of parallelization severely limits task\-solving efficiency\([Kimi\-Team et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib27)\)\. More critically, exploratory sub\-tasks often retrieve information\-intensive chunks that quickly bloat the context window, leading to severe decision distortion in subsequent steps\([Liu et al\., 2024](https://arxiv.org/html/2610.22869#bib.bib4);[Xue et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib5)\)\.

To address these challenges, mainstream harness frameworks \(e\.g\., Claude Code\([Anthropic, 2026b](https://arxiv.org/html/2610.22869#bib.bib11)\), Codex\([OpenAI, 2026a](https://arxiv.org/html/2610.22869#bib.bib14)\), Cursor\([Cursor, 2026a](https://arxiv.org/html/2610.22869#bib.bib15)\)\) integrate Model Context Protocol \(MCP\) servers\([Anthropic, 2024](https://arxiv.org/html/2610.22869#bib.bib10)\)to fulfill specialized tool demands, and support the autonomous multi\-agents runtime paradigm, Main\-Agent Delegation and Sub\-Agent Execution \(MAD\-SAE, details in §[3](https://arxiv.org/html/2610.22869#S3)\)\. In the paradigm, the main\-agent dynamically delegates sub\-tasks to specialized sub\-agents for parallel execution in isolated contexts, ultimately returning concise summaries\. Consequently, it is expected to alleviate the context burden of single\-agent, and accelerate task\-solving via parallel sub\-agent execution\. While widely adopted, the sub\-agent delegation and orachestration quality of MAD\-SAE has yet to be quantified\. This leaves a critical open problem:

*Can frontier LLMs in MAD\-SAE optimally route sub\-tasks at the right time for highly efficient and cost\-effective task completion?*

To facilitate this evaluation, we propose theMAD\-SAEEvaluation \(MS\-Eval\) kit, comprising three components: \(1\) a curated MCP\-intensive benchmark covering eight working scenarios, featuring tasks annotated with optimal delegation priors \(§[4\.1](https://arxiv.org/html/2610.22869#S4.SS1)\); \(2\) A controlled harness standardizing the execution principle and tool accessibility for MAD\-SAE and the Single\-Agent \(SA\) baseline \(§[4\.2](https://arxiv.org/html/2610.22869#S4.SS2)\); \(3\) tailored metrics evaluating MAD\-SAE on Delegation Appropriateness \(e\.g\., ODR, UDR, and Sub\-Agent Hit Scores\) and Orchestration Utilities \(performance, efficiency, and cost gains over SA\)\. Additionally, we propose the MAD\-SAE Quality \(MSQ\) index, a unified metric integrating both perspectives for a comprehensive assessment \(§[4\.3](https://arxiv.org/html/2610.22869#S4.SS3)\)\.

After evaluating on four implementation variants of MAD\-SAE \(e\.g\., Gen, Spec, Plan\-1st and Oracle, details in §[3\.2](https://arxiv.org/html/2610.22869#S3.SS2)\), and eight frontier models under the best MAD\-SAE practice, we reveal several critical insights:

- •Sub\-agent specialization \(Spec\) is the best MAD\-SAE practice\.Spec avoids the overly conservative delegation of Gen, and the aggressive over\-delegation of Plan\-1st, achieving the highest MSQ with 20\.8% among practical variants in Table[1](https://arxiv.org/html/2610.22869#S5.T1)\(a\)\.
- •Open\-source models lead delegation and orchestration quality\.Excluding Oracle, the top six MSQ settings \(Fig\.[1](https://arxiv.org/html/2610.22869#S1.F1)\) are all open\-source models, with GLM\-5\.3 \(Spec\) attaining the highest MSQ of 24\.6% and Kimi\-K3 \(Spec\) the only setting improving performance, efficiency, and cost simultaneously as shown in Table[1](https://arxiv.org/html/2610.22869#S5.T1)\(b\)\.
- •MAD\-SAE generally incurs higher cost, but its payoff is contingent on delegation quality\.Across the full benchmark𝒟\\mathcal\{D\}\(§[7](https://arxiv.org/html/2610.22869#S7)\), MAD\-SAE improves both performance and efficiency for most open\-source models at added cost, whereas closed\-source models see cost rise alongside degraded performance and efficiency\. Our case study traces these gains to more efficient single\-agent solutions, effective context offloading, and genuine sub\-task parallelism, while unnecessary, unverified, or redundant delegation turns the same cost into pure overhead\.

## 2Related Work

MCP\-Intensive Evaluation\.To empower LLMs as capable working assistants, modern harness frameworks\([OpenAI, 2026a](https://arxiv.org/html/2610.22869#bib.bib14);[Anthropic, 2026b](https://arxiv.org/html/2610.22869#bib.bib11);[Cursor, 2026a](https://arxiv.org/html/2610.22869#bib.bib15)\)increasingly integrate Model Context Protocol \(MCP\) servers\([Anthropic, 2024](https://arxiv.org/html/2610.22869#bib.bib10)\), motivating the growing demand of MCP\-intensive evaluation\. Existing MCP evaluations have progressively evolved along three directions\. Early efforts established large\-scale evaluation protocols focusing on tool retrieval, execution correctness, and automated judging mechanisms\([Liu et al\., 2025](https://arxiv.org/html/2610.22869#bib.bib30);[Mo et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib31);[Gao et al\., 2025](https://arxiv.org/html/2610.22869#bib.bib32)\)\. To improve reliability and reduce evaluation subjectivity, subsequent works introduced deterministic execution\-based verification and broader real\-world MCP environments\([Luo et al\., 2025](https://arxiv.org/html/2610.22869#bib.bib33);[Yin et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib34);[Guo et al\., 2025](https://arxiv.org/html/2610.22869#bib.bib35)\), with MCPWorld further incorporating semantic internal\-state verification through application\-level instrumentation\([Yan et al\., 2025](https://arxiv.org/html/2610.22869#bib.bib29)\)\. Recent benchmarks have pushed toward more realistic working scenarios, emphasizing long\-horizon CRUD workflows by MCPMark\([Wu et al\., 2025](https://arxiv.org/html/2610.22869#bib.bib37)\), cross\-application orchestration by MCP\-Atlas\([Bandi et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib36)\), and ambiguous user instructions across diverse applications by Toolathlon\([Li et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib6)\)\. Toolathlon\-Gym\([Zhang et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib13)\)inherits Toolathlon’s broad scenario coverage and provides a locally deployed MCP server and database environment, on which we build MS\-Eval\.

Multi\-Agent Orchestration\.Multi\-Agent Systems \(MAS\) are designed to overcome the limitations of single\-agent systems in context capacity\([Yu et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib40)\), insufficient exploration\([Gurjar et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib43)\), and inefficient serial execution\([Masterman et al\., 2024](https://arxiv.org/html/2610.22869#bib.bib42)\), through specialized collaboration\([Guo et al\., 2024](https://arxiv.org/html/2610.22869#bib.bib41)\)\. Early multi\-agent studies explored structured collaboration via role specialization, communication protocols, and iterative interaction among agents\([Li et al\., 2023](https://arxiv.org/html/2610.22869#bib.bib7);[Du et al\., 2023](https://arxiv.org/html/2610.22869#bib.bib38);[Park et al\., 2023](https://arxiv.org/html/2610.22869#bib.bib39)\)\. Recent efforts have shifted toward autonomous orchestration, enabling LLM agents to dynamically plan workflows, assign responsibilities, and coordinate heterogeneous agents or tools through harness frameworks\([Anthropic, 2026b](https://arxiv.org/html/2610.22869#bib.bib11);[OpenAI, 2026a](https://arxiv.org/html/2610.22869#bib.bib14);[Cursor, 2026a](https://arxiv.org/html/2610.22869#bib.bib15)\)\. More recent systems further scale this paradigm toward adaptive, large\-scale delegation, e\.g\., agent swarms of[Kimi\-Team et al\. \(2026\)](https://arxiv.org/html/2610.22869#bib.bib27)\. Despite this progress, existing MAS evaluations remain limited: MultiAgentBench\([Zhu et al\., 2025](https://arxiv.org/html/2610.22869#bib.bib8)\)assesses task completion, delegation, and coordination quality under plan\-first runtime constraints, but does not compare efficiency or cost against single\-agent execution;[Fu et al\. \(2026\)](https://arxiv.org/html/2610.22869#bib.bib44)conduct a controlled comparison between single\-agent and MAS in terms of accuracy and cost, yet focus on short\-horizon reasoning, coding, and instruction following, lacking standardized evaluation of delegation quality; ClawArena\-Team\([Xiong et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib9)\)considers sub\-agent orchestration but primarily studies a main agent creating its own sub\-agent team \(e\.g\., tool granting\), without assessing the rationality of sub\-agent delegation or collaborative gains\. Moreover, none of these works consider MCP\-intensive working scenarios\.

To our knowledge, we are the first to systematically evaluate sub\-agent delegation quality and orchestration versus single\-agent execution in performance, efficiency, and cost under MCP\-intensive working scenarios\.

## 3Main\-Agent Delegation and Sub\-Agent Execution

### 3\.1Definition and Property

Main\-Agent Delegation and Sub\-Agent Execution \(MAD\-SAE\) serves as an autonomous muti\-agent orchestration paradigm, being supported by mainstream harness frameworks, such as Claude Code\([Anthropic, 2026b](https://arxiv.org/html/2610.22869#bib.bib11)\), Codex\([OpenAI, 2026a](https://arxiv.org/html/2610.22869#bib.bib14)\), Cursor\([Cursor, 2026a](https://arxiv.org/html/2610.22869#bib.bib15)\), etc\. The paradigm transforms the classical Single\-Agent \(SA\) task\-solving runtime into a collaborative, hierarchical orchestration, decoupling long\-horizon contexts via two distinct roles:

Main\-Agent\.Governing global execution, the main\-agent is uniquely empowered withAgent\-Delegation \(AD\)tools \(i\.e\.,Agent, AgentSwarmin Fig\.[2](https://arxiv.org/html/2610.22869#S4.F2)\)\. With thses tool, the main\-agent dynamically decomposes tasks into sub\-tasks and delegates them to specialized sub\-agents\. Each concurrent execution of multiple specialized sub\-agents is called awave\. Depending on the delegation pattern, a runtime session may feature zero, one, or multiple sequential waves\.

Sub\-Agent\.Triggered within a wave, a sub\-agent is a specialized AI assistant responsible solely for its assigned sub\-task\. Each sub\-agent has its own type, operates within an isolated context window, is constrained by a specialized prompt and a tightly restricted tool allowlist, and ultimately returns its results to the main\-agent\.

Intuitively, MAD\-SAE is expected to have two advantages:\(1\) Higher efficiency: parallelizable sub\-agents enable concurrent sub\-task execution, potentially surpassing the serial SA baseline\([Kimi\-Team et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib27)\)\.\(2\) Performance gains: specialized sub\-agents may solve their expert sub\-tasks better and, via isolated contexts, offload intensive information \(e\.g\., exploration findings\) from the main\-agent, preventing context bloat from degrading decisions\([Yu et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib40);[Cursor, 2026b](https://arxiv.org/html/2610.22869#bib.bib16)\)\.

### 3\.2Variants

We consider three practical MAD\-SAE variants, General \(Gen\), Specialized \(Spec\), and Plan\-1st \(Plan\), plus an Oracle baseline for golden study \(see Appx\.[B\.4](https://arxiv.org/html/2610.22869#A2.SS4)for details\)\.Genlets the main\-agent autonomously decide when and how to delegate to generally defined sub\-agents: a read\-only Planner for decomposition and delegation, a read\-only Explorer for exploration, and a writable Coder for editing workflows\.Specretains Gen’s Planner but replaces the general sub\-agents with domain\-specialized ones, splitting exploration into four read\-only experts \(Academic\-Literature\-Researcher, Enterprise\-Data\-Analyst, Financial\-Market\-Analyst, and Web\-Domain\-Researcher\) and editing into two writable experts \(External\-Workflow\-Operator and Office\-Report\-Builder\)\.Plan\-1stinherits Gen’s typology but mandates calling the Planner for decomposition and delegation planning before execution\.Oracleinjects golden delegation priors into Gen’s prompt to test orchestration utilities under golden delegation\.

## 4MS\-Eval Kit

To comprehensively investigate the sub\-agent delegation appropriateness of main\-agent and the underlying orchestration utilities against the SA baseline, we introduce theMAD\-SAEEvaluation Kit \(MS\-Eval Kit\)\. This kit comprises three core components: a curated benchmark \(§[4\.1](https://arxiv.org/html/2610.22869#S4.SS1)\), a controlled harness \(§[4\.2](https://arxiv.org/html/2610.22869#S4.SS2)\), and tailored evaluation metrics \(§[4\.3](https://arxiv.org/html/2610.22869#S4.SS3)\)\.

![Refer to caption](https://arxiv.org/html/2610.22869v1/common/figures/new_benchmark.drawio_2.png)Figure 2:The Overview of MS\-Eval Kit\.Benchmarkexemplifiesarxiv\-survey\-presentationwith its Task Description and Delegation Priors with 3 waves and 9 sub\-agents annotation \(§[4\.1](https://arxiv.org/html/2610.22869#S4.SS1)\)\.Harnessdefines principles and tool permissions governing agent behavior across execution paradigms \(§[4\.2](https://arxiv.org/html/2610.22869#S4.SS2)\)\.Eval Metricsvisualizes MAD\-SAE metric computation \(§[4\.3](https://arxiv.org/html/2610.22869#S4.SS3)\)\.### 4\.1Benchmark

We sample, refine111We cover all Toolathlon\-Gym scenarios and align task descriptions with evaluation criteria to reduce false negatives\., and annotate 166 tasks from Toolathlon\-Gym\([Zhang et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib13)\), a large\-scale and long\-horizon environment covering eight real\-world work scenarios: Learning Management System \(LMS\), Enterprise Data Warehouse \(EDW\), E\-Commerce \(EC\), Stock Market \(SM\), Video Platform \(VP\), Railway System \(RS\), Academic Research \(AR\) and Food\-and\-Catering \(FC\)\.

To facilitate the assessment on delegation appropriateness, we establish Delegation Prior, a golden delegation plan, as the ground truth for each task\. As an example in Fig\.[2](https://arxiv.org/html/2610.22869#S4.F2), such prior specifies \(1\)whento delegate a wave of sub\-agents, \(2\)which typesof sub\-agents to assign, and \(3\)which sub\-tasksthey should handle to complete the task efficiently\. The annotations are derived from each task’s description, required tools, and expected deliverables, following two principles: \(i\) maximizing opportunities for parallel execution, and \(ii\) isolating exploratory sub\-tasks within sub\-agents to reduce context overload for main\-agent\. These principles echo expected advantages mentioned in §[3](https://arxiv.org/html/2610.22869#S3)\.

Based on the annotation, the entire dataset is divided into two splits: \(1\)𝒟no\-need\\mathcal\{D\}\_\{\\text\{no\-need\}\}with 77 tasks, where the main\-agent alone is already efficient enough, and \(2\)𝒟need\\mathcal\{D\}\_\{\\text\{need\}\}with 89 tasks, where the main\-agent should delegate sub\-tasks to sub\-agents for efficient and effective solutions\. Refer to Appx\.[A](https://arxiv.org/html/2610.22869#A1)for detailed sub\-agent delegation statistics\.

### 4\.2Harness

We develop MAD\-SAE and single\-agent \(SA\), two agent executation paradigms, on top of Kimi\-Code\([Moonshot AI, 2026a](https://arxiv.org/html/2610.22869#bib.bib18)\)\. Kimi\-Code is an open\-source framework with native sub\-agent support\. To rigorously isolate the effects of execution paradigms under specified working scenarios, we disable Kimi\-Code’s built\-in skills and strictly manage the executation principle and tool accessibility\. Refer to Appx\.[E\.2](https://arxiv.org/html/2610.22869#A5.SS2)for detials\.

Executation Principle\.Agent behaviors are standardized through tailored principles in system prompts\. For SA, the basic principle is enough to govern its behavior, by enforcing efficient task execution, strict workspace boundary restrictions, and a mandatory completion protocol\. For MAD\-SAE, the main\-agent inherits the basic principle of SA, and is further augmented with sub\-agent delegation norms, available sub\-agent profiles, and concrete few\-shot delegation examples to guide appropriate delegation\. Conversely, sub\-agents operate under specialized constraints: they are strictly confined to their isolated workspaces, mandated to return concise result summaries to the main\-agent, and explicitly prohibited from invoking the globalclaim\_donetermination signal\.

Tool accessibility\.SA is granted with basic tools, which includesTodoListfor progress tracking, MCP tools for specified task\-solving,claim\_donefor task termination andpython\_executefor isolated python coding\. Within MAD\-SAE, the main\-agent additionally accessesAgentandAgentSwarmbesides basic tools, to delegate sub\-agents for heterogeneous or homogeneous sub\-tasks, while sub\-agents receive role\-specific tool allowlists and runtime exclusions\.

### 4\.3Evaluation Metrics

Our evaluation considers two complementary perspectives: Delegation Appropriateness \(DA\) \(§[4\.3\.1](https://arxiv.org/html/2610.22869#S4.SS3.SSS1)\) to assess the sub\-agent delegation rationality, and Orchestration Utilities \(OU\) \(§[4\.3\.2](https://arxiv.org/html/2610.22869#S4.SS3.SSS2)\) to quantify performance, efficiency, and cost gains over the SA baseline\. §[4\.3\.3](https://arxiv.org/html/2610.22869#S4.SS3.SSS3)aggregates these into MAD\-SAE Quality \(MSQ\), a unified index discounting the average utility by delegation appropriateness\.

#### 4\.3\.1Delegation Appropriateness \(DA\)

Two questions are considered for evaluating DA: \(1\)*whether*the main agent delegates when sub\-agent assistance is warranted, and, \(2\) once it delegates,*how well*the executed delegations match the expected ones\. Accordingly, we introduce: Under\-Delegation Rate \(UDR\), Over\-Delegation Rate \(ODR\), and Sub\-agent Hit Scores withPhitP\_\{\\text\{hit\}\},RhitR\_\{\\text\{hit\}\}andℱ1 hit\\mathcal\{F\}\_\{\\text\{1 hit\}\}\.

UDR and ODRassess whether the main agent correctly determines when to delegate\. Let𝒟need\\mathcal\{D\}\_\{\\mathrm\{need\}\}and𝒟no​\-​need\\mathcal\{D\}\_\{\\mathrm\{no\\text\{\-\}need\}\}denote the sets of tasks annotated as requiring and not requiring sub\-agent assistance, respectively, and let𝒟del\\mathcal\{D\}\_\{\\mathrm\{del\}\}denote the tasks in which the main agent actually invokes at least one non\-planning sub\-agent\. We define

UDR=\|𝒟need∖𝒟del\|\|𝒟need\|,ODR=\|𝒟no​\-​need∩𝒟del\|\|𝒟no​\-​need\|\.\\mathrm\{UDR\}=\\frac\{\|\\mathcal\{D\}\_\{\\mathrm\{need\}\}\\setminus\\mathcal\{D\}\_\{\\mathrm\{del\}\}\|\}\{\|\\mathcal\{D\}\_\{\\mathrm\{need\}\}\|\},\\qquad\\mathrm\{ODR\}=\\frac\{\|\\mathcal\{D\}\_\{\\mathrm\{no\\text\{\-\}need\}\}\\cap\\mathcal\{D\}\_\{\\mathrm\{del\}\}\|\}\{\|\\mathcal\{D\}\_\{\\mathrm\{no\\text\{\-\}need\}\}\|\}\.\(1\)UDR measures missed delegation opportunities, whereas ODR measures unnecessary delegation\.

Sub\-agent Hit Scoresevaluate fine\-grained delegation quality by matching runtime\-delegated sub\-agents with annotated golden sub\-agents\. To match sub\-agents defined under different conventions \(e\.g\.,Coderof Gen v\.s\.Office\-Report\-Builderof Spec\), we functionally tag both golden and delegated sub\-agents based on their sub\-task semantics, using 24 labels \(details in Appx\.[B\.2](https://arxiv.org/html/2610.22869#A2.SS2)\) and DeepSeek\-V4\.1\-Flash\([DeepSeek\-AI, 2026b](https://arxiv.org/html/2610.22869#bib.bib28)\)\. Within a wave, tagged sub\-agents are grouped into function units that never span waves, with all sub\-agents solving the same sub\-task collapsing into one unit\. Typically, each writable sub\-agent forms its own unit, whereas read\-only exploratory sub\-agents form separate units only when handling different sub\-tasks\.

Treating function units as atomic delegation responsibilities, for taskttwe obtain the golden unit groupGtG\_\{t\}and delegated unit groupEtE\_\{t\}\. Using maximum bipartite matching, we findmtm\_\{t\}one\-to\-one matches \(mt≤min⁡\(\|Gt\|,\|Et\|\)m\_\{t\}\\leq\\min\(\|G\_\{t\}\|,\|E\_\{t\}\|\)\), determining which golden units are hit\. We then compute:

𝒫hit=∑tmt∑t\|Et\|,ℛhit=∑tmt∑t\|Gt\|,ℱ1 hit=2​𝒫hit​ℛhit𝒫hit\+ℛhit\.\\mathcal\{P\}\_\{\\text\{hit\}\}=\\frac\{\\sum\_\{t\}m\_\{t\}\}\{\\sum\_\{t\}\|E\_\{t\}\|\},\\qquad\\mathcal\{R\}\_\{\\text\{hit\}\}=\\frac\{\\sum\_\{t\}m\_\{t\}\}\{\\sum\_\{t\}\|G\_\{t\}\|\},\\qquad\\mathcal\{F\}\_\{\\text\{1 hit\}\}=\\frac\{2\\mathcal\{P\}\_\{\\text\{hit\}\}\\mathcal\{R\}\_\{\\text\{hit\}\}\}\{\\mathcal\{P\}\_\{\\text\{hit\}\}\+\\mathcal\{R\}\_\{\\text\{hit\}\}\}\.\(2\)where𝒫hit\\mathcal\{P\}\_\{\\text\{hit\}\}measures the precision of executed delegation units,ℛhit\\mathcal\{R\}\_\{\\text\{hit\}\}measures the coverage of golden units, andℱ1 hit\\mathcal\{F\}\_\{\\text\{1 hit\}\}balances the two\.

#### 4\.3\.2Orchestration Utilities \(OU\)

We quantify MAD\-SAE utilities over the SA baseline across three dimensions:performance\(pp\),efficiency\(ee\), andcost\(cc\)\. Specifically, performance is measured by Success Rates \(SR\) in Eq\.[5](https://arxiv.org/html/2610.22869#A2.E5)on𝒟del\\mathcal\{D\}\_\{\\mathrm\{del\}\}\(defined in §[4\.3\.1](https://arxiv.org/html/2610.22869#S4.SS3.SSS1)\)\. Conversely, efficiency and cost are strictly evaluated via Critical Steps \(Csteps\) in Eq\.[6](https://arxiv.org/html/2610.22869#A2.E6)and API billing estimated in Eq\.[7](https://arxiv.org/html/2610.22869#A2.E7), on the successfully completed subset𝒟del\+⊆𝒟del\\mathcal\{D\}\_\{\\mathrm\{del\}\}^\{\+\}\\subseteq\\mathcal\{D\}\_\{\\mathrm\{del\}\}, as resource savings are meaningful only for solved tasks\.

For each dimensioni∈\{p,e,c\}i\\in\\\{p,e,c\\\}, letvimadv\_\{i\}^\{\\mathrm\{mad\}\}andvisav\_\{i\}^\{\\mathrm\{sa\}\}denote the measurement for MAD\-SAE and SA of the specified dimensionii\. Because these metrics possess disparate scales and opposing optimization directions \(i\.e\., higher\-is\-better for performance, but lower\-is\-better for efficiency and cost\), we normalize them into a unified, higher\-is\-better\[0,1\]\[0,1\]indexUiU\_\{i\}\. This standardization seamlessly facilitates the MAD\-SAE quality \(MSQ\) aggregation in §[4\.3\.3](https://arxiv.org/html/2610.22869#S4.SS3.SSS3):

Ui=\{vimadvimad\+visa,i=p,visavimad\+visa,i∈\{e,c\}\.U\_\{i\}=\\begin\{cases\}\\displaystyle\\frac\{v\_\{i\}^\{\\mathrm\{mad\}\}\}\{v\_\{i\}^\{\\mathrm\{mad\}\}\+v\_\{i\}^\{\\mathrm\{sa\}\}\},&i=p,\\\\\[8\.0pt\] \\displaystyle\\frac\{v\_\{i\}^\{\\mathrm\{sa\}\}\}\{v\_\{i\}^\{\\mathrm\{mad\}\}\+v\_\{i\}^\{\\mathrm\{sa\}\}\},&i\\in\\\{e,c\\\}\.\\end\{cases\}\(3\)Under this formulation,Ui=50%U\_\{i\}=50\\%indicates parity between the two paradigms,Ui\>50%U\_\{i\}\>50\\%reflects a net advantage for MAD\-SAE, andUi<50%U\_\{i\}<50\\%reveals a negative orchestration utility\.

#### 4\.3\.3MAD\-SAE Quality

Building uponℱ1 hit\\mathcal\{F\}\_\{\\text\{1 hit\}\}of Sub\-Agent Hit Scores in §[4\.3\.1](https://arxiv.org/html/2610.22869#S4.SS3.SSS1)and normalized utilitiesUi,i∈\{p,e,c\}U\_\{i\},i\\in\\\{p,e,c\\\}in §[4\.3\.2](https://arxiv.org/html/2610.22869#S4.SS3.SSS2), we aggregate these metrics for a overall assessment of the MAD\-SAE quality \(MSQ\):

MSQ=ℱ1 hit⋅\(wp​Up\+we​Ue\+wc​Uc\),wp\+we\+wc=1\.\\mathrm\{MSQ\}=\\mathcal\{F\}\_\{\\text\{1 hit\}\}\\cdot\\left\(w\_\{p\}U\_\{p\}\+w\_\{e\}U\_\{e\}\+w\_\{c\}U\_\{c\}\\right\),\\qquad w\_\{p\}\+w\_\{e\}\+w\_\{c\}=1\.\(4\)Here,wpw\_\{p\},wew\_\{e\}, andwcw\_\{c\}dictate the relative emphasis on performance, efficiency, and cost\. While these weights can be flexibly tailored to specific priorities, we assign equal weights \(1/31/3\) in our experiments to ensure a balanced evaluation\. Crucially,ℱ1 hit\\mathcal\{F\}\_\{\\text\{1 hit\}\}acts as a multiplicative discount: high orchestration utility receives full credit only when driven by appropriate delegation\.

## 5Experiment Setup

Leveraging the MS\-Eval Kit mentioned in §[4](https://arxiv.org/html/2610.22869#S4), our experiments are structured to progressively address three core research questions: \(1\)RQ1:Among representative MAD\-SAE variants, which performs best, and what is its gap to the oracle upper bound? \(2\)RQ2:Under the optimal implementation, which frontier model achieves the highest MSQ, and how do their orchestration profiles differ? \(3\)RQ3:Ultimately, does MAD\-SAE actualize its expected utilities over the monolithic Single\-Agent \(SA\) paradigm, and what is the inherent trade\-off?

### 5\.1Models and Configuration

Models\.Multiple closed\-source and open\-source models are considered into our evaluation\. The closed\-source ones include GPT\-5\.6\-Sol\([OpenAI, 2026c](https://arxiv.org/html/2610.22869#bib.bib19)\)and Claude\-Sonnet\-5\([Anthropic, 2026c](https://arxiv.org/html/2610.22869#bib.bib20)\), while the open\-source ones cover models of varying sizes: Qwen3\.8\-Max\([Qwen Team, 2026](https://arxiv.org/html/2610.22869#bib.bib21)\), Kimi\-K3\([Team et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib22)\), GLM\-5\.3\([Z\.ai, 2026b](https://arxiv.org/html/2610.22869#bib.bib23)\), MiniMax\-M3\([MiniMax, 2026b](https://arxiv.org/html/2610.22869#bib.bib25)\), GLM\-5\.3\-Flash\([Z\.ai, 2026a](https://arxiv.org/html/2610.22869#bib.bib24)\), and Deepseek\-V4\-Flash\-0731\([DeepSeek\-AI, 2026c](https://arxiv.org/html/2610.22869#bib.bib26)\)\. Considering the high cost\-effectiveness of Deepseek\-V4\-Flash\-0731, we use it for RQ1 experiments\. See Appx\.[B\.3](https://arxiv.org/html/2610.22869#A2.SS3)for detialed model inference configurations\.

Runtime\.All experiments proceed as fully autonomous, single\-turn, multi\-step runs with zero human intervention\. To accommodate the complex, long\-horizon episode, we provision every main\-agent and sub\-agent with a 128K context window, a 100\-step interaction limit, and a 7,200\-second running\-time budget\. Following default Kimi\-Code settings,AgentSwarmis capped at 128 concurrent sub\-agents\. To prevent unintended system exploitation, built\-in terminal tools are globally disabled, and the terminal access is granted strictly per task and safely supported via a local mock terminal MCP server\. During execution, a session terminates normally if the agent invokesclaim\_doneor yields a pure\-text response without any tool calls\. Besides it, the execution is also terminated upon breaching the step/time budgets, entering a loop of six consecutive repetitive tool calls, or exceeding the maximum API retry limit of ten\.

Table 1:The Comparison of MAD\-SAE variants and frontier models\. \(a\) In thevariants comparison, shaded area distinguishes Oracle from practical variants; bold marks the best practical result\. \(b\)In themodel comparison, bold and underline denote the best and second\-best results, respectively\.SettingParam\.Delegation Appropriateness \(DA\)Orchestration Utilities \(OU\)MSQ↑\\uparrowUDR↓\\downarrowODR↓\\downarrow\|𝒟del\|\|\\mathcal\{D\}\_\{\\mathrm\{del\}\}\|𝒫hit↑\\mathcal\{P\_\{\\text\{hit\}\}\}\\uparrowℛhit↑\\mathcal\{R\_\{\\text\{hit\}\}\}\\uparrowℱ1 hit↑\\mathcal\{F\}\_\{\\text\{1 hit\}\}\\uparrowUp↑U\_\{p\}\\uparrowUe↑U\_\{e\}\\uparrowUc↑U\_\{c\}\\uparrowAvg\.↑\\uparrow\(a\) MAD\-SAE Variants Comparison on Deepseek\-V4\-Flash\-0731Gen284B\(A13B\)52\.8%9\.0%4948\.0%18\.8%27\.1%48\.7%48\.9%31\.2%42\.9%11\.6%Spec16\.8%57\.1%11841\.0%56\.2%47\.4%50\.4%48\.0%33\.1%43\.8%20\.8%Plan\-1st15\.7%61\.0%12238\.4%51\.7%44\.1%48\.0%42\.1%29\.4%39\.9%17\.6%Oracle2\.2%1\.3%8888\.8%96\.6%92\.5%49\.6%53\.5%23\.6%42\.2%39\.1%\(b\) Model Comparison with SpecClosed\-source ModelsClaude\-Sonnet\-5\-39\.3%29\.8%7731\.1%31\.7%31\.4%45\.9%46\.8%34\.1%42\.3%13\.2%GPT\-5\.6\-Sol\-7\.8%93\.5%15429\.6%66\.7%41\.0%42\.6%44\.0%27\.4%38\.0%15\.6%Open\-source ModelsQwen\-3\.8\-Max15\.7%59\.7%12146\.7%48\.6%47\.6%54\.4%49\.9%38\.4%47\.6%22\.7%GLM\-5\.38\.9%64\.9%13146\.4%56\.9%51\.1%50\.8%52\.7%40\.5%48\.0%24\.6%Kimi\-K350\.5%16\.8%5752\.7%25\.2%34\.1%56\.4%50\.3%50\.9%52\.6%17\.9%MiniMax\-M379\.7%9\.0%2529\.4%10\.5%15\.5%42\.8%42\.7%37\.6%41\.0%6\.3%GLM\-5\.3\-Flash26\.9%50\.6%10443\.2%32\.4%37\.0%51\.3%54\.0%32\.0%45\.8%16\.9%DeepSeek\-V4\-Flash\-073116\.8%57\.1%11841\.0%56\.2%47\.4%50\.4%48\.0%33\.1%43\.8%20\.8%

## 6Main Results

MAS\-SAE Variants Comparison\.As Table[1](https://arxiv.org/html/2610.22869#S5.T1)\(a\), Spec achieves the highestℱ1 hit\\mathcal\{F\}\_\{\\text\{1 hit\}\}and average utilities among three practical variants, yielding a peak MSQ of 20\.8%\. The result outperform Gen by 9\.2 points, the counterpart with general sub\-agent profiles\. Gen’s underperformance stems from overly conservative delegation: despite a leading𝒫hit\\mathcal\{P\_\{\\text\{hit\}\}\}\(48\.0%\), its lowℛhit\\mathcal\{R\_\{\\text\{hit\}\}\}\(18\.8%\) and high UDR \(52\.8%\) leave over half of𝒟need\\mathcal\{D\}\_\{\\mathrm\{need\}\}tasks without MAD\-SAE benefits\. Conversely, Plan\-1st suffers from aggressive over\-delegation, erroneously invoking sub\-agents in 61\.0% of𝒟no​\-​need\\mathcal\{D\}\_\{\\mathrm\{no\\text\{\-\}need\}\}tasks\. This fails to translate into gains over SA, with all utilities \(Up,Ue,UcU\_\{p\},U\_\{e\},U\_\{c\}\) falling below the 50% parity line\. Ultimately, sub\-agent specialization variant of MAD\-SAE,Spec, emerges as the best practice, navigating between the extremes of conservative and aggressive delegation\.

Guided by delegation priors, Oracle provides near\-perfect delegations with 92\.5% ofℱ1 hit\\mathcal\{F\}\_\{\\text\{1 hit\}\}, which exceeds Gen \(the prior\-free baseline\) by 65\.4 points\. On the OU metrics, Oracle attains 49\.6% ofUpU\_\{p\}and 53\.5% ofUeU\_\{e\}, matching the SA baseline in performance while realizing the expected higher efficiency discussed in §[3\.1](https://arxiv.org/html/2610.22869#S3.SS1)\. Meanwhile, itsUcU\_\{c\}of 23\.6%, well below 50%, indicates substantial additional cost relative to SA, leaving Oracle with an MSQ of only 39\.1%\. Under near\-perfect sub\-agent delegation, Oracle isolates the trade\-off of MAD\-SAE: extra cost buys efficiency, but not necessarily performance gains\.

Models Comparison\.Confirming Spec’s superiority, Table[1](https://arxiv.org/html/2610.22869#S5.T1)\(b\) summarizes the Spec\-based MAD\-SAE results across eight frontier models\.

GPT\-5\.6\-Sol exhibits the most aggressive delegation behavior, with a massive ODR of 93\.5%\. Crucially, such over\-delegation fails to translate into orchestration advantages, with all its utilities falling below the 50% parity line\. Claude\-Sonnet\-5 similarly suffers from negative orchestration returns across the board \(UpU\_\{p\}: 45\.9%,UeU\_\{e\}: 46\.8%,UcU\_\{c\}: 34\.1%\)\. Consequently, both models languish near the bottom of the MSQ leaderboard in Fig[1](https://arxiv.org/html/2610.22869#S1.F1), with GPT\-5\.6\-Sol \(15\.6%\) and Claude\-Sonnet\-5 \(13\.2%\) ranking third\-to\-last and second\-to\-last, respectively\.

GLM\-5\.3 tops the MSQ leaderboard at 24\.6% through active yet rational delegation: its UDR of 8\.9% covers 91\.1% of𝒟need\\mathcal\{D\}\_\{\\mathrm\{need\}\}tasks, and itsℱ1 hit\\mathcal\{F\}\_\{\\text\{1 hit\}\}of 51\.1% is the highest overall\. Meanwhile, it preserves task performance while ranking second in efficiency, with aUeU\_\{e\}of 52\.7%\. GLM\-5\.3\-Flash achieves the highest efficiency utility, reaching aUeU\_\{e\}of 54\.0%\. Kimi\-K3 is the only model to surpass SA on performance, efficiency, and cost alike, solving 11 more tasks with the highestUpU\_\{p\}of 56\.4%\. Yet its conservative delegation, with anℱ1 hit\\mathcal\{F\}\_\{\\text\{1 hit\}\}of only 34\.1%, drags its MSQ down to 17\.9% and places it fourth\. verall,open\-source models dominate MSQ, with more appropriate sub\-agent delegation and thus greater orchestration gains in performance and efficiency\.

![Refer to caption](https://arxiv.org/html/2610.22869v1/performance_cost_efficiency_2_1.png)Figure 3:Performance\-cost \(left\) and performance\-efficiency \(right\) trade\-offs of MAD\-SAE relative to SA across the full benchmark𝒟\\mathcal\{D\}\. Each point represents a model\-variant configuration, with changes computed as MAD\-SAE minus SA\. All tags are not explicitly stated are in the Spec mode\. Upper\-left regions indicate higher success rates with lower cost or fewer steps\.
## 7Analysis

The OU analysis in §[6](https://arxiv.org/html/2610.22869#S6)was confined to𝒟del\\mathcal\{D\}\_\{\\text\{del\}\}, the subset with sub\-agent invocation, to isolate delegation\-specific benefits\. We now turn to the full benchmark𝒟\\mathcal\{D\}for a global comparison between MAD\-SAE and SA, examining macroscopic performance–efficiency/cost trade\-offs and the distinct behaviors each execution principle induces\.

Trade\-off Analysis\.Across nearly all models, MAD\-SAE carries a cost premium over SA \(Fig\.[3](https://arxiv.org/html/2610.22869#S6.F3), left\), ranging from \+$0\.1 for GLM\-5\.3\-Flash to \+$4\.2 for GPT\-5\.6\-Sol per task\. For open\-source models under the Spec variant, this premium typically buys a performance gain: Qwen\-3\.8\-Max, for instance, trades a $1\.0 cost increase for a 10\.2\-point rise in Success Rates \(SR\)\. K3 \(Spec\) is the sole configuration to escape this trade\-off entirely, cutting cost by $0\.4 per task while still lifting SR by 10\.2 points\. By contrast, closed\-source models incur a double penalty: inflated cost alongside degraded performance, echoing the negative utilities reported in Table[1](https://arxiv.org/html/2610.22869#S5.T1)\. The same pattern recurs for efficiency \(Fig\.[3](https://arxiv.org/html/2610.22869#S6.F3), right\): most open\-source models \(K3, Qwen\-3\.8\-Max, GLM\-5\.3, GLM\-5\.3\-Flash\) attain both higher SR and fewer steps under MAD\-SAE, whereas closed\-source models cluster in the lower\-right quadrant, suffering both lower success and more steps\. Taken together, these results point to a clear practical takeaway: For most open\-source models, MAD\-SAE with Spec is the preferable configuration, delivering gains in both performance and efficiency with more cost\. For closed\-source models, in contrast, it tends to raise cost while degrading both performance and efficiency, making SA the more practical choice\.

Figure 4:The distribution of beneficial and adverse categories, alongside with their underlying reasons\.Case Study\.The trade\-off analysis shows that MAD\-SAE does not always realize its advantages, depending on the setting\. Pairing each MAD\-SAE \(Spec\) result with its SA counterpart across eight models yields 625beneficialand 703adversecases, revealing MAD\-SAE as a double\-edged sword whose net payoff hinges on orchestration quality\. The full taxonomy and underlying reasons are detailed in Appx\.[C\.1](https://arxiv.org/html/2610.22869#A3.SS1)and[C\.2](https://arxiv.org/html/2610.22869#A3.SS2)\. Detailed model\-wise case analysis refer to Appx\.[D](https://arxiv.org/html/2610.22869#A4)\.

Strikingly,Main\-Agent\-Only \(MAO\)\(see Fig\.[16](https://arxiv.org/html/2610.22869#A4.F16)\) behavior, where the main\-agent solves the task unassisted, dominates the beneficial partition\. It accounts for 305 cases \(48\.8% of all benefits\)\. MAO also appears in 254 adverse cases, but its beneficial occurrences still outnumber its adverse ones, suggesting that models prompted under MAD\-SAE occasionally overfit to its semantic patterns yet serendipitously converge on more efficient single\-agent solutions\.

Beyond MAO,Context Offloading\(see Fig\.[13](https://arxiv.org/html/2610.22869#A4.F13)\) andDelegated Parallelism\(see Fig\.[14](https://arxiv.org/html/2610.22869#A4.F14)\) are the primary benefit drivers\. Context Offloading accounts for 148 Cheaper\-or\-Efficient\-Success cases by letting isolated sub\-agent contexts absorb heavy work so the main\-agent can focus on core solving, while Delegated Parallelism drives 88 Cheaper\-or\-Efficient\-Success and 41 Failure\-to\-Success cases by converting an infeasible serial workload within a bounded context into a feasible parallel one\. Conversely, ill\-posed delegation backfires:Unnecessary Delegation\(see Fig\.[17](https://arxiv.org/html/2610.22869#A4.F17)\) inflates overhead in 124 cases despite preserving success,Delegated Failure Propagation\(see Fig\.[18](https://arxiv.org/html/2610.22869#A4.F18)\) accounts for over half of the Success\-to\-Failure cases, andExecution Bloat\(see Fig\.[20](https://arxiv.org/html/2610.22869#A4.F20)\) dominates the Costly\-and\-Inefficient\-Success cases\.

These findings provide two takeaways: \(1\) MAD\-SAE can unexpectedly casue a better single\-agent solution compared to the SA execution paradigm; \(2\) Context offloading and parallelism of sub\-agents convert extra cost into efficiency and performance gains, whereas unnecessary, unverified or redundant delegation turns the same cost into pure overhead\.

## 8Conclusion

We present MS\-Eval to quantify MAD\-SAE via a benchmark, controlled harness, delegation and orchestration metrics\. Across four variants and eight frontier models, we find \(1\) Spec is the best practice; \(2\) open\-source models are largely competitive, with few exceptions; and \(3\) MAD\-SAE’s higher cost yields contingent payoff, benefiting most open\-source models in both performance and efficiency while leaving closed\-source models with higher cost and degraded performance, with these gains traced to better single\-agent solutions, context offloading, and sub\-task parallelism rather than delegation for its own sake\. Thus, MS\-Eval assesses the sub\-agent delegation and orchestration comprehensively, revealing the benefits and drawbacks of MAD\-SAE in MCP\-intensive working scenarios\.

## References

- Alibaba CloudQwen3\.8\-max pricing\.Note:Qianwen AI Model DocumentationExternal Links:[Link](https://www.qianwenai.com/models/qwen3.8-max)Cited by:[§B\.3](https://arxiv.org/html/2610.22869#A2.SS3.p2.1)\.
- Anthropic \(2024\)AnthropicIntroducing the model context protocol\.Note:[https://www\.anthropic\.com/news/model\-context\-protocol](https://www.anthropic.com/news/model-context-protocol)Cited by:[§1](https://arxiv.org/html/2610.22869#S1.p3.1),[§2](https://arxiv.org/html/2610.22869#S2.p1.1)\.
- Anthropic \(2026a\)AnthropicClaude api pricing\.Note:Anthropic API DocumentationExternal Links:[Link](https://platform.claude.com/docs/en/about-claude/pricing)Cited by:[§B\.3](https://arxiv.org/html/2610.22869#A2.SS3.p2.1)\.
- Anthropic \(2026b\)AnthropicClaude code\.Note:Claude Code DocumentationAccessed: 2026\-09\-16External Links:[Link](https://code.claude.com/docs/en/overview)Cited by:[1st item](https://arxiv.org/html/2610.22869#A2.I1.i1.p1.1),[3rd item](https://arxiv.org/html/2610.22869#A2.I1.i3.p1.1),[4th item](https://arxiv.org/html/2610.22869#A2.I1.i4.p1.1),[§B\.4](https://arxiv.org/html/2610.22869#A2.SS4.p1.1),[§1](https://arxiv.org/html/2610.22869#S1.p3.1),[§2](https://arxiv.org/html/2610.22869#S2.p1.1),[§2](https://arxiv.org/html/2610.22869#S2.p2.1),[§3\.1](https://arxiv.org/html/2610.22869#S3.SS1.p1.1)\.
- Anthropic \(2026c\)AnthropicClaude sonnet 5\.External Links:[Link](https://www.anthropic.com/news/claude-sonnet-5)Cited by:[§5\.1](https://arxiv.org/html/2610.22869#S5.SS1.p1.1)\.
- Bandiet al\.\(2026\)C\. Bandi, R\. Dumitru, B\. Hertzberg, D\. Agarwal, G\. Boo, T\. Polakam, S\. Hassaan, J\. Da, H\. Kim, V\. Gupta, M\. Sharma, A\. Park, M\. Dimakis, E\. G\. H\. Montoya, D\. Rambado, I\. Salazar, R\. Cruz, M\. Rezaei, C\. Rane, B\. Levin, D\. Y\. Zhang, B\. Kenstler, and B\. LiuMCP\-atlas: a large\-scale benchmark for tool\-use competency with real mcp servers\.External Links:2602\.00933,[Link](https://arxiv.org/abs/2602.00933)Cited by:[§1](https://arxiv.org/html/2610.22869#S1.p2.1),[§2](https://arxiv.org/html/2610.22869#S2.p1.1)\.
- Cursor \(2026a\)CursorCursor\.Note:Cursor DocumentationAccessed: 2026\-09\-16External Links:[Link](https://cursor.com/en-US/docs)Cited by:[3rd item](https://arxiv.org/html/2610.22869#A2.I1.i3.p1.1),[4th item](https://arxiv.org/html/2610.22869#A2.I1.i4.p1.1),[§B\.4](https://arxiv.org/html/2610.22869#A2.SS4.p1.1),[§1](https://arxiv.org/html/2610.22869#S1.p3.1),[§2](https://arxiv.org/html/2610.22869#S2.p1.1),[§2](https://arxiv.org/html/2610.22869#S2.p2.1),[§3\.1](https://arxiv.org/html/2610.22869#S3.SS1.p1.1)\.
- Cursor \(2026b\)CursorSubagents\.External Links:[Link](https://cursor.com/docs/subagents)Cited by:[2nd item](https://arxiv.org/html/2610.22869#A2.I1.i2.p1.1),[§3\.1](https://arxiv.org/html/2610.22869#S3.SS1.p4.1)\.
- DeepSeek\-AI \(2026a\)DeepSeek\-AIDeepSeek api pricing\.Note:DeepSeek API DocumentationExternal Links:[Link](https://platform.deepseek.com/usage)Cited by:[§B\.3](https://arxiv.org/html/2610.22869#A2.SS3.p2.1)\.
- DeepSeek\-AI \(2026b\)DeepSeek\-AIDeepSeek\-v4\.1\-flash technical report\.Note:Technical ReportExternal Links:[Link](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/DeepSeek_V41_Tech_Report.pdf)Cited by:[§1](https://arxiv.org/html/2610.22869#S1.p1.1),[§4\.3\.1](https://arxiv.org/html/2610.22869#S4.SS3.SSS1.p3.1)\.
- DeepSeek\-AI \(2026c\)DeepSeek\-AIDeepSeek\-v4: towards highly efficient million\-token context intelligence\.Cited by:[§5\.1](https://arxiv.org/html/2610.22869#S5.SS1.p1.1)\.
- Dinget al\.\(2026\)B\. Ding, Y\. Chen, J\. Lyu, J\. Yuan, Q\. Zhu, S\. Tian, D\. Zhu, F\. Wang, H\. Deng, F\. Mi, L\. Shang, and T\. LinRethinking expert trajectory utilization in LLM post\-training for mathematical reasoning\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 33081–33106\.External Links:[Link](https://aclanthology.org/2026.acl-long.1528/),[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.1528),ISBN 979\-8\-89176\-390\-6Cited by:[§1](https://arxiv.org/html/2610.22869#S1.p1.1)\.
- Duet al\.\(2023\)Y\. Du, S\. Li, A\. Torralba, J\. B\. Tenenbaum, and I\. MordatchImproving factuality and reasoning in language models through multiagent debate\.External Links:2305\.14325,[Link](https://arxiv.org/abs/2305.14325)Cited by:[§2](https://arxiv.org/html/2610.22869#S2.p2.1)\.
- Fuet al\.\(2026\)Y\. Fu, R\. Fang, J\. Shao, H\. Zheng, Z\. Zhu, B\. Luo, and T\. LinDo more agents help? controlled and protocol\-aligned evaluation of llm agent workflows\.External Links:2606\.05670,[Link](https://arxiv.org/abs/2606.05670)Cited by:[§2](https://arxiv.org/html/2610.22869#S2.p2.1)\.
- Gaoet al\.\(2025\)X\. Gao, S\. Xie, J\. Zhai, S\. Ma, and C\. ShenMCP\-radar: a multi\-dimensional benchmark for evaluating tool use capabilities in large language models\.External Links:2505\.16700,[Link](https://arxiv.org/abs/2505.16700)Cited by:[§2](https://arxiv.org/html/2610.22869#S2.p1.1)\.
- GLM\-5\-Teamet al\.\(2026\)GLM\-5\-Team, :, A\. Zeng, X\. Lv, Z\. Hou, Z\. Du, Q\. Zheng, B\. Chen, D\. Yin, C\. Ge, C\. Huang, C\. Xie, C\. Zhu, C\. Yin, C\. Wang, G\. Pan, H\. Zeng, H\. Zhang, H\. Wang, H\. Chen, J\. Zhang, J\. Jiao, J\. Guo, J\. Wang, J\. Du, J\. Wu, K\. Wang, L\. Li, L\. Fan, L\. Zhong, M\. Liu, M\. Zhao, P\. Du, Q\. Dong, R\. Lu, Shuang\-Li, S\. Cao, S\. Liu, T\. Jiang, X\. Chen, X\. Zhang, X\. Huang, X\. Dong, Y\. Xu, Y\. Wei, Y\. An, Y\. Niu, Y\. Zhu, Y\. Wen, Y\. Cen, Y\. Bai, Z\. Qiao, Z\. Wang, Z\. Wang, Z\. Zhu, Z\. Liu, Z\. Li, B\. Wang, B\. Wen, C\. Huang, C\. Cai, C\. Yu, C\. Li, C\. Hu, C\. Zhang, D\. Zhang, D\. Lin, D\. Yang, D\. Wang, D\. Ai, E\. Zhu, F\. Yi, F\. Chen, G\. Wen, H\. Sun, H\. Zhao, H\. Hu, H\. Zhang, H\. Liu, H\. Zhang, H\. Peng, H\. Tai, H\. Zhang, H\. Liu, H\. Wang, H\. Yan, H\. Ge, H\. Liu, H\. Chu, J\. Zhao, J\. Wang, J\. Zhao, J\. Ren, J\. Wang, J\. Zhang, J\. Gui, J\. Zhao, J\. Li, J\. An, J\. Li, J\. Yuan, J\. Du, J\. Liu, J\. Zhi, J\. Duan, K\. Zhou, K\. Wei, K\. Wang, K\. Luo, L\. Zhang, L\. Sha, L\. Xu, L\. Wu, L\. Ding, L\. Chen, M\. Li, N\. Lin, P\. Ta, Q\. Zou, R\. Song, R\. Yang, S\. Tu, S\. Yang, S\. Wu, S\. Zhang, S\. Li, S\. Li, S\. Fan, W\. Qin, W\. Tian, W\. Zhang, W\. Yu, W\. Liang, X\. Kuang, X\. Cheng, X\. Li, X\. Yan, X\. Hu, X\. Ling, X\. Fan, X\. Xia, X\. Zhang, X\. Zhang, X\. Pan, X\. Zou, X\. Zhang, Y\. Liu, Y\. Wu, Y\. Li, Y\. Wang, Y\. Zhu, Y\. Tan, Y\. Zhou, Y\. Pan, Y\. Zhang, Y\. Su, Y\. Geng, Y\. Yan, Y\. Tan, Y\. Bi, Y\. Shen, Y\. Yang, Y\. Li, Y\. Liu, Y\. Wang, Y\. Li, Y\. Wu, Y\. Zhang, Y\. Duan, Y\. Zhang, Z\. Liu, Z\. Jiang, Z\. Yan, Z\. Zhang, Z\. Wei, Z\. Chen, Z\. Feng, Z\. Yao, Z\. Chai, Z\. Wang, Z\. Zhang, B\. Xu, M\. Huang, H\. Wang, J\. Li, Y\. Dong, and J\. TangGLM\-5: from vibe coding to agentic engineering\.External Links:2602\.15763,[Link](https://arxiv.org/abs/2602.15763)Cited by:[§1](https://arxiv.org/html/2610.22869#S1.p1.1)\.
- Guoet al\.\(2024\)T\. Guo, X\. Chen, Y\. Wang, R\. Chang, S\. Pei, N\. V\. Chawla, O\. Wiest, and X\. ZhangLarge language model based multi\-agents: a survey of progress and challenges\.InProceedings of the Thirty\-Third International Joint Conference on Artificial Intelligence, IJCAI\-24,K\. Larson \(Ed\.\),pp\. 8048–8057\.Note:Survey TrackExternal Links:[Document](https://dx.doi.org/10.24963/ijcai.2024/890),[Link](https://doi.org/10.24963/ijcai.2024/890)Cited by:[§2](https://arxiv.org/html/2610.22869#S2.p2.1)\.
- Guoet al\.\(2025\)Z\. Guo, B\. Xu, C\. Zhu, W\. Hong, X\. Wang, and Z\. MaoMCP\-agentbench: evaluating real\-world language agent performance with mcp\-mediated tools\.External Links:2509\.09734,[Link](https://arxiv.org/abs/2509.09734)Cited by:[§2](https://arxiv.org/html/2610.22869#S2.p1.1)\.
- Gurjaret al\.\(2026\)P\. Gurjar, M\. F\. Ishmam, and K\. MarinoDORA explorer: improving the exploration ability of llms without training\.External Links:2604\.17244,[Link](https://arxiv.org/abs/2604.17244)Cited by:[§2](https://arxiv.org/html/2610.22869#S2.p2.1)\.
- Kaplanet al\.\(2020\)J\. Kaplan, S\. McCandlish, T\. Henighan, T\. B\. Brown, B\. Chess, R\. Child, S\. Gray, A\. Radford, J\. Wu, and D\. AmodeiScaling laws for neural language models\.External Links:2001\.08361,[Link](https://arxiv.org/abs/2001.08361)Cited by:[§1](https://arxiv.org/html/2610.22869#S1.p1.1)\.
- Kimi\-Teamet al\.\(2026\)Kimi\-Team, T\. Bai, Y\. Bai, Y\. Bao, S\. H\. Cai, Y\. Cao, Z\. Chai, Y\. Charles, H\. S\. Che, C\. Chen, G\. Chen, H\. Chen, J\. Chen, J\. Chen, J\. Chen, K\. Chen, L\. Chen, R\. Chen, X\. Chen, Y\. Chen, Y\. Chen, Y\. Chen, Y\. Chen, Y\. Chen, Y\. Chen, Y\. Chen, Y\. Chen, Z\. Chen, Z\. Chen, D\. Cheng, Y\. Cheng, M\. Chu, J\. Cui, J\. Deng, M\. Diao, H\. Ding, M\. Dong, M\. Dong, Y\. Dong, Y\. Dong, A\. Du, C\. Du, D\. Du, L\. Du, Y\. Du, Y\. Fan, S\. Fang, Q\. Feng, Y\. Feng, G\. Fu, K\. Fu, H\. Gao, T\. Gao, Y\. Ge, S\. Geng, C\. Gong, X\. Gong, Z\. Gongque, Q\. Gu, X\. Gu, Y\. Gu, L\. Guan, S\. Guan, Y\. Guo, X\. Hao, D\. He, T\. He, W\. He, W\. He, Y\. He, Y\. He, C\. Hong, H\. Hu, J\. Hu, Y\. Hu, Z\. Hu, K\. Huang, R\. Huang, W\. Huang, Z\. Huang, C\. Jia, T\. Jiang, Z\. Jiang, X\. Jin, Y\. Jing, G\. Lai, A\. Li, C\. Li, C\. Li, F\. Li, G\. Li, G\. Li, H\. Li, H\. Li, J\. Li, J\. Li, J\. Li, L\. Li, M\. Li, W\. Li, W\. Li, X\. Li, X\. Li, Y\. Li, Y\. Li, Y\. Li, Y\. Li, Z\. Li, Z\. Li, Z\. Li, W\. Liao, J\. Lin, X\. Lin, Y\. Lin, Z\. Lin, Z\. Lin, C\. Liu, C\. Liu, H\. Liu, L\. Liu, S\. Liu, S\. Liu, S\. Liu, T\. Liu, T\. Liu, W\. Liu, X\. Liu, Y\. Liu, Y\. Liu, Y\. Liu, Y\. Liu, Z\. Liu, Z\. Liu, E\. Lu, H\. Lu, Z\. Lu, G\. Luo, J\. Luo, T\. Luo, Y\. Luo, L\. Ma, S\. Mao, Y\. Mei, X\. Men, F\. Meng, Z\. Meng, Y\. Miao, M\. Ni, K\. Ouyang, S\. Pan, B\. Pang, Y\. Qian, R\. Qin, Z\. Qin, J\. Qiu, B\. Qu, Z\. Shang, Y\. Shao, T\. Shen, Z\. Shen, J\. Shi, L\. Shi, S\. Shi, F\. Song, P\. Song, T\. Song, X\. Song, H\. Su, J\. Su, Z\. Su, L\. Sui, J\. Sun, J\. Sun, T\. Sun, F\. Sung, Y\. Tai, C\. Tang, H\. Tang, X\. Tang, Z\. Tang, J\. Tao, S\. Teng, C\. Tian, P\. Tian, B\. Wang, C\. Wang, C\. Wang, C\. Wang, D\. Wang, D\. Wang, D\. Wang, F\. Wang, H\. Wang, H\. Wang, H\. Wang, H\. Wang, H\. Wang, H\. Wang, J\. Wang, J\. Wang, J\. Wang, K\. Wang, L\. Wang, Q\. Wang, S\. Wang, S\. Wang, S\. Wang, W\. Wang, X\. Wang, X\. Wang, Y\. Wang, Y\. Wang, Y\. Wang, Y\. Wang, Y\. Wang, Y\. Wang, Z\. Wang, Z\. Wang, Z\. Wang, Z\. Wang, Z\. Wang, Z\. Wang, Z\. Wang, C\. Wei, M\. Wei, C\. Wen, Z\. Wen, C\. Wu, H\. Wu, J\. Wu, R\. Wu, W\. Wu, Y\. Wu, Y\. Wu, Y\. Wu, Z\. Wu, C\. Xiao, J\. Xie, X\. Xie, Y\. Xie, B\. Xing, B\. Xu, J\. Xu, J\. Xu, J\. Xu, L\. H\. Xu, L\. Xu, S\. Xu, W\. Xu, X\. Xu, X\. Xu, Y\. Xu, Y\. Xu, Y\. Xu, Z\. Xu, Z\. Xu, J\. Yan, Y\. Yan, G\. Yang, H\. Yang, J\. Yang, K\. Yang, N\. Yang, X\. Yang, X\. Yang, X\. Yang, Y\. Yang, Y\. Yang, Y\. Yang, Z\. Yang, Z\. Yang, Z\. Yang, H\. Yao, D\. Ye, H\. Ye, W\. Ye, Z\. Ye, P\. Yebo, B\. Yin, C\. Yu, L\. Yu, T\. Yu, T\. Yu, E\. Yuan, M\. Yuan, X\. Yuan, Y\. Yue, W\. Zeng, D\. Zha, H\. Zhan, D\. Zhang, H\. Zhang, J\. Zhang, P\. Zhang, Q\. Zhang, R\. Zhang, X\. Zhang, X\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Z\. Zhang, C\. Zhao, F\. Zhao, J\. Zhao, S\. Zhao, X\. Zhao, X\. Zhao, Y\. Zhao, Z\. Zhao, H\. Zheng, R\. Zheng, S\. Zheng, T\. Zheng, J\. Zhong, L\. Zhong, W\. Zhong, M\. Zhou, R\. Zhou, X\. Zhou, Z\. Zhou, J\. Zhu, L\. Zhu, X\. Zhu, Y\. Zhu, Z\. Zhu, J\. Zhuang, W\. Zhuang, Y\. Zou, and X\. ZuKimi k2\.5: visual agentic intelligence\.External Links:2602\.02276,[Link](https://arxiv.org/abs/2602.02276)Cited by:[§B\.1](https://arxiv.org/html/2610.22869#A2.SS1.p3.1),[§1](https://arxiv.org/html/2610.22869#S1.p2.1),[§2](https://arxiv.org/html/2610.22869#S2.p2.1),[§3\.1](https://arxiv.org/html/2610.22869#S3.SS1.p4.1)\.
- Liet al\.\(2023\)G\. Li, H\. Hammoud, H\. Itani, D\. Khizbullin, and B\. GhanemCAMEL: communicative agents for "Mind" exploration of large language model society\.InAdvances in Neural Information Processing Systems,Vol\.36\.External Links:[Document](https://dx.doi.org/10.52202/075280-2264)Cited by:[§2](https://arxiv.org/html/2610.22869#S2.p2.1)\.
- Liet al\.\(2026\)J\. Li, W\. Zhao, J\. Zhao, W\. Zeng, H\. Wu, X\. Wang, R\. Ge, Y\. Cao, Y\. Huang, W\. Liu, J\. Liu, Z\. Su, Y\. Guo, F\. Zhou, L\. Zhang, J\. Michelini, X\. Wang, X\. Yue, S\. Zhou, G\. Neubig, and J\. HeThe tool decathlon: benchmarking language agents for diverse, realistic, and long\-horizon task execution\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2610.22869#S1.p2.1),[§2](https://arxiv.org/html/2610.22869#S2.p1.1)\.
- Liuet al\.\(2024\)N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. LiangLost in the middle: how language models use long contexts\.Transactions of the Association for Computational Linguistics12,pp\. 157–173\.External Links:[Link](https://aclanthology.org/2024.tacl-1.9/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00638)Cited by:[§1](https://arxiv.org/html/2610.22869#S1.p2.1)\.
- Liuet al\.\(2025\)Z\. Liu, J\. Qiu, S\. Wang, J\. Zhang, Z\. Liu, R\. Ram, H\. Chen, W\. Yao, S\. Heinecke, S\. Savarese, H\. Wang, and C\. XiongMCPEval: automatic mcp\-based deep evaluation for ai agent models\.External Links:2507\.12806,[Link](https://arxiv.org/abs/2507.12806)Cited by:[§2](https://arxiv.org/html/2610.22869#S2.p1.1)\.
- Luoet al\.\(2025\)Z\. Luo, Z\. Shen, W\. Yang, Z\. Zhao, P\. Jwalapuram, A\. Saha, D\. Sahoo, S\. Savarese, C\. Xiong, and J\. LiMCP\-universe: benchmarking large language models with real\-world model context protocol servers\.External Links:2508\.14704,[Link](https://arxiv.org/abs/2508.14704)Cited by:[§2](https://arxiv.org/html/2610.22869#S2.p1.1)\.
- Mastermanet al\.\(2024\)T\. Masterman, S\. Besen, M\. Sawtell, and A\. ChaoThe landscape of emerging ai agent architectures for reasoning, planning, and tool calling: a survey\.External Links:2404\.11584,[Link](https://arxiv.org/abs/2404.11584)Cited by:[§2](https://arxiv.org/html/2610.22869#S2.p2.1)\.
- MiniMax \(2026a\)MiniMaxMiniMax api pay\-as\-you\-go pricing\.Note:MiniMax API DocumentationExternal Links:[Link](https://platform.minimax.com/docs/guides/pricing-paygo)Cited by:[§B\.3](https://arxiv.org/html/2610.22869#A2.SS3.p2.1)\.
- MiniMax \(2026b\)MiniMaxMiniMax M3: frontier coding, 1m context, native multimodality — all in one model\.Note:[https://www\.minimax\.io/blog/minimax\-m3](https://www.minimax.io/blog/minimax-m3)Accessed: 2026\-09\-13Cited by:[§5\.1](https://arxiv.org/html/2610.22869#S5.SS1.p1.1)\.
- Moet al\.\(2026\)G\. Mo, W\. Zhong, J\. Chen, Q\. Yuan, X\. Chen, Y\. Lu, H\. Lin, B\. He, X\. Han, and L\. SunLiveMCPBench: can agents navigate an ocean of mcp tools?\.External Links:2508\.01780,[Link](https://arxiv.org/abs/2508.01780)Cited by:[§2](https://arxiv.org/html/2610.22869#S2.p1.1)\.
- Moonshot AI \(2026a\)Moonshot AIBuilt\-in tools: agent and AgentSwarm\.External Links:[Link](https://moonshotai.github.io/kimi-code/en/reference/tools.html)Cited by:[1st item](https://arxiv.org/html/2610.22869#A2.I1.i1.p1.1),[3rd item](https://arxiv.org/html/2610.22869#A2.I1.i3.p1.1),[4th item](https://arxiv.org/html/2610.22869#A2.I1.i4.p1.1),[§B\.1](https://arxiv.org/html/2610.22869#A2.SS1.p4.1),[§B\.4](https://arxiv.org/html/2610.22869#A2.SS4.p1.1),[§4\.2](https://arxiv.org/html/2610.22869#S4.SS2.p1.1)\.
- Moonshot AI \(2026b\)Moonshot AIKimi k3 api pricing\.Note:Kimi API DocumentationExternal Links:[Link](https://platform.kimi.com/docs/pricing/chat-k3.md)Cited by:[§B\.3](https://arxiv.org/html/2610.22869#A2.SS3.p2.1)\.
- OpenAI \(2026a\)OpenAICodex\.Note:GitHub repositoryAccessed: 2026\-09\-16External Links:[Link](https://github.com/openai/codex)Cited by:[3rd item](https://arxiv.org/html/2610.22869#A2.I1.i3.p1.1),[4th item](https://arxiv.org/html/2610.22869#A2.I1.i4.p1.1),[§B\.4](https://arxiv.org/html/2610.22869#A2.SS4.p1.1),[§1](https://arxiv.org/html/2610.22869#S1.p3.1),[§2](https://arxiv.org/html/2610.22869#S2.p1.1),[§2](https://arxiv.org/html/2610.22869#S2.p2.1),[§3\.1](https://arxiv.org/html/2610.22869#S3.SS1.p1.1)\.
- OpenAI \(2026b\)OpenAIGPT\-5\.6 sol api pricing\.Note:OpenAI API DocumentationExternal Links:[Link](https://developers.openai.com/api/docs/models/gpt-5.6-sol)Cited by:[§B\.3](https://arxiv.org/html/2610.22869#A2.SS3.p2.1)\.
- OpenAI \(2026c\)OpenAIGPT\-5\.6: Frontier Intelligence That Scales with Your Ambition\.Note:[https://openai\.com/index/gpt\-5\-6/](https://openai.com/index/gpt-5-6/)Cited by:[§5\.1](https://arxiv.org/html/2610.22869#S5.SS1.p1.1)\.
- Parket al\.\(2023\)J\. S\. Park, J\. C\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. BernsteinGenerative agents: interactive simulacra of human behavior\.External Links:2304\.03442,[Link](https://arxiv.org/abs/2304.03442)Cited by:[§2](https://arxiv.org/html/2610.22869#S2.p2.1)\.
- Qwen Team \(2026\)Qwen TeamQwen3\.8\-max: a new bar for coding and cowork\.External Links:[Link](https://qwen.ai/blog?id=qwen3.8)Cited by:[§5\.1](https://arxiv.org/html/2610.22869#S5.SS1.p1.1)\.
- Teamet al\.\(2026\)K\. Team, T\. Bai, Y\. Bai, Y\. Bao, M\. C\., J\. Cai, X\. Cai, P\. Cao, Y\. Cao, Z\. Chai, Y\. Charles, H\. S\. Che, G\. Chen, G\. Chen, G\. Chen, H\. Chen, J\. Chen, J\. Chen, J\. Chen, K\. Chen, P\. Chen, R\. Chen, W\. Chen, X\. Chen, Y\. Chen, Y\. Chen, Y\. Chen, Y\. Chen, Y\. Chen, Y\. Chen, Y\. Chen, Z\. Chen, D\. Cheng, Y\. Cheng, J\. Cui, J\. Cui, A\. Dai, J\. Deng, H\. Ding, R\. Ding, S\. Ding, M\. Dong, M\. Dong, Y\. Dong, Y\. Dong, A\. Du, C\. Du, D\. Du, J\. Du, Y\. Du, Y\. Fan, J\. Feng, Q\. Feng, Y\. Feng, K\. Fu, Q\. Fu, F\. Gao, H\. Gao, J\. Gao, T\. Gao, W\. Gao, S\. Geng, J\. Gong, L\. Gong, S\. Gong, X\. Gong, Q\. Gu, Y\. Gu, S\. Guan, H\. Guo, S\. Guo, X\. Guo, Z\. Guo, B\. Hao, W\. Hao, X\. Hao, D\. He, H\. He, L\. He, Q\. He, W\. He, X\. He, X\. He, Y\. He, Y\. He, C\. Hong, T\. Hong, H\. Hu, J\. Hu, R\. Hu, W\. Hu, Y\. Hu, Z\. Hu, L\. Hua, J\. Huang, K\. Huang, R\. Huang, S\. Huang, W\. Huang, Y\. Huang, Z\. Huang, Z\. Huang, Y\. Hui, C\. Jia, Y\. Jiang, Z\. Jiang, Z\. Jiang, W\. Jin, X\. Jin, Y\. Jing, H\. Kong, G\. Lai, A\. Li, C\. Li, C\. Li, C\. Li, F\. Li, G\. Li, H\. Li, J\. Li, J\. Li, L\. Li, L\. Li, L\. Li, W\. Li, W\. Li, X\. Li, Y\. Li, Y\. Li, Y\. Li, Y\. Li, Z\. Li, Z\. Li, Z\. Li, Z\. Li, Z\. Li, J\. Lin, X\. Lin, Y\. Lin, Z\. Lin, Z\. Lin, B\. Liu, B\. Liu, C\. Liu, L\. Liu, S\. Liu, S\. Liu, S\. Liu, T\. Liu, W\. Liu, Y\. Liu, Y\. Liu, Y\. Liu, Y\. Liu, Z\. Liu, Z\. Liu, E\. Lu, H\. Lu, L\. Lu, T\. Lu, Z\. Lu, A\. Luo, G\. Luo, J\. Luo, Y\. Luo, B\. Lyu, W\. Lyu, S\. Mao, Y\. Mei, X\. Men, M\. Ni, Y\. Niu, S\. Pan, S\. Peng, Z\. Qi, R\. Qin, Z\. Qin, Z\. Qin, H\. Qiu, J\. Qiu, J\. Qiu, B\. Qu, Y\. Qu, Z\. Shang, Y\. Shao, H\. Shen, J\. Shi, J\. Shi, L\. Shi, S\. Shi, W\. Siu, P\. Song, X\. Song, J\. Su, Y\. Su, Z\. Su, L\. Sui, J\. Sun, J\. Sun, S\. Sun, S\. Sun, T\. Sun, Y\. Sun, Y\. Tai, C\. Tang, H\. Tang, S\. Tang, Z\. Tang, C\. Tian, R\. Tian, Y\. Tian, W\. Tu, C\. Wang, C\. Wang, C\. Wang, D\. Wang, F\. Wang, H\. Wang, H\. Wang, H\. Wang, H\. Wang, H\. Wang, H\. Wang, J\. Wang, J\. Wang, J\. Wang, J\. Wang, L\. Wang, S\. Wang, S\. Wang, S\. Wang, S\. Wang, S\. Wang, T\. Wang, W\. Wang, X\. Wang, X\. Wang, X\. Wang, X\. Wang, Y\. Wang, Y\. Wang, Y\. Wang, Y\. Wang, Y\. Wang, Y\. Wang, Y\. Wang, Y\. Wang, Z\. Wang, Z\. Wang, Z\. Wang, Z\. Wang, Z\. Wang, Z\. Wang, C\. Wei, M\. Wei, S\. Wei, Z\. Wen, F\. Wu, H\. Wu, R\. Wu, W\. Wu, X\. Wu, Y\. Wu, Y\. Wu, Y\. Wu, Z\. Wu, X\. Xian, C\. Xiang, Y\. Xiang, B\. Xiao, C\. Xiao, X\. Xiao, J\. Xie, X\. Xie, Y\. Xie, Z\. Xie, B\. Xing, Y\. Xiong, B\. Xu, B\. Xu, J\. Xu, J\. Xu, J\. Xu, J\. Xu, L\. H\. Xu, Q\. Xu, S\. Xu, S\. Xu, T\. Xu, T\. Xu, W\. Xu, X\. Xu, Y\. Xu, Y\. Xu, Y\. Xu, Z\. Xu, H\. Xue, J\. Yan, Y\. Yan, F\. Yang, G\. Yang, H\. Yang, J\. Yang, R\. Yang, W\. Yang, X\. Yang, X\. Yang, Y\. Yang, Y\. Yang, Y\. Yang, Y\. Yang, Z\. Yang, Z\. Yang, Z\. Yang, Z\. Yang, H\. Yao, D\. Ye, H\. Ye, W\. Ye, Z\. Ye, B\. Yin, H\. Yin, X\. Yin, C\. Yu, H\. Yu, L\. Yu, S\. Yu, S\. Yu, T\. Yu, E\. Yuan, M\. Yuan, T\. Yue, W\. Yue, Y\. Yue, D\. Zha, H\. Zhan, B\. H\. Zhang, D\. Zhang, F\. Zhang, H\. Zhang, H\. Zhang, H\. Zhang, J\. Zhang, J\. Zhang, J\. Zhang, K\. Zhang, M\. Zhang, P\. Zhang, Q\. Zhang, R\. Zhang, R\. Zhang, S\. Zhang, S\. Zhang, X\. Zhang, X\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Z\. Zhang, Z\. Zhang, B\. Zhao, C\. Zhao, F\. Zhao, J\. Zhao, J\. Zhao, S\. Zhao, W\. Zhao, X\. Zhao, X\. Zhao, Y\. Zhao, Z\. Zhao, H\. Zheng, H\. Zheng, R\. Zheng, S\. Zheng, T\. Zheng, H\. Zhong, L\. Zhong, L\. Zhong, M\. Zhou, Q\. Zhou, R\. Zhou, R\. Zhou, X\. Zhou, Y\. Zhou, Z\. Zhou, J\. Zhu, L\. Zhu, X\. Zhu, Y\. Zhu, Y\. Zhu, Z\. Zhu, C\. Zhuang, W\. Zhuang, and X\. ZuKimi k3: open frontier intelligence\.External Links:2607\.24653,[Link](https://arxiv.org/abs/2607.24653)Cited by:[§5\.1](https://arxiv.org/html/2610.22869#S5.SS1.p1.1)\.
- TRAE Team \(2026\)TRAE TeamTRAE: ai coding agent and development environment\.Note:[https://www\.trae\.ai/ide](https://www.trae.ai/ide)Cited by:[1st item](https://arxiv.org/html/2610.22869#A2.I1.i1.p1.1),[3rd item](https://arxiv.org/html/2610.22869#A2.I1.i3.p1.1),[4th item](https://arxiv.org/html/2610.22869#A2.I1.i4.p1.1),[§B\.4](https://arxiv.org/html/2610.22869#A2.SS4.p1.1)\.
- Wuet al\.\(2025\)Z\. Wu, X\. Liu, X\. Zhang, L\. Chen, F\. Meng, L\. Du, Y\. Zhao, F\. Zhang, Y\. Ye, J\. Wang, Z\. Wang, J\. Ni, Y\. Yang, A\. Xu, and M\. Q\. ShiehMCPMark: a benchmark for stress\-testing realistic and comprehensive mcp use\.External Links:2509\.24002,[Link](https://arxiv.org/abs/2509.24002)Cited by:[§2](https://arxiv.org/html/2610.22869#S2.p1.1)\.
- Xionget al\.\(2026\)K\. Xiong, H\. Ji, S\. Qiu, Z\. Zheng, C\. Xie, X\. Ye, and H\. YaoClawArena\-Team: benchmarking subagent orchestration and dynamic workflows in language\-model agents\.arXiv preprint arXiv:2606\.31174\.Cited by:[§2](https://arxiv.org/html/2610.22869#S2.p2.1)\.
- Xuet al\.\(2026\)F\. F\. Xu, Y\. Song, B\. Li, Y\. Tang, K\. Jain, M\. Bao, Z\. Wang, X\. Zhou, Z\. Guo, M\. Cao,et al\.Theagentcompany: benchmarking llm agents on consequential real world tasks\.Advances in Neural Information Processing Systems38\.Cited by:[§1](https://arxiv.org/html/2610.22869#S1.p1.1)\.
- Xueet al\.\(2026\)B\. Xue, B\. Wu, S\. Qiao, S\. Wang, R\. Wang, Y\. Du, H\. Wang, J\. Z\. Pan, E\. Yilmaz, K\. Wong, and A\. LipaniMitigating context interference for reliable and efficient search agents\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 3541–3558\.External Links:[Link](https://aclanthology.org/2026.acl-long.160/),[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.160),ISBN 979\-8\-89176\-390\-6Cited by:[§1](https://arxiv.org/html/2610.22869#S1.p2.1)\.
- Yanet al\.\(2025\)Y\. Yan, S\. Wang, J\. Du, Y\. Yang, Y\. Shan, Q\. Qiu, X\. Jia, X\. Wang, X\. Yuan, X\. Han, M\. Qin, Y\. Chen, C\. Peng, S\. Wang, and M\. XuMCPWorld: a unified benchmarking testbed for api, gui, and hybrid computer use agents\.External Links:2506\.07672,[Link](https://arxiv.org/abs/2506.07672)Cited by:[§2](https://arxiv.org/html/2610.22869#S2.p1.1)\.
- Yinet al\.\(2026\)M\. Yin, D\. Shen, S\. Xu, S\. Dong, M\. Zhang, Y\. Hu, S\. Liu, J\. Han, S\. Ma, S\. Wang, S\. R\. Indurthi, X\. Wang, Y\. Chen, and K\. SongLiveMCP\-101: stress testing and diagnosing mcp\-enabled agents on challenging queries\.External Links:2508\.15760,[Link](https://arxiv.org/abs/2508.15760)Cited by:[§2](https://arxiv.org/html/2610.22869#S2.p1.1)\.
- Yuet al\.\(2026\)W\. Yu, L\. Zhou, X\. Dong, S\. S\. Barath, D\. Staunton, B\. Yoon, X\. Qian, J\. Caverlee, and S\. JiWhen do multi\-agent systems help? an information bottleneck perspective\.External Links:2607\.16133,[Link](https://arxiv.org/abs/2607.16133)Cited by:[§2](https://arxiv.org/html/2610.22869#S2.p2.1),[§3\.1](https://arxiv.org/html/2610.22869#S3.SS1.p4.1)\.
- Z\.ai \(2026a\)Z\.aiGLM\-5\.3\-Flash: frontier intelligence, flash cost\.External Links:[Link](https://z.ai/blog/glm-5.3-flash)Cited by:[§5\.1](https://arxiv.org/html/2610.22869#S5.SS1.p1.1)\.
- Z\.ai \(2026b\)Z\.aiGLM\-5\.3: frontier coding with emergent cyber capabilities\.External Links:[Link](https://z.ai/blog/glm-5.3)Cited by:[§5\.1](https://arxiv.org/html/2610.22869#S5.SS1.p1.1)\.
- Z\.AI \(2026\)Z\.AIZ\.ai api pricing\.Note:Z\.AI Developer DocumentationExternal Links:[Link](https://docs.z.ai/guides/overview/pricing)Cited by:[§B\.3](https://arxiv.org/html/2610.22869#A2.SS3.p2.1)\.
- Zhanget al\.\(2026\)P\. Zhang, W\. Bai, W\. Fan, and G\. LiToolathlon\-GYM: Large\-Scale Long\-Horizon Environments for Tool\-Use Agents\.External Links:[Link](https://github.com/eigent-ai/toolathlon_gym)Cited by:[§1](https://arxiv.org/html/2610.22869#S1.p2.1),[§2](https://arxiv.org/html/2610.22869#S2.p1.1),[§4\.1](https://arxiv.org/html/2610.22869#S4.SS1.p1.1)\.
- Zhuet al\.\(2025\)K\. Zhu, H\. Du, Z\. Hong, X\. Yang, S\. Guo, Z\. Wang, Z\. Wang, C\. Qian, X\. Tang, H\. Ji, and J\. YouMultiAgentBench: evaluating the collaboration and competition of LLM agents\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 8580–8622\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.421)Cited by:[§2](https://arxiv.org/html/2610.22869#S2.p2.1)\.

## Appendix ASub\-agent Delegation Statistics

After annotating delegation priors for all tasks in MS\-Eval, we obtain 89 tasks in𝒟need\\mathcal\{D\}\_\{\\text\{need\}\}that require sub\-agents\. We group them by the sub\-agent types they demand\. Read\-Only tasks require only exploratory read\-only sub\-agents for information gathering, with no editing involved\. Writable tasks require only editing\-capable sub\-agents and need no dedicated exploration\. Read & Write tasks require both, exploratory read\-only sub\-agents for information gathering and editing sub\-agents for producing deliverables\.

Table[2](https://arxiv.org/html/2610.22869#A1.T2)summarizes their distribution across domains and these three groups\. Among the 89 tasks, Read & Write tasks are the most prevalent at 44, nearly half, followed by 23 Writable tasks and 22 Read\-Only tasks\. For the 22 Read\-Only tasks, each involves on average 1\.0 delegation wave and 8\.6 sub\-agents\. The 23 Writable tasks average 1\.0 wave and 2\.3 sub\-agents\. The 44 Read & Write tasks average 2\.2 waves and 7\.1 sub\-agents\.Notably, Read\-Only tasks demand far more sub\-agents than Writable tasks, indicating that exploration is inherently more parallelizable than editing, since it can be sharded across many independent information sources\.Read & Write tasks require roughly as many sub\-agents as Read\-Only tasks but twice as many waves, implying their combination of exploration and editing\.

Specifically, Enterprise Data Warehouse is dominated by Writable tasks \(10 of 15\), whereas Stock Market and Railway System consist mostly of Read & Write tasks \(4 of 5 each\)\. Learning Management System and Academic Research exhibit balanced mixtures of all three types\.

Table 2:Distribution of the 89 tasks in𝒟need\\mathcal\{D\}\_\{\\text\{need\}\}by domain and sub\-agent type\. Tasks are grouped into Read\-Only, Writable, and Read & Write\. For each group, we report the number of tasks and the min/max/mean of delegation waves and sub\-agents\.
## Appendix BFurther Implementation Details

### B\.1Measurements for Performance, Efficiency, and Cost

We evaluate MAD\-SAE utilities along three dimensions: performance, efficiency, and cost\. For each dimension, we define a corresponding measurement as follows\.

Success Rate \(SR\)\.We use Success Rate \(SR\) to measure task performance\. Given a task set𝒟\\mathcal\{D\}, an evaluator determines whether each task is successfully completed\. We denote the subset of successfully solved tasks as𝒟\+\\mathcal\{D\}^\{\+\}, and compute SR as:

S​R=\|𝒟\+\|\|𝒟\|\.SR=\\frac\{\|\\mathcal\{D\}^\{\+\}\|\}\{\|\\mathcal\{D\}\|\}\.\(5\)For evaluating the performance utilityUpU\_\{p\}, we instantiate𝒟\\mathcal\{D\}as𝒟del\\mathcal\{D\}\_\{\\mathrm\{del\}\}, the subset of tasks where MAD\-SAE actually invokes at least one non\-planning sub\-agent\. This isolates the performance impact introduced by sub\-agent orchestration from tasks that are solved entirely by the main\-agent\.

Critical Steps \(CSteps\)\.Following Kimi\-K2\.5\([Kimi\-Team et al\., 2026](https://arxiv.org/html/2610.22869#bib.bib27)\), we use Critical Steps \(CSteps\)\. A step is counted whenever tools are called or a textual response is returned\. An episode consists ofTTsequential stages: each main\-agent step forms one stage, and each parallel sub\-agent wave is treated as a stage\. For stagett, letSmain\(t\)S\_\{\\mathrm\{main\}\}^\{\(t\)\}denote the number of main\-agent steps andSsub,i\(t\)S\_\{\\mathrm\{sub\},i\}^\{\(t\)\}the steps taken by theii\-th sub\-agent in the corresponding wave\. CSteps are computed as:

CSteps=∑t=1T\(Smain\(t\)\+maxi⁡Ssub,i\(t\)\)\.\\mathrm\{CSteps\}=\\sum\_\{t=1\}^\{T\}\\left\(S\_\{\\mathrm\{main\}\}^\{\(t\)\}\+\\max\_\{i\}S\_\{\\mathrm\{sub\},i\}^\{\(t\)\}\\right\)\.\(6\)For SA, CSteps reduces to the total number of sequential steps\. For MAD\-SAE, it captures the bottleneck effect of parallel execution, where each wave completes only when its slowest sub\-agent finishes\. Since successful task completion is the primary objective, we evaluate efficiency utilityUeU\_\{e\}as the average CSteps over successfully completed tasks𝒟\+\\mathcal\{D\}^\{\+\}for each model\-variant setting\.

API Billing\.We use the official API pricing for each model to compute the financial cost of all evaluation experiments\. As shown in Table[3](https://arxiv.org/html/2610.22869#A2.T3), input, cached input, and output tokens are billed separately\. Using the token counts from Kimi\-Code\([Moonshot AI, 2026a](https://arxiv.org/html/2610.22869#bib.bib18)\)for input, cached input, and output tokens \(Ninput,Ncached,NoutputN\_\{\\text\{input\}\},N\_\{\\text\{cached\}\},N\_\{\\text\{output\}\}\), we compute the total financial cost as:

cost=∑i∈\{input,cached,output\}Ni⋅Pricei\.\\text\{cost\}=\\sum\_\{i\\in\\\{\\text\{input\},\\text\{cached\},\\text\{output\}\\\}\}N\_\{i\}\\cdot\\text\{Price\}\_\{i\}\.\(7\)Following Kimi’s official pricing scheme, we do not billinputCacheCreation\. However, since it essentially writes prefixes into the KV cache and thus occupies the context window, it is counted toward the cumulative input tokens, affecting tiered\-pricing models such as GPT\-5\.6\-Sol and MiniMax\-M3\.

### B\.2Function Labeling for Sub\-agent Hit

Computing Sub\-agent Hit Scores in §[4\.3](https://arxiv.org/html/2610.22869#S4.SS3)requires constructing an atomic matching unit, termed*sub\-task unit*, with specific function labels on both golden and delegated sub\-agents\. This design avoids penalizing semantically equivalent delegations that differ only in sub\-agent count or specialization\. In other words, we match*delegation responsibilities*rather than raw sub\-agent calls\.

We provide 14 function labels for read\-only sub\-tasks in𝒱read\\mathcal\{V\}\_\{\\mathrm\{read\}\}:

𝒱read=\{\\displaystyle\\mathcal\{V\}\_\{\\mathrm\{read\}\}=\\\{rail\_12306,woocommerce,yahoo\-finance,snowflake,\\displaystyle\\texttt\{rail\\\_12306\},\\texttt\{woocommerce\},\\texttt\{yahoo\-finance\},\\texttt\{snowflake\},\(8\)scholarly\-arxiv,external\-fetch,notion,youtube,\\displaystyle\\texttt\{scholarly\-arxiv\},\\texttt\{external\-fetch\},\\texttt\{notion\},\\texttt\{youtube\},playwright\-web,filesystem\-workspace,canvas,howtocook,\\displaystyle\\texttt\{playwright\-web\},\\texttt\{filesystem\-workspace\},\\texttt\{canvas\},\\texttt\{howtocook\},emails,github\}\.\\displaystyle\\texttt\{emails\},\\texttt\{github\}\\\}\.Each label corresponds to a specific domain of exploratory and investigative sub\-tasks\.

We also provide 11 function labels for editing and writing sub\-tasks in𝒱write\\mathcal\{V\}\_\{\\mathrm\{write\}\}:

𝒱write=\{\\displaystyle\\mathcal\{V\}\_\{\\mathrm\{write\}\}=\\\{excel,word,ppt,notion,email,calendar,\\displaystyle\\texttt\{excel\},\\texttt\{word\},\\texttt\{ppt\},\\texttt\{notion\},\\texttt\{email\},\\texttt\{calendar\},\(9\)gsheet,gform,pdf,processor,other\-write\}\.\\displaystyle\\texttt\{gsheet\},\\texttt\{gform\},\\texttt\{pdf\},\\texttt\{processor\},\\texttt\{other\-write\}\\\}\.Each label corresponds to a writing sub\-task targeting a specific object, withother\-writeserving as a fallback when no other writing\-related label applies\. Theplanersub\-agent is not included in the matching process, considering it does not resolve any specific sub\-tasks by design\.

### B\.3Model Details

Table[3](https://arxiv.org/html/2610.22869#A2.T3)summarizes the inference configuration of each model, with reasoning effort set to each official API’s default\. It also lists the official API prices, which we use in Eq\.[7](https://arxiv.org/html/2610.22869#A2.E7)to estimate the cost of our experiments\.

For DeepSeek\-V4\-Flash\-0731, we adopt the peak\-hour pricing effective from 00:00, August 17, 2026 \(Beijing time\)\([DeepSeek\-AI, 2026a](https://arxiv.org/html/2610.22869#bib.bib45)\)\. For MiniMax\-M3, we use the permanent 50% discount offered on the official website under the standard service\([MiniMax, 2026a](https://arxiv.org/html/2610.22869#bib.bib49)\)\. All other models use the non\-discounted standard prices from their official documentation\([Alibaba Cloud, 2026](https://arxiv.org/html/2610.22869#bib.bib46);[Z\.AI, 2026](https://arxiv.org/html/2610.22869#bib.bib47);[Moonshot AI, 2026b](https://arxiv.org/html/2610.22869#bib.bib48);[Anthropic, 2026a](https://arxiv.org/html/2610.22869#bib.bib50);[OpenAI, 2026b](https://arxiv.org/html/2610.22869#bib.bib51)\)\. For models billed in CNY, we convert to USD at 1 USD = 6\.71 CNY\.

Table 3:Inference configuration and official API pricing for each model\. Prices are in USD per 1M tokens\. For tiered models, we reportlow/highprices, where tiers are triggered when input tokens exceed 272K for GPT\-5\.6\-Sol and 512K for MiniMax\-M3\.
### B\.4MAD\-SAE Variants Illustration

Our experiments include a Single\-Agent \(SA\) baseline and four MAD\-SAE variants: General \(Gen\), Specialized \(Spec\), Plan\-1st, and Oracle\. All configurations correspond to execution strategies supported by mainstream agent harness frameworks\([Moonshot AI, 2026a](https://arxiv.org/html/2610.22869#bib.bib18);[OpenAI, 2026a](https://arxiv.org/html/2610.22869#bib.bib14);[Anthropic, 2026b](https://arxiv.org/html/2610.22869#bib.bib11);[Cursor, 2026a](https://arxiv.org/html/2610.22869#bib.bib15);[TRAE Team, 2026](https://arxiv.org/html/2610.22869#bib.bib17)\)\. Table[4](https://arxiv.org/html/2610.22869#A2.T4)provides an intuitive comparison of these execution paradigms and variants\.

Table 4:The outline of common MAD\-SAE variants\.Orangedenotes read\-only sub\-agents;bluedenotes writable sub\-agents\.Naturally, the SA paradigm, serving as the most direct runtime solution, is universally supported\. The MAD\-SAE variants are defined as follows:

- •General \(Gen\):Employs three generally defined sub\-agents: a read\-onlyPlannerfor strategy formulation, a read\-onlyExplorerfor investigation, and a writableCoderfor execution and delivery\. This aligns with Kimi\-Code’s built\-in taxonomy\([Moonshot AI, 2026a](https://arxiv.org/html/2610.22869#bib.bib18)\)and corresponds to thegeneral\-purposesub\-agents in Claude Code\([Anthropic, 2026b](https://arxiv.org/html/2610.22869#bib.bib11)\)and TRAE\([TRAE Team, 2026](https://arxiv.org/html/2610.22869#bib.bib17)\)designed for complex research and code modification\.
- •Specialized \(Spec\):Retains the GenPlannerbut granularizes the other agents into domain\-specific roles, an approach advocated by Cursor\([Cursor, 2026b](https://arxiv.org/html/2610.22869#bib.bib16)\)\. TheExploreris divided into four read\-only investigators: Academic\-Literature\-Researcher \(ALR\), Enterprise\-Data\-Analyst \(EDA\), Financial\-Market\-Analyst \(FMA\), and Web\-Domain\-Researcher \(WDR\)\. Meanwhile, theCoderis split into an Office\-Report\-Builder \(for Office documents\) and an External\-Workflow\-Operator \(for targets like Notion and Google Calendar\)\.
- •Plan\-1st:Adopts the Gen sub\-agent trio but enforces a mandatory initial step where thePlannerdetermines task decomposition and delegation\. This planning\-first mechanism is supported across all harness frameworks\([Moonshot AI, 2026a](https://arxiv.org/html/2610.22869#bib.bib18);[OpenAI, 2026a](https://arxiv.org/html/2610.22869#bib.bib14);[Anthropic, 2026b](https://arxiv.org/html/2610.22869#bib.bib11);[Cursor, 2026a](https://arxiv.org/html/2610.22869#bib.bib15);[TRAE Team, 2026](https://arxiv.org/html/2610.22869#bib.bib17)\)\.
- •Oracle:Utilizes the Gen trio but injects golden priors, specifying exactly which sub\-agent handles which sub\-task and when, directly into the user query\. This aligns with the explicit sub\-agent invocation feature highlighted in the official documentation of various frameworks\([Moonshot AI, 2026a](https://arxiv.org/html/2610.22869#bib.bib18);[OpenAI, 2026a](https://arxiv.org/html/2610.22869#bib.bib14);[Anthropic, 2026b](https://arxiv.org/html/2610.22869#bib.bib11);[Cursor, 2026a](https://arxiv.org/html/2610.22869#bib.bib15);[TRAE Team, 2026](https://arxiv.org/html/2610.22869#bib.bib17)\)\.

## Appendix CDetails for Case Study

### C\.1Taxonomy

We perform pairwise comparisons between the Spec variant of MAD\-SAE and Single\-Agent \(SA\) across all tasks and eight models\. Based on whether MAD\-SAE brings performance, efficiency, or cost benefits over SA that would motivate replacing SA, we partition all cases into beneficial and adverse cases, as shown in Table[5](https://arxiv.org/html/2610.22869#A3.T5)\.

The 625 beneficial cases fall into two categories:

1. 1\.Failure\-to\-Success \(F2S\)includes 130 cases where MAD\-SAE succeeds while SA fails\. We assign all such cases to this category regardless of any efficiency or cost advantage, since task success is the first principle\.
2. 2\.Cheaper\-or\-Efficient Success \(CES\)includes 495 cases where both MAD\-SAE and SA succeed, and MAD\-SAE is either more efficient \(strictly fewer critical steps\) or cheaper \(strictly lower economic cost\) than SA\.

However, MAD\-SAE \(Spec\) does not always realize advantages over SA\. We therefore have 703 adverse cases, which fall into four categories:

1. 1\.Success\-to\-Failure \(S2F\)includes 124 cases where MAD\-SAE fails on tasks that SA solves correctly, indicating a performance regression\. Costly\-and\-Inefficient Success \(CIS\)includes 349 cases where both succeed, but MAD\-SAE incurs strictly higher cost than SA and achieves no reduction in critical steps, adding overhead without either efficiency or cost savings\.
2. 2\.Attempt\-for\-Efficiency\-or\-Saving \(AES\)includes 88 cases where both fail, but MAD\-SAE attempts more efficient or cheaper strategies that proved effective in beneficial cases\.
3. 3\.Costly\-or\-Inefficient Failure \(CIF\)includes 142 cases where both fail, but MAD\-SAE uses strictly more steps or incurs strictly higher cost than SA\.

Table 5:The taxonomy of MAD\-SAE \(Spec\) cases\. Across both partitions,✓\\checkmarkdenotes strictly fewer critical steps \(Efficiency\) or strictly lower cost \(Cost\), and×\\timesdenotes the absence of that gain \(equality or an increase\);==denotes equality\. CIS additionally requires strictly higher cost, and CIF requires a strict increase in critical steps or cost\.PartitionCategorySuccess SituationEfficiency BenefitsCost BenefitsBeneficialCases \(625\)Failure\-to\-Success \(F2S, 130\)MAD\-SAE success,SA failure✓\\checkmark✓\\checkmark✓\\checkmark×\\times×\\times✓\\checkmark×\\times×\\timesCheaper\-or\-Efficient Success \(CES, 495\)MAD\-SAE success,SA success✓\\checkmark×\\times✓\\checkmark✓\\checkmark×\\times✓\\checkmarkAdverseCases \(703\)Success\-to\-Failure \(S2F, 124\)MAD\-SAE failure,SA success✓\\checkmark✓\\checkmark✓\\checkmark×\\times×\\times✓\\checkmark×\\times×\\timesMAD\-SAE success,SA success×\\times×\\timesCostly\-and\-Inefficient Success \(CIS, 349\)Attempt\-for\-Efficiency\-or\-Saving \(AES, 88\)MAD\-SAE failure,SA failure✓\\checkmarkor==✓\\checkmarkCostly\-or\-Inefficient Failure \(CIF, 142\)✓\\checkmark×\\times×\\times✓\\checkmark×\\times×\\times

### C\.2Underlying Reasons

To understand why MAD\-SAE yields benefits or drawbacks, we conduct a case study\. For beneficial cases, we identify five underlying reasons:

1. 1\.Main\-Agent\-Only \(MAO\):Under MAD\-SAE, no sub\-agent is delegated\. The benefit comes from the main\-agent finding a correct, cheaper, or more step\-efficient solution than SA\.
2. 2\.Context Offloading \(CO\):Isolated sub\-agent contexts absorb heavy tasks that would otherwise occupy the SA context, allowing the main\-agent to focus on the core of the task and thereby gain benefits\.
3. 3\.Delegated Parallelism \(DP\):MAD\-SAE enables parallel sub\-agent delegation, decomposing what would be serial work into concurrently executed sub\-tasks and thus offering efficiency gains\.
4. 4\.Delegated Recovery \(DR\):Sub\-agents introduce verifying and recovery operations rarely seen in SA execution, validating deliverables and fixing bugs during the runtime session, which improves robustness\.
5. 5\.Others \(OTH\):Cases not covered by the four categories above\.

For adverse cases, the reasons are more varied and fall into twelve categories:

1. 1\.Main\-Agent\-Only \(MAO\):Under MAD\-SAE, no sub\-agent is delegated\. The drawback comes from the main\-agent finding an incorrect, costlier, or more step\-consuming solution than SA\.
2. 2\.Delegated Context Amplification \(DCA\):Each sub\-agent repeatedly carries a large context, multiplying token costs\.
3. 3\.Delegated Failure Propagation \(DFP\):Failures, corrupted outputs, or unverified outputs in the delegation chain propagate and degrade the overall result\.
4. 4\.Unnecessary Delegation \(UD\):Delegation is not necessary for the task, and the orchestration, handoff, and verification costs exceed the actual gains\.
5. 5\.Misdirected Delegation \(MD\):Delegated content mismatches task requirements, such as specification drift, capability mismatch, wrong tool use, or incorrect hierarchy construction\.
6. 6\.Execution Bloat \(EB\):Redundant verification, rework, repeated operations, or detours inflate steps or costs\.
7. 7\.Unrealized Delegated Parallelism \(UDP\):The run incurs the intended delegation overhead, but the delegated parallel work does not deliver the expected efficiency benefit\.
8. 8\.Context Offloading Attempt \(COA\):Specific to AES cases\. Under MAD\-SAE, the model attempts to share the main\-agent’s context burden by delegating sub\-agents, but this does not turn SA failure into MAD\-SAE success\.
9. 9\.Delegated Parallelism Attempt \(DPA\):Specific to AES cases\. Under MAD\-SAE, the model attempts to delegate sub\-agents in parallel and improve the task solving efficiency, but this does not turn SA failure into MAD\-SAE success\.
10. 10\.Delegated Recovery Attempt \(DRA\):Specific to AES cases\. Under MAD\-SAE, sub\-agents attempt verifying and recovery operations, but this does not turn SA failure into MAD\-SAE success\.
11. 11\.Missing Final Integration \(MFI\):Specific to AES cases\. Delegation completes the main work, but the main\-agent fails to finish final integration, summarization, or acceptance\.
12. 12\.Others \(OTH\):Cases not covered by the categories above\.

The distribution of underlying reasons varies across categories\. We visualize and analyze this in the case study in §[7](https://arxiv.org/html/2610.22869#S7)\.

## Appendix DFurther Case Study

§[7](https://arxiv.org/html/2610.22869#S7)quantified the overall benefits and drawbacks of MAD\-SAE \(Spec\) relative to SA across the eight models in Table[3](https://arxiv.org/html/2610.22869#A2.T3)and summarized the underlying mechanisms in Fig\.[4](https://arxiv.org/html/2610.22869#S7.F4)\. Here we ask a more precise question: for each model, do the observed gains or regressions arise from the main agent’s own execution or from sub\-agent delegation? This distinction matters because Fig\.[4](https://arxiv.org/html/2610.22869#S7.F4)shows that many beneficial and adverse cases derive from Main\-Agent\-Only \(MAO\) behavior, with 305 such cases among the beneficial cases and 254 among the adverse cases\. We aim to determine whether this pattern is specific to certain models or shared across all models, which would clarify the source of MAD\-SAE’s overall advantage or disadvantage\.

To support this analysis, Table[6](https://arxiv.org/html/2610.22869#A4.T6)reports the number of beneficial and adverse cases per model, together with a breakdown into MAO and Non\-MAO cases\. Non\-MAO cases are those whose underlying reasons are not labeled as MAO and whose outcomes mainly arise from sub\-agent delegation\. We also compare Failure\-to\-Success \(F2S\) and Success\-to\-Failure \(S2F\) cases per model, since these flip the success outcome when switching from SA to MAD\-SAE and thus drive changes in the success rate\. We discuss models with more beneficial than adverse cases in Appx\.[D\.1](https://arxiv.org/html/2610.22869#A4.SS1), and the reverse in Appx\.[D\.2](https://arxiv.org/html/2610.22869#A4.SS2)\.

Table 6:Per\-model case composition and delegation\-quality metrics under MAD\-SAE \(Spec\); UDR, ODR, andUpU\_\{p\}/UeU\_\{e\}/UcU\_\{c\}are reproduced from Table[1](https://arxiv.org/html/2610.22869#S5.T1)\.B/AB/A: beneficial/adverse case counts, with the larger side of each pair bolded \(ties unbolded\)\.Δ​SR\\Delta\\mathrm\{SR\}: success\-rate change on𝒟\\mathcal\{D\}from converting SA to MAD\-SAE \(Spec\)\. Bold/underline mark best/second\-best for UDR and ODR\.ModelCase Composition \(full𝒟\\mathcal\{D\}\)Δ​SR\\Delta\\mathrm\{SR\}Delegation AppropriatenessOrchestration UtilitiesB/AB/AMAO \(B/AB/A\)Non\-MAO \(B/AB/A\)F2S/S2FUDR↓\\downarrowODR↓\\downarrowUp↑U\_\{p\}\\uparrowUe↑U\_\{e\}\\uparrowUc↑U\_\{c\}\\uparrowGLM\-5\.389/7725/2164/5616/9\+4\.2\+4\.28\.9%64\.9%50\.8%52\.7%40\.5%GLM\-5\.3\-Flash85/8132/4153/407/5\+1\.2\+1\.226\.9%50\.6%51\.3%54\.0%32\.0%Kimi\-K3130/3689/2241/1426/9\+10\.2\+10\.250\.5%16\.8%56\.4%50\.3%50\.9%Qwen\-3\.8\-Max80/8632/2248/6420/3\+10\.2\+10\.215\.7%59\.7%54\.4%49\.9%38\.4%DeepSeek\-V4\-Flash\-073175/9126/1949/7211/110\.00\.016\.8%57\.1%50\.4%48\.0%33\.1%Claude\-Sonnet\-574/9246/4328/4915/22−4\.2\-4\.239\.3%29\.8%45\.9%46\.8%34\.1%GPT\-5\.6\-Sol29/1370/729/1309/45−21\.7\-21\.77\.8%93\.5%42\.6%44\.0%27\.4%MiniMax\-M363/10355/798/2426/20\+3\.6\+3\.679\.7%9\.0%42\.8%42\.7%37\.6%

### D\.1MAD\-SAE \(Spec\) Benefits Outweigh Drawbacks

GLM\-5\.3: gains driven by effective delegation, still at a cost\.Delegation clearly accounts for 63 of the 89 beneficial cases \(70\.8%\), excluding one Non\-MAO case labeled Other, withContext Offloading\(Fig\.[13](https://arxiv.org/html/2610.22869#A4.F13)\) andDelegated Parallelism\(Fig\.[14](https://arxiv.org/html/2610.22869#A4.F14)\) contributing 31 and 27 cases, respectively\. Similarly, 11 of the 16 Failure\-to\-Success cases stem from delegationX\. GLM\-5\.3’s improvement therefore mainly reflects sub\-agents absorbing context and executing in parallel, while Main\-Agent\-Only improvement plays a smaller role\. This aligns with its high efficiency utility \(Ue=52\.7%U\_\{e\}=52\.7\\%\) in Table[6](https://arxiv.org/html/2610.22869#A4.T6)\. ItsUc=40\.5%U\_\{c\}=40\.5\\%further shows that these gains still come at added cost, consistent with the higher\-cost, higher\-success pattern it occupies, e\.g\., Costly\-and\-Inefficient\-Success, in Fig\.[3](https://arxiv.org/html/2610.22869#S6.F3)\.

Figure 5:The distribution of beneficial and adverse categories, alongside with their underlying reasons of GLM\-5\.3\.GLM\-5\.3\-Flash: a narrow edge carried entirely by delegation\.Its Main\-Agent\-Only cases split 32 beneficial to 41 adverse, indicating that MAO does not provide notably better solutions in this setting\. Its overall edge over SA therefore comes entirely from delegation\. All 53 beneficial Non\-MAO cases carry an identified delegation mechanism \(Fig\.[6](https://arxiv.org/html/2610.22869#A4.F6)\), with 48 falling under Cheaper\-or\-Efficient\-Success: 33 from Context Offloading and 15 from Delegated Parallelism\. Given a net gain of only 2 successful tasks by comparing Failure\-to\-Success \(7 cases\) with Success\-to\-Failure \(5 cases\), GLM\-5\.3\-Flash’s more salient benefit is execution efficiency, matching its top\-rankedUe=54\.0%U\_\{e\}=54\.0\\%in Table[6](https://arxiv.org/html/2610.22869#A4.T6)\. ItsUc=32\.0%U\_\{c\}=32\.0\\%indicates that this efficiency still comes at extra cost\.

Figure 6:The distribution of beneficial and adverse categories, alongside with their underlying reasons of GLM\-5\.3\-Flash\.K3: main\-agent improvement leads, but delegation contributes clearly too\.Main\-Agent\-Only \(MAO\) accounts for 89 of K3’s 130 beneficial cases \(68\.5%\), including 76 Cheaper\-or\-Efficient\-Success and 13 Failure\-to\-Success cases\. MAO yields a net advantage of 67 cases \(89 beneficial vs\. 22 adverse\), and beneficial cases outnumber adverse cases by 94 overall\. This indicates much of K3’s MAD\-SAE gain comes from the main agent’s own execution path\. Nevertheless, the remaining 41 beneficial cases are all attributed to an identified delegation mechanism, e\.g\., Context Offloading, Delegated Parallelism, etc\. On top of that, K3’s Failure\-to\-Success cases significantly outnumber its Success\-to\-Failure cases, primarily from 13 MAO and 9Delegated Parallelism\(Fig\.[15](https://arxiv.org/html/2610.22869#A4.F15)\) cases\. These results demonstrate clearly that K3’s MAD\-SAE benefits are jointly supported by both the better main\-agent execution and the high\-quality delegation, which explain K3’s broad improvement in performance, cost, and efficiency in Fig\.[3](https://arxiv.org/html/2610.22869#S6.F3), as well as three orachestration utilities all exceeding 50% in Table[6](https://arxiv.org/html/2610.22869#A4.T6)\. However, broad gains do not imply that appropriate delegation is fully recalled, according to K3’s 50\.5% UDR\.

Figure 7:The distribution of beneficial and adverse categories, alongside with their underlying reasons of Kimi\-K3\.
### D\.2MAD\-SAE \(Spec\) Drawbacks Outweigh Benefits

Qwen\-3\.8\-Max: more adverse cases, yet the largest success\-rate gain\.Although adverse cases outnumber beneficial ones by 6, Qwen\-3\.8\-Max achieves the largest success\-rate gain \(\+10\.2 points\), because its 20 Failure\-to\-Success \(F2S\) cases far exceed its 3 Success\-to\-Failure \(S2F\) cases\. Delegation plays a central role in these flips: 14 of the 20 F2S cases are delegation\-attributed \(Fig\.[8](https://arxiv.org/html/2610.22869#A4.F8)\), led by Delegated Recovery \(7\) and Delegated Parallelism \(5\)\. This gain, however, comes with overhead: 25 of its 48 Costly\-and\-Inefficient\-Success cases are attributed to Unnecessary Delegation\. Delegation thus simultaneously drives additional successes and introduces redundancy on a subset of tasks, a pattern consistent with its utilities in Table[6](https://arxiv.org/html/2610.22869#A4.T6):Up=54\.4%U\_\{p\}=54\.4\\%,Uc=38\.4%U\_\{c\}=38\.4\\%, andUe=49\.9%U\_\{e\}=49\.9\\%, indicating higher success and higher cost with roughly flat efficiency\. The step\-count reduction over the full benchmark is examined separately in Fig\.[3](https://arxiv.org/html/2610.22869#S6.F3)\.

Figure 8:The distribution of beneficial and adverse categories, alongside with their underlying reasons of Qwen\-3\.8\-Max\.DeepSeek\-V4\-Flash\-0731: a flat success rate masking two offsetting effects\.Failure\-to\-Success \(F2S\) and Success\-to\-Failure \(S2F\) are tied at 11 cases each, matching its zero success\-rate change in Fig\.[3](https://arxiv.org/html/2610.22869#S6.F3)\. This balance arises from two offsetting effects: Main\-Agent\-Only contributes a net loss \(F2S 4 v\.s\. S2F 6\), while delegation\-attributed cases yield a net gain of 2 \(F2S 7 v\.s\. S2F 5\), so the two roughly cancel\. This is consistent with a slightly positiveUp=50\.4%U\_\{p\}=50\.4\\%in Table[6](https://arxiv.org/html/2610.22869#A4.T6)\. Its adverse cases are heavily concentrated in Costly\-and\-Inefficient\-Success \(71 of 91, 78\.0%; Fig\.[9](https://arxiv.org/html/2610.22869#A4.F9)\), of which 40 stem fromUnnecessary Delegation\(Fig\.[17](https://arxiv.org/html/2610.22869#A4.F17)\), the single largest source of drawbacks across all its adverse cases \(43 occurrences in total\)\. Overall, delegation provides only a limited correctness benefit here, while extensive redundant delegation still adds overhead, consistent withUe=48\.0%U\_\{e\}=48\.0\\%andUc=33\.1%U\_\{c\}=33\.1\\%in the same table\.

Figure 9:The distribution of beneficial and adverse categories, alongside with their underlying reasons of DeepSeek\-V4\-Flash\-0731\.Claude\-Sonnet\-5: failure propagation outweighs main\-agent gains\.MAO’s beneficial and adverse counts are nearly balanced \(46 vs\. 43\), with its Failure\-to\-Success \(F2S\) and Success\-to\-Failure \(S2F\) both equal to 7, so MAO contributes no net change\. Claude\-Sonnet\-5’s overall success\-rate still falls by 7 tasks\. The driver is Delegated Failure Propagation, responsible for 14 of its 22 S2F cases \(Fig\.[10](https://arxiv.org/html/2610.22869#A4.F10)\), showing that insufficiently verified delegation outcomes are the main source of the regression\. Its overhead problem, however, is not purely a delegation story: 31 of its 48 Costly\-and\-Inefficient\-Success cases are labeled MAO\. Claude\-Sonnet\-5 is thus hurt by both delegation failures and inefficient main\-agent execution, consistent with all three utilities in Table[6](https://arxiv.org/html/2610.22869#A4.T6)falling below 50% and with its more expensive, more inefficient, lower success position in Fig\.[3](https://arxiv.org/html/2610.22869#S6.F3)\.

Figure 10:The distribution of beneficial and adverse categories, alongside with their underlying reasons of Claude\-Sonnet\-5\.GPT\-5\.6\-Sol: over\-delegation undermines both correctness and efficiency\.With 137 adverse cases against only 29 beneficial ones, and all 7 of its Main\-Agent\-Only \(MAO\) cases falling on the adverse side, GPT\-5\.6\-Sol’s problems are concentrated in delegation\.Delegated Failure Propagation\(Fig\.[18](https://arxiv.org/html/2610.22869#A4.F18)\) accounts for 33 of its 45 Success\-to\-Failure cases \(Fig\.[11](https://arxiv.org/html/2610.22869#A4.F11)\); among its 55 Costly\-and\-Inefficient\-Success cases,Unnecessary DelegationandUnrealized Delegated Parallelism\(Fig\.[19](https://arxiv.org/html/2610.22869#A4.F19)\) contribute 25 and 20 respectively\. Failure propagation explains the correctness regression, and redundant, unrealized delegation explains the execution overhead\. This is consistent with its 93\.5% ODR, all three utilities below 50% in Table[6](https://arxiv.org/html/2610.22869#A4.T6), and its pronounced regression in Fig\.[3](https://arxiv.org/html/2610.22869#S6.F3): frequent sub\-agent invocation here never converts into effective collaboration\.

Figure 11:The distribution of beneficial and adverse categories, alongside with their underlying reasons of GPT\-5\.6\-Sol\.MiniMax\-M3: outcomes dominated by main\-agent behavior\.Main\-Agent\-Only \(MAO\)\(Fig\.[16](https://arxiv.org/html/2610.22869#A4.F16)\) dominates both sides \(Fig\.[12](https://arxiv.org/html/2610.22869#A4.F12)\), accounting for 55 of 63 beneficial and 79 of 103 adverse cases\. Of its 26 Failure\-to\-Success \(F2S\) cases, 24 are labeled MAO\. Together with 14 MAO\-labeled Success\-to\-Failure \(S2F\) cases, these MAO cases net a gain of 10 successful tasks, offsetting a net loss of 4 from the remaining Non\-MAO successes and yielding the model’s overall success\-rate gain of 3\.6 points\. At the same time, 33 of its 42 Costly\-and\-Inefficient\-Success cases are also labeled MAO, indicating that the main\-agent likewise generates many successes with added overhead and no efficiency gain\. This composition echoes its UDR of 79\.7% in Table[6](https://arxiv.org/html/2610.22869#A4.T6), although a high MAO share cannot itself substitute for UDR, which is computed from task\-level priors\. Its performance gain on𝒟\\mathcal\{D\}and itsUp=42\.8%<50%U\_\{p\}=42\.8\\%<50\\%in the same table are not in tension: main\-agent improvement masks a regression within the delegation subset, and the larger adverse count overall cannot be attributed to delegation overhead alone\.

Figure 12:The distribution of beneficial and adverse categories, alongside with their underlying reasons of MiniMax\-M3\.Takeaway\.This model\-wise analysis shows that MAD\-SAE’s overall value rests on the joint contribution of main\-agent change and delegation mechanisms, and their relative weight is highly model\-dependent\. Deciding whether to adopt MAD\-SAE should draw on full\-benchmark success rate, cost, and efficiency together; judging whether delegation itself is effective requires specific mechanisms and delegation\-subset metrics in tandem\.

### D\.3Typical Cases Illustration

We select several typical cases to illustrate the benefits of MAD\-SAE over the SA baseline in Appx\.[D\.3\.1](https://arxiv.org/html/2610.22869#A4.SS3.SSS1)and its drawbacks in Appx\.[D\.3\.2](https://arxiv.org/html/2610.22869#A4.SS3.SSS2)\.

#### D\.3\.1Beneficial Cases

Context Offloadingis the primary component of GLM\-5\.3’s Cheaper\-or\-Efficient\-Success category \(see Fig\.[5](https://arxiv.org/html/2610.22869#A4.F5)\)\. We use thecanvas\-course\-notion\-wikicase from this category for illustration in Figure[13](https://arxiv.org/html/2610.22869#A4.F13)\. SA encounters numerous Notion API issues when solving the task, whereas Spec resolves these Notion API bugs more efficiently through a dedicated sub\-agent\. This offloads the error messages that would otherwise occupy the main\-agent’s context, keeping it cleaner and allowing it to focus on core task\-solving logic, thereby improving efficiency\.

Delegated Parallelismis an important component of GLM\-5\.3’s Cheaper\-or\-Efficient\-Success category \(see Fig\.[5](https://arxiv.org/html/2610.22869#A4.F5)\)\. We use thecanvas\-enrollment\-ppt\-gsheetcase from this category for illustration in Fig\.[14](https://arxiv.org/html/2610.22869#A4.F14)\. The optimal solution annotated in our Delegation Priors is to use parallelized explorers to read course information separately, followed by three dedicated writable sub\-agents responsible for producing and delivering xlsx, pptx, and Google Sheet files\. Spec perfectly matches this optimal solution, totaling 23 critical steps and saving 13 steps compared to SA\. SA suffers from serial execution, where xlsx, pptx, and Google Sheet must be completed one after another, which weakens execution efficiency\.

For Kimi\-K3, Delegated Parallelism helps correct failures \(see Fig\.[7](https://arxiv.org/html/2610.22869#A4.F7)\)\. Fig\.[15](https://arxiv.org/html/2610.22869#A4.F15)shows theterminal\-yf\-canvas\-excel\-gcal\-emailtask from Kimi\-K3’s Failure\-to\-Success category\. SA’s serial solution hits the 100\-step limit and fails to deliver, whereas Spec distributes the execution pressure of a single agent through reasonably parallelized paginated course exploration, completing a successful delivery in only 67 steps\.

Better Main\-Agent\-Only Solutionsdominates Minimax\-M3’s Failure\-to\-Success category and is the main reason MAD\-SAE \(Spec\) improves this model’s performance \(see Fig\.[12](https://arxiv.org/html/2610.22869#A4.F12)\)\. As shown in Fig\.[16](https://arxiv.org/html/2610.22869#A4.F16)\(scholarly\-arxiv\-survey\-word\-notion\-gcal\), although the Delegation Prior expects parallel paper explorers and writable sub\-agents to solve the problem more efficiently, in Spec the main\-agent completes the task alone\. By applying some verification during problem solving, it reduces the SA approach that hit the 100\-step limit to a success in 48 steps\.

Figure 13:The Illustration of the Context Offloading benefit on GLM\-5\.3\.Figure 14:The Illustration of the Delegated Parallelism benefit on GLM\-5\.3\.Figure 15:The Illustration of the Delegated Parallelism benefit on Kimi\-K3\.Figure 16:The Illustration of the Better Main\-Agent\-Only solution on Minimax\-M3\.
#### D\.3\.2Adverse Cases

Unnecessary Delegationplays an important role in DeepSeek\-V4\-Flash\-0731’s Costly\-and\-Inefficient\-Success category \(see Fig\.[9](https://arxiv.org/html/2610.22869#A4.F9)\)\. We select theacademic\-literature\-reviewcase \(Fig\.[17](https://arxiv.org/html/2610.22869#A4.F17)\) from this category for a comparative illustration between MAD\-SAE \(Spec\) and the SA baseline\. The optimal solution in our annotated delegation priors requires only one wave of exploratory sub\-agents to conduct parallel exploration of five non\-overlapping RAG topics\. The subsequent process of producing Literature\_Review\.docx is not cumbersome and can be handled by the main\-agent itself\. Spec perfectly uses five academic\-literature\-researcher sub\-agents for parallel paper exploration, but then assigns an additional office\-report\-builder specifically to generate the docx report\. This operation is semantically valid and reasonable, yet the office\-report\-builder consumes 70 steps, more than SA’s total critical steps \(CSteps\) of 51\. Combined with the main\-agent’s execution steps, its total CSteps reach 97, nearly twice that of SA\. This significantly affects execution efficiency and increases execution cost\.

Delegated Failure Propagationdominates GPT\-5\.6\-Sol’s Success\-to\-Failure cases \(see Fig\.[11](https://arxiv.org/html/2610.22869#A4.F11)\)\. We usecanvas\-ta\-workload\-excel\-emailfor visualizatio in Fig\.[18](https://arxiv.org/html/2610.22869#A4.F18)\. Here, Spec’senterprise\-data\-analystruns only a few steps and returns a compressed, lossy summary, which the downstream Excel, Notion, Email builders directly trust such the summary\. This error propagation causes GPT\-5\.6\-Sol to fail a task that SA solves correctly with 21 CSteps, despite spending 41 CSteps\.

Unrealized Delegated Parallelismis the second most common factor behind GPT\-5\.6\-Sol’s Costly\-and\-Inefficient\-Success \(see Fig\.[11](https://arxiv.org/html/2610.22869#A4.F11)\)\. As shown in Fig\.[19](https://arxiv.org/html/2610.22869#A4.F19)\(canvas\-enrollment\-ppt\-gsheet\), ideally, course information acquisition and deliverable generation should be completed synchronously by parallelized sub\-agents to accelerate task solving\. However, GPT\-5\.6\-Sol calls four sub\-agents and executes them serially\. This unnecessary and unreasonable delegation makes Spec require 63 steps for a task that SA can complete in 25 critical steps\. This contrasts sharply with GLM\-5\.3 \(Fig\.[14](https://arxiv.org/html/2610.22869#A4.F14)\), which efficiently solves the same task by reasonably parallelizing sub\-agent scheduling\.

Execution Bloataccounts for a considerable proportion of GPT\-5\.6\-Sol’s Costly\-and\-Inefficient\-Success category \(see Fig\.[11](https://arxiv.org/html/2610.22869#A4.F11)\)\. We use theterminal\-arxiv\-scholarly\-notion\-word\-excelcase in this setting for illustration in Fig\.[20](https://arxiv.org/html/2610.22869#A4.F20)\. In the Delegation Priors, we expect parallelized exploratory sub\-agents to first collect information, followed by parallelized writable sub\-agents responsible for editing and delivering Notion, Word, and Excel files\. However, this Spec generates five serial single\-agent waves, including a resume of an academic\-literature\-researcher\. The sub\-agent delegation here achieves no parallelism and instead introduces rework costs\. This makes task solving more complex, requiring 18 more steps than the SA solution\.

Figure 17:The Illustration of the Unnecessary Delegation drawback on DeepSeek\-V4\-Flash\-0731\.Figure 18:The Illustration of the Delegated Falure Propagation drawback on GPT\-5\.6\-Sol\.Figure 19:The Illustration of the Unrealized Delegated Parallelism drawback on GPT\-5\.6\-Sol\.Figure 20:The Illustration of the Execution Bloat drawback on GPT\-5\.6\-Sol\.

## Appendix EMS\-Eval Kit Details

Fig\.[2](https://arxiv.org/html/2610.22869#S4.F2)shows the overall architecture of MS\-Eval with the example ofarxiv\-survey\-presentation\. We continue with this task and provide more complete information to facilitate understanding of the MS\-Eval Benchmark \(Appx\.[E\.1](https://arxiv.org/html/2610.22869#A5.SS1)\) and Harness \(Appx\.[E\.2](https://arxiv.org/html/2610.22869#A5.SS2)\)\. In Appx\.[E\.1](https://arxiv.org/html/2610.22869#A5.SS1), we provide the full task description and delegation prior forarxiv\-survey\-presentation, supplementing the summarized content in Fig\.[2](https://arxiv.org/html/2610.22869#S4.F2)\(Benchmark\)\. In Appx\.[E\.2](https://arxiv.org/html/2610.22869#A5.SS2), we provide the principles and tools for the basic, main\-agent, and sub\-agent levels, illustrating the core constraints needed to govern agent behavior across different paradigms in working scenarios\.

### E\.1Benchmark

#### E\.1\.1Task Description

We provide the full task description forarxiv\-survey\-presentationin Table[7](https://arxiv.org/html/2610.22869#A5.T7)\. This is a representative working case from the Academic Research scenario\. It requires the following MCP servers:playwright\_with\_chunkfor web browsing,terminalfor command\-line operations, file inspection, and verification,arxiv\-latexfor searching, listing, reading, and downloading arXiv papers, andexcelandpptxfor creating and writing xlsx and pptx files\. Using these MCP servers, the agent must complete three milestone sub\-tasks: \(1\) inspect survey\-methodology pages to understand the local arXiv candidate space relevant to the task, \(2\) select exactly five source\-supported papers, and \(3\) create the two required files Survey\_Data\.xlsx and NLP\_Survey\.pptx\. This illustrates the reliance of working scenarios on domain\-specific tools and the complexity of tasks involving multiple sub\-tasks\.

#### E\.1\.2Delegation Priors

We exhibit the detailed and specific delegation prior for each task, as shown in Table[8](https://arxiv.org/html/2610.22869#A5.T8)\. We assign three waves of sub\-agents to complete the three milestone sub\-tasks mentioned in Appx\.[E\.1\.1](https://arxiv.org/html/2610.22869#A5.SS1.SSS1)\. The first wave consists of two exploratory sub\-agents that perform read\-only tasks in parallel\. One inspects the survey\-methodology page, and the other scouts the task\-relevant local arXiv candidate space\. This allows the agent to fully understand the current environment state\. The second wave consists of five read\-only exploratory sub\-agents running in parallel\. Each sub\-agent investigates one paper and returns compact records\. After obtaining these records, the third wave assigns two writable sub\-agents to simultaneously edit the deliverables\. They are responsible for Survey\_Data\.xlsx and NLP\_Survey\.pptx, respectively\. Such detailed delegation priors not only indicate whether the current task requires delegating sub\-agents, but also provide golden priors for the Oracle Study\. This helps us understand MAD\-SAE’s utility under near\-perfect delegation\.

### E\.2Harness

For the single\-agent \(SA\) paradigm, we use the Basic Principles & Tools in Appx\.[E\.2\.1](https://arxiv.org/html/2610.22869#A5.SS2.SSS1)to govern agent behavior\. For the MAD\-SAE paradigm, the main\-agent inherits these Basic Principles & Tools while adding variant\-specific sub\-agent delegation information for different MAD\-SAE variants \(Appx\.[E\.2\.2](https://arxiv.org/html/2610.22869#A5.SS2.SSS2)\)\. This includes adding Delegation Norms, Delegation Examples, and Sub\-Agent Introduction to the principles, as mentioned in Fig\.[2](https://arxiv.org/html/2610.22869#S4.F2)\(Harness\)\. For the tools, we addAgentandAgentSwarmto enable the main\-agent to delegate sub\-agents\. For sub\-agent principles and tools \(Appx\.[E\.2\.3](https://arxiv.org/html/2610.22869#A5.SS2.SSS3)\), different sub\-agent types may have different allowed MCP tools, but none are granted access toAgent,AgentSwarm, or the task\-termination toolclaim\_done\.

#### E\.2\.1Basic Principles & Tools

Table[9](https://arxiv.org/html/2610.22869#A5.T9)presents the Basic Principles & Tools that govern SA agents and also underlie main\-agent control in MAD\-SAE\. The runtime profile specifies name, description, and override, where override determines whether to replace a same\-named built\-in profile\. tools grants all task\-authorized MCP tools viamcp\_\_\*, while disallowedTools disables Kimi\-Code built\-ins, ensuring all functionality is accessed only throughmcp\_\_\*\. subagents is set to None because SA requires no sub\-agent calls\. The profile further defines the accessible workspace,Core Responsibilities, Visible Boundary, and Completion Protocol, which guide valid task solving, restrict resource access, and specify termination\. Notably, many models fail to callmcp\_\_local\_\_claim\_donedespite this requirement\.

#### E\.2\.2Main\-Agent Principles & Tools

General \(Gen\)is a MAD\-SAE variant that defines three sub\-agents: a read\-onlyPlanfor strategy planning, a read\-onlyExplorefor investigation, and a writableCoderfor editing\. As shown in Table[10](https://arxiv.org/html/2610.22869#A5.T10), thesubagentsfield in the Runtime Profile changes fromNonein Table[9](https://arxiv.org/html/2610.22869#A5.T9)to names of three sub\-agents\. Relative to Table[9](https://arxiv.org/html/2610.22869#A5.T9), Gen adds twoCore Responsibilities: assessing whether to delegate sub\-agents and reusing sub\-agent outputs under their return contracts\. It further introduces Subagent Delegation and Sub\-Agent Orchestration Rules, collectively calledDelegation Norms, which specify when and how to schedule sub\-agents\. The Available Sub\-Agent Types section, referred to asSub\-Agent Introduction, briefly describes each sub\-agent type and its strengths\. Finally, the Examples section, termedDelegation Examples, illustrates in what contexts to delegate and which sub\-agents to schedule\.

Specialized \(Spec\)is a MAD\-SAE variant that inherits Gen’sPlanner, while Gen’sExplorerandCoderare split into five and two more domain\-specific sub\-agents, respectively\. As shown in Table[11](https://arxiv.org/html/2610.22869#A5.T11), Main\-Agent Principles & Tools \(Spec\) and Main\-Agent Principles & Tools \(Gen\) in Table[10](https://arxiv.org/html/2610.22869#A5.T10)are largely identical, except that the subagents parameter, the corresponding entries in Sub\-Agent Orchestration Rules, Available Sub\-Agent Types, and Examples are replaced with Spec\-specific content\.

Plan\-1stextends Gen’s sub\-agent definitions with a mandatory initial planning step: the first action must be aPlannercall that determines sub\-task partitioning and subsequent sub\-agent scheduling\. Consequently, relative to Gen \(Table[10](https://arxiv.org/html/2610.22869#A5.T10)\), Plan\-1st adds a Plan\-First Protocol \(Mandatory\) to enforce this initial planning, as shown in Table[12](https://arxiv.org/html/2610.22869#A5.T12)\.

Oracleserves as the variant for the golden study under MAD\-SAE\. It fully inherits Main\-Agent Principles & Tools \(Gen\) in Table[10](https://arxiv.org/html/2610.22869#A5.T10), except that it appends the delegation prior directly to the end of the task description\.

#### E\.2\.3Sub\-Agent Principles & Tools

Tables[13](https://arxiv.org/html/2610.22869#A5.T13),[14](https://arxiv.org/html/2610.22869#A5.T14),[15](https://arxiv.org/html/2610.22869#A5.T15)present the principles and tools for thePlanner,Explorer, andCodersub\-agents, while Tables[16](https://arxiv.org/html/2610.22869#A5.T16),[17](https://arxiv.org/html/2610.22869#A5.T17),[18](https://arxiv.org/html/2610.22869#A5.T18),[19](https://arxiv.org/html/2610.22869#A5.T19),[20](https://arxiv.org/html/2610.22869#A5.T20),[21](https://arxiv.org/html/2610.22869#A5.T21)present those for the Academic\-Literature\-Researcher, Enterprise\-Data\-Analyst, Financial\-Market\-Analyst, Web\-Domain\-Researcher, External\-Workflow\-Operator, and Office\-Report\-Builder sub\-agents\. For all sub\-agents, the Runtime Profile shares the same parameter schema as the Basic Principles & Tools in Table[9](https://arxiv.org/html/2610.22869#A5.T9), with an additionalwhenToUsefield specifying when the sub\-agent is dispatched\. ThetoolsanddisallowedToolsfields define which tools are available or unavailable, determined by the sub\-agent’s read\-only or writable nature and its domain\-specialized role\. In particular,Planner,Explorer, Academic\-Literature\-Researcher, Enterprise\-Data\-Analyst, Financial\-Market\-Analyst, and Web\-Domain\-Researcher are read\-only, whereasCoder, External\-Workflow\-Operator, and Office\-Report\-Builder are writable\. The Constraints section restricts each sub\-agent to domain\-specific read\-only or writable operations, and the Visible Boundary limits the scope of resources accessible to the sub\-agent\.

Task DescriptionI am a PhD student preparing a survey presentation on recent advances in natural language processing, specifically focusing on large language models and their reasoning capabilities\. I need to gather information from multiple sources, organize it systematically, and create both a data spreadsheet and a presentation\.First, visit the survey methodology guide at[http://localhost:30231](http://localhost:30231/)to learn about how to structure a literature survey\. The main page explains the overall approach including comparison criteria and taxonomy development\. There is also a guide page at[http://localhost:30231/guide\.html](http://localhost:30231/guide.html)with tips on presenting survey results effectively\. Take notes on the recommended structure and presentation tips\.Next, search the arxiv paper database for recent papers related to NLP, language models, and reasoning\. You should find around five relevant papers on these topics\. For each paper, look up its full details including the abstract and read its LaTeX source to extract the key methodology, the datasets used in experiments, and the main findings\. The LaTeX source of each paper \(its section headings and content\) is available through the arxiv\-latex tools \(list\_paper\_sections, get\_paper\_section, get\_paper\_prompt\)\. In the content returned by these tools, each section heading appears on its own line starting with ‘\#’, so you can count the sections of a paper by counting those heading lines \(or by reading the sections list returned by list\_paper\_sections\)\. Count the LaTeX sections of each paper, because the survey needs the average number of sections across papers\.After gathering all the paper information, create an Excel workbook called Survey\_Data\.xlsx in the workspace with three sheets\. The first sheet should be named “Paper Comparison” with columns Paper\_ID, Title, Year, Method, Dataset, and Key\_Finding\. The Paper\_ID column must contain each paper’s arXiv identifier in the standard form YYMM\.NNNNN \(for example 2404\.00001\), with no prefix such as “arXiv:” and no version suffix such as “v1”\. The Method column should describe the core approach in a few words, the Dataset column should list the primary evaluation dataset, and Key\_Finding should summarize the main result\. The second sheet should be named “Taxonomy” with columns Category and Papers\. Group the papers into meaningful categories based on their approaches such as prompting methods, training methods, and architectural innovations\. The third sheet should be named “Summary Statistics” with columns Metric and Value, including Total\_Papers, Most\_Common\_Category, Year\_Range, and Average\_Sections \(average number of LaTeX sections across papers\)\. All values in the Value column must be written as literal text or numbers in the cell itself – do NOT write Excel formulas \(for example, do not enter “=COUNTA\(…\)” or “=AVERAGE\(…\)”; instead write the plain number, e\.g\. 5 or 4\.0\)\.Then create a PowerPoint presentation called NLP\_Survey\.pptx in the workspace\. The presentation should have at least six slides\. The first slide should be a title slide with a title containing the words “NLP Survey” or “Language Model Survey” or similar\. The second slide should present the survey methodology and structure based on what you learned from the methodology guide\. Then include one slide for each of at least three key papers, showing the paper title, its method, dataset, and key finding\. End with a summary slide that provides a taxonomy overview and identifies key trends across the surveyed papers\.Please review the survey outline and presentation style guide files in the workspace for additional context about the expected structure and formatting guidelines\.Important coordination note: this is a multi\-agent task and all agents share the same workspace\. Each of the two final files \(Survey\_Data\.xlsx and NLP\_Survey\.pptx\) must be written to disk by a single sub\-agent\. Never have more than one agent write the same file at the same time – concurrent writes to the same file overwrite each other\. Coordinate your work so that, after the research information has been gathered, one designated sub\-agent is responsible for writing Survey\_Data\.xlsx and one \(or the same agent\) is responsible for writing NLP\_Survey\.pptx\.Table 7:The full task description ofarxiv\-survey\-presentationDelegation PriorTask\-Solving Rationable with Sub\-task Decomposition:To solve this task efficiently, follow this exact orchestration plan\. A wave must finish and return all stated handoffs before the next wave begins\. Within a wave, dispatch exactly one sub\-agent for each numbered sub\-task and run those sub\-agents concurrently\.Before Wave 1, the main agent must list and read all task\-visible survey\-outline and presentation\-style reference files in read\-only mode, without assuming their filenames, and freeze the survey comparison schema, taxonomy/statistics rules, at\-least\-six\-slide contract, and single\-writer ownership of Survey\_Data\.xlsx and NLP\_Survey\.pptx\. It must resolve the task\-visible workspace directory from the runtime allowed\-directory listing \(or literal accessible\-workspace directory already supplied by the harness\), freeze ‘<workspace\_root\>’, and reject bare ‘/workspace’ and dumps/run\-specific paths\. It must then inspect the granted runtime tool catalog and verify that the task\-required LaTeX section operations can actually read section lists and contents for discovered paper IDs\. If those operations are unavailable or no authorized source can return the required paper bodies, the main agent must stop before delegation, report the exact missing\-capability blocker, and must not fabricate Method, Dataset, Key\_Finding, section counts, or either dependent deliverable\. The following waves apply only when the required source capability is present; all paper facts must come from task\-visible runtime sources\.1\. Wave 1 – dispatch exactly 2 sub\-agents in parallel, one for each numbered sub\-task:\(1\)\.Explorer 0\(Web\-Domain\-Researcher 0\): Use one explore sub\-agent to inspect both task\-provided survey\-methodology pages in read\-only mode and combine their source\-backed guidance with the frozen local\-reference packet supplied by the main agent\. Use only the granted non\-interactive browser navigation, snapshot, search, and wait operations; do not click, type, evaluate scripts, select options, invoke terminal, or reread workspace files\. Return a compact source\-referenced methodology packet covering comparison criteria, taxonomy guidance, recommended presentation structure, slide tips, and per\-page/reference coverage\. Do not write deliverables or alter the pages\.\(2\)\.Explorer 1\(Academic\-Literature\-Researcher 0\): Use one explore sub\-agent to scout the task\-relevant local arXiv candidate space read\-only for recent NLP, language\-model, and reasoning papers\. Use complementary queries at the supported per\-call result bound, union and deduplicate by stable ID because no search cursor is exposed, validate relevance from title/abstract, and return a candidate manifest with authors/date, abstract, available arxiv\-latex status, per\-query/union controls, and rejected distractors\. Do not read full manuscripts, write files, or create deliverables\.After Wave 1, the main agent must apply the task’s relevance rule, select exactly five source\-supported papers, and freeze an explicit five\-row assignment table containing ‘agent\_no’, stable ID, and exact title, with one distinct paper per row\.Before dispatching Wave 2, render each numbered prompt into a complete self\-contained instruction that includes that row’s literal stable ID and exact title, the permitted read\-only paper/LaTeX calls, the complete return schema and all missing/conflict controls\. Never send an agent only an ordinal such as “paper row 3” or require it to infer fields from another prompt\.2\. Wave 2 – dispatch exactly 5 explore sub\-agents in parallel, one for each frozen paper:\(1\)\.Explorer 0\(Academic\-Literature\-Researcher 0\): Use one explore sub\-agent to own only paper row 1 from the frozen assignment table\.\(2\)\.Explorer 1\(Academic\-Literature\-Researcher 1\): Use one explore sub\-agent to own only paper row 2 from the frozen assignment table\.\(3\)\.Explorer 2\(Academic\-Literature\-Researcher 2\): Use one explore sub\-agent to own only paper row 3 from the frozen assignment table\.\(4\)\.Explorer 3\(Academic\-Literature\-Researcher 3\): Use one explore sub\-agent to own only paper row 4 from the frozen assignment table\.\(5\)\.Explorer 4\(Academic\-Literature\-Researcher 4\): Use one explore sub\-agent to own only paper row 5 from the frozen assignment table\.Each Wave 2 sub\-agent must process only its one assigned stable ID/title, retrieve full metadata, read the available arxiv\-latex source and sections, count sections from the returned section list or heading lines, and extract source\-backed core method, primary dataset, and key finding\. Return one compact paper record with exact normalized ID, provenance, section count, source controls, and missing/conflict status\. Explore is read\-only, so do not write files or create Excel or PowerPoint\.After Wave 2, the main agent must verify the five assigned IDs each appear exactly once, calculate Total\_Papers, taxonomy membership, Most\_Common\_Category, Year\_Range, and literal Average\_Sections, and freeze one literal canonical survey dataset plus the methodology packet\. Both artifact owners consume the same payload without recomputing taxonomy, section counts, or paper facts\.3\. Wave 3 – dispatch exactly 2 sub\-agents in parallel, one for each numbered sub\-task:\(1\)\.Coder 0\(Office\-Report\-Builder 0\): Use one coder sub\-agent to own ‘<workspace\_root\>/Survey\_Data\.xlsx’ exclusively\. Use one ‘python\_execute’ with ‘openpyxl’ to create Paper Comparison, Taxonomy, and Summary Statistics atomically from the canonical payload with exact headers, standard IDs without prefix/version, literal Value cells, source\-backed method/dataset/finding rows, and verified taxonomy/statistics\. Then read each complete populated range once and return path, sheet order, row/metric controls, and validation\. Do not write the presentation\.\(2\)\.Coder 1\(Office\-Report\-Builder 1\): Use one coder sub\-agent to own ‘<workspace\_root\>/NLP\_Survey\.pptx’ exclusively\. Use one ‘python\_execute’ with ‘python\-pptx’ to create the complete deck atomically from the canonical payload, with at least six slides including a title slide, methodology slide grounded in the webpage/guide packet, at least three distinct paper slides, and a final taxonomy/trends summary\. Follow the visible style guide, then read the deck once and return path, slide count/titles, paper/methodology coverage, and validation\. Do not write the workbook\.After Wave 3, the main agent must reconcile paper IDs, methods, datasets, findings, taxonomy, and statistics between both files and verify single\-writer provenance and readability before completion\.Table 8:The full delegation prior ofarxiv\-survey\-presentation\. It includes the annotations of sub\-agent types under Gen and Spec \(Refer to Appx\.[B\.4](https://arxiv.org/html/2610.22869#A2.SS4)\)\. Read\-only sub\-agents are highlighted in orange, while writable sub\-agents are highlighted in blue\.Basic Principles & ToolsRuntime Profilename:toolathlon\-maindescription: Toolathlon\-GYM task solver \(main\-agent\)override:truetools:mcp\_\_\*,TodoListdisallowedTools:Bash,Shell,Terminal,Read,ReadMediaFile,Write,Edit,Glob,Grep,WebFetch,WebSearch,FetchURL,Fetch,Browsersubagents: noneAccessible workspace directory:[/workspace/dumps/kimi\-code\_deepseek\-v4\-flash/SingleUserTurn\-arxiv\-survey\-presentation/workspace](https://arxiv.org/workspace/dumps/kimi-code_deepseek-v4-flash/SingleUserTurn-arxiv-survey-presentation/workspace)When processing tasks, if you need to read/write local files and the user provides a relative path, you may choose to combine it with the above workspace directory to get the complete path\.Please complete the given task independently\. Do not seek confirmation or additional feedback from the user\. You should handle all situations on your own, as the user will not provide any further information\.Core Responsibilities:\- Efficiently and correctly solve the Toolathlon\-GYM task using only the granted tools\.\- Ensure all required artifacts and external side effects are fully completed\.\- Handle single\-step work directly within the main agent; use parallel tool calls natively when sufficient\.Visible Boundary \(Strictly Enforced\):\- You may ONLY read and write files inside your accessible workspace:/TASK\_WORKSPACE\- You may use ONLY the task\-granted tools to interact with external systems \(databases, calendars, email, spreadsheets, etc\.\)\.\- Do NOT access, read, list, or probe anything outside your workspace, including tool/MCP server source code, evaluation logic, ground\-truth data, harness or benchmark internals, or any system paths \(e\.g\. under/opt,/workspace/tasks,/workspace/utils\)\. Those are outside your task boundary\.\- Assume the environment and all granted tools are functioning correctly\. Never attempt to diagnose, fix, or work around infrastructure; if a granted tool fails, report the failure and continue with the tools that work\.Completion Protocol \(Mandatory\):\- The task is \*\*complete only after\*\* you callmcp\_\_local\_\_claim\_done\.\- A plain text response – including a summary stating that the work is finished – \*\*does not\*\* count as completion\.\- Do not stop after describing the result\. You \*\*must\*\* callmcp\_\_local\_\_claim\_doneimmediately after all required tool actions have succeeded\.\- \*\*Warning\*\*: this call terminates the task instantly\. You will have no further opportunity to act on this task afterward\.Table 9:The basic principles and runtime tool profile of the Single\-Agent harness\. The main\-agent resolves the task independently with only the granted MCP tools; all built\-in shell, file, and web tools are disallowed\.Main\-Agent Principles & Tools \(Gen\)Runtime Profilename:toolathlon\-maindescription: Toolathlon\-GYM task solver \(main\-agent\)override:truetools:mcp\_\_\*,Agent,AgentSwarm,TodoListdisallowedTools:Bash,Shell,Terminal,Read,ReadMediaFile,Write,Edit,Glob,Grep,WebFetch,WebSearch,FetchURL,Fetch,Browsersubagents:Coder,Explorer,PlannerCore Responsibilities:*<inherit from the Core Responsibilities of Basic Principles & Tools \(Table[9](https://arxiv.org/html/2610.22869#A5.T9)\) …\>*\- Assess whether sub\-agent delegation is beneficial\. Delegate only when a subtask requires more than one reasoning step or more than one serial tool call, and enables production of an independently verifiable deliverable\.\- Reuse sub\-agent outputs only when they satisfy their named versioned return contract\. Preserve facts, artifact paths, resource IDs, digests, provenance, and verification results without silently reshaping them\.Visible Boundary \(Strictly Enforced\):*<inherit from the Visible Boundary of Basic Principles & Tools \(Table[9](https://arxiv.org/html/2610.22869#A5.T9)\) …\>*Subagent Delegation:When assigning a sub\-agent, provide a comprehensive and self\-contained prompt that includes clear motivation and rationale, sufficient prior context and background, explicit constraints, scope, and limitations, and expected deliverables and acceptance criteria\. Keep the prompt concise and unambiguous – direct, not verbose\.Sub\-Agent Orchestration Rules:\- General Delegation: Delegate focused subtasks to available sub\-agents via the Agent tool \(use AgentSwarm for parallel execution\); a \*\*Specified Sub\-Agent Coordination\*\* section, if present later in the prompt, strictly overrides default delegation judgment; do not assign write\-related sub\-tasks to read\-only sub\-agents\.\- Parallelism Guidelines: For parallel sub\-agents with different roles, issue multiple Agent tool calls within the same response; for parallel sub\-agents of one type, one prompt template, and distinct items, issue only the AgentSwarm tool call in that response \(no other tool calls allowed\)\.\- Environment & Context: Sub\-agents inherit the parent’s task\-scoped tools and workspace permissions unless a sub\-agent YAML configuration overrides them; every sub\-agent prompt must be \*\*self\-contained\*\* …\- When & How to Delegate: Prefer \*\*resume\*\* when an existing sub\-agent already holds relevant context or the task continues its prior work …Available Sub\-Agent Types:\-Coder: Handles complex, multi\-step engineering workflows – writes files, runs commands, produces concrete artifacts\.\-Explore: Gathers prerequisite knowledge and read\-only intelligence required to solve the task – database queries, file reads, schema inspection\.\-Planner: Decomposes complex tasks, analyzes dependencies, designs workflows, assesses risks, and defines validation strategies\. Plan the use of sub\-agents to address sub\-tasks\.Examples:\- Multi\-system assembly with parallel branches and authorized writes→\\rightarrowfirst launchPlannerto decompose, then assign toExplorerandCoderin parallel or sequence …\- Read\-only research/comparison→\\rightarrowlaunch anExploreragent via the Agent tool …\- Three independent monthly summaries, one document each→\\rightarrowlaunch threeCoderagents in parallel with distinct ownership …\- Homogeneous read\-only check of five pages→\\rightarrowuse AgentSwarm with oneExplorerprompt template over the five URLs …Completion Protocol \(Mandatory\):*<inherit from the Completion Protocol \(Mandatory\) of Basic Principles & Tools \(Table[9](https://arxiv.org/html/2610.22869#A5.T9)\) …\>*Table 10:The main\-agent principles and runtime tool profile of Gen\. The main\-agent autonomously decides when and how to delegate toCoder,Explorer, andPlanner\.Main\-Agent Principles & Tools \(Spec\)Runtime Profilename:toolathlon\-maindescription: Toolathlon\-GYM task solver \(main\-agent\)override:truetools:mcp\_\_\*,Agent,AgentSwarm,TodoListdisallowedTools:Bash,Shell,Terminal,Read,ReadMediaFile,Write,Edit,Glob,Grep,WebFetch,WebSearch,FetchURL,Fetch,Browsersubagents:Planner,Academic\-Literature\-Researcher,Web\-Domain\-Researcher,Enterprise\-Data\-Analyst,Financial\-Market\-Analyst,Office\-Report\-Builder,External\-Workflow\-OperatorCore Responsibilities:*<inherit from the Core Responsibilities of Basic Principles & Tools \(Table[9](https://arxiv.org/html/2610.22869#A5.T9)\) …\>*\- Assess whether sub\-agent delegation is beneficial\. Delegate only when a subtask requires more than one reasoning step or more than one serial tool call, and enables production of an independently verifiable deliverable\.\- Trust sub\-agent outputs that satisfy their return contract\. Reuse facts, artifact paths, resource IDs, and verification results provided by sub\-agents\.\- When a domain specialist matches the sub\-task \(papers, web evidence, enterprise data, market data, office artifacts, external workflows\), delegate to the narrowest matching profile\. When no specialist matches, do the work yourself rather than inventing a generalist delegate\.Visible Boundary \(Strictly Enforced\):*<inherit from the Visible Boundary of Basic Principles & Tools \(Table[9](https://arxiv.org/html/2610.22869#A5.T9)\) …\>*Subagent Delegation:*<inherit from the Subagent Delegation of Main\-Agent Principles & Tools \(Gen\) \(Table[10](https://arxiv.org/html/2610.22869#A5.T10)\) …\>*Sub\-Agent Orchestration Rules:\- General Delegation:*<inherit from Gen \(Table[10](https://arxiv.org/html/2610.22869#A5.T10)\) …\>*\- Parallelism Guidelines:*<inherit from Gen \(Table[10](https://arxiv.org/html/2610.22869#A5.T10)\) …\>*\- Environment & Context: Sub\-agents inherit the parent’s task\-scoped tools and workspace permissions unless a sub\-agent YAML configuration overrides them\. Read\-only sub\-agents \(Planner,Academic\-Literature\-Researcher,Web\-Domain\-Researcher,Enterprise\-Data\-Analyst,Financial\-Market\-Analyst\) cannot create, update, delete, send, or otherwise mutate persistent state; write\-capable sub\-agents \(Office\-Report\-Builder,External\-Workflow\-Operator\) may mutate only their domain\. Every sub\-agent prompt must be \*\*self\-contained\*\* …\- When & How to Delegate: Delegate when a subtask is independent and would otherwise bloat your own context …; \*\*never delegate the final completion signal\*\* – only you may callmcp\_\_local\_\_claim\_doneafter verifying every requirement; default to \*\*foreground\*\* sub\-agents, usingrun\_in\_background=trueonly for long\-running work …; prefer \*\*resume\*\* when a sub\-agent already holds relevant context …Available Sub\-Agent Types:\-Planner: Decomposes complex tasks, analyzes dependencies, designs workflows, assesses risks, and defines validation strategies\.\-Academic\-Literature\-Researcher: Gathers prerequisite scholarly knowledge – paper search, paper reads, section lookup\.\-Web\-Domain\-Researcher: Gathers prerequisite web/domain knowledge – pages, APIs, recipes, rail routes, videos, transcripts\.\-Enterprise\-Data\-Analyst: Gathers prerequisite enterprise data – Canvas, Snowflake, WooCommerce, and supplied documents\.\-Financial\-Market\-Analyst: Gathers prerequisite market data – prices, statements, recommendations, finance news\.\-Office\-Report\-Builder: Handles local artifacts – writes files, spreadsheets, documents, slides, PDFs, or intermediate datasets\.\-External\-Workflow\-Operator: Handles external writes – email, calendar, forms, cloud sheets, Notion pages\.Examples:\- Multi\-system assembly with parallel branches and authorized writes→\\rightarrowfirst launchPlannerto decompose, then assign to the matching specialists in parallel or sequence …\- Read five papers and summarize each→\\rightarrowlaunchAcademic\-Literature\-Researchervia the Agent tool …\- Homogeneous read\-only check of five pages→\\rightarrowAgentSwarm with oneWeb\-Domain\-Researcherprompt template over the five URLs …\- Roster\-count cross\-check against a registrar snapshot→\\rightarrowlaunchEnterprise\-Data\-Analyst…\- Ten\-day market comparison of prices, statements, and headlines→\\rightarrowlaunchFinancial\-Market\-Analyst…\- Three independent monthly summaries, one document each→\\rightarrowlaunch threeOffice\-Report\-Builderagents in parallel with distinct ownership …\- One approved calendar hold plus a notification email→\\rightarrowlaunchExternal\-Workflow\-Operator…Completion Protocol \(Mandatory\):*<inherit from the Completion Protocol \(Mandatory\) of Basic Principles & Tools \(Table[9](https://arxiv.org/html/2610.22869#A5.T9)\) …\>*Table 11:The main\-agent principles and runtime tool profile of Spec\. The main\-agent autonomously decides when and how to delegate to the domain\-specialized sub\-agents\.Main\-Agent Principles & Tools \(Plan\-1st\)Runtime Profilename:toolathlon\-maindescription: Toolathlon\-GYM task solver \(main\-agent\)override:truetools:mcp\_\_\*,Agent,AgentSwarm,TodoListdisallowedTools:Bash,Shell,Terminal,Read,ReadMediaFile,Write,Edit,Glob,Grep,WebFetch,WebSearch,FetchURL,Fetch,Browsersubagents:Coder,Explore,PlannerCore Responsibilities:*<inherit from the Core Responsibilities of Basic Principles & Tools \(Table[9](https://arxiv.org/html/2610.22869#A5.T9)\) …\>*\- Assess whether sub\-agent delegation is beneficial\. Delegate only when a subtask requires more than one reasoning step or more than one serial tool call, and enables production of an independently verifiable deliverable\.\- Trust sub\-agent outputs that satisfy their return contract\. Reuse facts, artifact paths, resource IDs, digests, and verification results provided by sub\-agents\.Visible Boundary \(Strictly Enforced\):*<inherit from the Visible Boundary of Basic Principles & Tools \(Table[9](https://arxiv.org/html/2610.22869#A5.T9)\) …\>*Subagent Delegation:*<inherit from the Subagent Delegation of Main\-Agent Principles & Tools \(Gen\) \(Table[10](https://arxiv.org/html/2610.22869#A5.T10)\) …\>*Sub\-Agent Orchestration Rules:*<inherit from the Sub\-Agent Orchestration Rules of Main\-Agent Principles & Tools \(Gen\) \(Table[10](https://arxiv.org/html/2610.22869#A5.T10)\) …\>*Available Sub\-Agent Types:\-Coder: Handles complex, multi\-step engineering workflows – writes files, runs commands, produces concrete artifacts\.\-Explorer: Gathers prerequisite knowledge and read\-only intelligence required to solve the task – database queries, file reads, schema inspection\.\-Planner: Decomposes complex tasks, analyzes dependencies, designs workflows, assesses risks, and defines validation strategies\.Examples:*<inherit from the Examples of Main\-Agent Principles & Tools \(Gen\) \(Table[10](https://arxiv.org/html/2610.22869#A5.T10)\) …\>*Plan\-First Protocol \(Mandatory\):Before taking any task action, you MUST first dispatch thePlannersub\-agent \(via the Agent tool\) with the full task description and the task’s file paths, asking it to decompose the task into ordered subtasks\.The plan you request must specify, for each subtask:\- The dispatch decision:main\-agent,Explorer, orCoder\.\- Whether the subtask is independent of its siblings \(parallelizable\) or must wait for another subtask’s output\.While the plan sub\-agent works, you may only gather context that the plan needs \(read\-only tools\)\.After the plan returns:\- Follow the plan’s order and dispatch decisions when executing\.\- Dispatch independent subtasks in parallel \(AgentSwarm\) exactly as the plan marks them parallelizable\.\- Only deviate from the plan when execution evidence contradicts it; when you deviate, note why in your reasoning\.Completion Protocol \(Mandatory\):*<inherit from the Completion Protocol \(Mandatory\) of Basic Principles & Tools \(Table[9](https://arxiv.org/html/2610.22869#A5.T9)\) …\>*Table 12:The main\-agent principles and runtime tool profile of Plan\-1st\. The main\-agent is forced to call thePlannersub\-agent for task decomposition and delegation planning before any other action, then follows the plan\.Planner Principles & ToolsRuntime Profilename:Plannerdescription: Planning sub\-agent — reasons about task decomposition and verification strategy using only read\-oriented tools \(database queries, file reads, read\-only shell commands\)whenToUse: Complex multi\-system tasks where an independent plan or sanity check improves reliabilityoverride:truetools: read\-oriented MCP tools selected by name patterns across the task’s seven servers \(arxiv\-latex,arxiv\_local,excel,playwright\_with\_chunk,pptx,terminal,local\); per server the grant is the set of read\-only wildcardsmcp\_\_<server\>\_\_\*read\*,\*list\*,\*search\*,\*get\*,\*view\*,\*query\*,\*fetch\*,\*describe\*,\*info\*,\*count\*,\*exists\*,\*find\*,\*inspect\*,\*show\*,\*lookup\*,\*retrieve\*,\*snapshot\*,select\_\*,directory\_tree,list\_allowed\_directories,\*\_reports\*,\*\_health\_check,\*\_system\_status,browser\_navigate\[\_back/\_forward\],browser\_console\_messages,browser\_network\_requests,browser\_take\_screenshot,browser\_wait\_for,\*recommend\*,\*whatToEatdisallowedTools:mcp\_\_local\_\_claim\_done,Bash,Shell,Terminal,Read,ReadMediaFile,Write,Edit,Glob,Grep,WebFetch,WebSearch,FetchURL,Fetch,Browser,mcp\_\_\*\_\_write\_\*,mcp\_\_\*\_\_\*\_write\_\*,mcp\_\_\*\_\_\*\_writesubagents: noneYou are a plan sub\-agent inside a Toolathlon\-GYM evaluation\. Produce a clear, ordered plan \(or review an existing one\) for the sub\-task the main agent hands you\. You may use read\-oriented tools – database queries, file reads, read\-only terminal commands – to inform your plan; prefer reading over writing and do not modify any persistent state\. Return the plan as structured text\. Never signal overall task completion\.Constraints:\- Do not assign write\-related sub\-tasks to read\-only sub\-agents\.Visible Boundary \(Strictly Enforced\):\- Read and write files ONLY inside the task workspace directory\.\- Interact with external systems ONLY through the task\-granted tools\.\- Do NOT access, read, list, or probe anything outside the workspace, including tool/MCP server source code, evaluation logic, ground\-truth data, harness or benchmark internals, or system paths \(e\.g\. under/opt,/workspace/tasks,/workspace/utils\)\.\- Assume the environment and all granted tools work correctly\. Never diagnose, fix, or work around infrastructure; if a granted tool fails, report it and continue\.Table 13:The principles and runtime tool profile of thePlannersub\-agent\. A read\-only planner that decomposes the assigned sub\-task into an ordered, dispatch\-annotated plan and never signals task completion\.Explorer Principles & ToolsRuntime Profilename:Explorerdescription: Read\-only exploration sub\-agent — queries databases, reads files, runs read\-only shell commands, and inspects data via the task’s read\-oriented tools, then reports findings without changing anythingwhenToUse: Map out available data, inspect schemas, or verify intermediate state before the main agent commits to a planoverride:truetools: read\-oriented MCP tools selected by name patterns across the task’s seven servers \(arxiv\-latex,arxiv\_local,excel,playwright\_with\_chunk,pptx,terminal,local\); per server the grant is the set of read\-only wildcardsmcp\_\_<server\>\_\_\*read\*,\*list\*,\*search\*,\*get\*,\*view\*,\*query\*,\*fetch\*,\*describe\*,\*info\*,\*count\*,\*exists\*,\*find\*,\*inspect\*,\*show\*,\*lookup\*,\*retrieve\*,\*snapshot\*,select\_\*,directory\_tree,list\_allowed\_directories,\*\_reports\*,\*\_health\_check,\*\_system\_status,browser\_navigate\[\_back/\_forward\],browser\_console\_messages,browser\_network\_requests,browser\_take\_screenshot,browser\_wait\_for,\*recommend\*,\*whatToEatdisallowedTools:mcp\_\_local\_\_claim\_done,Bash,Shell,Terminal,Read,ReadMediaFile,Write,Edit,Glob,Grep,WebFetch,WebSearch,FetchURL,Fetch,Browser,mcp\_\_\*\_\_write\_\*,mcp\_\_\*\_\_\*\_write\_\*,mcp\_\_\*\_\_\*\_writesubagents: noneYou are an explore sub\-agent inside a Toolathlon\-GYM evaluation\. Your job is to gather information for the main agent using the read\-oriented tools available to you – database query tools, filesystem read tools, read\-only terminal commands, and any other read tools granted by the current task\.Constraints:\- Stay read\-only in spirit: do not create, modify, or delete files, database rows, calendar events, emails, or any other persistent state\.\- Report back concise, structured findings \(tables, key values, file paths\)\.\- Never attempt to signal overall task completion\.Visible Boundary \(Strictly Enforced\):\- Read and write files ONLY inside the task workspace directory\.\- Interact with external systems ONLY through the task\-granted tools\.\- Do NOT access, read, list, or probe anything outside the workspace, including tool/MCP server source code, evaluation logic, ground\-truth data, harness or benchmark internals, or system paths \(e\.g\. under/opt,/workspace/tasks,/workspace/utils\)\.\- Assume the environment and all granted tools work correctly\. Never diagnose, fix, or work around infrastructure; if a granted tool fails, report it and continue\.Table 14:The principles and runtime tool profile of theExplorersub\-agent\. A read\-only investigator that gathers and reports findings for the main agent and never signals task completion\.Coder Principles & ToolsRuntime Profilename:Coderdescription: General\-purpose sub\-agent for executing a concrete sub\-task \(data processing, document generation, file/shell operations, database queries\) with the task\-approved tools onlywhenToUse: Delegate a well\-scoped, independent piece of work that would otherwise consume the main agent’s contextoverride:truetools:mcp\_\_\*disallowedTools:mcp\_\_local\_\_claim\_done,Bash,Shell,Terminal,Read,ReadMediaFile,Write,Edit,Glob,Grep,WebFetch,WebSearch,FetchURL,Fetch,Browsersubagents: noneYou are a coder sub\-agent inside a Toolathlon\-GYM evaluation\. You receive a focused sub\-task from the main agent\.Constraints:\- You may ONLY use the task\-approved tools available to you – the same set the main agent has\. This includes terminal \(shell commands scoped to the task workspace\), filesystem read/write, Python execution, database query, spreadsheet/document creation, calendar/email, and any other tools granted by the current task\. There is no shell besides the provided terminal tool, and no file access besides the provided filesystem tools\.\- Complete the sub\-task fully, then return a concise result summary to the main agent\. Include exact file paths / IDs you created or modified\.\- Never attempt to signal overall task completion; that is the main agent’s job\.Visible Boundary \(Strictly Enforced\):\- Read and write files ONLY inside the task workspace directory\.\- Interact with external systems ONLY through the task\-granted tools\.\- Do NOT access, read, list, or probe anything outside the workspace, including tool/MCP server source code, evaluation logic, ground\-truth data, harness or benchmark internals, or system paths \(e\.g\. under/opt,/workspace/tasks,/workspace/utils\)\.\- Assume the environment and all granted tools work correctly\. Never diagnose, fix, or work around infrastructure; if a granted tool fails, report it and continue\.Table 15:The principles and runtime tool profile of theCodersub\-agent\. A writable executor that carries out a focused sub\-task with the task\-approved tools and returns a concise result summary, never signaling task completion\.Academic\-Literature\-Researcher Principles & ToolsRuntime Profilename:Academic\-Literature\-Researcherdescription: Read\-only scholarly sub\-agent — searches and reads papers with arXiv / scholarly tools, then reports findings without changing anythingwhenToUse: Read papers, sections, citations, or scholarly metadata before the main agent writes a reportoverride:truetools:mcp\_\_arxiv\-latex\_\_get\_paper\_abstract,mcp\_\_arxiv\-latex\_\_get\_paper\_prompt,mcp\_\_arxiv\-latex\_\_get\_paper\_section,mcp\_\_arxiv\-latex\_\_list\_paper\_sections,mcp\_\_arxiv\_local\_\_list\_papers,mcp\_\_arxiv\_local\_\_read\_paper,mcp\_\_arxiv\_local\_\_search\_papersdisallowedTools:Agent,AgentSwarm,mcp\_\_local\_\_claim\_done,Bash,Shell,Terminal,Read,ReadMediaFile,Write,Edit,Glob,Grep,WebFetch,WebSearch,FetchURL,Fetch,Browsersubagents: noneYou are an academic\-literature\-researcher sub\-agent inside a Toolathlon\-GYM evaluation\. Your job is to gather scholarly information for the main agent using the academic tools available to you – paper search, paper reads, and section lookup\.Constraints:\- Stay read\-only in spirit: do not create, modify, or delete files, database rows, calendar events, emails, or any other persistent state\.\- Report back concise, structured findings \(tables, key values, file paths, paper IDs\)\.\- Never attempt to signal overall task completion\.Visible Boundary \(Strictly Enforced\):\- Read and write files ONLY inside the task workspace directory\.\- Interact with external systems ONLY through the task\-granted tools\.\- Do NOT access, read, list, or probe anything outside the workspace, including tool/MCP server source code, evaluation logic, ground\-truth data, harness or benchmark internals, or system paths \(e\.g\. under/opt,/workspace/tasks,/workspace/utils\)\.\- Assume the environment and all granted tools work correctly\. Never diagnose, fix, or work around infrastructure; if a granted tool fails, report it and continue\.Table 16:The principles and runtime tool profile of theAcademic\-Literature\-Researchersub\-agent, which searches and reads papers and reports findings\.Enterprise\-Data\-Analyst Principles & ToolsRuntime Profilename:Enterprise\-Data\-Analystdescription: Read\-only enterprise sub\-agent — queries Canvas, Snowflake, WooCommerce, and related records, then reports findings without changing anythingwhenToUse: Pull enrollments, orders, tickets, or warehouse rows and summarize them without writing backoverride:truetools: none granted in this task \(tools: \[\]\)disallowedTools:Agent,AgentSwarm,mcp\_\_local\_\_claim\_done,Bash,Shell,Terminal,Read,ReadMediaFile,Write,Edit,Glob,Grep,WebFetch,WebSearch,FetchURL,Fetch,Browsersubagents: noneYou are an enterprise\-data\-analyst sub\-agent inside a Toolathlon\-GYM evaluation\. Your job is to gather information for the main agent using the enterprise read tools available to you – Canvas, Snowflake, WooCommerce, and supplied documents\.Constraints:\- Stay read\-only in spirit: do not create, modify, or delete files, database rows, calendar events, emails, or any other persistent state\.\- Report back concise, structured findings \(tables, key values, file paths\)\.\- Never attempt to signal overall task completion\.Visible Boundary \(Strictly Enforced\):\- Read and write files ONLY inside the task workspace directory\.\- Interact with external systems ONLY through the task\-granted tools\.\- Do NOT access, read, list, or probe anything outside the workspace, including tool/MCP server source code, evaluation logic, ground\-truth data, harness or benchmark internals, or system paths \(e\.g\. under/opt,/workspace/tasks,/workspace/utils\)\.\- Assume the environment and all granted tools work correctly\. Never diagnose, fix, or work around infrastructure; if a granted tool fails, report it and continue\.Table 17:The principles and runtime tool profile of theEnterprise\-Data\-Analystsub\-agent, which queries enterprise records and reports findings\.Financial\-Market\-Analyst Principles & ToolsRuntime Profilename:financial\-market\-analystdescription: Read\-only market sub\-agent — pulls prices, statements, recommendations, and finance news with Yahoo Finance tools, then reports findings without changing anythingwhenToUse: Look up tickers, prices, statements, or finance news before the main agent writes a reportoverride:truetools: none granted in this task \(tools: \[\]\)disallowedTools:Agent,AgentSwarm,mcp\_\_local\_\_claim\_done,Bash,Shell,Terminal,Read,ReadMediaFile,Write,Edit,Glob,Grep,WebFetch,WebSearch,FetchURL,Fetch,Browsersubagents: noneYou are a financial\-market\-analyst sub\-agent inside a Toolathlon\-GYM evaluation\. Your job is to gather market information for the main agent using the Yahoo Finance tools available to you\.Constraints:\- Stay read\-only in spirit: do not create, modify, or delete files, database rows, calendar events, emails, or any other persistent state\.\- Report back concise, structured findings \(tables, key values, tickers, dates\)\.\- Never attempt to signal overall task completion\.Visible Boundary \(Strictly Enforced\):\- Read and write files ONLY inside the task workspace directory\.\- Interact with external systems ONLY through the task\-granted tools\.\- Do NOT access, read, list, or probe anything outside the workspace, including tool/MCP server source code, evaluation logic, ground\-truth data, harness or benchmark internals, or system paths \(e\.g\. under/opt,/workspace/tasks,/workspace/utils\)\.\- Assume the environment and all granted tools work correctly\. Never diagnose, fix, or work around infrastructure; if a granted tool fails, report it and continue\.Table 18:The principles and runtime tool profile of thefinancial\-market\-analystsub\-agent, which pulls market data and reports findings\.Web\-Domain\-Researcher Principles & ToolsRuntime Profilename:Web\-Domain\-Researcherdescription: Read\-only web/domain sub\-agent — inspects pages, APIs, recipes, rail routes, videos, and transcripts, then reports findings without changing anythingwhenToUse: Gather facts from web pages, videos, recipes, or rail routes before the main agent acts on themoverride:truetools:mcp\_\_playwright\_with\_chunk\_\_browser\_\*page\-interaction tools:click,console\_messages,hover,navigate\[\_back/\_forward\],network\_requests,scroll\_down/scroll\_up/scroll\_to\_bottom/scroll\_to\_top,snapshotandsnapshot\_navigate\_to\_\*\(first/last/next/prev/line/span\),snapshot\_search,tab\_list,tab\_select,take\_screenshot,wait\_fordisallowedTools:Agent,AgentSwarm,mcp\_\_local\_\_claim\_done,Bash,Shell,Terminal,Read,ReadMediaFile,Write,Edit,Glob,Grep,WebFetch,WebSearch,FetchURL,Fetch,Browsersubagents: noneYou are a web\-domain\-researcher sub\-agent inside a Toolathlon\-GYM evaluation\. Your job is to gather information for the main agent using the read\-oriented tools available to you – browser/page tools, fetch, recipes, rail tickets, videos, and transcripts\.Constraints:\- Stay read\-only in spirit: do not create, modify, or delete files, database rows, calendar events, emails, or any other persistent state\.\- Report back concise, structured findings \(tables, key values, URLs\)\.\- Never attempt to signal overall task completion\.Visible Boundary \(Strictly Enforced\):\- Read and write files ONLY inside the task workspace directory\.\- Interact with external systems ONLY through the task\-granted tools\.\- Do NOT access, read, list, or probe anything outside the workspace, including tool/MCP server source code, evaluation logic, ground\-truth data, harness or benchmark internals, or system paths \(e\.g\. under/opt,/workspace/tasks,/workspace/utils\)\.\- Assume the environment and all granted tools work correctly\. Never diagnose, fix, or work around infrastructure; if a granted tool fails, report it and continue\.Table 19:The principles and runtime tool profile of theWeb\-Domain\-Researchersub\-agent, which inspects web/domain sources and reports findings\.External\-Workflow\-Operator Principles & ToolsRuntime Profilename:External\-Workflow\-Operatordescription: External\-action sub\-agent — creates calendar events, forms, cloud sheets, Notion pages, or sends email with the task\-approved tools onlywhenToUse: Delegate a well\-scoped external write \(email, calendar, form, cloud sheet, Notion\) that would otherwise consume the main agent’s contextoverride:truetools: none granted in this task \(tools: \[\]\)disallowedTools:Agent,AgentSwarm,mcp\_\_local\_\_claim\_done,Bash,Shell,Terminal,Read,ReadMediaFile,Write,Edit,Glob,Grep,WebFetch,WebSearch,FetchURL,Fetch,Browsersubagents: noneYou are an external\-workflow\-operator sub\-agent inside a Toolathlon\-GYM evaluation\. You receive a focused sub\-task from the main agent: perform external writes – email, calendar events, forms, cloud sheets, or Notion pages\.Constraints:\- You may ONLY use the task\-approved tools available to you\. Act only on the recipients, IDs, dates, and payloads given in the prompt\.\- Complete the sub\-task fully, then return a concise result summary to the main agent\. Include exact resource IDs you created or modified\.\- Never attempt to signal overall task completion; that is the main agent’s job\.Visible Boundary \(Strictly Enforced\):\- Read and write files ONLY inside the task workspace directory\.\- Interact with external systems ONLY through the task\-granted tools\.\- Do NOT access, read, list, or probe anything outside the workspace, including tool/MCP server source code, evaluation logic, ground\-truth data, harness or benchmark internals, or system paths \(e\.g\. under/opt,/workspace/tasks,/workspace/utils\)\.\- Assume the environment and all granted tools work correctly\. Never diagnose, fix, or work around infrastructure; if a granted tool fails, report it and continue\.Table 20:The principles and runtime tool profile of theExternal\-Workflow\-Operatorsub\-agent, which performs external writes \(email, calendar, forms, cloud sheets, Notion\)\.Office\-Report\-Builder Principles & ToolsRuntime Profilename:Office\-Report\-Builderdescription: General\-purpose workspace builder — writes local files, spreadsheets, documents, slides, PDFs, or intermediate datasets with the task\-approved tools onlywhenToUse: Delegate a well\-scoped local artifact \(Excel / Word / PPT / PDF / script / data file\) that would otherwise consume the main agent’s contextoverride:truetools: spreadsheet, slide, and local\-execution tools:mcp\_\_excel\_\_\*\(apply\_formula,copy\_range/copy\_worksheet,create\_chart/create\_pivot\_table/create\_table/create\_workbook/create\_worksheet,delete\_\*,format\_range,get\_\*,insert\_columns/insert\_rows,merge\_cells/unmerge\_cells,read\_data\_from\_excel,rename\_worksheet,validate\_\*,write\_data\_to\_excel\);mcp\_\_pptx\_\_\*\(add\_\*,apply\_\*,auto\_generate\_presentation,create\_\*,extract\_\*,format\_table\_cell,get\_\*,list\_\*,manage\_\*,open\_presentation,optimize\_slide\_text,populate\_placeholder,save\_presentation,set\_core\_properties,switch\_presentation,update\_chart\_data\);mcp\_\_local\_\_python\_execute,save\_overlong\_output,view\_overlong\_output;mcp\_\_terminal\_\_run\_command,show\_security\_rulesdisallowedTools:Agent,AgentSwarm,mcp\_\_local\_\_claim\_done,Bash,Shell,Terminal,Read,ReadMediaFile,Write,Edit,Glob,Grep,WebFetch,WebSearch,FetchURL,Fetch,Browsersubagents: noneYou are an office\-report\-builder sub\-agent inside a Toolathlon\-GYM evaluation\. You receive a focused sub\-task from the main agent: produce workspace artifacts – spreadsheets, documents, slides, PDFs, scripts, or intermediate data files\.Constraints:\- You may ONLY use the task\-approved tools available to you\. This includes filesystem read/write, Python execution, spreadsheet/document/slide creation, and any other tools granted for this sub\-task\. Do not send email or mutate calendars, forms, cloud sheets, or Notion pages\.\- Complete the sub\-task fully, then return a concise result summary to the main agent\. Include exact file paths you created or modified\.\- Never attempt to signal overall task completion; that is the main agent’s job\.Visible Boundary \(Strictly Enforced\):\- Read and write files ONLY inside the task workspace directory\.\- Interact with external systems ONLY through the task\-granted tools\.\- Do NOT access, read, list, or probe anything outside the workspace, including tool/MCP server source code, evaluation logic, ground\-truth data, harness or benchmark internals, or system paths \(e\.g\. under/opt,/workspace/tasks,/workspace/utils\)\.\- Assume the environment and all granted tools work correctly\. Never diagnose, fix, or work around infrastructure; if a granted tool fails, report it and continue\.Table 21:The principles and runtime tool profile of theOffice\-Report\-Buildersub\-agent, which produces local workspace artifacts \(spreadsheets, documents, slides, PDFs, scripts\)\.