Learning What to Retain: Gated-Memory Routing for Efficient Collaboration in Multi-Agent LLM Systems

# Learning What to Retain: Gated-Memory Routing for Efficient Collaboration in Multi-Agent LLM Systems
Source: [https://arxiv.org/html/2609.00237](https://arxiv.org/html/2609.00237)
###### Abstract

Large language model \(LLM\)–based multi\-agent systems tackle complex reasoning by orchestrating how multiple agents are configured and how they collaborate\. A central challenge is to adapt orchestration to the evolving collaboration state\. Routing from the query alone cannot adapt to intermediate progress or errors, which hurts accuracy\. Routing from the complete execution history supplies this missing context, but forces later decisions to process every prior step, including redundant or low\-utility ones\. This creates an*execution\-history overload*that inflates cost\. Effective orchestration instead requires a compact state that captures useful progress without accumulating redundant context\. We proposeGated\-Memory Routing, which conditions each decision on the query and a learned execution memory\. A learned*Memory Write Gate*commits only non\-redundant reasoning steps, and a learned*Retrieval Gate*supplies each agent a compact, relevant subset, so every decision conditions on a clean, informative state\. At each step, the system selects the next role and backbone from this memory, while an*Adaptive Halting Controller*stops execution once the memory contains sufficient evidence for answering\. Across five reasoning and code\-generation benchmarks, our framework is both effective and efficient: it attains the best average accuracy, exceeding the strongest baseline by2\.442\.44points, while reducing HumanEval inference cost by31\.9%31\.9\\%relative to that baseline\. Code is available at[https://github\.com/rajibrhasan/gated\-memory\-routing](https://github.com/rajibrhasan/gated-memory-routing)\.

## 1Introduction

\(a\)Full\-History Routing![Refer to caption](https://arxiv.org/html/2609.00237v1/gated_memory_routing.png)\(b\)Gated\-Memory Routing

Figure 1:Full\-history routing feeds each decision the entire trajectory, raising context length and cost\. Gated\-memory routing instead maintains a selective memory through write and retrieval gates and halts when further collaboration is unlikely to add value, keeping context relevant and cost low\.Large language model \(LLM\)–based multi\-agent systems have become a powerful paradigm for complex reasoning: by decomposing a problem across agents that specialize, critique, or extend one another’s reasoning, they achieve strong performance on reasoning and code\-generation benchmarks[Chen et al\. \(2024c\)](https://arxiv.org/html/2609.00237#bib.bib22);[Liang et al\. \(2024\)](https://arxiv.org/html/2609.00237#bib.bib11);[Zhang et al\. \(2024\)](https://arxiv.org/html/2609.00237#bib.bib13);[Qian et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib12)\. Their effectiveness depends not only on the underlying LLMs but on how execution is*orchestrated*: how many agents to invoke, which roles to assign, which backbone powers each agent, and what each agent may see\. These decisions constitute the*routing*problem, and how well they are made determines whether collaboration yields genuine reasoning gains or redundant computation\.

Early multi\-agent systems address routing through fixed, hand\-engineered designs, instantiating predetermined agents with manually specified roles and static collaboration patterns that remain constant across queries[Hong et al\. \(2024\)](https://arxiv.org/html/2609.00237#bib.bib21);[Qian et al\. \(2024\)](https://arxiv.org/html/2609.00237#bib.bib20);[Li et al\. \(2023\)](https://arxiv.org/html/2609.00237#bib.bib14);[Du et al\. \(2024\)](https://arxiv.org/html/2609.00237#bib.bib19);[Wu et al\. \(2024\)](https://arxiv.org/html/2609.00237#bib.bib15)\. To relax this rigidity, later work makes the interaction structure adaptive by optimizing or pruning the communication graph[Zhuge et al\. \(2024\)](https://arxiv.org/html/2609.00237#bib.bib4);[Zhang et al\. \(2025a\)](https://arxiv.org/html/2609.00237#bib.bib5);[Wang et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib6);[Liu et al\. \(2024\)](https://arxiv.org/html/2609.00237#bib.bib23), and end\-to\-end routers configure agent count, roles, and backbones directly from the query[Yue et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib10)\. These*query\-only*routers commit routing decisions before any agent produces reasoning, so they cannot respond to partial progress, errors, or emerging gaps in the solution\. A recent alternative conditions routing on the unfolding execution, sequencing agents over the evolving task state[Dang et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib30)\. When that evolving state is carried forward with little filtering, however, each decision must reason over an undifferentiated, ever\-growing context that mixes useful evidence with redundant or erroneous reasoning; this*full\-history routing*exposes downstream agents to low\-value context and inflates inference cost\.

These two approaches have opposite limitations: conditioning on the query alone observes too little, committing before execution begins, which hurts accuracy, while conditioning on the full execution history \(Figure[1](https://arxiv.org/html/2609.00237#S1.F1)[1\(a\)](https://arxiv.org/html/2609.00237#S1.F1.sf1)\) observes too much, forcing each decision to process an unfiltered and growing trajectory, which inflates cost\. A router should instead learn what to retain, what to surface, and how to act on the resulting state\. This motivates conditioning routing on a*structured, selective memory*\(Figure[1](https://arxiv.org/html/2609.00237#S1.F1)[1\(b\)](https://arxiv.org/html/2609.00237#S1.F1.sf2)\): the system dynamically routes based on a gated memory, exposes only step\-relevant context to downstream agents, and commits only the reasoning steps worth preserving\. These decisions are tightly coupled: low\-quality writes pollute later retrieval, overly broad retrieval distracts downstream agents, and fixed\-depth execution either stops prematurely or continues spending computation after additional reasoning is unlikely to help\.

Based on this view, we introduceGated\-Memory Routing, a framework in which a learned, gated execution memory serves as the shared state that every routing decision conditions on\. A single jointly trained router acts on it through complementary components: a*History\-Aware Role Allocator*and an*LLM Router*choose the agent and its backbone, a*Retrieval Gate*surfaces a compact, step\-relevant subset of memory, a*Memory Write Gate*commits only high\-utility, non\-redundant reasoning steps, and an*Adaptive Halting Controller*decides when the gated state is sufficient to stop\. Because routing is driven by this gated memory rather than the raw history, the interaction structure emerges dynamically during execution\.

#### Contributions\.

- •Routing over a learned gated memory\.We recast multi\-agent orchestration as routing over a learned, selective execution memory: each step conditions on a filtered task\-relevant state rather than the query alone \(query\-only\) or the raw execution history \(full\-history\)\.
- •Memory curation and budget\-aware halting\.Learned write and retrieval gates control what memory stores and surfaces, keeping the state compact and high\-signal, while a budget\-aware halting policy consumes that clean state to control reasoning depth and thus cost, all trained end\-to\-end with role and backbone routing under a group\-relative, cost\-aware objective\.
- •Empirical effectiveness and efficiency\.Across MATH, GSM\-Hard, MBPP, HumanEval, and MMLU\-Pro, Gated\-Memory Routing attains the best average accuracy, exceeding the strongest baseline by2\.442\.44points while cutting HumanEval inference cost by31\.9%31\.9\\%relative to that baseline\.

## 2Related Work

#### Fixed and Role\-Based Multi\-Agent Systems\.

Representative LLM\-based multi\-agent systems design collaboration through predefined roles and static interaction graphs\. MetaGPT[Hong et al\. \(2024\)](https://arxiv.org/html/2609.00237#bib.bib21)enforces role\-based workflows through Standard Operating Procedures, while ChatDev[Qian et al\. \(2024\)](https://arxiv.org/html/2609.00237#bib.bib20)models software development as a fixed linear chain\. These systems demonstrate the value of specialization, but their communication patterns are fixed before execution begins and cannot adapt the participating agents, context, or depth as execution unfolds\.

#### Adaptive Topology and Graph Optimization\.

Several methods make workflows more flexible by adapting interaction structures or optimizing communication graphs\. GPTSwarm[Zhuge et al\. \(2024\)](https://arxiv.org/html/2609.00237#bib.bib4)optimizes orchestration through learnable prompting, and AFlow[Zhang et al\. \(2025b\)](https://arxiv.org/html/2609.00237#bib.bib18)searches over executable workflows\. AgentPrune[Zhang et al\. \(2025a\)](https://arxiv.org/html/2609.00237#bib.bib5)and AgentDropout[Wang et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib6)induce sparsity by pruning connections or agents from predefined templates\. DyLAN[Liu et al\. \(2024\)](https://arxiv.org/html/2609.00237#bib.bib23)goes further by pruning low\-contribution agents during inference using peer evaluation and consensus, and OMAC[Li et al\. \(2026\)](https://arxiv.org/html/2609.00237#bib.bib37)jointly optimizes agent functionality and collaboration structure\. These methods improve flexibility by optimizing which agents participate and how they are connected, but they do not learn which intermediate information subsequent orchestration decisions should condition on; DyLAN, for example, adapts by removing agents while still forwarding the unfiltered outputs of the remaining ones\.

#### Query\-Level LLM and MAS Routing\.

LLM routing optimizes the trade\-off between model cost and capability\. RouteLLM[Ong et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib8), RouterDC[Chen et al\. \(2024b\)](https://arxiv.org/html/2609.00237#bib.bib9), R2\-Router[Xue et al\. \(2026\)](https://arxiv.org/html/2609.00237#bib.bib40), HW\-Router[Kabir et al\. \(2026\)](https://arxiv.org/html/2609.00237#bib.bib41), Securerouter[Zhang et al\. \(2026\)](https://arxiv.org/html/2609.00237#bib.bib42)and FrugalGPT[Chen et al\. \(2024a\)](https://arxiv.org/html/2609.00237#bib.bib7)route each query to a cost\-effective model\. In multi\-agent systems, MASRouter[Yue et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib10)trains an RL policy to configure execution components, including agent count, role assignment, and backbone selection, from the input query\. These methods make routing more cost\-aware and task\-adaptive, but their decisions are primarily conditioned on the query rather than on the unfolding execution trajectory\. They therefore cannot revise orchestration based on the quality, novelty, or redundancy of intermediate trajectory elements\.

#### Routing over the Execution History\.

Recent work addresses the query\-only routing limitation by conditioning orchestration on the unfolding execution state\. Evolving Orchestration[Dang et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib30)uses RL to train a centralized orchestrator that selects agents from the evolving execution state and terminates through a designated stopping action\. This makes routing responsive to execution, but the state retains the accumulated trajectory with limited filtering, so intermediate elements are carried forward to later decisions\. Such full\-history routing can lead to execution\-history overload as execution deepens, forcing later decisions to process redundant, noisy, or low\-utility intermediate content\. Our framework instead routes through a gated execution memory, conditioning each decision on a filtered state rather than the unfiltered history, and outperforms Evolving Orchestration by2\.442\.44average points across the benchmarks in Table[1](https://arxiv.org/html/2609.00237#S4.T1)\. Our adaptive halting also relates to adaptive computation[Graves \(2016\)](https://arxiv.org/html/2609.00237#bib.bib36)and early\-exit inference, which learn when to stop computing within a single model; in contrast, it halts multi\-agent collaboration based on the gated memory state\.

#### Memory\-Optimized Agent Architectures\.

Memory management is a core component of agent systems\. MemGPT[Packer et al\. \(2023\)](https://arxiv.org/html/2609.00237#bib.bib24)pages information in and out of context through a virtual memory hierarchy, AIOS[Mei et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib25)manages memory as a shared OS\-level resource for agents, and systems such as A\-mem[Xu et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib26)and MM[Hatalis et al\. \(2023\)](https://arxiv.org/html/2609.00237#bib.bib27)organize memory through categorization and retrieval\. Memory\-R1[Yan et al\. \(2026\)](https://arxiv.org/html/2609.00237#bib.bib38)uses reinforcement learning to train a single agent’s policies for writing to and retrieving from a long\-term memory over an external store\. These systems primarily treat memory as a storage and retrieval substrate, optimizing what information should be preserved or recalled under a context budget\. In contrast, Gated\-Memory Routing treats the within\-query execution memory of a multi\-agent trajectory as an active control signal: write and retrieval decisions are trained jointly with role allocation, backbone routing, and halting under the task\-level reward, so memory not only supplies context but also determines who acts next, what they read, and when collaboration stops\.

## 3Methodology

Figure 2:Gated\-Memory Routing framework\.At each steptt, the system constructs a statest=\(q,St−1\)s\_\{t\}=\(q,S\_\{t\-1\}\)from the query and current execution memory\. A role allocator and LLM router configure the next agent, a retrieval gate supplies relevant memory context, and a write gate decides whether the new execution record enters memory\. The halting controller either continues the loop or passes the stored records to an Aggregator LLM for the final answer\.### 3\.1Problem Definition

We formalize gated\-memory routing as a sequential decision process over a set of agent specializationsℛ\\mathcal\{R\}, a pool of heterogeneous LLM backbonesℳ\\mathcal\{M\}, and an execution memoryStS\_\{t\}maintained within a single query execution\. At steptt, the observable state isst=\(q,St−1\)s\_\{t\}=\(q,S\_\{t\-1\}\), whereSt−1S\_\{t\-1\}holds the retained execution records from prior steps\. The system selects a specializationrt∈ℛr\_\{t\}\\in\\mathcal\{R\}and a backbonemt∈ℳm\_\{t\}\\in\\mathcal\{M\}, retrieves a context subsetCt⊆St−1C\_\{t\}\\subseteq S\_\{t\-1\}, generates a reasoning stepyty\_\{t\}, and decides whether to commit the recordet=\(rt,mt,yt\)e\_\{t\}=\(r\_\{t\},m\_\{t\},y\_\{t\}\), so memory evolves as a gated subset of the trajectory,St⊆\{ei\}i=1tS\_\{t\}\\subseteq\\\{e\_\{i\}\\\}\_\{i=1\}^\{t\}\. The process terminates at a realized depthL∈\{1,…,ϕ\}L\\in\\\{1,\\dots,\\phi\\\}when the halting policy fires or the maximum depthϕ\\phiis reached; because halting is evaluated after each step, at least one agent always acts\. An aggregator then produces the final answer from the terminal stateSLS\_\{L\}\. Throughout, boldface denotes the frozen sentence embedding of a quantity \(e\.g\.,𝐪=enc⁡\(q\)\\mathbf\{q\}=\\operatorname\{enc\}\(q\)\)\.

Writingτ<t=\{ei\}i=1t−1\\tau\_\{<t\}=\\\{e\_\{i\}\\\}\_\{i=1\}^\{t\-1\}for the full raw trajectory before steptt, routing strategies differ in the state each decision sees:*query\-only*routing conditions onqqalone,*full\-history*routing onqqand the entireτ<t\\tau\_\{<t\}, and*gated\-memory*routing onst=\(q,St−1\)s\_\{t\}=\(q,S\_\{t\-1\}\), pairing the query with a learned selective memory derived fromτ<t\\tau\_\{<t\}that keeps only selected records\.

### 3\.2Framework Overview

Figure[2](https://arxiv.org/html/2609.00237#S3.F2)illustrates Gated\-Memory Routing\. Its central object is the gated execution memoryStS\_\{t\}that every decision conditions on\. A single jointly trained router acts over it through complementary components: a*Memory Write Gate*\(Section[3\.6](https://arxiv.org/html/2609.00237#S3.SS6)\) and*Retrieval Gate*\(Section[3\.5](https://arxiv.org/html/2609.00237#S3.SS5)\) keep the memory high\-signal, so the*History\-Aware Role Allocator*\(Section[3\.3](https://arxiv.org/html/2609.00237#S3.SS3)\) and*LLM Router*\(Section[3\.4](https://arxiv.org/html/2609.00237#S3.SS4)\) choose from a gated execution state, while the*Adaptive Halting Controller*\(Section[3\.7](https://arxiv.org/html/2609.00237#S3.SS7)\) reads that state to decide when to stop, making depth, and thus cost, adaptive\. Because the memory stays compact and trustworthy, it can halt early without the overload full\-history routing incurs\. The trajectory is trained end\-to\-end with a single reward trading answer quality against cost \(Section[3\.8](https://arxiv.org/html/2609.00237#S3.SS8)\); encoders and representations appear in Appendix[8](https://arxiv.org/html/2609.00237#S8)\.

### 3\.3History\-Aware Role Allocator

Role allocation chooses which agent specialization acts at each step\. Rather than committing from the query alone or reacting to the full raw trajectory, our allocator conditions on the current statest=\(q,St−1\)s\_\{t\}=\(q,S\_\{t\-1\}\), so the next specialization complements the retained solution state rather than every intermediate step produced\.

Because specializations such as a critic, verifier, or debugger carry semantic structure rather than being interchangeable IDs, we represent each role through its textual description rather than a categorical label\. The description is encoded and projected through a Variational Autoencoder \(VAE\)[Kingma and Welling \(2014\)](https://arxiv.org/html/2609.00237#bib.bib28)into a continuous latent role embedding𝐫i\\mathbf\{r\}\_\{i\}\. A state encoder mapsst=\(q,St−1\)s\_\{t\}=\(q,S\_\{t\-1\}\)to a context vector𝐜t\\mathbf\{c\}\_\{t\}, and the allocator defines a stochastic policy over the available rolesℛ\\mathcal\{R\}:

πr​\(ri∣st\)=exp⁡\(𝐜t⊤​𝐫i\)∑rj∈ℛexp⁡\(𝐜t⊤​𝐫j\)\.\\pi\_\{r\}\(r\_\{i\}\\mid s\_\{t\}\)=\\frac\{\\exp\(\\mathbf\{c\}\_\{t\}^\{\\top\}\\mathbf\{r\}\_\{i\}\)\}\{\\sum\_\{r\_\{j\}\\in\\mathcal\{R\}\}\\exp\(\\mathbf\{c\}\_\{t\}^\{\\top\}\\mathbf\{r\}\_\{j\}\)\}\.This policy selects the specialization whose latent description best matches the query and current memory state, supporting transfer across related roles and extension to newly described specializations without changing the routing interface\.

### 3\.4LLM Router

Backbone selection matches the current reasoning need to a model’s capability and cost\. Because this need depends on the selected role and the progress already in memory \(a difficult step may warrant a stronger model, while memory may already supply enough guidance for a cheaper one\), we model LLM routing as a role\-conditioned policy over the current statests\_\{t\}rather than a query\-level choice\.

The router combines the state context𝐜t\\mathbf\{c\}\_\{t\}with the selected role representation via a learned projection to form a role\-conditioned context𝐮t\\mathbf\{u\}\_\{t\}\. Each candidate LLM is represented by a continuous latent capability vector𝐦i\\mathbf\{m\}\_\{i\}, derived from its natural\-language description using the same description\-encoding scheme as the role allocator, rather than by a fixed index\. The router samples the backbone according to

πm​\(mi∣st,rt\)=exp⁡\(𝐮t⊤​𝐦i\)∑mj∈ℳexp⁡\(𝐮t⊤​𝐦j\)\.\\pi\_\{m\}\(m\_\{i\}\\mid s\_\{t\},r\_\{t\}\)=\\frac\{\\exp\(\\mathbf\{u\}\_\{t\}^\{\\top\}\\mathbf\{m\}\_\{i\}\)\}\{\\sum\_\{m\_\{j\}\\in\\mathcal\{M\}\}\\exp\(\\mathbf\{u\}\_\{t\}^\{\\top\}\\mathbf\{m\}\_\{j\}\)\}\.This lets the system route each step to a model whose described capabilities match the current role and memory state\.

### 3\.5Retrieval Gate

Even a selective memory should not be exposed wholesale to every agent: many retained records belong to other subproblems or roles irrelevant to the agent about to act, so passing all of them inflates token cost and buries the entries that matter\. A fixed top\-kkretriever is equally rigid, since some steps need broad context while others need one record or none\. The*Retrieval Gate*therefore treats context construction as a step\-adaptive selection, deciding independently for each record whether to surface it\.

Each stored record is summarized by an embedding𝐯j\\mathbf\{v\}\_\{j\}that fuses its role latent, backbone latent, and a projection of its response, and the current step forms a retrieval query𝐩t\\mathbf\{p\}\_\{t\}by the analogous fusion of the query with the role and backbone just selected\. The gate therefore asks which prior records a step of this configuration should read: for recordjj, the retrieval logit and binary decision are

ℓt,j=𝐩t⊤​𝐯j,zt,j∼Bernoulli⁡\(σ⁡\(ℓt,j\)\),\\ell\_\{t,j\}=\\mathbf\{p\}\_\{t\}^\{\\top\}\\mathbf\{v\}\_\{j\},\\qquad z\_\{t,j\}\\sim\\mathrm\{Bernoulli\}\(\\sigma\(\\ell\_\{t,j\}\)\),and the retrieved context is

Ct=\{ej∈St−1:zt,j=1\}\.C\_\{t\}=\\\{e\_\{j\}\\in S\_\{t\-1\}:z\_\{t,j\}=1\\\}\.Because the draws are independent,CtC\_\{t\}varies in size, contracting toward empty when memory is largely irrelevant and expanding when several records bear on the step\. Independence keeps each decision local; cross\-record redundancy is instead suppressed at write time \(Section[3\.6](https://arxiv.org/html/2609.00237#S3.SS6)\)\. The gate has no cost term of its own: retrieving more records lengthens the agent prompt and is paid for only through the trajectory reward \(Section[3\.8](https://arxiv.org/html/2609.00237#S3.SS8)\)\.

### 3\.6Memory Write Gate

What the system writes to memory determines the state on which all later decisions are based\. If every intermediate element is appended, gated memory collapses back into full\-history routing, and redundant records crowd memory and dilute retrieval\. The*Memory Write Gate*therefore treats memory construction as a selective decision rather than a default append: a new record should enter memory only when it is both relevant to the task and novel with respect to the records already stored\.

We score this relevance–novelty trade\-off with a Maximal Marginal Relevance criterion[Carbonell and Goldstein \(1998\)](https://arxiv.org/html/2609.00237#bib.bib29), but replace its fixed components with learned, stochastic ones\. Let𝐜t\\mathbf\{c\}\_\{t\}denote the current context state \(Section[3\.3](https://arxiv.org/html/2609.00237#S3.SS3)\),𝐲t\\mathbf\{y\}\_\{t\}the embedding of the new reasoning step, and\{𝐲j\}\\\{\\mathbf\{y\}\_\{j\}\\\}the embeddings of stored steps, and letsim⁡\(⋅,⋅\)\\mathrm\{sim\}\(\\cdot,\\cdot\)be a cosine similarity taken in a learned projection of these frozen embeddings, so the relevance and novelty geometry is trained rather than prescribed\. The write score is

ωt=λsim\(𝐲t,𝐜t\)−\(1−λ\)maxj:ej∈St−1sim\(𝐲t,𝐲j\),\\omega\_\{t\}=\\lambda\\,\\mathrm\{sim\}\(\\mathbf\{y\}\_\{t\},\\mathbf\{c\}\_\{t\}\)\-\(1\-\\lambda\)\\max\_\{j:e\_\{j\}\\in S\_\{t\-1\}\}\\mathrm\{sim\}\(\\mathbf\{y\}\_\{t\},\\mathbf\{y\}\_\{j\}\),with a learned coefficientλ∈\(0,1\)\\lambda\\in\(0,1\)balancing relevance against redundancy with stored records; the novelty term is dropped when memory is empty, so the first record is admitted on relevance alone\. Rather than greedily keeping the top\-scoring record as in deterministic MMR, we sample the write decisionwt∼Bernoulli⁡\(σ⁡\(ωt\)\)w\_\{t\}\\sim\\mathrm\{Bernoulli\}\(\\sigma\(\\omega\_\{t\}\)\), keeping memory construction trainable under the same policy objective as the routing decisions\. The novelty criterion controls cross\-record redundancy, allowing the retrieval gate \(Section[3\.5](https://arxiv.org/html/2609.00237#S3.SS5)\) to score records independently\. As a result, memory remains a compact representation of execution progress rather than accumulating into a transcript\.

### 3\.7Adaptive Halting Controller

Reasoning depth is a major driver of cost, yet a depth fixed from the query commits this budget before the system observes how execution unfolds\. We therefore make depth a consequence of execution: after each step the*Adaptive Halting Controller*decides whether the memory holds sufficient evidence for aggregation or another agent should act\.

Whether recent steps still add evidence is a trend no single\-step snapshot captures, so the controller carries a recurrent state𝐡thalt=GRU⁡\(𝐡t−1halt,enc⁡\(St\)\)\\mathbf\{h\}^\{\\mathrm\{halt\}\}\_\{t\}=\\mathrm\{GRU\}\(\\mathbf\{h\}^\{\\mathrm\{halt\}\}\_\{t\-1\},\\mathrm\{enc\}\(S\_\{t\}\)\)that integrates a pooled summary of the post\-write memory across steps, kept separate from the query\-attended routing context𝐜t\\mathbf\{c\}\_\{t\}\. It then samples a halt action

ht∼Bernoulli⁡\(σ⁡\(MLP⁡\(𝐡thalt\)\)\)\.h\_\{t\}\\sim\\mathrm\{Bernoulli\}\\\!\\left\(\\sigma\\\!\\left\(\\mathrm\{MLP\}\(\\mathbf\{h\}^\{\\mathrm\{halt\}\}\_\{t\}\)\\right\)\\right\)\.Because the decision is evaluated only after the step’s agent has executed, at least one agent always runs\. Ifht=1h\_\{t\}=1the stored records pass to the Aggregator LLM; otherwise execution continues to the maximum depthϕ\\phi\. The halt action is optimized under the same trajectory\-level reward as the other decisions \(Section[3\.8](https://arxiv.org/html/2609.00237#S3.SS8)\), so reasoning depth co\-adapts with routing\.

### 3\.8Optimization

The routing stack couples several discrete stochastic decisions \(role, backbone, retrieval, write, and halt\), and their number varies with the halt\-determined depth, so we optimize the trajectory with policy gradients rather than backpropagating through sampled actions\. Because queries differ widely in difficulty, a single trajectory baseline is noisy; we instead train all policies jointly with a group\-relative advantage in the spirit of GRPO[Shao et al\. \(2024\)](https://arxiv.org/html/2609.00237#bib.bib31), usingGGrollouts of each query as a per\-query baseline without a learned critic\.

For each query we sampleGGtrajectories that differ only in their sampled decisions\. Trajectoryτi\\tau\_\{i\}has utilityui=Ri−λc​Cost​\(τi\)u\_\{i\}=R\_\{i\}\-\\lambda\_\{c\}\\,\\mathrm\{Cost\}\(\\tau\_\{i\}\), whereRi∈\{0,1\}R\_\{i\}\\in\\\{0,1\\\}is task success,Cost⁡\(τi\)\\mathrm\{Cost\}\(\\tau\_\{i\}\)sums the per\-step backbone costs, andλc\\lambda\_\{c\}sets the accuracy–cost trade\-off \(Appendix[9](https://arxiv.org/html/2609.00237#S9)\)\. Centering each utility within its group yields the advantage

Ai=ui−1G​∑i′∈𝒢⁡\(i\)ui′,A\_\{i\}=u\_\{i\}\-\\frac\{1\}\{G\}\\sum\_\{i^\{\\prime\}\\in\\mathcal\{G\}\(i\)\}u\_\{i^\{\\prime\}\},a per\-query baseline that cancels difficulty without a critic\. We deliberately drop the usual within\-group standard\-deviation normalization: when rollouts for a query share near\-identical rewards, the standard deviation approaches zero and inflates the advantage from negligible cost differences, whereas the centered advantage remains well\-scaled\. The same advantage weights every sampled action\. Withlog⁡πi\\log\\pi\_\{i\}the summed log\-probability of all role, backbone, retrieval, write, and halt actions inτi\\tau\_\{i\}, the objective is

ℒ=−meani​\[log⁡πi​sg​\(Ai\)\]\+α​ℒVAE−cH​H¯​\(π\),\\mathcal\{L\}=\-\\mathrm\{mean\}\_\{i\}\\\!\\left\[\\log\\pi\_\{i\}\\,\\mathrm\{sg\}\(A\_\{i\}\)\\right\]\+\\alpha\\,\\mathcal\{L\}\_\{\\mathrm\{VAE\}\}\-c\_\{H\}\\,\\bar\{H\}\(\\pi\),wheresg\\mathrm\{sg\}is stop\-gradient,ℒVAE\\mathcal\{L\}\_\{\\mathrm\{VAE\}\}regularizes the role and backbone encoders \(Appendix[8](https://arxiv.org/html/2609.00237#S8)\), andH¯​\(π\)\\bar\{H\}\(\\pi\)is the mean per\-step entropy of the routing and halt policies\. The entropy term discourages premature deterministic behavior; Appendix[12](https://arxiv.org/html/2609.00237#S12)verifies empirically that the trained gates do not collapse to trivial always\-write, never\-write, retrieve\-all, or retrieve\-none solutions\. The LLM backbones and sentence encoder are frozen, so the router is the only trained component, updated jointly by a single optimizer\.

## 4Experimental Setup

#### Datasets and Benchmarks\.

We evaluate on five benchmarks spanning mathematical reasoning, program synthesis, and knowledge\-intensive question answering: GSM\-Hard[Gao et al\. \(2023\)](https://arxiv.org/html/2609.00237#bib.bib32), MATH[Hendrycks et al\. \(2021\)](https://arxiv.org/html/2609.00237#bib.bib1), HumanEval[Chen et al\. \(2021\)](https://arxiv.org/html/2609.00237#bib.bib2), MBPP[Austin et al\. \(2021\)](https://arxiv.org/html/2609.00237#bib.bib3), and MMLU\-Pro[Wang et al\. \(2024\)](https://arxiv.org/html/2609.00237#bib.bib33)\.

#### Baselines\.

Our baselines span single\-model inference; single\-agent reasoning \(CoT[Wei et al\. \(2022\)](https://arxiv.org/html/2609.00237#bib.bib16), Complex\-CoT[Fu et al\. \(2023\)](https://arxiv.org/html/2609.00237#bib.bib17)\); fixed\-topology multi\-agent collaboration \(MacNet\-Chain, \-Tree, and \-Complete[Qian et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib12)\); adaptive workflow optimization \(AFlow[Zhang et al\. \(2025b\)](https://arxiv.org/html/2609.00237#bib.bib18)\); evolving orchestration \(Puppeteer[Dang et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib30)\); and query\-only routing \(MASRouter[Yue et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib10)\)\.

#### LLM Backbones\.

We route over five open\-weight LLMs: llama\-3\.2\-3B, llama\-3\.1\-8B[Grattafiori et al\. \(2024\)](https://arxiv.org/html/2609.00237#bib.bib34), mistral\-nemo\-12B[Mistral AI Team \(2024\)](https://arxiv.org/html/2609.00237#bib.bib43), qwen\-2\.5\-14B, and qwen\-2\.5\-32B[Yang et al\. \(2024\)](https://arxiv.org/html/2609.00237#bib.bib35)\. This 3B–32B heterogeneous pool allows the router to trade off capability and inference cost\. To quantify the accuracy–cost trade\-off, we assign each backbone a price proportional to its parameter count, a hardware\-independent proxy for per\-token inference compute \(Appendix[9](https://arxiv.org/html/2609.00237#S9)\)\. Backbone capability descriptions are listed in Appendix[16](https://arxiv.org/html/2609.00237#S16)\.

### 4\.1Implementation Details

We train with Adam optimizer using learning rate0\.010\.01, GRPO group sizeG=6G=6, and1616queries per batch\. We set maximum depthϕ=6\\phi=6, cost penaltyλc∈\{10,20,50\}\\lambda\_\{c\}\\in\\\{10,20,50\\\}, entropy coefficientcH=0\.01c\_\{H\}=0\.01, and VAE regularizer weightα=0\.001\\alpha=0\.001\. For the fixed\-topology multi\-agent baselines, we set the number of agents equal to our maximum depthϕ=6\\phi=6, so every multi\-agent method operates under the same agent budget\. We use the same2626heterogeneous role profiles as MASRouter[Yue et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib10), spanning programming agents with compiler access to research\-oriented agents with external knowledge sources; full descriptions are in Appendix[14](https://arxiv.org/html/2609.00237#S14)\. For final aggregation, we use the LLM backbone selected most frequently across the query’s reasoning steps\. The aggregation prompt is provided in Appendix[15](https://arxiv.org/html/2609.00237#S15)\.

MethodLLMMul\.Rout\.MATHGSM\-HardMBPPHumanEvalMMLU\-ProAvg\.Single LLMllama\-3\.2\-3B✗✗43\.2725\.8552\.2062\.7934\.0043\.62llama\-3\.1\-8B✗✗43\.7539\.8757\.6069\.7848\.2551\.85mistral\-nemo\-12B✗✗42\.0730\.1156\.0068\.2227\.7544\.83qwen\-2\.5\-14B✗✗75\.7264\.5871\.8082\.9565\.2572\.06qwen\-2\.5\-32B✗✗78\.3761\.5279\.0084\.3766\.0273\.86CoT[Wei et al\. \(2022\)](https://arxiv.org/html/2609.00237#bib.bib16)qwen\-2\.5\-14B✗✗77\.8867\.5270\.8082\.1762\.1672\.11qwen\-2\.5\-32B✗✗78\.8564\.5677\.2085\.1568\.4174\.83Complex\-CoT[Fu et al\. \(2023\)](https://arxiv.org/html/2609.00237#bib.bib17)qwen\-2\.5\-14B✗✗75\.9666\.7670\.4080\.6263\.0771\.36qwen\-2\.5\-32B✗✗77\.1665\.3278\.4085\.4768\.4174\.95MacNet\-Chain[Qian et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib12)qwen\-2\.5\-14B✓✗73\.8059\.7582\.6686\.8261\.4872\.90qwen\-2\.5\-32B✓✗77\.6461\.9380\.7383\.8067\.9574\.41MacNet\-Tree[Qian et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib12)qwen\-2\.5\-14B✓✗76\.6860\.3279\.8482\.9560\.6872\.09qwen\-2\.5\-32B✓✗77\.8862\.3180\.7186\.0267\.7374\.93MacNet\-Complete[Qian et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib12)qwen\-2\.5\-14B✓✗77\.1659\.0080\.6584\.5061\.4872\.56qwen\-2\.5\-32B✓✗79\.0962\.2281\.2086\.0567\.1675\.14AFlow[Zhang et al\. \(2025b\)](https://arxiv.org/html/2609.00237#bib.bib18)qwen\-2\.5\-14B✓✗63\.4658\.2470\.0085\.2764\.6668\.33qwen\-2\.5\-32B✓✗77\.6467\.0576\.4084\.5068\.4174\.80Puppeteer[Dang et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib30)qwen\-2\.5\-14B✓✗75\.0065\.9172\.5883\.5965\.2372\.46qwen\-2\.5\-32B✓✗79\.0968\.7574\.8085\.1668\.6475\.29MASRouter[Yue et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib10)LLM Pool✓✓74\.3166\.0079\.2085\.1666\.7074\.27OursLLM Pool✓✓79\.3370\.5579\.6089\.8469\.3277\.73Table 1:Test accuracy \(%\) across five benchmarks; MBPP and HumanEval report pass@1\. Mul\. indicates multi\-agent execution, and Rout\. indicates routing over the LLM backbone pool\. Best results are inbold, and second\-best results areunderlined\. “LLM Pool” denotes the full five\-model pool\.

## 5Results

### 5\.1Performance Analysis

Table[1](https://arxiv.org/html/2609.00237#S4.T1)shows that Gated\-Memory Routing achieves the strongest overall performance: it attains the best average accuracy and the top result on four of the five benchmarks, improving on the strongest baseline by2\.442\.44points on average\. The gains are most pronounced on GSM\-Hard and HumanEval, where intermediate reasoning quality and selective context matter most; the one exception is MBPP, where a fixed multi\-agent chain topology powered by qwen\-2\.5\-14B edges ahead at pass@1 but trails on every other benchmark and on average\. On MBPP our method is comparable to MASRouter \(79\.6079\.60vs\.79\.2079\.20\) and ahead of Puppeteer\-32B \(74\.8074\.80\), while several fixed MacNet configurations score highest\. MBPP’s short, relatively uniform problems appear to benefit from a fixed pipeline built on a strong code model, leaving less room for adaptive orchestration to help; the benefit of gated memory is most visible where reasoning is longer and more heterogeneous\.

The comparison is sharpest against the two paradigms our method is built to improve on\. MASRouter routes from the query alone, committing every decision before any reasoning exists, and trails by3\.463\.46points\. Puppeteer orchestrates agents over the evolving execution state but does not route over the pool, running every step on the largest backbone in the pool \(qwen\-2\.5\-32B\), and still trails by2\.442\.44points\. Gated\-memory routing conditions each decision on a filtered state instead, and is the most accurate of the three on every benchmark, even though it routes over a heterogeneous pool rather than relying on the largest model alone\. The gain is therefore not a matter of raw scale but of how well roles, backbones, and memory are coordinated as the trajectory unfolds\. Appendix[11](https://arxiv.org/html/2609.00237#S11)reports the mean and standard deviation over three training seeds for our method and both routing baselines; the ranking of the three methods is identical under every seed\.

Figure 3:Accuracy versus inference token cost on HumanEval, with cost on a log scale\. Our method lies on the high\-accuracy end of the Pareto frontier, achieving the highest accuracy among compared methods while using substantially less inference cost than the strongest multi\-agent and routing baselines\.
### 5\.2Cost Analysis

Figure[3](https://arxiv.org/html/2609.00237#S5.F3)places our method on the Pareto frontier of accuracy against inference cost on HumanEval, where it costs43\.86%43\.86\\%less than MASRouter and31\.9%31\.9\\%less than Puppeteer \(qwen\-2\.5\-32B\) while remaining more accurate than both, gaining on cost and accuracy at once rather than trading one for the other\. The GSM\-Hard frontier in Appendix[10](https://arxiv.org/html/2609.00237#S10)shows the same pattern\. Appendix[9](https://arxiv.org/html/2609.00237#S9)additionally reports approximate FLOPs and batch\-amortized wall\-clock time per query on HumanEval and MBPP, which give the same ordering\.

These savings come from where computation is spent\. Because each decision conditions on a compact, gated memory rather than the full execution history, the halting controller can stop once that memory carries enough evidence, so easy queries terminate early and only the hard ones run deep\. The gates contribute indirectly: a clean memory makes halting trustworthy and spares downstream agents the redundant context that inflates prompts under full\-history routing\. Section[5\.5](https://arxiv.org/html/2609.00237#S5.SS5)compares the three paradigms directly: gated\-memory routing is at least as accurate as full\-history routing while costing appreciably less, and is both more accurate and cheaper than query\-only routing\.

### 5\.3Ablation Study

ConfigurationGSM\-HardHumanEvalAcc\.CostAcc\.CostFull system70\.550\.58789\.840\.032w/o role allocator68\.370\.40085\.940\.043w/o LLM router56\.530\.55571\.880\.026w/o retrieval gate69\.220\.48885\.940\.034w/o write gate66\.670\.62184\.380\.037w/o halting70\.080\.84086\.720\.086w/o both memory gates67\.990\.44783\.590\.033Table 2:Leave\-one\-out ablation\. Each row disables one routing module while keeping the others active\. We report accuracy \(%\) and total test\-set cost on GSM\-Hard and HumanEval\.Table[2](https://arxiv.org/html/2609.00237#S5.T2)reports a leave\-one\-out ablation, each variant replacing one module with a default:*w/o role allocator*uses random role selection,*w/o LLM router*random backbone selection,*w/o retrieval gate*exposes the full memory to each agent,*w/o write gate*writes every response to memory, and*w/o halting*runs every query to the maximum depthϕ\\phi\. The effects split cleanly along the division of labor the method is built around\. The two routing decisions carry accuracy: dropping the LLM router is by far the most damaging change \(a14\.014\.0\-point drop on GSM\-Hard\), and dropping the role allocator costs a smaller but consistent amount, so matching model capacity and specialization to the current state is what drives answer quality\.

The memory gates also serve accuracy through the quality of the memory state: removing either lowers accuracy, because unfiltered context dilutes what each agent and the router condition on\. Disabling both gates at once \(*w/o both memory gates*\) lowers accuracy by2\.562\.56points on GSM\-Hard and6\.256\.25on HumanEval, so selective memory matters even with the router and role allocat Halting is the lever that controls cost: without it, inference cost rises by over40%40\\%on GSM\-Hard and more than doubles on HumanEval, while accuracy also slips, since most queries are solved before the depth limit and the extra steps add computation without improving answers\. The*w/o halting*variant also provides an equal\-depth comparison with the fixed\-topology baselines: executing all six steps, it reaches70\.0870\.08on GSM\-Hard, above every MacNet\-32B variant \(61\.9361\.93–62\.3162\.31, Table[1](https://arxiv.org/html/2609.00237#S4.T1)\) at lower cost \(Appendix[10](https://arxiv.org/html/2609.00237#S10)\), because each step is still routed to an appropriate backbone rather than the largest one\. Halting also reduces training\-time computation: disabling it raises the number of training rollout steps by about1\.8×1\.8\\timeson both benchmarks \(Appendix[13](https://arxiv.org/html/2609.00237#S13)\)\.

Figure 4:Effect of the maximum depthϕ\\phion MATH: test accuracy \(left axis\) and per\-query inference token cost \(right axis\)\.ϕ=6\\phi=6is the default used elsewhere in the paper\.
### 5\.4Effect of the Depth Budget

Figure[4](https://arxiv.org/html/2609.00237#S5.F4)varies the maximum depthϕ\\phion MATH with adaptive halting left in place\. Accuracy increases with the budget but saturates: most of the gain is realized byϕ=6\\phi=6, with under a point more fromϕ=6\\phi=6toϕ=10\\phi=10as cost keeps climbing\. Small budgets truncate useful reasoning on the harder problems, while larger ones mostly add cost, since halting already stops easy queries early regardless of the ceiling\. The defaultϕ=6\\phi=6thus sits near the knee of the accuracy–cost curve, capturing nearly all the attainable accuracy before the budget begins buying cost more than accuracy\.

### 5\.5Routing\-Paradigm Comparison

We contrast Gated\-Memory Routing with the two routing paradigms it is built to improve on\. Query\-only routing fixes every decision from the input query before execution begins; we instantiate it with MASRouter, the strongest query\-only router among our baselines\. Under full\-history routing, the per\-step role and backbone allocation is conditioned on the accumulated history, and all intermediate reasoning is forwarded as context to the next agent\. Gated\-memory routing is our full system\. Table[3](https://arxiv.org/html/2609.00237#S5.T3)compares the three\.

GSM\-HardHumanEvalRouting paradigmAcc\. \(%\)CostAcc\. \(%\)CostQuery\-only \(MASRouter\)66\.001\.01285\.160\.057Full\-history70\.270\.98589\.060\.068Gated\-memory \(ours\)70\.550\.58789\.840\.032Table 3:Comparison of the three routing paradigms on GSM\-Hard and HumanEval\.Full\-history routing conditions each decision on the entire accumulated trajectory, increasing context length and reasoning overhead\. The additional context does not appear to improve accuracy here: full\-history routing is roughly on par with the curated memory \(70\.2770\.27vs70\.5570\.55on GSM\-Hard and89\.0689\.06vs89\.8489\.84on HumanEval\), while the gated memory attains comparable accuracy at appreciably lower cost \(about40%40\\%on GSM\-Hard and over50%50\\%on HumanEval\)\. Query\-only routing tends to be the least accurate of the three\. These results suggest that, in our setting, curating the memory rather than forwarding the full history largely preserves accuracy while reducing cost\.

## 6Conclusion

We presentedGated\-Memory Routing, a framework for dynamic multi\-agent coordination that conditions routing decisions on a learned, gated execution memory rather than on the query alone or the full raw history\. A Memory Write Gate and a Retrieval Gate keep this memory compact and task\-relevant, enabling role and backbone routing to operate over a filtered execution state\. An Adaptive Halting Controller uses the same memory to decide when to stop, adapting reasoning depth and cost to each query\. Across five benchmarks, our method attains the best average accuracy, exceeding the strongest baseline by2\.442\.44points, while reducing HumanEval inference cost by31\.9%31\.9\\%relative to that baseline\. Future work will extend gated\-memory routing to open\-ended generation, larger\-scale agent ecosystems, and pools of more recent backbone models, and explore richer reward signals beyond binary task success\.

## Limitations

Our evaluation uses closed\-domain tasks with verifiable answers, enabling automatic reward computation; open\-ended generation would require reward models, human evaluation, or other supervision\. Our efficiency analysis: parameter count provides a hardware\-independent compute proxy but does not capture latency, memory pressure, or batching, while FLOPs and wall\-clock measurements in Appendix[9](https://arxiv.org/html/2609.00237#S9)cover only two benchmarks\. Finally, we evaluate open\-weight Qwen2\.5, Llama\-3, and Mistral models; backbones can be substituted through their capability profiles and per\-token costs without changing the framework\.

## References

- Austinet al\.\(2021\)J\. Austin, A\. Odena, M\. Nye, M\. Bosma, H\. Michalewski, D\. Dohan, E\. Jiang, C\. Cai, M\. Terry, Q\. Le,et al\.Program synthesis with large language models\.arXiv preprint arXiv:2108\.07732\.External Links:[Link](https://arxiv.org/abs/2108.07732)Cited by:[§4](https://arxiv.org/html/2609.00237#S4.SS0.SSS0.Px1.p1.1)\.
- Carbonell and Goldstein \(1998\)J\. Carbonell and J\. GoldsteinThe use of mmr, diversity\-based reranking for reordering documents and producing summaries\.InProceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval,SIGIR ’98,New York, NY, USA,pp\. 335–336\.External Links:ISBN 1581130155,[Link](https://doi.org/10.1145/290941.291025),[Document](https://dx.doi.org/10.1145/290941.291025)Cited by:[§3\.6](https://arxiv.org/html/2609.00237#S3.SS6.p2.1)\.
- Chenet al\.\(2024a\)L\. Chen, M\. Zaharia, and J\. ZouFrugalGPT: how to use large language models while reducing cost and improving performance\.Transactions on Machine Learning Research\.Note:Featured CertificationExternal Links:ISSN 2835\-8856,[Link](https://openreview.net/forum?id=cSimKw5p6R)Cited by:[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px3.p1.1)\.
- Chenet al\.\(2021\)M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. de Oliveira Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman, A\. Ray, R\. Puri, G\. Krueger, M\. Petrov, H\. Khlaaf, G\. Sastry, P\. Mishkin, B\. Chan, S\. Gray, N\. Ryder, M\. Pavlov, A\. Power, L\. Kaiser, M\. Bavarian, C\. Winter, P\. Tillet, F\. P\. Such, D\. Cummings, M\. Plappert, F\. Chantzis, E\. Barnes, A\. Herbert\-Voss, W\. H\. Guss, A\. Nichol, A\. Paino, N\. Tezak, J\. Tang, I\. Babuschkin, S\. Balaji, S\. Jain, W\. Saunders, C\. Hesse, A\. N\. Carr, J\. Leike, J\. Achiam, V\. Misra, E\. Morikawa, A\. Radford, M\. Knight, M\. Brundage, M\. Murati, K\. Mayer, P\. Welinder, B\. McGrew, D\. Amodei, S\. McCandlish, I\. Sutskever, and W\. ZarembaEvaluating large language models trained on code\.CoRRabs/2107\.03374\.External Links:[Link](https://arxiv.org/abs/2107.03374),2107\.03374Cited by:[§4](https://arxiv.org/html/2609.00237#S4.SS0.SSS0.Px1.p1.1)\.
- Chenet al\.\(2024b\)S\. Chen, W\. Jiang, B\. Lin, J\. Kwok, and Y\. ZhangRouterDC: query\-based router by dual contrastive learning for assembling large language models\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=7RQvjayHrM)Cited by:[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px3.p1.1)\.
- Chenet al\.\(2024c\)W\. Chen, Y\. Su, J\. Zuo, C\. Yang, C\. Yuan, C\. Chan, H\. Yu, Y\. Lu, Y\. Hung, C\. Qian, Y\. Qin, X\. Cong, R\. Xie, Z\. Liu, M\. Sun, and J\. ZhouAgentVerse: facilitating multi\-agent collaboration and exploring emergent behaviors\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024,External Links:[Link](https://openreview.net/forum?id=EHg5GDnyq1)Cited by:[§1](https://arxiv.org/html/2609.00237#S1.p1.1)\.
- Danget al\.\(2025\)Y\. Dang, C\. Qian, X\. Luo, J\. Fan, Z\. Xie, R\. Shi, W\. Chen, C\. Yang, X\. Che, Y\. Tian, X\. Xiong, L\. Han, Z\. Liu, and M\. SunMulti\-agent collaboration via evolving orchestration\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=L0xZPXT3le)Cited by:[§1](https://arxiv.org/html/2609.00237#S1.p2.1),[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px4.p1.1),[§4](https://arxiv.org/html/2609.00237#S4.SS0.SSS0.Px2.p1.1),[Table 1](https://arxiv.org/html/2609.00237#S4.T1.2.19.1.1)\.
- Duet al\.\(2024\)Y\. Du, S\. Li, A\. Torralba, J\. B\. Tenenbaum, and I\. MordatchImproving factuality and reasoning in language models through multiagent debate\.InForty\-first International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=zj7YuTE4t8)Cited by:[§1](https://arxiv.org/html/2609.00237#S1.p2.1)\.
- Fuet al\.\(2023\)Y\. Fu, H\. Peng, A\. Sabharwal, P\. Clark, and T\. KhotComplexity\-based prompting for multi\-step reasoning\.InThe Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1\-5, 2023,External Links:[Link](https://openreview.net/forum?id=yf1icZHC-l9)Cited by:[§4](https://arxiv.org/html/2609.00237#S4.SS0.SSS0.Px2.p1.1),[Table 1](https://arxiv.org/html/2609.00237#S4.T1.2.9.1.1)\.
- Gaoet al\.\(2023\)L\. Gao, A\. Madaan, S\. Zhou, U\. Alon, P\. Liu, Y\. Yang, J\. Callan, and G\. NeubigPAL: program\-aided language models\.InProceedings of the 40th International Conference on Machine Learning,ICML’23\.External Links:[Link](https://openreview.net/forum?id=M1fd9Z00sj)Cited by:[§4](https://arxiv.org/html/2609.00237#S4.SS0.SSS0.Px1.p1.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan, A\. Yang, A\. Fan, A\. Goyal, A\. Hartshorn, A\. Yang, A\. Mitra, A\. Sravankumar, A\. Korenev, A\. Hinsvark, A\. Rao, A\. Zhang, A\. Rodriguez, A\. Gregerson, A\. Spataru, B\. Roziere, B\. Biron, B\. Tang, B\. Chern, C\. Caucheteux, C\. Nayak, C\. Bi, C\. Marra, C\. McConnell, C\. Keller, C\. Touret, C\. Wu, C\. Wong, C\. C\. Ferrer, C\. Nikolaidis, D\. Allonsius, D\. Song, D\. Pintz, D\. Livshits, D\. Wyatt, D\. Esiobu, D\. Choudhary, D\. Mahajan, D\. Garcia\-Olano, D\. Perino, D\. Hupkes, E\. Lakomkin, E\. AlBadawy, E\. Lobanova, E\. Dinan, E\. M\. Smith, F\. Radenovic, F\. Guzmán, F\. Zhang, G\. Synnaeve, G\. Lee, G\. L\. Anderson, G\. Thattai, G\. Nail, G\. Mialon, G\. Pang, G\. Cucurell, H\. Nguyen, H\. Korevaar, H\. Xu, H\. Touvron, I\. Zarov, I\. A\. Ibarra, I\. Kloumann, I\. Misra, I\. Evtimov, J\. Zhang, J\. Copet, J\. Lee, J\. Geffert, J\. Vranes, J\. Park, J\. Mahadeokar, J\. Shah, J\. van der Linde, J\. Billock, J\. Hong, J\. Lee, J\. Fu, J\. Chi, J\. Huang, J\. Liu, J\. Wang, J\. Yu, J\. Bitton, J\. Spisak, J\. Park, J\. Rocca, J\. Johnstun, J\. Saxe, J\. Jia, K\. V\. Alwala, K\. Prasad, K\. Upasani, K\. Plawiak, K\. Li, K\. Heafield, K\. Stone, K\. El\-Arini, K\. Iyer, K\. Malik, K\. Chiu, K\. Bhalla, K\. Lakhotia, L\. Rantala\-Yeary, L\. van der Maaten, L\. Chen, L\. Tan, L\. Jenkins, L\. Martin, L\. Madaan, L\. Malo, L\. Blecher, L\. Landzaat, L\. de Oliveira, M\. Muzzi, M\. Pasupuleti, M\. Singh, M\. Paluri, M\. Kardas, M\. Tsimpoukelli, M\. Oldham, M\. Rita, M\. Pavlova, M\. Kambadur, M\. Lewis, M\. Si, M\. K\. Singh, M\. Hassan, N\. Goyal, N\. Torabi, N\. Bashlykov, N\. Bogoychev, N\. Chatterji, N\. Zhang, O\. Duchenne, O\. Çelebi, P\. Alrassy, P\. Zhang, P\. Li, P\. Vasic, P\. Weng, P\. Bhargava, P\. Dubal, P\. Krishnan, P\. S\. Koura, P\. Xu, Q\. He, Q\. Dong, R\. Srinivasan, R\. Ganapathy, R\. Calderer, R\. S\. Cabral, R\. Stojnic, R\. Raileanu, R\. Maheswari, R\. Girdhar, R\. Patel, R\. Sauvestre, R\. Polidoro, R\. Sumbaly, R\. Taylor, R\. Silva, R\. Hou, R\. Wang, S\. Hosseini, S\. Chennabasappa, S\. Singh, S\. Bell, S\. S\. Kim, S\. Edunov, S\. Nie, S\. Narang, S\. Raparthy, S\. Shen, S\. Wan, S\. Bhosale, S\. Zhang, S\. Vandenhende, S\. Batra, S\. Whitman, S\. Sootla, S\. Collot, S\. Gururangan, S\. Borodinsky, T\. Herman, T\. Fowler, T\. Sheasha, T\. Georgiou, T\. Scialom, T\. Speckbacher, T\. Mihaylov, T\. Xiao, U\. Karn, V\. Goswami, V\. Gupta, V\. Ramanathan, V\. Kerkez, V\. Gonguet, V\. Do, V\. Vogeti, V\. Albiero, V\. Petrovic, W\. Chu, W\. Xiong, W\. Fu, W\. Meers, X\. Martinet, X\. Wang, X\. Wang, X\. E\. Tan, X\. Xia, X\. Xie, X\. Jia, X\. Wang, Y\. Goldschlag, Y\. Gaur, Y\. Babaei, Y\. Wen, Y\. Song, Y\. Zhang, Y\. Li, Y\. Mao, Z\. D\. Coudert, Z\. Yan, Z\. Chen, Z\. Papakipos, A\. Singh, A\. Srivastava, A\. Jain, A\. Kelsey, A\. Shajnfeld, A\. Gangidi, A\. Victoria, A\. Goldstand, A\. Menon, A\. Sharma, A\. Boesenberg, A\. Baevski, A\. Feinstein, A\. Kallet, A\. Sangani, A\. Teo, A\. Yunus, A\. Lupu, A\. Alvarado, A\. Caples, A\. Gu, A\. Ho, A\. Poulton, A\. Ryan, A\. Ramchandani, A\. Dong, A\. Franco, A\. Goyal, A\. Saraf, A\. Chowdhury, A\. Gabriel, A\. Bharambe, A\. Eisenman, A\. Yazdan, B\. James, B\. Maurer, B\. Leonhardi, B\. Huang, B\. Loyd, B\. D\. Paola, B\. Paranjape, B\. Liu, B\. Wu, B\. Ni, B\. Hancock, B\. Wasti, B\. Spence, B\. Stojkovic, B\. Gamido, B\. Montalvo, C\. Parker, C\. Burton, C\. Mejia, C\. Liu, C\. Wang, C\. Kim, C\. Zhou, C\. Hu, C\. Chu, C\. Cai, C\. Tindal, C\. Feichtenhofer, C\. Gao, D\. Civin, D\. Beaty, D\. Kreymer, D\. Li, D\. Adkins, D\. Xu, D\. Testuggine, D\. David, D\. Parikh, D\. Liskovich, D\. Foss, D\. Wang, D\. Le, D\. Holland, E\. Dowling, E\. Jamil, E\. Montgomery, E\. Presani, E\. Hahn, E\. Wood, E\. Le, E\. Brinkman, E\. Arcaute, E\. Dunbar, E\. Smothers, F\. Sun, F\. Kreuk, F\. Tian, F\. Kokkinos, F\. Ozgenel, F\. Caggioni, F\. Kanayet, F\. Seide, G\. M\. Florez, G\. Schwarz, G\. Badeer, G\. Swee, G\. Halpern, G\. Herman, G\. Sizov, Guangyi, Zhang, G\. Lakshminarayanan, H\. Inan, H\. Shojanazeri, H\. Zou, H\. Wang, H\. Zha, H\. Habeeb, H\. Rudolph, H\. Suk, H\. Aspegren, H\. Goldman, H\. Zhan, I\. Damlaj, I\. Molybog, I\. Tufanov, I\. Leontiadis, I\. Veliche, I\. Gat, J\. Weissman, J\. Geboski, J\. Kohli, J\. Lam, J\. Asher, J\. Gaya, J\. Marcus, J\. Tang, J\. Chan, J\. Zhen, J\. Reizenstein, J\. Teboul, J\. Zhong, J\. Jin, J\. Yang, J\. Cummings, J\. Carvill, J\. Shepard, J\. McPhie, J\. Torres, J\. Ginsburg, J\. Wang, K\. Wu, K\. H\. U, K\. Saxena, K\. Khandelwal, K\. Zand, K\. Matosich, K\. Veeraraghavan, K\. Michelena, K\. Li, K\. Jagadeesh, K\. Huang, K\. Chawla, K\. Huang, L\. Chen, L\. Garg, L\. A, L\. Silva, L\. Bell, L\. Zhang, L\. Guo, L\. Yu, L\. Moshkovich, L\. Wehrstedt, M\. Khabsa, M\. Avalani, M\. Bhatt, M\. Mankus, M\. Hasson, M\. Lennie, M\. Reso, M\. Groshev, M\. Naumov, M\. Lathi, M\. Keneally, M\. Liu, M\. L\. Seltzer, M\. Valko, M\. Restrepo, M\. Patel, M\. Vyatskov, M\. Samvelyan, M\. Clark, M\. Macey, M\. Wang, M\. J\. Hermoso, M\. Metanat, M\. Rastegari, M\. Bansal, N\. Santhanam, N\. Parks, N\. White, N\. Bawa, N\. Singhal, N\. Egebo, N\. Usunier, N\. Mehta, N\. P\. Laptev, N\. Dong, N\. Cheng, O\. Chernoguz, O\. Hart, O\. Salpekar, O\. Kalinli, P\. Kent, P\. Parekh, P\. Saab, P\. Balaji, P\. Rittner, P\. Bontrager, P\. Roux, P\. Dollar, P\. Zvyagina, P\. Ratanchandani, P\. Yuvraj, Q\. Liang, R\. Alao, R\. Rodriguez, R\. Ayub, R\. Murthy, R\. Nayani, R\. Mitra, R\. Parthasarathy, R\. Li, R\. Hogan, R\. Battey, R\. Wang, R\. Howes, R\. Rinott, S\. Mehta, S\. Siby, S\. J\. Bondu, S\. Datta, S\. Chugh, S\. Hunt, S\. Dhillon, S\. Sidorov, S\. Pan, S\. Mahajan, S\. Verma, S\. Yamamoto, S\. Ramaswamy, S\. Lindsay, S\. Lindsay, S\. Feng, S\. Lin, S\. C\. Zha, S\. Patil, S\. Shankar, S\. Zhang, S\. Zhang, S\. Wang, S\. Agarwal, S\. Sajuyigbe, S\. Chintala, S\. Max, S\. Chen, S\. Kehoe, S\. Satterfield, S\. Govindaprasad, S\. Gupta, S\. Deng, S\. Cho, S\. Virk, S\. Subramanian, S\. Choudhury, S\. Goldman, T\. Remez, T\. Glaser, T\. Best, T\. Koehler, T\. Robinson, T\. Li, T\. Zhang, T\. Matthews, T\. Chou, T\. Shaked, V\. Vontimitta, V\. Ajayi, V\. Montanez, V\. Mohan, V\. S\. Kumar, V\. Mangla, V\. Ionescu, V\. Poenaru, V\. T\. Mihailescu, V\. Ivanov, W\. Li, W\. Wang, W\. Jiang, W\. Bouaziz, W\. Constable, X\. Tang, X\. Wu, X\. Wang, X\. Wu, X\. Gao, Y\. Kleinman, Y\. Chen, Y\. Hu, Y\. Jia, Y\. Qi, Y\. Li, Y\. Zhang, Y\. Zhang, Y\. Adi, Y\. Nam, Yu, Wang, Y\. Zhao, Y\. Hao, Y\. Qian, Y\. Li, Y\. He, Z\. Rait, Z\. DeVito, Z\. Rosnbrick, Z\. Wen, Z\. Yang, Z\. Zhao, and Z\. MaThe llama 3 herd of models\.External Links:2407\.21783,[Link](https://arxiv.org/abs/2407.21783)Cited by:[§4](https://arxiv.org/html/2609.00237#S4.SS0.SSS0.Px3.p1.1)\.
- Graves \(2016\)A\. GravesAdaptive computation time for recurrent neural networks\.ArXivabs/1603\.08983\.External Links:[Link](https://api.semanticscholar.org/CorpusID:8224916)Cited by:[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px4.p1.1)\.
- Hataliset al\.\(2023\)K\. Hatalis, D\. Christou, J\. Myers, S\. J\. Jones, K\. Lambert, A\. Amos\-Binks, Z\. A\. Dannenhauer, and D\. DannenhauerMemory matters: the need to improve long\-term memory in llm\-agents\.InProceedings of the 2023 AAAI Fall Symposia, Arlington, Virginia, USA, October 25\-27, 2023,C\. W\. Geib and R\. P\. A\. Petrick \(Eds\.\),pp\. 277–280\.External Links:[Link](https://doi.org/10.1609/aaaiss.v2i1.27688),[Document](https://dx.doi.org/10.1609/AAAISS.V2I1.27688)Cited by:[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px5.p1.1)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. SteinhardtMeasuring mathematical problem solving with the MATH dataset\.InThirty\-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track \(Round 2\),External Links:[Link](https://openreview.net/forum?id=7Bywt2mQsCe)Cited by:[§4](https://arxiv.org/html/2609.00237#S4.SS0.SSS0.Px1.p1.1)\.
- Honget al\.\(2024\)S\. Hong, M\. Zhuge, J\. Chen, X\. Zheng, Y\. Cheng, J\. Wang, C\. Zhang, Z\. Wang, S\. K\. S\. Yau, Z\. Lin, L\. Zhou, C\. Ran, L\. Xiao, C\. Wu, and J\. SchmidhuberMetaGPT: meta programming for a multi\-agent collaborative framework\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=VtmBAGCN7o)Cited by:[§1](https://arxiv.org/html/2609.00237#S1.p2.1),[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px1.p1.1)\.
- Kabiret al\.\(2026\)A\. Kabir, J\. Xue, M\. Zheng, and Q\. LouHW\-Router: hardware\-aware routing for scalable multi\-LLM serving\.InProceedings of the 63rd Design Automation Conference \(DAC\),Cited by:[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px3.p1.1)\.
- Kaplanet al\.\(2020\)J\. Kaplan, S\. McCandlish, T\. Henighan, T\. B\. Brown, B\. Chess, R\. Child, S\. Gray, A\. Radford, J\. Wu, and D\. AmodeiScaling laws for neural language models\.CoRRabs/2001\.08361\.External Links:[Link](https://arxiv.org/abs/2001.08361),2001\.08361Cited by:[§9](https://arxiv.org/html/2609.00237#S9.p2.1)\.
- Kingma and Welling \(2014\)D\. P\. Kingma and M\. WellingAuto\-encoding variational bayes\.In2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14\-16, 2014, Conference Track Proceedings,Y\. Bengio and Y\. LeCun \(Eds\.\),External Links:[Link](http://arxiv.org/abs/1312.6114)Cited by:[§3\.3](https://arxiv.org/html/2609.00237#S3.SS3.p2.1)\.
- Liet al\.\(2023\)G\. Li, H\. A\. A\. K\. Hammoud, H\. Itani, D\. Khizbullin, and B\. GhanemCAMEL: communicative agents for ”mind” exploration of large language model society\.InThirty\-seventh Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=3IyL2XWDkG)Cited by:[§1](https://arxiv.org/html/2609.00237#S1.p2.1)\.
- Liet al\.\(2026\)S\. Li, H\. Hasson, and J\. GhoshOMAC: a holistic optimization framework for LLM\-based multi\-agent collaboration\.InForty\-third International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=6C4YQcq8YX)Cited by:[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px2.p1.1)\.
- Lianget al\.\(2024\)T\. Liang, Z\. He, W\. Jiao, X\. Wang, Y\. Wang, R\. Wang, Y\. Yang, S\. Shi, and Z\. TuEncouraging divergent thinking in large language models through multi\-agent debate\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 17889–17904\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.992/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.992)Cited by:[§1](https://arxiv.org/html/2609.00237#S1.p1.1)\.
- Liuet al\.\(2024\)Z\. Liu, Y\. Zhang, P\. Li, Y\. Liu, and D\. YangA dynamic LLM\-powered agent network for task\-oriented agent collaboration\.InFirst Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=XII0Wp1XA9)Cited by:[§1](https://arxiv.org/html/2609.00237#S1.p2.1),[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px2.p1.1)\.
- Meiet al\.\(2025\)K\. Mei, X\. Zhu, W\. Xu, M\. Jin, W\. Hua, Z\. Li, S\. Xu, R\. Ye, Y\. Ge, and Y\. ZhangAIOS: LLM agent operating system\.InSecond Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=L4HHkCDz2x)Cited by:[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px5.p1.1)\.
- Mistral AI Team \(2024\)Mistral AI TeamMistral nemo\.Note:[https://mistral\.ai/news/mistral\-nemo/](https://mistral.ai/news/mistral-nemo/)Accessed: 2026\-08\-28Cited by:[§4](https://arxiv.org/html/2609.00237#S4.SS0.SSS0.Px3.p1.1)\.
- Onget al\.\(2025\)I\. Ong, A\. Almahairi, V\. Wu, W\. Chiang, T\. Wu, J\. E\. Gonzalez, M\. W\. Kadous, and I\. StoicaRouteLLM: learning to route LLMs from preference data\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=8sSqNntaMr)Cited by:[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px3.p1.1)\.
- Packeret al\.\(2023\)C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. GonzalezMemGPT: towards llms as operating systems\.arXiv preprint arXiv:2310\.08560\.External Links:[Link](https://arxiv.org/abs/2310.08560)Cited by:[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px5.p1.1)\.
- Qianet al\.\(2024\)C\. Qian, W\. Liu, H\. Liu, N\. Chen, Y\. Dang, J\. Li, C\. Yang, W\. Chen, Y\. Su, X\. Cong, J\. Xu, D\. Li, Z\. Liu, and M\. SunChatDev: communicative agents for software development\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 15174–15186\.External Links:[Link](https://aclanthology.org/2024.acl-long.810/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.810)Cited by:[§1](https://arxiv.org/html/2609.00237#S1.p2.1),[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px1.p1.1)\.
- Qianet al\.\(2025\)C\. Qian, Z\. Xie, Y\. Wang, W\. Liu, K\. Zhu, H\. Xia, Y\. Dang, Z\. Du, W\. Chen, C\. Yang, Z\. Liu, and M\. SunScaling large language model\-based multi\-agent collaboration\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28, 2025,External Links:[Link](https://openreview.net/forum?id=K3n5jPkrU6)Cited by:[§1](https://arxiv.org/html/2609.00237#S1.p1.1),[§4](https://arxiv.org/html/2609.00237#S4.SS0.SSS0.Px2.p1.1),[Table 1](https://arxiv.org/html/2609.00237#S4.T1.2.11.1.1),[Table 1](https://arxiv.org/html/2609.00237#S4.T1.2.13.1.1),[Table 1](https://arxiv.org/html/2609.00237#S4.T1.2.15.1.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. GuoDeepSeekMath: pushing the limits of mathematical reasoning in open language models\.CoRRabs/2402\.03300\.External Links:[Link](https://doi.org/10.48550/arXiv.2402.03300),[Document](https://dx.doi.org/10.48550/ARXIV.2402.03300),2402\.03300Cited by:[§3\.8](https://arxiv.org/html/2609.00237#S3.SS8.p1.1)\.
- Wanget al\.\(2024\)Y\. Wang, X\. Ma, G\. Zhang, Y\. Ni, A\. Chandra, S\. Guo, W\. Ren, A\. Arulraj, X\. He, Z\. Jiang, T\. Li, M\. Ku, K\. Wang, A\. Zhuang, R\. Fan, X\. Yue, and W\. ChenMMLU\-pro: a more robust and challenging multi\-task language understanding benchmark\.InThe Thirty\-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track,External Links:[Link](https://openreview.net/forum?id=y10DM6R2r3)Cited by:[§4](https://arxiv.org/html/2609.00237#S4.SS0.SSS0.Px1.p1.1)\.
- Wanget al\.\(2025\)Z\. Wang, Y\. Wang, X\. Liu, L\. Ding, M\. Zhang, J\. Liu, and M\. ZhangAgentDropout: dynamic agent elimination for token\-efficient and high\-performance LLM\-based multi\-agent collaboration\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 24013–24035\.External Links:[Link](https://aclanthology.org/2025.acl-long.1170/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.1170),ISBN 979\-8\-89176\-251\-0Cited by:[§1](https://arxiv.org/html/2609.00237#S1.p2.1),[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px2.p1.1)\.
- Weiet al\.\(2022\)J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, brian ichter, F\. Xia, E\. H\. Chi, Q\. V\. Le, and D\. ZhouChain of thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems,A\. H\. Oh, A\. Agarwal, D\. Belgrave, and K\. Cho \(Eds\.\),External Links:[Link](https://openreview.net/forum?id=_VjQlMeSB_J)Cited by:[§4](https://arxiv.org/html/2609.00237#S4.SS0.SSS0.Px2.p1.1),[Table 1](https://arxiv.org/html/2609.00237#S4.T1.2.7.1.1)\.
- Wuet al\.\(2024\)Q\. Wu, G\. Bansal, J\. Zhang, Y\. Wu, B\. Li, E\. Zhu, L\. Jiang, X\. Zhang, S\. Zhang, J\. Liu, A\. H\. Awadallah, R\. W\. White, D\. Burger, and C\. WangAutoGen: enabling next\-gen LLM applications via multi\-agent conversations\.InFirst Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=BAakY1hNKS)Cited by:[§1](https://arxiv.org/html/2609.00237#S1.p2.1)\.
- Xuet al\.\(2025\)W\. Xu, Z\. Liang, K\. Mei, H\. Gao, J\. Tan, and Y\. ZhangA\-mem: agentic memory for LLM agents\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=FiM0M8gcct)Cited by:[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px5.p1.1)\.
- Xueet al\.\(2026\)J\. Xue, Q\. Lou, J\. Xing, and H\. HuangR2\-router: a new paradigm for LLM routing with reasoning\.InForty\-third International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=S3m1tSp8F4)Cited by:[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px3.p1.1)\.
- Yanet al\.\(2026\)S\. Yan, X\. Yang, Z\. Huang, E\. Nie, Z\. Ding, Z\. Li, X\. Ma, J\. Bi, K\. Kersting, J\. Z\. Pan, H\. Schuetze, V\. Tresp, and Y\. MaMemory\-r1: enhancing large language model agents to manage and utilize memories via reinforcement learning\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 12805–12825\.External Links:[Link](https://aclanthology.org/2026.acl-long.583/),[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.583),ISBN 979\-8\-89176\-390\-6Cited by:[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px5.p1.1)\.
- Yanget al\.\(2024\)A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. QiuQwen2\.5 technical report\.CoRRabs/2412\.15115\.External Links:[Link](https://doi.org/10.48550/arXiv.2412.15115),[Document](https://dx.doi.org/10.48550/ARXIV.2412.15115),2412\.15115Cited by:[§4](https://arxiv.org/html/2609.00237#S4.SS0.SSS0.Px3.p1.1)\.
- Yueet al\.\(2025\)Y\. Yue, G\. Zhang, B\. Liu, G\. Wan, K\. Wang, D\. Cheng, and Y\. QiMasRouter: learning to route LLMs for multi\-agent systems\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 15549–15572\.External Links:[Link](https://aclanthology.org/2025.acl-long.757/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.757),ISBN 979\-8\-89176\-251\-0Cited by:[§1](https://arxiv.org/html/2609.00237#S1.p2.1),[§14](https://arxiv.org/html/2609.00237#S14.p1.1),[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px3.p1.1),[§4](https://arxiv.org/html/2609.00237#S4.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2609.00237#S4.SS1.p1.1),[Table 1](https://arxiv.org/html/2609.00237#S4.T1.2.21.1)\.
- Zhanget al\.\(2025a\)G\. Zhang, Y\. Yue, Z\. Li, S\. Yun, G\. Wan, K\. Wang, D\. Cheng, J\. X\. Yu, and T\. ChenCut the crap: an economical communication pipeline for LLM\-based multi\-agent systems\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=LkzuPorQ5L)Cited by:[§1](https://arxiv.org/html/2609.00237#S1.p2.1),[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px2.p1.1)\.
- Zhanget al\.\(2025b\)J\. Zhang, J\. Xiang, Z\. Yu, F\. Teng, X\. Chen, J\. Chen, M\. Zhuge, X\. Cheng, S\. Hong, J\. Wang, B\. Zheng, B\. Liu, Y\. Luo, and C\. WuAFlow: automating agentic workflow generation\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=z5uVAKwmjf)Cited by:[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2609.00237#S4.SS0.SSS0.Px2.p1.1),[Table 1](https://arxiv.org/html/2609.00237#S4.T1.2.17.1.1)\.
- Zhanget al\.\(2024\)J\. Zhang, X\. Xu, N\. Zhang, R\. Liu, B\. Hooi, and S\. DengExploring collaboration mechanisms for LLM agents: a social psychology view\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 14544–14607\.External Links:[Link](https://aclanthology.org/2024.acl-long.782/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.782)Cited by:[§1](https://arxiv.org/html/2609.00237#S1.p1.1)\.
- Zhanget al\.\(2026\)Y\. Zhang, M\. Zheng, and Q\. LouSecureRouter: encrypted routing for efficient secure inference\.CoRRabs/2604\.15499\.External Links:[Link](https://doi.org/10.48550/arXiv.2604.15499),[Document](https://dx.doi.org/10.48550/ARXIV.2604.15499),2604\.15499Cited by:[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px3.p1.1)\.
- Zhugeet al\.\(2024\)M\. Zhuge, W\. Wang, L\. Kirsch, F\. Faccio, D\. Khizbullin, and J\. SchmidhuberGPTSwarm: language agents as optimizable graphs\.InForty\-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21\-27, 2024,R\. Salakhutdinov, Z\. Kolter, K\. A\. Heller, A\. Weller, N\. Oliver, J\. Scarlett, and F\. Berkenkamp \(Eds\.\),Proceedings of Machine Learning Research, Vol\.235,pp\. 62743–62767\.External Links:[Link](https://proceedings.mlr.press/v235/zhuge24a.html)Cited by:[§1](https://arxiv.org/html/2609.00237#S1.p2.1),[§2](https://arxiv.org/html/2609.00237#S2.SS0.SSS0.Px2.p1.1)\.

## 7Algorithm

We provide detailed pseudocode for Gated\-Memory Routing in Algorithm[1](https://arxiv.org/html/2609.00237#alg1)\.

Algorithm 1Gated\-Memory Routing1:Input:Query

qq, role set

ℛ\\mathcal\{R\}, model pool

ℳ\\mathcal\{M\}, maximum depth

ϕ\\phi
2:Output:Final answer

yfinaly\_\{\\mathrm\{final\}\}
3:Encode query embedding

𝐪←enc⁡\(q\)\\mathbf\{q\}\\leftarrow\\operatorname\{enc\}\(q\)
4:Initialize memory

S0←∅S\_\{0\}\\leftarrow\\emptyset, halting state

𝐡0halt←𝟎\\mathbf\{h\}^\{\\mathrm\{halt\}\}\_\{0\}\\leftarrow\\mathbf\{0\}
5:Initialize

t←0t\\leftarrow 0,

h0←0h\_\{0\}\\leftarrow 0
6:while

ht=0h\_\{t\}=0and

t<ϕt<\\phido

7:

t←t\+1t\\leftarrow t\+1
8:Form state

st←\(q,St−1\)s\_\{t\}\\leftarrow\(q,S\_\{t\-1\}\)
9:// Role and model routing

10:Sample role

rt∼πr\(⋅∣st\)r\_\{t\}\\sim\\pi\_\{r\}\(\\cdot\\mid s\_\{t\}\)
11:Sample LLM backbone

mt∼πm\(⋅∣st,rt\)m\_\{t\}\\sim\\pi\_\{m\}\(\\cdot\\mid s\_\{t\},r\_\{t\}\)
12:// Retrieval gate

13:Form retrieval vector

𝐩t←Linear⁡\(\[𝐪​‖𝐫rt‖​𝐦mt\]\)\\mathbf\{p\}\_\{t\}\\leftarrow\\operatorname\{Linear\}\(\[\\mathbf\{q\}\\,\\\|\\,\\mathbf\{r\}\_\{r\_\{t\}\}\\,\\\|\\,\\mathbf\{m\}\_\{m\_\{t\}\}\]\)
14:for all

ej∈St−1e\_\{j\}\\in S\_\{t\-1\}do

15:Sample

zt,j∼Bernoulli⁡\(σ⁡\(ℓt,j\)\)z\_\{t,j\}\\sim\\mathrm\{Bernoulli\}\(\\sigma\(\\ell\_\{t,j\}\)\)
16:endfor

17:Retrieve context

Ct←\{ej∈St−1:zt,j=1\}C\_\{t\}\\leftarrow\\\{e\_\{j\}\\in S\_\{t\-1\}:z\_\{t,j\}=1\\\}
18:// Agent execution

19:Generate reasoning step

yt←Agent⁡\(q,rt,mt,Ct\)y\_\{t\}\\leftarrow\\operatorname\{Agent\}\(q,r\_\{t\},m\_\{t\},C\_\{t\}\)
20:Form execution record

et←\(rt,mt,yt\)e\_\{t\}\\leftarrow\(r\_\{t\},m\_\{t\},y\_\{t\}\)
21:// Memory write gate

22:Compute write score

ωt\\omega\_\{t\}using the query,

yty\_\{t\}, and stored records in

St−1S\_\{t\-1\}
23:Sample

wt∼Bernoulli⁡\(σ⁡\(ωt\)\)w\_\{t\}\\sim\\mathrm\{Bernoulli\}\\\!\\left\(\\sigma\(\\omega\_\{t\}\)\\right\)
24:if

wt=1w\_\{t\}=1then

25:

St←St−1∪\{et\}S\_\{t\}\\leftarrow S\_\{t\-1\}\\cup\\\{e\_\{t\}\\\}
26:else

27:

St←St−1S\_\{t\}\\leftarrow S\_\{t\-1\}
28:endif

29:// Adaptive halting

30:

𝐡thalt←GRU⁡\(𝐡t−1halt,enc⁡\(St\)\)\\mathbf\{h\}^\{\\mathrm\{halt\}\}\_\{t\}\\leftarrow\\operatorname\{GRU\}\(\\mathbf\{h\}^\{\\mathrm\{halt\}\}\_\{t\-1\},\\operatorname\{enc\}\(S\_\{t\}\)\)
31:Sample

ht∼Bernoulli⁡\(σ⁡\(MLP⁡\(𝐡thalt\)\)\)h\_\{t\}\\sim\\mathrm\{Bernoulli\}\\\!\\left\(\\sigma\(\\operatorname\{MLP\}\(\\mathbf\{h\}^\{\\mathrm\{halt\}\}\_\{t\}\)\)\\right\)
32:endwhile

33:

L←tL\\leftarrow t
34:Select aggregator backbone

maggm\_\{\\mathrm\{agg\}\}as the most frequently selected model in

\{mi\}i=1L\\\{m\_\{i\}\\\}\_\{i=1\}^\{L\}
35:

yfinal←Aggregator⁡\(q,SL,magg\)y\_\{\\mathrm\{final\}\}\\leftarrow\\operatorname\{Aggregator\}\(q,S\_\{L\};m\_\{\\mathrm\{agg\}\}\)

## 8Representations, Encoders, and Training Interface

This appendix specifies the encoders and representations underlying the routing modules of Section[3](https://arxiv.org/html/2609.00237#S3), and how the policy is trained against the frozen backbones\.

#### Sentence encoder\.

All text is embedded by a single frozen sentence encoderenc⁡\(⋅\)\\operatorname\{enc\}\(\\cdot\)\(all\-MiniLM\-L6\-v2\), producing384384\-dimensional vectors; following the convention of Section[3](https://arxiv.org/html/2609.00237#S3),𝐱=enc⁡\(x\)\\mathbf\{x\}=\\operatorname\{enc\}\(x\)for any textxx, so𝐪\\mathbf\{q\}is the query embedding and𝐲t\\mathbf\{y\}\_\{t\}the embedding of reasoning stepyty\_\{t\}\. The same encoder embeds the query, the role and backbone descriptions, and every agent response\. Its parameters are not updated during training, and response embeddings are detached before they enter memory, so no gradient flows back through the encoder or through the agent outputs\.

#### Role and backbone latents\.

Each role profile and each backbone description is embedded once byenc\\operatorname\{enc\}and mapped to a continuous latent by a variational encoder \(a small VAE with a128128\-dimensional latent\), yielding the role embeddings𝐫i\\mathbf\{r\}\_\{i\}and backbone embeddings𝐦i\\mathbf\{m\}\_\{i\}used by the role allocator \(Section[3\.3](https://arxiv.org/html/2609.00237#S3.SS3)\) and LLM router \(Section[3\.4](https://arxiv.org/html/2609.00237#S3.SS4)\)\. The reconstruction and KL terms of these encoders form theℒVAE\\mathcal\{L\}\_\{\\mathrm\{VAE\}\}regularizer in the training objective\.

#### State encoder\.

The statest=\(q,St−1\)s\_\{t\}=\(q,S\_\{t\-1\}\)is encoded into the context vector𝐜t\\mathbf\{c\}\_\{t\}used by the routing modules\. The query is linearly projected to a128128\-dimensional vector𝐪^\\hat\{\\mathbf\{q\}\}, and each retained stepj<tj<tbecomes a token𝐮j\\mathbf\{u\}\_\{j\}formed from its role latent𝐫rj\\mathbf\{r\}\_\{r\_\{j\}\}and backbone latent𝐦mj\\mathbf\{m\}\_\{m\_\{j\}\}, additively modulated by a sigmoid gate over the response embedding𝐲j\\mathbf\{y\}\_\{j\}, so that response content, and not only routing identity, informs the state\. A two\-layer Transformer encoder contextualizes the tokens\{𝐮j\}j<t\\\{\\mathbf\{u\}\_\{j\}\\\}\_\{j<t\}, the query attends over them to produce a pooled history vector𝐡t\\mathbf\{h\}\_\{t\}, and𝐜t=\[𝐪^∥𝐡t\]\\mathbf\{c\}\_\{t\}=\[\\,\\hat\{\\mathbf\{q\}\}\\,\\\|\\,\\mathbf\{h\}\_\{t\}\\,\]\. The execution memory is therefore encoded as an ordered set of per\-step records rather than a single averaged vector\.

#### Memory\-record embedding\.

The retrieval gate \(Section[3\.5](https://arxiv.org/html/2609.00237#S3.SS5)\) forms the query vector𝐩t=Linear⁡\(\[𝐪​‖𝐫rt‖​𝐦mt\]\)\\mathbf\{p\}\_\{t\}=\\operatorname\{Linear\}\(\[\\mathbf\{q\}\\,\\\|\\,\\mathbf\{r\}\_\{r\_\{t\}\}\\,\\\|\\,\\mathbf\{m\}\_\{m\_\{t\}\}\]\)and, for each stored recordjj, an embedding𝐯j=Linear⁡\(\[𝐫rj​‖𝐦mj‖​𝐲~j\]\)\\mathbf\{v\}\_\{j\}=\\operatorname\{Linear\}\(\[\\mathbf\{r\}\_\{r\_\{j\}\}\\,\\\|\\,\\mathbf\{m\}\_\{m\_\{j\}\}\\,\\\|\\,\\tilde\{\\mathbf\{y\}\}\_\{j\}\]\), where𝐲~j\\tilde\{\\mathbf\{y\}\}\_\{j\}is a learned projection of𝐲j\\mathbf\{y\}\_\{j\}\. The retrieval logit is the scaled cosine similarityℓt,j=scos\(𝐩t,𝐯j\)\+b\\ell\_\{t,j\}=s\\,\\cos\(\\mathbf\{p\}\_\{t\},\\mathbf\{v\}\_\{j\}\)\+bwith learnable scalessand biasbb, and recordjjis admitted byzt,j∼Bernoulli⁡\(σ⁡\(ℓt,j\)\)z\_\{t,j\}\\sim\\mathrm\{Bernoulli\}\(\\sigma\(\\ell\_\{t,j\}\)\)\. Memory holds at mostϕ\\phirecords, soCtC\_\{t\}ranges from empty toϕ\\phientries, withbbsetting its typical length\.

#### Write\-gate similarities\.

For the Memory Write Gate \(Section[3\.6](https://arxiv.org/html/2609.00237#S3.SS6)\), the projection underlyingsim⁡\(⋅,⋅\)\\mathrm\{sim\}\(\\cdot,\\cdot\), the relevance–novelty weightλ\\lambda, the write thresholdθ\\theta, and a logit scaleβ\\betaare learned jointly with the routing policies, and the write decision iswt∼Bernoulli⁡\(σ⁡\(β⁡\(ωt−θ\)\)\)w\_\{t\}\\sim\\mathrm\{Bernoulli\}\\\!\\left\(\\sigma\(\\beta\(\\omega\_\{t\}\-\\theta\)\)\\right\), which Section[3\.6](https://arxiv.org/html/2609.00237#S3.SS6)abstracts asσ⁡\(ωt\)\\sigma\(\\omega\_\{t\}\)\.

#### Training interface\.

The backbones are frozen and queried as an environment: at each step the selected model is called through an OpenAI\-compatible vLLM server, and the returned text is embedded and detached as described above\. The router is optimized purely by the score\-function \(policy\-gradient\) estimator: the group\-relative advantageAiA\_\{i\}\(Section[3\.8](https://arxiv.org/html/2609.00237#S3.SS8)\) weights the trajectory log\-probabilitylog⁡πi=∑t=1L\(log⁡πr​\(rt∣st\)\+log⁡πm​\(mt∣st,rt\)\+log⁡p⁡\(zt\)\+log⁡p⁡\(wt\)\+log⁡p⁡\(ht\)\)\\log\\pi\_\{i\}=\\sum\_\{t=1\}^\{L\}\\big\(\\log\\pi\_\{r\}\(r\_\{t\}\\mid s\_\{t\}\)\+\\log\\pi\_\{m\}\(m\_\{t\}\\mid s\_\{t\},r\_\{t\}\)\+\\log p\(z\_\{t\}\)\+\\log p\(w\_\{t\}\)\+\\log p\(h\_\{t\}\)\\big\), whereztz\_\{t\}collects the step\-ttretrieval decisions\. Gradients reach only the routing, gating, halting, and embedding parameters; a single optimizer updates all of them jointly, and the backbones are never differentiated through\.

## 9Cost Calculation for Open\-Weight Backbones

Because the five backbones are open\-weight and served locally, they do not carry a public per\-token price\. To give the cost\-aware objective a hardware\-independent and monotonic cost signal, we set each model’s per\-token reference price proportional to its \(active\) parameter countNNin billions: an input rate of0\.003​N0\.003\\,Nand an output rate of0\.010​N0\.010\\,Nper million tokens, with output weighted more heavily to reflect the higher cost of autoregressive decoding\. For dense transformers, per\-token inference compute \(FLOPs\) grows approximately linearly withNN, so pricing each token by model size makes the reported cost proportional to a size\-weighted token count: a hardware\-independent proxy for relative inference compute across the pool, rather than the dollar price of any specific API or the wall\-clock cost on particular hardware\. Both the training cost term of Section[3\.8](https://arxiv.org/html/2609.00237#S3.SS8)and the reported test cost of a trajectory are the additive sumCost⁡\(τ\)=∑tFt\\mathrm\{Cost\}\(\\tau\)=\\sum\_\{t\}F\_\{t\}of its per\-step backbone costs, whereFtF\_\{t\}is the cost of steptt’s call \(its input and output tokens times these rates\)\. As a relative\-compute proxy it abstracts away attention cost that scales with context length, batching, and memory bandwidth\. Table[4](https://arxiv.org/html/2609.00237#S9.T4)lists the backbone pool and the resulting rates\.

BackboneSizeCost in/outllama\-3\.2\-3B3B0\.009 / 0\.03llama\-3\.1\-8B8B0\.024 / 0\.08mistral\-nemo\-12B12B0\.036 / 0\.12qwen\-2\.5\-14B14B0\.042 / 0\.14qwen\-2\.5\-32B32B0\.096 / 0\.32Table 4:Open\-weight backbone pool and size\-based reference rates \(input=0\.003​N=0\.003\\,N, output=0\.010​N=0\.010\\,Nper11M tokens, withNNthe parameter count in billions\), shown as input/output\.To corroborate the size\-weighted proxy with hardware\-relevant measurements, Table[5](https://arxiv.org/html/2609.00237#S9.T5)reports, for HumanEval and MBPP, an approximate FLOPs count and the batch\-amortized wall\-clock time per query for our method and the two strongest baselines\. FLOPs are estimated from the per\-call token records as2​N2NFLOPs per processed token[Kaplan et al\. \(2020\)](https://arxiv.org/html/2609.00237#bib.bib39), summed over all calls withNNthe parameter count of the model serving each call; this omits the sequence\-length\-dependent attention term\. Wall\-clock time is the end\-to\-end inference time for the test set divided by the number of queries, measured under the same serving configuration for all methods\. Our method has the lowest FLOPs and wall\-clock time on both benchmarks, and the ordering matches the proxy\.

HumanEvalMBPPMethodPFLOPs/queryTime/query \(s\)PFLOPs/queryTime/query \(s\)Ours0\.1071\.50\.1452\.6MASRouter0\.2004\.40\.2896\.4Puppeteer\-32B0\.1443\.00\.1633\.2Table 5:Approximate inference FLOPs and batch\-amortized wall\-clock time per query on HumanEval and MBPP\.
## 10Per\-Dataset Accuracy and Cost

Tables[6](https://arxiv.org/html/2609.00237#S10.T6)and[7](https://arxiv.org/html/2609.00237#S10.T7)report per\-method accuracy and total test\-set cost on GSM\-Hard and HumanEval, the data underlying the Pareto comparison of Figure[3](https://arxiv.org/html/2609.00237#S5.F3)\. Cost is the size\-based reference cost of Appendix[9](https://arxiv.org/html/2609.00237#S9)\. Figure[5](https://arxiv.org/html/2609.00237#S10.F5)shows the GSM\-Hard accuracy–cost frontier, the companion to the HumanEval frontier in the main text\.

Figure 5:Accuracy versus inference token cost on GSM\-Hard, with cost on a log scale\. As on HumanEval, our method lies on the high\-accuracy end of the Pareto frontier, reaching the highest accuracy among compared methods at substantially lower cost than the strongest multi\-agent and routing baselines\.MethodLLMAcc\. \(%\)CostSingle LLMllama\-3\.2\-3B25\.850\.025llama\-3\.1\-8B39\.870\.046mistral\-nemo\-12B30\.110\.050qwen\-2\.5\-14B64\.580\.047qwen\-2\.5\-32B61\.520\.071CoTqwen\-2\.5\-14B67\.520\.076qwen\-2\.5\-32B64\.560\.167Complex\-CoTqwen\-2\.5\-14B66\.760\.108qwen\-2\.5\-32B65\.320\.230MacNet\-Chainqwen\-2\.5\-14B59\.750\.629qwen\-2\.5\-32B61\.933\.421MacNet\-Treeqwen\-2\.5\-14B60\.320\.696qwen\-2\.5\-32B62\.312\.433MacNet\-Completeqwen\-2\.5\-14B59\.000\.933qwen\-2\.5\-32B62\.224\.099AFlowqwen\-2\.5\-14B58\.240\.134qwen\-2\.5\-32B67\.052\.165Puppeteerqwen\-2\.5\-14B65\.910\.232qwen\-2\.5\-32B68\.750\.445MASRouterLLM Pool66\.001\.012OursLLM Pool70\.550\.587Table 6:Accuracy and total test\-set cost on GSM\-Hard\.MethodLLMAcc\. \(%\)CostSingle LLMllama\-3\.2\-3B62\.790\.001llama\-3\.1\-8B69\.780\.0025mistral\-nemo\-12B68\.220\.002qwen\-2\.5\-14B82\.950\.003qwen\-2\.5\-32B84\.370\.009CoTqwen\-2\.5\-14B82\.170\.004qwen\-2\.5\-32B85\.150\.012Complex\-CoTqwen\-2\.5\-14B80\.620\.005qwen\-2\.5\-32B85\.470\.014MacNet\-Chainqwen\-2\.5\-14B86\.820\.019qwen\-2\.5\-32B83\.800\.059MacNet\-Treeqwen\-2\.5\-14B82\.950\.024qwen\-2\.5\-32B86\.020\.084MacNet\-Completeqwen\-2\.5\-14B84\.500\.033qwen\-2\.5\-32B86\.050\.119AFlowqwen\-2\.5\-14B85\.270\.014qwen\-2\.5\-32B84\.500\.224Puppeteerqwen\-2\.5\-14B83\.590\.029qwen\-2\.5\-32B85\.160\.047MASRouterLLM Pool85\.160\.057OursLLM Pool89\.840\.032Table 7:Accuracy and total test\-set cost on HumanEval\.
## 11Variance across Training Seeds

Table[1](https://arxiv.org/html/2609.00237#S4.T1)reports one training run per method\. To quantify run\-to\-run variability, we retrained our method and the two strongest baselines with three independent seeds each and report the per\-benchmark mean and sample standard deviation in Table[8](https://arxiv.org/html/2609.00237#S11.T8), together with the overall mean \(the macro\-average over benchmarks for each seed, then mean and standard deviation across seeds\)\. Our method has the highest mean on all five benchmarks and improves the overall mean by4\.914\.91points over MASRouter and2\.102\.10over Puppeteer\-32B\. The ranking is identical under every seed: the lowest overall score among our seeds \(76\.6276\.62\) exceeds the highest among Puppeteer’s \(75\.7975\.79\) and MASRouter’s \(74\.2774\.27\)\.

MethodGSM\-HardMATHMBPPHumanEvalMMLU\-ProOverallMASRouter65\.37±\\pm0\.5574\.37±\\pm2\.2876\.87±\\pm2\.2181\.51±\\pm3\.9464\.09±\\pm4\.8372\.44±\\pm1\.88Puppeteer\-32B66\.98±\\pm1\.8777\.17±\\pm4\.4176\.41±\\pm2\.2986\.72±\\pm2\.0668\.98±\\pm1\.0175\.25±\\pm0\.52Ours69\.76±\\pm1\.4578\.37±\\pm1\.8880\.40±\\pm1\.2188\.54±\\pm1\.1969\.62±\\pm1\.1777\.35±\\pm0\.63Table 8:Mean±\\pmsample standard deviation of test accuracy \(%\) over three training seeds per method\. Overall is the macro\-average over the five benchmarks\.
## 12Gate Behavior at Test Time

Table[9](https://arxiv.org/html/2609.00237#S12.T9)checks that the trained gates do not collapse to trivial policies\. The write rate is the fraction of reasoning steps committed to memory; the retrieved fraction is the mean number of retrieved items divided by the number available at that step \(the memory is empty at step 0\)\. Write rates lie strictly inside\(0,1\)\(0,1\)on all three benchmarks, and the retrieval gate becomes more selective as memory grows rather than exposing all or none of it\. The per\-step entropy of the gate distributions also stays well above zero throughout training on all three benchmarks\.

BenchmarkMean depthWrite rateRetrieved fraction, steps 1–5GSM\-Hard2\.9239\.6%16 / 24 / 14 / 12 / 11%MBPP3\.7932\.8%9 / 30 / 20 / 15 / 12%HumanEval3\.3036\.5%91 / 48 / 33 / 24 / 21%Table 9:Test\-time gate behavior of the full system: mean realized depth, fraction of steps written to memory, and fraction of available memory items retrieved at each step\.
## 13Halting and Training\-Time Computation

Adaptive halting also reduces training\-time computation, since shorter trajectories mean fewer rollout steps per update\. Table[10](https://arxiv.org/html/2609.00237#S13.T10)compares the mean realized depth and the total number of rollout steps over training with and without halting\.

BenchmarkDepth, haltingDepth, no haltingRollout steps, halting vs\. noneGSM\-Hard3\.46\.026\.5k vs\. 47\.9k \(1\.81×1\.81\\times\)HumanEval3\.36\.06\.3k vs\. 11\.5k \(1\.83×1\.83\\times\)Table 10:Effect of adaptive halting on training\-time computation\.
## 14Agent Role Profiles

The role allocator selects among the same2626heterogeneous role profiles used by MASRouter[Yue et al\. \(2025\)](https://arxiv.org/html/2609.00237#bib.bib10), grouped into three domains:77code\-oriented roles,77commonsense/knowledge roles, and1212mathematical\-reasoning roles\. Each profile is a short natural\-language description that we encode and project into the latent role space \(Section[3\.3](https://arxiv.org/html/2609.00237#S3.SS3)\)\. At instantiation, this description, together with the role’s reasoning strategy and the benchmark’s output\-format requirements, forms the agent’s system prompt, while the user prompt supplies the query and any retrieved context\. The descriptions are listed below\. The imbalance across domains reflects the original MASRouter catalog, which we adopt unchanged so that both methods select from the same roles\. To test dependence on the full catalog, we randomly subsampled it to1313roles, keeping roughly half of each domain, and retrained the full system: accuracy is70\.7470\.74on GSM\-Hard and89\.0689\.06on HumanEval, against70\.5570\.55and89\.8489\.84with all2626roles, so performance does not hinge on the full role set\.

#### Code\.

- •AlgorithmDesigner\.Specifies the design of the algorithm, including explanations, usage instructions, and API references, optionally giving pseudocode for the main logic; replies concisely\.
- •ProgrammingExpert\.A programming expert who, given a function signature and docstring, writes the full implementation \(restating the signature\) in a single Python code block\.
- •BugFixer\.A programming expert who restates the signature and returns a corrected full implementation in a Python code block\.
- •ReflectProgrammer\.A programming expert who reflects on prior attempts and returns a full implementation in a Python code block\.
- •PlanSolver\.Produces the pseudocode of the target function\.
- •ProjectManager\.Oversees the overall code structure, suggests optimal design patterns for maintainability and flexibility, and avoids over\-engineering; replies concisely\.
- •TestAnalyst\.Identifies problems in the current code from test data and feedback, supplies special cases and boundary conditions to watch, and points out potential errors; replies concisely\.

#### Commonsense and Knowledge\.

- •KnowledgeExpert\.A knowledgeable question\-answering expert who analyzes step by step and selects the correct answer\.
- •Reflector\.Re\-examines the question\-answering process step by step and selects the correct answer\.
- •Critic\.Points out potential issues in other agents’ analyses point by point and gives a critical opinion before the final result\.
- •Scientist\.A scientist with natural\-science knowledge who provides a thorough solving process, including necessary proofs and explanations\.
- •Economist\.An experienced economist \(macroeconomics, microeconomics, financial markets\) who gives well\-reasoned, evidence\-based answers\.
- •Historian\.Analyzes cultural, economic, political, and social events from primary sources to reason about the past\.
- •WikiSearcher\.Lists the key Wikipedia entities that should be looked up to solve the problem\.

#### Mathematical Reasoning\.

- •MathSolver\.A math expert who produces a solving process from the hints supplied by other agents\.
- •Mathematician\.A mathematician skilled at math games, arithmetic, and long\-horizon planning\.
- •MathTeacher\.Teaches the solution step by step as if to a student\.
- •MathAnalyst\.First derives the solution symbolically \(variables as letters\), then substitutes values to compute the result\.
- •Inspector\.Checks whether the problem\-solving logic, calculations, and any accompanying code are correct and consistent, then gives its own step\-by\-step solution\.
- •AlgorithmEngineer\.Integrates step\-by\-step reasoning with Python code to solve the problem\.
- •ProgrammingExpert\.Analyzes the problem and writes functions, combining reasoning with Python code\.
- •SoftwareDeveloper\.Designs efficient solutions and provides clear, concise functions\.
- •Engineer\.An experienced engineer who solves the problem from engineering knowledge\.
- •Scientist\.Provides a detailed solving process with necessary proofs and explanations\.
- •Economist\.Applies economic reasoning to give evidence\-based answers\.
- •CertifiedAccountant\.Analyzes financial problems and returns correct calculations and solutions\.

## 15Aggregator Prompt

Once the halting controller stops the trajectory, the Aggregator LLM receives the query and the retained execution records and produces the final answer\. The aggregator is instantiated with the backbone the router selected most frequently during the trajectory \(Section[4\.1](https://arxiv.org/html/2609.00237#S4.SS1)\)\. Across benchmarks the aggregator shares a common framing in its system prompt, namely to weigh the analyses and results of the other agents, identify errors, and commit to a single most\-reliable answer, while the user prompt enforces the benchmark\-specific output format\. The boxes below paraphrase the system and user prompts for each benchmark\.

Aggregator prompt: MATHSystem\.Weigh the other agents’ analyses and code, justify the decision, and report the final numeric answer as a boxed value with no units \(e\.g\., “The answer is140\\boxed\{140\}”\)\. User\.Give the final answer in the form “The answer is140\\boxed\{140\}”\.

Aggregator prompt: GSM\-HardSystem\.Weigh the other agents’ analyses and code and commit to the most reliable answer, with reasons\. Report a single pure number, with no units, symbols, commas, or scientific notation, and no rounding unless the problem asks; the last line must read exactlyThe answer is <number\>\. User\.Give the final answer with a last line of the formThe answer is <number\>\.

Aggregator prompt: HumanEval and MBPPSystem\.Act as the decision\-maker over the other agents’ outputs, identify errors, and return the answer as a single Python code block containing nothing else\. User\.Given the function signature and docstring, with any prior designs or implementations, write the full implementation \(restating the signature\) in one Python code block\.

Aggregator prompt: MMLU\-ProSystem\.Act as the decision\-maker over the other agents’ answers and analyses and identify errors\. User\.Exactly one of up to ten options \(A to J\) is correct; end with a last line of the form “The answer isX\\boxed\{X\}”, where X is one of A to J\.

## 16Backbone LLM Profiles

The LLM router selects among the five open\-weight backbones through their natural\-language capability descriptions, which a variational encoder maps into the latent backbone space \(Section[3\.4](https://arxiv.org/html/2609.00237#S3.SS4)\)\. The descriptions below are the text encoded for each model; each description also states the model’s reference input/output price \(Table[4](https://arxiv.org/html/2609.00237#S9.T4)\), so the encoded text covers both capability and cost\.

- •llama\-3\.2\-3B\.Meta’s compact 3\-billion\-parameter, text\-only instruction\-tuned model with a 128k\-token context window\. The cheapest and fastest choice in the pool, with solid general reasoning and instruction following; suited to easy queries where quality is less critical\.
- •llama\-3\.1\-8B\.Meta’s widely used 8\-billion\-parameter instruction\-tuned model, offering solid general\-purpose reasoning and instruction following at very low cost; a reliable baseline across a broad range of tasks\.
- •mistral\-nemo\-12B\.Mistral AI’s 12\-billion\-parameter dense model built in partnership with NVIDIA, with strong multilingual performance, a 128k\-token context window, and solid general reasoning and coding at modest cost; a reliable mid\-size workhorse\.
- •qwen\-2\.5\-14B\.Alibaba Cloud’s 14\-billion\-parameter dense instruction\-tuned model with a 128k\-token context window; strong on mathematics, coding, and general reasoning, punching above its size class as a capable mid\-tier option for moderately hard queries\.
- •qwen\-2\.5\-32B\.Alibaba Cloud’s flagship dense model in this pool, with frontier\-level performance among open\-weight models in its size class on mathematics, coding, and complex reasoning; reserved for the hardest queries where smaller models fall short\.