NeuroMAS: Multi-Agent Systems as Neural Networks with Joint Reinforcement Learning

# NeuroMAS: Multi-Agent Systems as Neural Networks with Joint Reinforcement Learning
Source: [https://arxiv.org/html/2605.16757](https://arxiv.org/html/2605.16757)
Haoran Lu Department of Statistics University of Georgia Athens, GA 30602 haoran\.lu@uga\.edu&Luyang Fang Department of Statistics University of Georgia Athens, GA 30602 luyang\.fang@uga\.edu&Wenxuan Zhong Department of Statistics University of Georgia Athens, GA 30602 wenxuan@uga\.edu&Ping Ma11footnotemark:1 Department of Statistics University of Georgia Athens, GA 30602 pingma@uga\.edu

###### Abstract

Multi\-agent language systems are often built as hand\-designed workflows, where agents are assigned semantic roles and communication protocols are specified in advance\. We proposeNeuroMAS, a method that first treats a multi\-agent language system as a trainable and scalable neural\-network\-like architecture with LLM agents as nodes and intermediate textual signals as edges\. In NeuroMAS, agent nodes are role\-free but structure\-aware: the topology only determines how information can flow in general, while reinforcement learning training determines how nodes communicate, specialize, and coordinate\. This formulation shifts multi\-agent design from workflow engineering toward architecture design, where depth, width, connectivity, and growth protocol become scalable sources of capability\. Further, we provide a theoretical perspective showing why such modular textual computation is more parameter\-efficient when tasks admit hierarchical decompositions\. Experiments show that NeuroMAS improves significantly over both inference\-time and trained multi\-agent baselines\. We further find that organizational scaling is path\-dependent: larger systems can be challenging to train from scratch, but become feasible when grown progressively from smaller trained systems\. These results suggest that learned neural multi\-agent systems are a promising scaling axis for LLMs\.

## 1Introduction

Large language model \(LLM\)\-based agents have become increasingly capable in coding, reasoning, mathematics, writing, and tool\-use tasks, as illustrated by recent systems for agentic coding, task automation, and mathematical problem solving\(Anthropic,[2025](https://arxiv.org/html/2605.16757#bib.bib56); OpenAI,[2025](https://arxiv.org/html/2605.16757#bib.bib57); OpenClaw,[2026](https://arxiv.org/html/2605.16757#bib.bib58); Nous Research,[2026](https://arxiv.org/html/2605.16757#bib.bib66); Google DeepMind,[2025](https://arxiv.org/html/2605.16757#bib.bib59)\)\. Despite this progress, current LLM agents still struggle with long\-horizon execution, complex reasoning, and domain\-specific adaptation\(Yaoet al\.,[2023b](https://arxiv.org/html/2605.16757#bib.bib32); Liuet al\.,[2023](https://arxiv.org/html/2605.16757#bib.bib60); Wanget al\.,[2023a](https://arxiv.org/html/2605.16757#bib.bib61)\)\. A common response to these limitations is to scale the underlying model: increase the parameter count, training data, training compute, or inference\-time computation\(Kaplanet al\.,[2020](https://arxiv.org/html/2605.16757#bib.bib10); Brownet al\.,[2020](https://arxiv.org/html/2605.16757#bib.bib42); Hoffmannet al\.,[2022](https://arxiv.org/html/2605.16757#bib.bib11); OpenAI,[2023](https://arxiv.org/html/2605.16757#bib.bib30)\)\. This strategy remains powerful, but it is also increasingly expensive\. Empirical scaling laws suggest that gains from larger models are sublinear in compute and model size, so continued improvement often requires disproportionately larger resources\(Kaplanet al\.,[2020](https://arxiv.org/html/2605.16757#bib.bib10); Hoffmannet al\.,[2022](https://arxiv.org/html/2605.16757#bib.bib11); Sardanaet al\.,[2024](https://arxiv.org/html/2605.16757#bib.bib45)\)\. For large frontier\-scale backbones, brute\-force scaling also increases training and inference cost\(Brownet al\.,[2020](https://arxiv.org/html/2605.16757#bib.bib42); Chowdheryet al\.,[2023](https://arxiv.org/html/2605.16757#bib.bib43); Feduset al\.,[2022](https://arxiv.org/html/2605.16757#bib.bib44)\)\. These trends motivate a complementary question: rather than only scaling the model itself, can we scale the organization of computation around a fixed model?

Multi\-agent systems \(MAS\) provide a natural path toward this alternative axis of scale\. Instead of relying on one monolithic generation process, multiple LLM agents can decompose a problem, explore different reasoning paths, exchange intermediate information, and aggregate partial solutions\. Related inference\-time reasoning methods already show that structuring computation can improve language\-model reasoning, for example through chain\-of\-thought prompting, least\-to\-most decomposition, tree search, or debate\-style interaction\(Weiet al\.,[2022](https://arxiv.org/html/2605.16757#bib.bib31); Zhouet al\.,[2023](https://arxiv.org/html/2605.16757#bib.bib46); Yaoet al\.,[2023a](https://arxiv.org/html/2605.16757#bib.bib33); Duet al\.,[2024](https://arxiv.org/html/2605.16757#bib.bib47)\)\. LLM\-based multi\-agent systems \(MAS\) extend this idea by turning reasoning from a single\-agent generation problem into a distributed computation process involving multiple communicating agents\(Wuet al\.,[2024](https://arxiv.org/html/2605.16757#bib.bib15); Honget al\.,[2024](https://arxiv.org/html/2605.16757#bib.bib34); Qianet al\.,[2024](https://arxiv.org/html/2605.16757#bib.bib17)\)\. Recent work further shows that the organization of these systems matters: changing the number of agents, communication pattern, or interaction topology can substantially affect performance\(Liet al\.,[2024](https://arxiv.org/html/2605.16757#bib.bib18); Zhugeet al\.,[2024](https://arxiv.org/html/2605.16757#bib.bib20); Zhouet al\.,[2026](https://arxiv.org/html/2605.16757#bib.bib22)\)\. Thus, capability may come not only from the scale of an individual backbone model, but also from how multiple model instances are organized\.

However, most existing multi\-agent systems still treat organization primarily as a design choice rather than as a scalable object\. Prompt\-based and workflow\-based systems assign agents human\-written roles such as planner, solver, critic, verifier, or reviewer, and specify communication protocols by hand\(Wuet al\.,[2024](https://arxiv.org/html/2605.16757#bib.bib15); Honget al\.,[2024](https://arxiv.org/html/2605.16757#bib.bib34); Qianet al\.,[2024](https://arxiv.org/html/2605.16757#bib.bib17); Wanget al\.,[2025](https://arxiv.org/html/2605.16757#bib.bib19)\)\. More recent methods optimize parts of this design space, including prompts, communication edges, workflows, or topologies\(Zhugeet al\.,[2024](https://arxiv.org/html/2605.16757#bib.bib20); Zhouet al\.,[2026](https://arxiv.org/html/2605.16757#bib.bib22); Yanget al\.,[2025b](https://arxiv.org/html/2605.16757#bib.bib23); Motwaniet al\.,[2025](https://arxiv.org/html/2605.16757#bib.bib24)\)\. These methods demonstrate that organization is important, but they usually separate the design of the organization from the training of the agents inside it\. In many cases, the workflow is fixed by the designer and learning adapts agents within that scaffold; in others, topology or prompt search is performed separately from end\-to\-end agent training\. This leaves a central gap: we lack a unified formulation in which the multi\-agent organization and the trainable agent parameters are optimized as one system\.

![Refer to caption](https://arxiv.org/html/2605.16757v1/fig/masdual.png)Figure 1:\(a\) Overview of the NeuroMAS Method\. \(b\) Conceptual landscape of LLM multi\-agent systems\. The horizontal axis represents frozen versus trainable within\-agent parameters, and the vertical axis represents fixed versus scalable and/or flexible between\-agent topology\. Positions are illustrative rather than quantitative\.We address this gap withNeural Multi\-Agent System \(NeuroMAS\), a framework that treats a multi\-agent language system as a trainable and scalable architecture with textual intermediate signals\. As illustrated in Figure[1](https://arxiv.org/html/2605.16757#S1.F1)\(a\), NeuroMAS is a system in which LLM agent nodes are connected by text\-carrying edges\. Each node receives the task input together with messages from upstream nodes, generates textual outputs for downstream nodes, and is optimized jointly with the rest of the nodes using the reward from final tasks\. Unlike conventional multi\-agent workflows, NeuroMAS does not assign nodes with hand\-written semantic roles such as planner, critic, verifier, or judge\. Instead, nodes receive minimal role information by beingrole\-free but structure\-aware: each node knows its position in the architecture and the format of the messages it must produce, but its functional specialization is not prescribed in advance\. The topology defines what information can flow where; training determines how each node uses incoming messages, what it sends forward, and how the system coordinates to produce the final answer\.

This formulation shifts MAS design from workflow engineering towardneural topology design\. The related existing MAS are summarized conceptually in Figure[1](https://arxiv.org/html/2605.16757#S1.F1)\(b\), which will be discussed in detail in Section[2](https://arxiv.org/html/2605.16757#S2), NeuroMAS targets a distinct and underexplored regime: the within\-agent LLM backbone models are trainable, while the between\-agent topology provides a scalable and flexible architectural structure rather than a fixed workflow\. In a conventional workflow, the designer specifies the roles and aggregation rules\. In NeuroMAS, the graph plays a role analogous to architecture in neural networks: depth, width, connectivity, and growth protocol determine the space of possible computations, while learning determines the effective behavior of the modules inside that architecture\. This view is inspired by the success of architectural design in neural networks, where depth, width, modularity, skip connections, graph\-structured computation, and architecture search can strongly shape performance\(LeCunet al\.,[1998](https://arxiv.org/html/2605.16757#bib.bib65); Heet al\.,[2016](https://arxiv.org/html/2605.16757#bib.bib40); Vaswaniet al\.,[2017](https://arxiv.org/html/2605.16757#bib.bib64); Shazeeret al\.,[2017](https://arxiv.org/html/2605.16757#bib.bib27); Elskenet al\.,[2019](https://arxiv.org/html/2605.16757#bib.bib41)\)\. NeuroMAS brings this important perspective to MAS\. The goal is not to hand\-code the best agent roles, but to provide a scalable structure in which communication, specialization, and coordination can emerge through reinforcement learning\. In this sense, NeuroMAS frames aBitter Lessonperspective for multi\-agent systems: durable progress should come from general trainable mechanisms that can scale with computation, rather than from increasingly elaborate human\-authored workflows\(Sutton,[2019](https://arxiv.org/html/2605.16757#bib.bib12)\)\.

We evaluate NeuroMAS in a deliberately resource\-constrained regime\. All agents share a small frozen backbone model, and each node is adapted only through lightweight trainable parameters\. This setting is designed to test whether learned organizations can improve LLM reasoning without relying on a stronger base model\. Across reasoning and coding benchmarks, NeuroMAS improves over human\-engineered collaboration baselines and single\-model or fixed\-topology trained methods\. Parameter\-count controls further suggest that the gains are not explained merely by adding more trainable adapter parameters to one model\. We also find that organizational scaling is possible but not automatic: larger systems trained from scratch can be unstable, whereas progressively growing a trained smaller system yields stronger and more reliable performance\. These results support the view that the organization around a language model can itself become a trainable source of capability\.

#### Contributions\.

This work makes the following major contributions\.Methodologically, we introduce NeuroMAS as a step toward treating multi\-agent language systems as scalable trainable architectures, rather than as separate prompting, routing, or coordination procedures\. In this view, communication and specialization are learned properties of the system, not fixed design choices\.Algorithmically, we show that multi\-agent systems can be scaled through progressive growth, where a smaller trained architecture is expanded into a larger one while preserving useful learned behavior\.Theoretically, we explain why distributing computation across organized interacting agents can require fewer parameters than forcing a single unstructured model to solve the whole problem\.Empirically, we show that learned organization produces consistent gains across reasoning, domain\-knowledge, and code\-generation tasks, outperforming prompt\-only collaboration, single\-model training controls, and recent trained multi\-agent baselines, and revealing that successful multi\-agent scaling depends on how the system is grown\.

## 2Related Work

Structured inference with a single model\.Many methods improve LLM reasoning by changing test\-time computation around a single frozen model\. Chain\-of\-thought exposes intermediate reasoning, self\-consistency samples multiple reasoning paths and aggregates answers, and Tree\-of\-Thought searches over candidate trajectories\(Weiet al\.,[2022](https://arxiv.org/html/2605.16757#bib.bib31); Wanget al\.,[2023b](https://arxiv.org/html/2605.16757#bib.bib69); Yaoet al\.,[2023a](https://arxiv.org/html/2605.16757#bib.bib33)\)\. Self\-Refine adds feedback\-and\-revision loops, while Self\-Check verifies step\-by\-step reasoning without external tools or supervision\(Madaanet al\.,[2023](https://arxiv.org/html/2605.16757#bib.bib13); Miaoet al\.,[2024](https://arxiv.org/html/2605.16757#bib.bib50)\)\. These methods show that structured inference can improve reasoning, but the structure is manually specified and used only at inference time\. NeuroMAS instead studies trainable organization around a shared backbone, where multiple language policies exchange textual states within a scalable network architecture\.

Engineered multi\-agent workflows\.Another line of work distributes reasoning across multiple LLM agents while keeping model parameters fixed\. AutoGen, MetaGPT, and ChatDev use explicit agent identities, role descriptions, hand\-designed communication protocols, and software\-style workflows\(Wuet al\.,[2024](https://arxiv.org/html/2605.16757#bib.bib15); Honget al\.,[2024](https://arxiv.org/html/2605.16757#bib.bib34); Qianet al\.,[2024](https://arxiv.org/html/2605.16757#bib.bib17)\)\. Mixture\-of\-Agents and Agent Forest use multiple calls or agents to generate, compare, refine, aggregate, or vote over answers\(Wanget al\.,[2025](https://arxiv.org/html/2605.16757#bib.bib19); Liet al\.,[2024](https://arxiv.org/html/2605.16757#bib.bib18)\)\. These systems show that agent organization affects performance, but roles, prompts, edges, and coordination rules are mainly human\-specified\. NeuroMAS differs by making the agent modules trainable components of the system rather than fixed participants in a hand\-authored workflow\.

Optimizing agent organization\.Recent methods reduce manual design by optimizing parts of the multi\-agent organization\. GPTSwarm optimizes agent graphs, including node prompts and graph connectivity\(Zhugeet al\.,[2024](https://arxiv.org/html/2605.16757#bib.bib20)\)\. MASS searches over prompts and topologies through staged optimization\(Zhouet al\.,[2026](https://arxiv.org/html/2605.16757#bib.bib22)\)\. Graph of Agents builds input\-dependent collaboration structures for long\-context modeling, and AgentNet studies decentralized coordination with dynamically evolving connectivity\(Jooet al\.,[2025](https://arxiv.org/html/2605.16757#bib.bib49); Yanget al\.,[2025b](https://arxiv.org/html/2605.16757#bib.bib23)\)\. These works show that topology and communication are important design variables\. However, the optimization usually targets prompts, graphs, routing rules, search procedures, or external controllers, while the underlying LLM agents remain frozen\. NeuroMAS is complementary: for a given topology, it trains the agent modules themselves as stochastic language policies, and studies how performance changes across scalable topology families\.

Training multi\-agent collaboration\.The closest work directly trains multi\-agent reasoning systems\. MALT post\-trains a sequential generator\-verifier\-refiner pipeline for mathematical and commonsense reasoning\(Motwaniet al\.,[2025](https://arxiv.org/html/2605.16757#bib.bib24)\)\. CoLLM\-CC trains decentralized LLM collaboration with multi\-agent actor\-critic learning and a centralized critic for sparse or long\-horizon rewards\(Liuet al\.,[2026](https://arxiv.org/html/2605.16757#bib.bib4)\)\. NeuroMAS also trains collaboration, but differs in two key ways\. First, its nodes arerole\-free: they are not assigned semantic identities such as generator, verifier, refiner, critic, or judge\. Second, its architecture is not tied to a single fixed role\-based pipeline; it can be instantiated with different depths, widths, and growth protocols\.

Taken together, these lines of work define the conceptual landscape in Figure[1](https://arxiv.org/html/2605.16757#S1.F1)\(b\)\. The horizontal axis represents frozen versus trainable within\-agent parameters, and the vertical axis represents fixed versus scalable and/or flexible between\-agent topology\. The placements are illustrative rather than quantitative\. NeuroMAS combines trainable within\-agent language policies with scalable multi\-agent topology design, whereas prior methods mainly engineer frozen\-agent workflows, optimize external organizational structures, or train collaboration within fixed role\-based scaffolds\.

## 3Preliminaries

We briefly introduce the notation used to formulate NeuroMAS as a reward\-trained language architecture\. The key point is that final answers and intermediate messages are discrete text sequences, so the system is optimized at the trajectory level rather than by backpropagating through sampled tokens\.

Text policies and task objective\.Let𝒱\\mathcal\{V\}denote the token vocabulary, and let𝒯\\mathcal\{T\}denote the set of all finite token sequences over𝒱\\mathcal\{V\}\. We consider text\-input, text\-output tasks with observed examples

\{\(Xi,Yi\)\}i=1n,\(Xi,Yi\)∈𝒯×𝒯,\\\{\(X\_\{i\},Y\_\{i\}\)\\\}\_\{i=1\}^\{n\},\\qquad\(X\_\{i\},Y\_\{i\}\)\\in\\mathcal\{T\}\\times\\mathcal\{T\},whereXiX\_\{i\}is an input query andYiY\_\{i\}is the desired answer\. A language model with parametersθ\\thetadefines a stochastic policy over output text,

Y^∼pθ\(⋅∣X\)\.\\widehat\{Y\}\\sim p\_\{\\theta\}\(\\cdot\\mid X\)\.Under the standard autoregressive factorization,

pθ​\(Y^∣X\)=∏t=1Spθ​\(Y^t∣X,Y^<t\),p\_\{\\theta\}\(\\widehat\{Y\}\\mid X\)=\\prod\_\{t=1\}^\{S\}p\_\{\\theta\}\(\\widehat\{Y\}\_\{t\}\\mid X,\\widehat\{Y\}\_\{<t\}\),whereY^=\(Y^1,…,Y^S\)\\widehat\{Y\}=\(\\widehat\{Y\}\_\{1\},\\ldots,\\widehat\{Y\}\_\{S\}\)is the generated sequence andY^<t\\widehat\{Y\}\_\{<t\}denotes its prefix before tokentt\.

A generated answer is evaluated by a task\-specific reward

r​\(Y,Y^\)∈\[0,1\]\.r\(Y,\\widehat\{Y\}\)\\in\[0,1\]\.In the reasoning benchmarks considered in this paper, we use binary rewards based on answer correctness\. For exact\-match tasks,r​\(Y,Y^\)=𝟏​\{Y^=Y\},r\(Y,\\widehat\{Y\}\)=\\mathbf\{1\}\\\{\\widehat\{Y\}=Y\\\},after applying the task\-specific answer canonicalization described in the experimental protocol\.

For a single language policy, the empirical expected reward objective is

Jn​\(θ\)=1n​∑i=1n𝔼Y^i∼pθ\(⋅∣Xi\)​\[r​\(Yi,Y^i\)\]\.J\_\{n\}\(\\theta\)=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\mathbb\{E\}\_\{\\widehat\{Y\}\_\{i\}\\sim p\_\{\\theta\}\(\\cdot\\mid X\_\{i\}\)\}\\left\[r\(Y\_\{i\},\\widehat\{Y\}\_\{i\}\)\\right\]\.The observed input\-output pairs are treated as fixed, and the randomness comes from the sampled model outputs\.

Score\-function optimization\.Because generated text is discrete, the reward cannot be differentiated through the sampled tokens\. We therefore use the score\-function identity, also known as REINFORCE\. For a fixed example\(X,Y\)\(X,Y\),

∇θ𝔼Y^∼pθ\(⋅∣X\)​\[r​\(Y,Y^\)\]=𝔼Y^∼pθ\(⋅∣X\)​\[r​\(Y,Y^\)​∇θlog⁡pθ​\(Y^∣X\)\]\.\\nabla\_\{\\theta\}\\mathbb\{E\}\_\{\\widehat\{Y\}\\sim p\_\{\\theta\}\(\\cdot\\mid X\)\}\\left\[r\(Y,\\widehat\{Y\}\)\\right\]=\\mathbb\{E\}\_\{\\widehat\{Y\}\\sim p\_\{\\theta\}\(\\cdot\\mid X\)\}\\left\[r\(Y,\\widehat\{Y\}\)\\nabla\_\{\\theta\}\\log p\_\{\\theta\}\(\\widehat\{Y\}\\mid X\)\\right\]\.Equivalently, with any baselineb​\(X,Y\)b\(X,Y\)that does not depend on the sampled outputY^\\widehat\{Y\},

∇θ𝔼Y^∼pθ\(⋅∣X\)​\[r​\(Y,Y^\)\]=𝔼Y^∼pθ\(⋅∣X\)​\[\(r​\(Y,Y^\)−b​\(X,Y\)\)​∇θlog⁡pθ​\(Y^∣X\)\]\.\\nabla\_\{\\theta\}\\mathbb\{E\}\_\{\\widehat\{Y\}\\sim p\_\{\\theta\}\(\\cdot\\mid X\)\}\\left\[r\(Y,\\widehat\{Y\}\)\\right\]=\\mathbb\{E\}\_\{\\widehat\{Y\}\\sim p\_\{\\theta\}\(\\cdot\\mid X\)\}\\left\[\\bigl\(r\(Y,\\widehat\{Y\}\)\-b\(X,Y\)\\bigr\)\\nabla\_\{\\theta\}\\log p\_\{\\theta\}\(\\widehat\{Y\}\\mid X\)\\right\]\.The baseline changes the variance of the gradient estimator but not its expectation\. Implementation details, including the sampled surrogate loss, token\-level log\-probability computation, and baseline update rule, are provided in Appendix[A\.1](https://arxiv.org/html/2605.16757#A1.SS1)\.

From one policy to a system of policies\.NeuroMAS extends this single\-policy objective to a collection of stochastic language\-agent nodes\. A sampled trajectory now includes not only the final answerY^\\widehat\{Y\}, but also the intermediate text generated by hidden nodes\. Since these intermediate texts are also discrete, we do not backpropagate through their contents\. Instead, the system is optimized through the joint probability of the sampled trajectory: each node receives the same terminal rewardr​\(Y,Y^\)r\(Y,\\widehat\{Y\}\), while its parameters are updated according to the log\-probability of its own sampled output\. The next section defines this multi\-node trajectory distribution and the corresponding trainable language architecture\.

## 4Method

NeuroMAS treats a multi\-agent language system as a trainable*text\-valued neural architecture*\. Instead of generating an answer with a single language policy, NeuroMAS composes multiple stochastic language\-agent nodes\. Each hidden node receives an input context, generates text, and sends textual messages to downstream nodes\. The output node aggregates the final messages and generates the answer\. The analogy to a neural network is architectural rather than algebraic: information flows through layers of trainable modules, but the hidden states are discrete text sequences rather than continuous vectors\.

The method has three defining features\. First, nodes are*role\-free*: no node is manually assigned a semantic role such as planner, solver, critic, verifier, or judge\. Second, nodes are*structure\-aware*: each node is informed of its position in the architecture and the message format required by the topology\. Third, all node policies are trained from the same terminal reward on the final answer, so any useful specialization or coordination must arise because it improves task performance\.

Letτ=\(n1,…,nL\)\\tau=\(n\_\{1\},\\ldots,n\_\{L\}\)denote a layered topology withLLhidden layers, wherenℓn\_\{\\ell\}is the number of nodes in hidden layerℓ\\ell\. The topology is followed by a single output aggregation node\. Given an input queryXX, NeuroMAS induces a conditional distribution over final answers,

Y^∼πΘ\(⋅∣X;τ\),\\widehat\{Y\}\\sim\\pi\_\{\\Theta\}\(\\cdot\\mid X;\\tau\),whereΘ\\Thetadenotes all trainable node parameters\. The degenerate topologyτ=∅\\tau=\\emptysetrecovers a single language policy, which corresponds to the single\-model reinforcement\-learning baseline used in our experiments\.

### 4\.1Text\-Valued Layered Architecture

As illustrated in Figure[1](https://arxiv.org/html/2605.16757#S1.F1)\(a\), NeuroMAS is a feed\-forward layered architecture whose edges carry text\. We writevℓ,jv\_\{\\ell,j\}for thejj\-th hidden node in layerℓ\\ell, andvoutv\_\{\\mathrm\{out\}\}for the output aggregation node\.

Each node is a stochastic language policy\. For a hidden nodevℓ,jv\_\{\\ell,j\}, letCℓ,jC\_\{\\ell,j\}denote its input context and letZℓ,jZ\_\{\\ell,j\}denote the generated text:

Zℓ,j∼πθℓ,j\(⋅∣Cℓ,j\),ℓ=1,…,L,j=1,…,nℓ\.Z\_\{\\ell,j\}\\sim\\pi\_\{\\theta\_\{\\ell,j\}\}\(\\cdot\\mid C\_\{\\ell,j\}\),\\qquad\\ell=1,\\ldots,L,\\quad j=1,\\ldots,n\_\{\\ell\}\.The output node similarly generates the final answer

Y^∼πθout\(⋅∣Cout\)\.\\widehat\{Y\}\\sim\\pi\_\{\\theta\_\{\\mathrm\{out\}\}\}\(\\cdot\\mid C\_\{\\mathrm\{out\}\}\)\.Hereθℓ,j\\theta\_\{\\ell,j\}andθout\\theta\_\{\\mathrm\{out\}\}are the trainable parameters of the corresponding node policies\.

The forward pass proceeds layer by layer\. A first\-layer node has no incoming messages, so its input context is built from the original queryXX, its structural position, and the topologyτ\\tau\. For a later hidden nodevℓ,jv\_\{\\ell,j\}, the input context contains the original query together with all messages sent to it from the previous layer:

ℐℓ,j=\{Mℓ−1,k→ℓ,j:k=1,…,nℓ−1\},ℓ=2,…,L\.\\mathcal\{I\}\_\{\\ell,j\}=\\left\\\{M\_\{\\ell\-1,k\\rightarrow\\ell,j\}:k=1,\\ldots,n\_\{\\ell\-1\}\\right\\\},\\qquad\\ell=2,\\ldots,L\.A fixed node prompt template converts this information intoCℓ,jC\_\{\\ell,j\}\. The template provides the task input, the node’s layer and position, the topology, and the incoming messages, but it does not assign a semantic role such as planner, solver, critic, or verifier\.

After generatingZℓ,jZ\_\{\\ell,j\}, each non\-output node sends messages to the next layer\. The generated text follows a fixed structured format, and a deterministic parser extracts recipient\-specific messages\. Forℓ<L\\ell<L,

σℓ,j​\(Zℓ,j\)=\(Mℓ,j→ℓ\+1,1,…,Mℓ,j→ℓ\+1,nℓ\+1\),\\sigma\_\{\\ell,j\}\(Z\_\{\\ell,j\}\)=\\left\(M\_\{\\ell,j\\rightarrow\\ell\+1,1\},\\ldots,M\_\{\\ell,j\\rightarrow\\ell\+1,n\_\{\\ell\+1\}\}\\right\),whereMℓ,j→ℓ\+1,kM\_\{\\ell,j\\rightarrow\\ell\+1,k\}is the message sent from nodevℓ,jv\_\{\\ell,j\}to nodevℓ\+1,kv\_\{\\ell\+1,k\}\. For the final hidden layer, the parser extracts the message sent to the output aggregation node:

ML,j→out=σL,jout​\(ZL,j\)\.M\_\{L,j\\rightarrow\\mathrm\{out\}\}=\\sigma^\{\\mathrm\{out\}\}\_\{L,j\}\(Z\_\{L,j\}\)\.The output node then forms its input contextCoutC\_\{\\mathrm\{out\}\}from the original queryXX, the topologyτ\\tau, and the final\-layer messages

\{ML,j→out\}j=1nL,\\\{M\_\{L,j\\rightarrow\\mathrm\{out\}\}\\\}\_\{j=1\}^\{n\_\{L\}\},and generates the final answerY^\\widehat\{Y\}\. Thus, a NeuroMAS forward pass can be summarized as a stochastic text\-valued composition:

H0=X,Hℓ∼Φθℓ\(⋅∣X,Hℓ−1,τ\),ℓ=1,…,L,H\_\{0\}=X,\\qquad H\_\{\\ell\}\\sim\\Phi\_\{\\theta\_\{\\ell\}\}\(\\cdot\\mid X,H\_\{\\ell\-1\},\\tau\),\\quad\\ell=1,\\ldots,L,followed by

Y^∼πθout\(⋅∣X,HL,τ\),\\widehat\{Y\}\\sim\\pi\_\{\\theta\_\{\\mathrm\{out\}\}\}\(\\cdot\\mid X,H\_\{L\},\\tau\),whereHℓ=\(Zℓ,1,…,Zℓ,nℓ\)H\_\{\\ell\}=\(Z\_\{\\ell,1\},\\ldots,Z\_\{\\ell,n\_\{\\ell\}\}\)denotes the generated texts from all nodes in layerℓ\\ell, andΦθℓ\\Phi\_\{\\theta\_\{\\ell\}\}denotes the collection of node policies in that layer\. This composition emphasizes that the final answer is produced by layered text\-valued computation rather than by a single monolithic language policy\.

The prompt templates and parsers are fixed implementation components and are described in Appendix[A](https://arxiv.org/html/2605.16757#A1)\.

### 4\.2Role\-Free but Structure\-Aware Nodes

The main design choice in NeuroMAS is to separate*structural position*from*semantic role*\. Conventional multi\-agent workflows often define agents by roles such as planner, solver, critic, verifier, or judge\. NeuroMAS removes these hand\-authored role descriptions\. A node is specified only by its location in the topology, its incoming messages, and the output format required for downstream routing\.

This design makes the architecture role\-free but structure\-aware\. The topology determines what information each node can access and where its output can be sent\. Training determines how the node uses that information\. Thus, node specialization is not imposed by the prompt designer; it is induced by the interaction between architecture and terminal reward\. Two nodes with the same frozen backbone can learn different behavior because they occupy different structural positions, receive different messages, and have different trainable adapters\.

### 4\.3Joint Training from Terminal Reward

NeuroMAS communicates through sampled text\. Since intermediate messages are discrete, we do not backpropagate through message contents\. Instead, we view every node as a stochastic policy and optimize the full architecture using the terminal reward\. This extends the single\-policy REINFORCE formulation in Section[3](https://arxiv.org/html/2605.16757#S3)to a multi\-node computation trace\.

For a fixed input queryXX, let

ξ=\{\(Cℓ,j,Zℓ,j\):ℓ=1,…,L,j=1,…,nℓ\}∪\{\(Cout,Y^\)\}\\xi=\\left\\\{\(C\_\{\\ell,j\},Z\_\{\\ell,j\}\):\\ell=1,\\ldots,L,\\ j=1,\\ldots,n\_\{\\ell\}\\right\\\}\\cup\\\{\(C\_\{\\mathrm\{out\}\},\\widehat\{Y\}\)\\\}denote the full sampled computation trace\. Since each node input context is determined by the original query and upstream generated messages, the probability of the trace factorizes as

pΘ​\(ξ∣X\)=\[∏ℓ=1L∏j=1nℓπθℓ,j​\(Zℓ,j∣Cℓ,j\)\]​πθout​\(Y^∣Cout\)\.p\_\{\\Theta\}\(\\xi\\mid X\)=\\left\[\\prod\_\{\\ell=1\}^\{L\}\\prod\_\{j=1\}^\{n\_\{\\ell\}\}\\pi\_\{\\theta\_\{\\ell,j\}\}\(Z\_\{\\ell,j\}\\mid C\_\{\\ell,j\}\)\\right\]\\pi\_\{\\theta\_\{\\mathrm\{out\}\}\}\(\\widehat\{Y\}\\mid C\_\{\\mathrm\{out\}\}\)\.
The empirical expected reward objective is

Jn​\(Θ\)=1n​∑i=1n𝔼ξi∼pΘ\(⋅∣Xi\)​\[r​\(Yi,Y^i\)\],J\_\{n\}\(\\Theta\)=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\mathbb\{E\}\_\{\\xi\_\{i\}\\sim p\_\{\\Theta\}\(\\cdot\\mid X\_\{i\}\)\}\\left\[r\(Y\_\{i\},\\widehat\{Y\}\_\{i\}\)\\right\],\(1\)whereΘ=\{θℓ,j\}ℓ,j∪\{θout\}\\Theta=\\\{\\theta\_\{\\ell,j\}\\\}\_\{\\ell,j\}\\cup\\\{\\theta\_\{\\mathrm\{out\}\}\\\}collects all trainable node parameters\.

Applying the score\-function identity to the full trace gives REINFORCE\-style policy gradients\(Williams,[1992](https://arxiv.org/html/2605.16757#bib.bib1)\)\. For a sampled trajectory on\(X,Y\)\(X,Y\), all trainable nodes receive the same terminal rewardr​\(Y,Y^\)r\(Y,\\widehat\{Y\}\), but each node updates only its own parameters according to the log\-probability of its own sampled output\. Specifically, the gradient contribution for hidden nodevℓ,jv\_\{\\ell,j\}is

r​\(Y,Y^\)​∇θℓ,jlog⁡πθℓ,j​\(Zℓ,j∣Cℓ,j\),r\(Y,\\widehat\{Y\}\)\\nabla\_\{\\theta\_\{\\ell,j\}\}\\log\\pi\_\{\\theta\_\{\\ell,j\}\}\(Z\_\{\\ell,j\}\\mid C\_\{\\ell,j\}\),and the output node is updated analogously byr​\(Y,Y^\)​∇θoutlog⁡πθout​\(Y^∣Cout\)\.r\(Y,\\widehat\{Y\}\)\\nabla\_\{\\theta\_\{\\mathrm\{out\}\}\}\\log\\pi\_\{\\theta\_\{\\mathrm\{out\}\}\}\(\\widehat\{Y\}\\mid C\_\{\\mathrm\{out\}\}\)\.Thus, the parameters of different nodes are updated separately, but they are optimized under a shared system\-level objective and a shared terminal reward\. There are no intermediate labels, role\-specific rewards, or node\-specific supervision signals\. The sampled loss used to implement these node\-wise policy\-gradient updates is described in Appendix[A\.1](https://arxiv.org/html/2605.16757#A1.SS1)\.

This shared\-reward training design is consistent with recent multi\-agent reinforcement learning approaches for LLM collaboration\. For example,Liuet al\.\([2026](https://arxiv.org/html/2605.16757#bib.bib4)\)study centralized and decentralized value\-sharing strategies for multi\-agent LLM training, and find that sharing a common value signal across agents can be more effective in long\-horizon or sparse\-reward settings\. NeuroMAS follows the same cooperative principle in a simpler form: multiple nodes are trained from a shared global reward rather than from separately engineered node\-level rewards\.

#### Parameter\-Efficient Node Instantiation

The NeuroMAS formulation does not require a particular parameterization of the node policies\. In principle, each node could have an independent language model\. In our experiments, however, all nodes share the same frozen backbone model and differ only through node\-specific LoRA adapters\(Huet al\.,[2022](https://arxiv.org/html/2605.16757#bib.bib3)\)\. This keeps training feasible while allowing nodes to specialize according to their structural positions\.

Letπ0\\pi\_\{0\}denote the shared frozen backbone\. For each nodevv, we attach a node\-specific LoRA adapter with parametersαv\\alpha\_\{v\}\. The node policy is

πv\(⋅∣Cv\)=π0\(⋅∣Cv;αv\),\\pi\_\{v\}\(\\cdot\\mid C\_\{v\}\)=\\pi\_\{0\}\(\\cdot\\mid C\_\{v\};\\alpha\_\{v\}\),where the backbone parameters are fixed and onlyαv\\alpha\_\{v\}is trained\. The output aggregation node also has its own adapter\. Thus, a topologyτ=\(n1,…,nL\)\\tau=\(n\_\{1\},\\ldots,n\_\{L\}\)uses1\+∑ℓ=1Lnℓ1\+\\sum\_\{\\ell=1\}^\{L\}n\_\{\\ell\}node\-specific adapters and the same number of LLM calls per forward pass\. Further implementation details are provided in Appendix[A\.3](https://arxiv.org/html/2605.16757#A1.SS3)\.

### 4\.4Progressive Growth

We propose progressive growth as the scaling protocol used to expand a trained NeuroMAS topology into a larger one\. Rather than initializing the larger system entirely from scratch, we reuse the trainable parameters of nodes that have corresponding positions in the expanded topology and initialize only the newly added nodes\.

Suppose a trained topologyτ\\tauis expanded to a larger topologyτ′\\tau^\{\\prime\}\. For each node inτ\\tauthat is retained inτ′\\tau^\{\\prime\}, we copy its trainable parameters to the corresponding node in the expanded architecture\. Newly added nodes are initialized separately\. The expanded system is then trained further using the same terminal\-reward objective in Equation \([1](https://arxiv.org/html/2605.16757#S4.E1)\)\. In the LoRA implementation, reusing a node means copying its node\-specific adapter parameters\.

This protocol preserves the learned behavior of the smaller topology while giving the larger topology additional capacity for communication and specialization\. Detailed choices for inherited nodes, new\-node initialization, and continued training are provided in Appendix[A\.4](https://arxiv.org/html/2605.16757#A1.SS4)\.

## 5Theoretical analysis from a compositional\-efficiency perspective

We now give a theoretical perspective on why a trainable multi\-agent architecture can be parameter\-efficient\. The argument is not a training guarantee and does not claim that every multi\-agent system is better than every single\-agent system\. Instead, it isolates a representational advantage: when a task admits a stable hierarchical decomposition, a modular architecture can allocate parameters to simpler local components and compose their outputs, whereas an unstructured single policy must represent the full solution process at once\.

This perspective parallels classical results showing that neural networks can approximate compositional function classes more efficiently when their architecture exposes the underlying hierarchical composition structure\(Bauer and Kohler,[2019](https://arxiv.org/html/2605.16757#bib.bib55); Fan and Gu,[2024](https://arxiv.org/html/2605.16757#bib.bib62)\)\. NeuroMAS provides an analogous architecture for language generation: width allows different nodes to process different subproblems, depth allows intermediate messages to be combined or refined, and the output node aggregates the resulting textual states\.

#### Fixed\-input exact\-match error\.

We use the notation from Section[3](https://arxiv.org/html/2605.16757#S3)\. Consider a fixed inputX∈𝒯,X\\in\\mathcal\{T\},with a unique desired answerY∗​\(X\)∈𝒯\.Y^\{\*\}\(X\)\\in\\mathcal\{T\}\.A conditional generatorπ\(⋅∣X\)\\pi\(\\cdot\\mid X\)samples a candidate answerY^\\widehat\{Y\}\. We evaluate it by exact\-match error,

ℰX​\(π\)=PrY^∼π\(⋅∣X\)⁡\(Y^≠Y∗​\(X\)\)\.\\mathcal\{E\}\_\{X\}\(\\pi\)=\\Pr\_\{\\widehat\{Y\}\\sim\\pi\(\\cdot\\mid X\)\}\\left\(\\widehat\{Y\}\\neq Y^\{\*\}\(X\)\\right\)\.This fixed\-input formulation keeps the analysis conditional on a given problem instance\. The result below is a capacity statement in terms of trainable parameter budget, not a sample\-complexity or optimization theorem\.

LetΠmulti,q\\Pi\_\{\\mathrm\{multi\},q\}denote a class of modular multi\-agent generators with at mostqqtrainable parameters, and letΠsingle,q\\Pi\_\{\\mathrm\{single\},q\}denote a class of unstructured single\-agent generators with at mostqqtrainable parameters\. Define the smallest achievable exact\-match errors

εmulti,q=infπ∈Πmulti,qℰX​\(π\),εsingle,q=infπ∈Πsingle,qℰX​\(π\)\.\\varepsilon\_\{\\mathrm\{multi\},q\}=\\inf\_\{\\pi\\in\\Pi\_\{\\mathrm\{multi\},q\}\}\\mathcal\{E\}\_\{X\}\(\\pi\),\\qquad\\varepsilon\_\{\\mathrm\{single\},q\}=\\inf\_\{\\pi\\in\\Pi\_\{\\mathrm\{single\},q\}\}\\mathcal\{E\}\_\{X\}\(\\pi\)\.Thus,ℰX​\(π\)\\mathcal\{E\}\_\{X\}\(\\pi\)is the error of one generator, whileεmulti,q\\varepsilon\_\{\\mathrm\{multi\},q\}andεsingle,q\\varepsilon\_\{\\mathrm\{single\},q\}are the best errors achievable by the corresponding architecture classes under the same trainable parameter budget\.

We assume that the ideal solution process forXXhas a useful hierarchical decomposition\.

###### Assumption 1\(Hierarchical compositional solution\)\.

The ideal generatorπ∗\\pi^\{\*\}can be represented by composingKKsimpler generative components

ψ1∗,…,ψK∗,\\psi\_\{1\}^\{\*\},\\ldots,\\psi\_\{K\}^\{\*\},arranged in a parallel or hierarchical computation graph\.

Each component may correspond to a subquestion, an intermediate computation, an aggregation step, or a refinement step\. This assumption is natural for many reasoning, planning, mathematical, and coding tasks, but may fail for tasks that require one indivisible global judgment\.

###### Assumption 2\(Local learnability\)\.

For componentii, letηi​\(qi\)\\eta\_\{i\}\(q\_\{i\}\)be the smallest component\-level exact\-match error achievable withqiq\_\{i\}trainable parameters\. There exist constantsCloc\>0C\_\{\\mathrm\{loc\}\}\>0anda\>0a\>0such that

ηi​\(qi\)≤Cloc​qi−1/a\\eta\_\{i\}\(q\_\{i\}\)\\leq C\_\{\\mathrm\{loc\}\}q\_\{i\}^\{\-1/a\}for alli=1,…,Ki=1,\\ldots,K\.

The exponentaameasures the effective complexity of the local subtasks\. Smalleraacorresponds to easier local approximation\. The assumption says that decomposed components are learnable with increasing parameter budget\.

###### Assumption 3\(Stable composition\)\.

There exists a constantCstab\>0C\_\{\\mathrm\{stab\}\}\>0such that, if componentiiis approximated with error at mosteie\_\{i\}, then the composed generator has final exact\-match error at mostCstab​∑i=1Kei\.C\_\{\\mathrm\{stab\}\}\\sum\_\{i=1\}^\{K\}e\_\{i\}\.

This assumption requires local errors not to amplify too quickly when components are composed\. It is appropriate for stable aggregation or refinement, but may fail when a single early mistake can derail the entire solution\.

###### Theorem 5\.1\(Compositional parameter\-efficiency bound\)\.

Suppose Assumptions[1](https://arxiv.org/html/2605.16757#Thmassumption1)–[3](https://arxiv.org/html/2605.16757#Thmassumption3)hold\. Then the modular multi\-agent class satisfies

εmulti,q=O​\(K1\+1/a​q−1/a\)\.\\varepsilon\_\{\\mathrm\{multi\},q\}=O\\left\(K^\{1\+1/a\}q^\{\-1/a\}\\right\)\.

The theorem shows that a modular architecture pays a polynomial overhead in the number of componentsKK, but its error decreases according to the local subtask complexity exponentaa\. The proof is given in Appendix[B](https://arxiv.org/html/2605.16757#A2)\. The intuition is simple: allocate approximatelyq/Kq/Kparameters to each component\. By Assumption[2](https://arxiv.org/html/2605.16757#Thmassumption2), each component then has error on the order of\(q/K\)−1/a\(q/K\)^\{\-1/a\}\. By Assumption[3](https://arxiv.org/html/2605.16757#Thmassumption3), composing theKKcomponents gives total error on the order of

K​\(q/K\)−1/a=K1\+1/a​q−1/a\.K\(q/K\)^\{\-1/a\}=K^\{1\+1/a\}q^\{\-1/a\}\.
To compare with a single\-agent generator, we add a condition that the unstructured global problem is harder than the local components\.

###### Assumption 4\(Unstructured global difficulty\)\.

For the corresponding single\-agent class, there exist constantscsingle\>0c\_\{\\mathrm\{single\}\}\>0andb\>ab\>asuch that, for sufficiently largeqq,

εsingle,q≥csingle​q−1/b\.\\varepsilon\_\{\\mathrm\{single\},q\}\\geq c\_\{\\mathrm\{single\}\}q^\{\-1/b\}\.

This is a comparison assumption, not a universal claim about all single models\. It states that if the compositional structure is not exposed, the effective approximation exponent is worse\.

###### Corollary 1\(Comparison with a single\-agent generator\)\.

Suppose Assumptions[1](https://arxiv.org/html/2605.16757#Thmassumption1)–[4](https://arxiv.org/html/2605.16757#Thmassumption4)hold, andKKis fixed\. Then there existsq0\>0q\_\{0\}\>0such that, for allq≥q0q\\geq q\_\{0\},

εmulti,q<εsingle,q\.\\varepsilon\_\{\\mathrm\{multi\},q\}<\\varepsilon\_\{\\mathrm\{single\},q\}\.That is, under the stated compositional conditions, the best modular multi\-agent generator achieves smaller exact\-match error than the best unstructured single\-agent generator with the same sufficiently large trainable parameter budget\.

The fixed\-error view is equivalent\. Letqmulti​\(δ\)q\_\{\\mathrm\{multi\}\}\(\\delta\)andqsingle​\(δ\)q\_\{\\mathrm\{single\}\}\(\\delta\)denote the minimum trainable parameter budgets required to reach exact\-match error at mostδ\\delta\. Under the same assumptions,

qmulti​\(δ\)=O​\(K1\+a​δ−a\),qsingle​\(δ\)=Ω​\(δ−b\)\.q\_\{\\mathrm\{multi\}\}\(\\delta\)=O\\left\(K^\{1\+a\}\\delta^\{\-a\}\\right\),\\qquad q\_\{\\mathrm\{single\}\}\(\\delta\)=\\Omega\\left\(\\delta^\{\-b\}\\right\)\.Sinceb\>ab\>a, the modular generator requires fewer trainable parameters for sufficiently small target errorδ\\delta, up to constants and the polynomial overhead inKK\. The proof is given in Appendix[B](https://arxiv.org/html/2605.16757#A2)\.

#### Implications for NeuroMAS\.

This result explains why NeuroMAS can be parameter\-efficient when a task has useful compositional structure\. A wider topology can allocate different nodes to different intermediate computations, while a deeper topology can combine and refine textual states across layers\. Importantly, the theorem does not require these nodes to be assigned semantic roles by hand\. The graph only needs to expose a structure through which specialization is possible; training may then induce useful node behavior through the shared terminal reward\.

The limitations of the result are equally important\. It is an architectural capacity statement, not a guarantee that the empirical algorithm will discover the best decomposition\. The advantage may disappear if the task has no useful compositional structure, if the chosen topology fails to expose that structure, if the parameter budget is split too aggressively across weak nodes, or if a single model internally learns the same decomposition\. The result also concerns trainable parameter complexity rather than inference latency or total computation\. These caveats motivate the empirical study in the next section, where we test whether learned organization provides gains beyond single\-model reinforcement learning and parameter\-count\-matched controls\.

## 6Experiments

We design the experiments to evaluate three aspects of NeuroMAS\. First, we test whether learned multi\-agent organization improves over fixed\-backbone collaboration methods and single\-model reinforcement learning across reasoning, domain knowledge, and code\-generation tasks\. Second, we examine whether this advantage is robust to changing the backbone from Qwen3\-0\.6B to Gemma\-3\-1B\-IT\. Third, we study organizational scaling and show that larger topologies do not improve automatically, but can become effective when grown progressively from smaller trained systems\.

### 6\.1Experimental Setup

#### Backbone models and training\.

We use two frozen backbone models in our experiments: Qwen3\-0\.6B\(Yanget al\.,[2025a](https://arxiv.org/html/2605.16757#bib.bib6)\)for the main experiments and Gemma\-3\-1B\-IT\(Gemma Teamet al\.,[2025](https://arxiv.org/html/2605.16757#bib.bib68)\)for the backbone\-robustness study\. In all experiments, the backbone parameters are kept frozen\. Trainable parameters are introduced through LoRA adapters\. Trained methods are optimized with REINFORCE using the reward defined in Section[3](https://arxiv.org/html/2605.16757#S3)\. Evaluation uses greedy decoding, and accuracy is measured after canonicalizing outputs into the expected answer format\. Additional implementation details are provided in Appendix[C](https://arxiv.org/html/2605.16757#A3)\.

#### NeuroMAS settings\.

We writeNeuroMAS\-​c\\text\{NeuroMAS\-\}cfor a NeuroMAS topology that usescctotal language\-model calls per forward pass, including the output aggregation node\. Equivalently, if the hidden topology isτ=\(n1,…,nL\)\\tau=\(n\_\{1\},\\ldots,n\_\{L\}\), thenc=1\+∑ℓ=1Lnℓc=1\+\\sum\_\{\\ell=1\}^\{L\}n\_\{\\ell\}\. Thus, NeuroMAS\-3 corresponds toτ=\[1,1\]\\tau=\[1,1\], with two hidden nodes and one output node; NeuroMAS\-5 corresponds toτ=\[2,2\]\\tau=\[2,2\]; and NeuroMAS\-7 corresponds toτ=\[2,2,2\]\\tau=\[2,2,2\]\. Each node has its own LoRA adapter, so the number of adapters equals the number of LLM calls\.

#### Benchmarks\.

We evaluate on six tasks spanning reasoning and coding: ARC\-Challenge for science question answering\(Clarket al\.,[2018](https://arxiv.org/html/2605.16757#bib.bib7)\), Navigate for spatial navigation reasoning from the BIG\-Bench Hard problem set\(Suzgunet al\.,[2023](https://arxiv.org/html/2605.16757#bib.bib8)\), three subjects from Massive Multitask Language Understanding \(MMLU\), including Abstract Algebra for mathematical reasoning, College Physics for physical reasoning, and Professional Medicine for biomedical knowledge\(Hendryckset al\.,[2021](https://arxiv.org/html/2605.16757#bib.bib9)\), and HumanEval for code generation\(Chenet al\.,[2021](https://arxiv.org/html/2605.16757#bib.bib67)\)\. This suite places the relatively weak backbone in a nontrivial operating regime: direct prompting is far from saturated, but the model remains capable enough for reward\-based improvement to be measurable\.

#### Baselines\.

We compare against two groups of methods\. The first group consists of fixed\-backbone baselines, where the underlying LLM parameters remain frozen: direct prompting, Self\-Refine\(Madaanet al\.,[2023](https://arxiv.org/html/2605.16757#bib.bib13)\), Self\-Check\(Miaoet al\.,[2024](https://arxiv.org/html/2605.16757#bib.bib50)\), MoA\(Wanget al\.,[2025](https://arxiv.org/html/2605.16757#bib.bib19)\), GoA\(Jooet al\.,[2025](https://arxiv.org/html/2605.16757#bib.bib49)\), AgentNet\(Yanget al\.,[2025b](https://arxiv.org/html/2605.16757#bib.bib23)\), and GPTSwarm\(Zhugeet al\.,[2024](https://arxiv.org/html/2605.16757#bib.bib20)\)\. These baselines test whether NeuroMAS improves beyond prompt\-based, self\-refinement, and graph\-based collaboration without updating model parameters\. The second group consists of trained\-backbone baselines, where model parameters are updated during training: Single\-LLM RL, MALT\(Motwaniet al\.,[2025](https://arxiv.org/html/2605.16757#bib.bib24)\), CoLLM\-CC\(Liuet al\.,[2026](https://arxiv.org/html/2605.16757#bib.bib4)\), and NeuroMAS variants\. Single\-LLM RL uses the same REINFORCE training procedure as NeuroMAS but with a single LLM agent, and can be viewed as the one\-agent version of NeuroMAS\. These baselines test whether the gain comes from scalable multi\-agent organization rather than reinforcement learning alone\.

### 6\.2Main Results

Table[1](https://arxiv.org/html/2605.16757#S6.T1)reports accuracy and computational cost\. The most direct NeuroMAS comparison is NeuroMAS\-3, which uses two hidden nodes, one output node, three LoRA adapters, and three LLM calls per example\.

Table 1:Accuracy and computational cost across reasoning, domain knowledge, and code\-generation benchmarks\. NeuroMAS\-ccdenotes a NeuroMAS topology withccagents, including the output aggregation agent\. LLM Calls reports the number of model invocations used per input example\.Accuracy \(%\)Computation CostMethodARCNavigateAlgebraPhysicsMedicineHumanEvalTrainable ParamsLLM CallsFixed backbone modelsDirect prompting24\.540\.522\.017\.624\.011\.001Self\-Refine37\.540\.525\.024\.537\.521\.302Self\-Check38\.544\.526\.022\.532\.524\.402MoA34\.544\.524\.022\.540\.512\.205GoA39\.540\.528\.024\.534\.020\.703GPTSwarm46\.040\.525\.025\.543\.512\.8011AgentNet30\.540\.535\.024\.039\.018\.3011Trained backbone modelsMALT53\.543\.035\.031\.438\.029\.36\.9M3CoLLM\-CC55\.040\.041\.035\.333\.029\.54\.6M2Single\-LLM RL46\.042\.229\.024\.543\.017\.76\.9M1NeuroMAS\-356\.545\.539\.044\.148\.030\.56\.9M3NeuroMAS\-554\.048\.039\.039\.041\.529\.911\.5M5NeuroMAS\-753\.551\.042\.039\.543\.531\.716\.1M7NeuroMAS\-3 outperforms the strongest fixed\-backbone baseline on all six benchmarks\. Compared with the best fixed\-backbone result in each column, NeuroMAS\-3 improves by 10\.5 percentage points on ARC, 1\.0 on Navigate, 4\.0 on Algebra, 18\.6 on Physics, 4\.5 on Professional Medicine, and 6\.1 on HumanEval\. Because the fixed\-backbone baselines use prompting, refinement, sampling, or graph\-based orchestration without updating model parameters, these gains show that NeuroMAS provides benefits beyond repeatedly querying the same frozen model\.

NeuroMAS\-3 also outperforms the single\-model RL control on every benchmark in Table[1](https://arxiv.org/html/2605.16757#S6.T1)\. This is a key matched comparison: Single\-LLM RL and NeuroMAS\-3 use the same frozen backbone, the same reinforcement\-learning objective, and the same trainable\-parameter budget of 6\.9M LoRA parameters\. The improvements are 10\.5 percentage points on ARC, 3\.3 on Navigate, 10\.0 on Algebra, 19\.6 on Physics, 5\.0 on Professional Medicine, and 12\.8 on HumanEval\. The difference lies in the organization of computation: Single\-LLM RL uses one model call, whereas NeuroMAS\-3 distributes computation across two hidden nodes and one output node\. This parameter\-matched improvement therefore supports the claim that learned multi\-agent organization contributes capability beyond reinforcement learning alone\.

Among trained baselines with comparable call budgets, NeuroMAS\-3 obtains the strongest results on ARC, Navigate, Physics, Professional Medicine, and HumanEval\. CoLLM\-CC performs better on Algebra than NeuroMAS\-3, while NeuroMAS\-7 achieves the best Algebra result among all reported methods\. Across the full table, a NeuroMAS variant obtains the best result on every benchmark: NeuroMAS\-3 on ARC, Physics, and Professional Medicine, and NeuroMAS\-7 on Navigate, Algebra, and HumanEval\. We therefore do not interpret Table[1](https://arxiv.org/html/2605.16757#S6.T1)as showing that one fixed topology universally dominates\. The consistent pattern is instead that NeuroMAS improves over prompt\-only collaboration and single\-model RL, while larger NeuroMAS topologies can provide further gains when organizational scaling is successfully optimized\.

### 6\.3Backbone Robustness

To test whether the gains of NeuroMAS depend on a specific backbone, we repeat the ARC\-Challenge evaluation using Gemma\-3\-1B\-IT\(Gemma Teamet al\.,[2025](https://arxiv.org/html/2605.16757#bib.bib68)\)\. Table[2](https://arxiv.org/html/2605.16757#S6.T2)compares fixed\-backbone methods, which keep the underlying LLM parameters frozen, with trained\-backbone methods, which optimize trainable adapters during post\-training\.

Table 2:Backbone robustness results on ARC\-Challenge using Gemma\-3\-1B\-IT as the backbone\(Gemma Teamet al\.,[2025](https://arxiv.org/html/2605.16757#bib.bib68)\)\. Fixed\-backbone methods do not update the underlying LLM parameters, while trained\-backbone methods optimize trainable adapters during post\-training\.NeuroMAS remains effective under the Gemma backbone\. Among fixed\-backbone methods, GPTSwarm obtains the strongest accuracy at 43\.0%, but requires 11 LLM calls\. Self\-Refine is also competitive, reaching 41\.0% accuracy with two calls\. NeuroMAS\-3 reaches 44\.5% accuracy with three LLM calls, outperforming the strongest fixed\-backbone baseline while using substantially fewer calls than GPTSwarm\.

Among trained\-backbone methods, NeuroMAS\-3 achieves the best result\. It outperforms MALT by 9\.5 percentage points, CoLLM\-CC by 4\.0 points, and Single\-LLM RL by 8\.5 points\. NeuroMAS\-5 remains close at 44\.0%, indicating that the result is not isolated to one topology\. These results suggest that the advantage of learned multi\-agent organization is not specific to Qwen3\-0\.6B, and also appears when the backbone is changed to Gemma\-3\-1B\-IT\.

Increasing the number of agents does not monotonically improve performance in this setting\. NeuroMAS\-5 remains close to NeuroMAS\-3, while NeuroMAS\-7 drops to 41\.0%\. This pattern is consistent with the main results: larger organizations can help, but scaling depends on the interaction among topology, optimization, and training protocol\.

### 6\.4Ablation Study: Progressive Growth

We conduct an ablation study to examine whether larger NeuroMAS topologies benefit from progressive growth rather than from increased topology size alone\. Table[3](https://arxiv.org/html/2605.16757#S6.T3)compares two training protocols on Navigate, where the full sequence of NeuroMAS\-3, NeuroMAS\-5, and NeuroMAS\-7 is evaluated\. The from\-scratch protocol initializes each topology directly, whereas progressive growth expands a trained smaller topology by transferring inherited adapters, initializing newly added nodes separately, and continuing training with the same terminal\-reward objective\.

Table 3:Ablation of progressive growth on Navigate\. From\-scratch training initializes each topology directly\. Progressive growth expands a trained smaller topology by transferring inherited adapters, initializing new nodes separately, and continuing training under the same terminal\-reward objective\.The ablation shows that larger topology size alone does not guarantee better performance\. When trained from scratch, accuracy drops from 45\.5% for NeuroMAS\-3 to 41\.0% for NeuroMAS\-5 and 40\.5% for NeuroMAS\-7\. In contrast, progressive growth improves performance from 45\.5% to 48\.0% and then to 51\.0%\. The gains are substantial for larger systems: progressive growth improves NeuroMAS\-5 by 7\.0 percentage points and NeuroMAS\-7 by 10\.5 percentage points over from\-scratch training\. These results suggest that scaling NeuroMAS depends not only on the final topology, but also on the optimization path used to reach it\. Progressive growth provides a stable learned core around which larger organizations can be expanded\.

#### Summary of findings\.

The experiments support three conclusions\. First, learned multi\-agent organization improves over prompt\-engineered collaboration and single\-model reinforcement learning in a weak\-backbone regime\. Second, the parameter\-matched comparison with Single\-LLM RL in Table[1](https://arxiv.org/html/2605.16757#S6.T1)suggests that the gains are not merely due to adding more LoRA parameters\. Third, the progressive\-growth ablation shows that organizational scaling is path\-dependent: larger systems can degrade when trained from scratch, but improve when expanded from smaller trained systems\.

## 7Discussion

We introduced NeuroMAS, a framework that treats a multi\-agent system as a trainable architecture with textual intermediate messages\. Rather than assigning agents hand\-written semantic roles, NeuroMAS uses role\-free but structure\-aware nodes connected by text\-carrying edges\. Each node is adapted through lightweight trainable parameters, and the entire system is optimized at the trajectory level from the same terminal reward on the final answer\. In a weak\-backbone reasoning regime, this design improves over prompt\-only collaboration and single\-model reinforcement learning, suggesting that the organization around a language model can itself be a source of capability\.

The experiments support three main conclusions\. First, effective multi\-agent behavior does not require manually specified roles such as planner, solver, critic, or verifier\. A layered architecture with structurally positioned nodes is sufficient to induce useful specialization through reward\-based training\. Second, the gains are not explained by trainable parameter count alone: a parameter\-matched single\-model adapter remains weaker than NeuroMAS\-3 on the dedicated parameter\-control tasks\. This suggests that distributing computation across interacting modules is not equivalent to placing the same adapter budget into one larger single\-model policy\. Third, organizational scaling is path\-dependent\. Larger topologies can degrade when trained from scratch, but improve when grown progressively from a smaller trained system\. Thus, scaling a multi\-agent organization is not merely a question of adding more nodes; it is also an optimization problem\.

These findings point to a broader view of multi\-agent language systems\. Existing MAS methods often treat organization as workflow engineering: roles, communication rules, and aggregation mechanisms are designed by humans\. NeuroMAS instead treats organization as an architectural object\. The topology defines possible routes of information flow, while training determines how nodes use those routes\. This perspective makes multi\-agent systems closer to neural architectures, where depth, width, connectivity, and growth protocol shape the computation performed by the model\.

The results also clarify what NeuroMAS does not claim\. The framework is not evidence that more agents are always better\. The advantage depends on the task having useful decomposable structure, the topology exposing that structure, and optimization successfully inducing coordination among nodes\. The current experiments use a small frozen backbone, exact\-match reasoning benchmarks, and relatively small evaluation subsets\. In addition, NeuroMAS increases inference cost because each forward pass requires multiple LLM calls\. These limitations mean that the present results should be interpreted as evidence for learned organization as a promising scaling axis, not as a universal replacement for single\-model scaling\.

## Acknowledgments

This work was partially supported by the U\.S\. National Science Foundation DMS\-1925066, DMS\-1903226, DMS\-2124493, DMS\-2311297, DMS\-2319279, DMS\-2318809 and the National Institutes of Health NIH R01GM152814\.

## References

- Anthropic \(2025\)Claude code\.Note:[https://www\.anthropic\.com/product/claude\-code](https://www.anthropic.com/product/claude-code)Accessed: 2026\-05\-09Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p1.1)\.
- B\. Bauer and M\. Kohler \(2019\)On deep learning as a remedy for the curse of dimensionality in nonparametric regression\.The Annals of Statistics47\(4\),pp\. 2261–2285\.External Links:[Document](https://dx.doi.org/10.1214/18-AOS1747)Cited by:[§5](https://arxiv.org/html/2605.16757#S5.p2.1)\.
- T\. B\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell,et al\.\(2020\)Language models are few\-shot learners\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 1877–1901\.Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p1.1)\.
- M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. d\. O\. Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman,et al\.\(2021\)Evaluating large language models trained on code\.arXiv preprint arXiv:2107\.03374\.Cited by:[§6\.1](https://arxiv.org/html/2605.16757#S6.SS1.SSS0.Px3.p1.1)\.
- A\. Chowdhery, S\. Narang, J\. Devlin, M\. Bosma, G\. Mishra, A\. Roberts, P\. Barham, H\. W\. Chung, C\. Sutton, S\. Gehrmann,et al\.\(2023\)PaLM: scaling language modeling with pathways\.Journal of Machine Learning Research24\(240\),pp\. 1–113\.Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p1.1)\.
- P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. Tafjord \(2018\)Think you have solved question answering? try ARC, the AI2 reasoning challenge\.arXiv preprint arXiv:1803\.05457\.Cited by:[§6\.1](https://arxiv.org/html/2605.16757#S6.SS1.SSS0.Px3.p1.1)\.
- Y\. Du, S\. Li, A\. Torralba, J\. B\. Tenenbaum, and I\. Mordatch \(2024\)Improving factuality and reasoning in language models through multiagent debate\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 11733–11763\.Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p2.1)\.
- T\. Elsken, J\. H\. Metzen, and F\. Hutter \(2019\)Neural architecture search: a survey\.Journal of Machine Learning Research20\(55\),pp\. 1–21\.Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p5.1)\.
- J\. Fan and Y\. Gu \(2024\)Factor augmented sparse throughput deep ReLU neural networks for high dimensional regression\.Journal of the American Statistical Association119\(548\),pp\. 2680–2694\.Cited by:[§5](https://arxiv.org/html/2605.16757#S5.p2.1)\.
- W\. Fedus, B\. Zoph, and N\. Shazeer \(2022\)Switch transformers: scaling to trillion parameter models with simple and efficient sparsity\.Journal of Machine Learning Research23\(120\),pp\. 1–39\.Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p1.1)\.
- Gemma Team, A\. Kamath, J\. Ferret, S\. Pathak, N\. Vieillard, R\. Merhej, S\. Perrin,et al\.\(2025\)Gemma 3 technical report\.arXiv preprint arXiv:2503\.19786\.Cited by:[§6\.1](https://arxiv.org/html/2605.16757#S6.SS1.SSS0.Px1.p1.1),[§6\.3](https://arxiv.org/html/2605.16757#S6.SS3.p1.1),[Table 2](https://arxiv.org/html/2605.16757#S6.T2)\.
- Google DeepMind \(2025\)Advanced version of Gemini with deep think officially achieves gold\-medal standard at the international mathematical olympiad\.Note:[https://deepmind\.google/blog/advanced\-version\-of\-gemini\-with\-deep\-think\-officially\-achieves\-gold\-medal\-standard\-at\-the\-international\-mathematical\-olympiad/](https://deepmind.google/blog/advanced-version-of-gemini-with-deep-think-officially-achieves-gold-medal-standard-at-the-international-mathematical-olympiad/)Accessed: 2026\-05\-09Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p1.1)\.
- K\. He, X\. Zhang, S\. Ren, and J\. Sun \(2016\)Deep residual learning for image recognition\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition,pp\. 770–778\.Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p5.1)\.
- D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. Steinhardt \(2021\)Measuring massive multitask language understanding\.InInternational Conference on Learning Representations,Cited by:[§6\.1](https://arxiv.org/html/2605.16757#S6.SS1.SSS0.Px3.p1.1)\.
- J\. Hoffmann, S\. Borgeaud, A\. Mensch, E\. Buchatskaya, T\. Cai, E\. Rutherford, D\. de Las Casas, L\. A\. Hendricks, J\. Welbl, A\. Clark, T\. Hennigan, E\. Noland, K\. Millican, G\. van den Driessche, B\. Damoc, A\. Guy, S\. Osindero, K\. Simonyan, E\. Elsen, J\. W\. Rae, O\. Vinyals, and L\. Sifre \(2022\)Training compute\-optimal large language models\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 30016–30030\.Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p1.1)\.
- S\. Hong, M\. Zhuge, J\. Chen, X\. Zheng, Y\. Cheng, J\. Wang, C\. Zhang, Z\. Wang, S\. K\. S\. Yau, Z\. Lin, L\. Zhou, C\. Ran, L\. Xiao, C\. Wu, and J\. Schmidhuber \(2024\)MetaGPT: meta programming for a multi\-agent collaborative framework\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p2.1),[§1](https://arxiv.org/html/2605.16757#S1.p3.1),[§2](https://arxiv.org/html/2605.16757#S2.p2.1)\.
- E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen \(2022\)LoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations,Cited by:[§4\.3](https://arxiv.org/html/2605.16757#S4.SS3.SSS0.Px1.p1.1)\.
- T\. Joo, S\. Ishida, I\. Sosnovik, B\. Lim, S\. Rezaei\-Shoshtari, A\. Gaier, and R\. Giaquinto \(2025\)Graph of agents: principled long context modeling by emergent multi\-agent collaboration\.arXiv preprint arXiv:2509\.21848\.Cited by:[§2](https://arxiv.org/html/2605.16757#S2.p3.1),[§6\.1](https://arxiv.org/html/2605.16757#S6.SS1.SSS0.Px4.p1.1)\.
- J\. Kaplan, S\. McCandlish, T\. Henighan, T\. B\. Brown, B\. Chess, R\. Child, S\. Gray, A\. Radford, J\. Wu, and D\. Amodei \(2020\)Scaling laws for neural language models\.arXiv preprint arXiv:2001\.08361\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2001.08361)Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p1.1)\.
- Y\. LeCun, L\. Bottou, Y\. Bengio, and P\. Haffner \(1998\)Gradient\-based learning applied to document recognition\.Proceedings of the IEEE86\(11\),pp\. 2278–2324\.Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p5.1)\.
- J\. Li, Q\. Zhang, Y\. Yu, Q\. Fu, and D\. Ye \(2024\)More agents is all you need\.Transactions on Machine Learning Research\.Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p2.1),[§2](https://arxiv.org/html/2605.16757#S2.p2.1)\.
- S\. Liu, T\. Chen, R\. Amiri, and C\. Amato \(2026\)Learning decentralized LLM collaboration with multi\-agent actor critic\.arXiv preprint arXiv:2601\.21972\.Cited by:[§2](https://arxiv.org/html/2605.16757#S2.p4.1),[§4\.3](https://arxiv.org/html/2605.16757#S4.SS3.p5.1),[§6\.1](https://arxiv.org/html/2605.16757#S6.SS1.SSS0.Px4.p1.1)\.
- X\. Liu, H\. Yu, H\. Zhang, Y\. Xu, X\. Lei, H\. Lai, Y\. Gu, H\. Ding, K\. Men, K\. Yang,et al\.\(2023\)AgentBench: evaluating LLMs as agents\.arXiv preprint arXiv:2308\.03688\.Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p1.1)\.
- A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang, S\. Gupta, B\. P\. Majumder, K\. Hermann, S\. Welleck, A\. Yazdanbakhsh, and P\. Clark \(2023\)Self\-Refine: iterative refinement with self\-feedback\.InAdvances in Neural Information Processing Systems,Vol\.36,pp\. 46534–46562\.Cited by:[§2](https://arxiv.org/html/2605.16757#S2.p1.1),[§6\.1](https://arxiv.org/html/2605.16757#S6.SS1.SSS0.Px4.p1.1)\.
- N\. Miao, Y\. W\. Teh, and T\. Rainforth \(2024\)SelfCheck: using LLMs to zero\-shot check their own step\-by\-step reasoning\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2605.16757#S2.p1.1),[§6\.1](https://arxiv.org/html/2605.16757#S6.SS1.SSS0.Px4.p1.1)\.
- S\. R\. Motwani, C\. Smith, R\. J\. Das, R\. Rafailov, P\. H\. S\. Torr, I\. Laptev, F\. Pizzati, R\. Clark, and C\. Schroeder de Witt \(2025\)MALT: improving reasoning with multi\-agent LLM training\.InProceedings of the Second Conference on Language Modeling,Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p3.1),[§2](https://arxiv.org/html/2605.16757#S2.p4.1),[§6\.1](https://arxiv.org/html/2605.16757#S6.SS1.SSS0.Px4.p1.1)\.
- Nous Research \(2026\)Hermes Agent: open\-source autonomous AI Agent\.Note:[https://hermes\-agent\.org/](https://hermes-agent.org/)Accessed: 2026\-05\-15Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p1.1)\.
- OpenAI \(2023\)GPT\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p1.1)\.
- OpenAI \(2025\)Introducing codex\.Note:[https://openai\.com/index/introducing\-codex/](https://openai.com/index/introducing-codex/)Accessed: 2026\-05\-09Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p1.1)\.
- OpenClaw \(2026\)OpenClaw: personal AI assistant\.Note:[https://openclaw\.ai/](https://openclaw.ai/)Accessed: 2026\-05\-09Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p1.1)\.
- C\. Qian, W\. Liu, H\. Liu, N\. Chen, Y\. Dang, J\. Li, C\. Yang, W\. Chen, Y\. Su, X\. Cong, J\. Xu, D\. Li, Z\. Liu, and M\. Sun \(2024\)ChatDev: communicative agents for software development\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Bangkok, Thailand,pp\. 15174–15186\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.810)Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p2.1),[§1](https://arxiv.org/html/2605.16757#S1.p3.1),[§2](https://arxiv.org/html/2605.16757#S2.p2.1)\.
- N\. Sardana, J\. Portes, S\. Doubov, and J\. Frankle \(2024\)Beyond chinchilla\-optimal: accounting for inference in language model scaling laws\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 43445–43460\.Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p1.1)\.
- N\. Shazeer, A\. Mirhoseini, K\. Maziarz, A\. Davis, Q\. V\. Le, G\. E\. Hinton, and J\. Dean \(2017\)Outrageously large neural networks: the sparsely\-gated mixture\-of\-experts layer\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p5.1)\.
- R\. S\. Sutton \(2019\)The Bitter Lesson\.Note:[http://www\.incompleteideas\.net/IncIdeas/BitterLesson\.html](http://www.incompleteideas.net/IncIdeas/BitterLesson.html)Online essayCited by:[§1](https://arxiv.org/html/2605.16757#S1.p5.1)\.
- M\. Suzgun, N\. Scales, N\. Schärli, S\. Gehrmann, Y\. Tay, H\. W\. Chung, A\. Chowdhery, Q\. V\. Le, E\. H\. Chi, D\. Zhou, and J\. Wei \(2023\)Challenging BIG\-Bench tasks and whether chain\-of\-thought can solve them\.InFindings of the Association for Computational Linguistics: ACL 2023,pp\. 13003–13051\.Cited by:[§6\.1](https://arxiv.org/html/2605.16757#S6.SS1.SSS0.Px3.p1.1)\.
- A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin \(2017\)Attention is all you need\.InAdvances in Neural Information Processing Systems,Vol\.30\.Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p5.1)\.
- G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. Anandkumar \(2023a\)Voyager: an open\-ended embodied agent with large language models\.arXiv preprint arXiv:2305\.16291\.Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p1.1)\.
- J\. Wang, J\. Wang, B\. Athiwaratkun, C\. Zhang, and J\. Zou \(2025\)Mixture\-of\-agents enhances large language model capabilities\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p3.1),[§2](https://arxiv.org/html/2605.16757#S2.p2.1),[§6\.1](https://arxiv.org/html/2605.16757#S6.SS1.SSS0.Px4.p1.1)\.
- X\. Wang, J\. Wei, D\. Schuurmans, Q\. V\. Le, E\. H\. Chi, S\. Narang, A\. Chowdhery, and D\. Zhou \(2023b\)Self\-consistency improves chain of thought reasoning in language models\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2605.16757#S2.p1.1)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. H\. Chi, Q\. V\. Le, and D\. Zhou \(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 24824–24837\.Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p2.1),[§2](https://arxiv.org/html/2605.16757#S2.p1.1)\.
- R\. J\. Williams \(1992\)Simple statistical gradient\-following algorithms for connectionist reinforcement learning\.Machine Learning8\(3\),pp\. 229–256\.External Links:[Document](https://dx.doi.org/10.1007/BF00992696)Cited by:[§4\.3](https://arxiv.org/html/2605.16757#S4.SS3.p4.3)\.
- Q\. Wu, G\. Bansal, J\. Zhang, Y\. Wu, B\. Li, E\. Zhu, L\. Jiang, X\. Zhang, S\. Zhang, J\. Liu, A\. H\. Awadallah, R\. W\. White, D\. Burger, and C\. Wang \(2024\)AutoGen: enabling next\-gen LLM applications via multi\-agent conversations\.InProceedings of the First Conference on Language Modeling,Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p2.1),[§1](https://arxiv.org/html/2605.16757#S1.p3.1),[§2](https://arxiv.org/html/2605.16757#S2.p2.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng,et al\.\(2025a\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§6\.1](https://arxiv.org/html/2605.16757#S6.SS1.SSS0.Px1.p1.1)\.
- Y\. Yang, H\. Chai, S\. Shao, Y\. Song, S\. Qi, R\. Rui, and W\. Zhang \(2025b\)AgentNet: decentralized evolutionary coordination for LLM\-based multi\-agent systems\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p3.1),[§2](https://arxiv.org/html/2605.16757#S2.p3.1),[§6\.1](https://arxiv.org/html/2605.16757#S6.SS1.SSS0.Px4.p1.1)\.
- S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. L\. Griffiths, Y\. Cao, and K\. Narasimhan \(2023a\)Tree of thoughts: deliberate problem solving with large language models\.InAdvances in Neural Information Processing Systems,Vol\.36\.Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p2.1),[§2](https://arxiv.org/html/2605.16757#S2.p1.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao \(2023b\)ReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p1.1)\.
- D\. Zhou, N\. Schärli, L\. Hou, J\. Wei, N\. Scales, X\. Wang, D\. Schuurmans, C\. Cui, O\. Bousquet, Q\. V\. Le, and E\. H\. Chi \(2023\)Least\-to\-most prompting enables complex reasoning in large language models\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p2.1)\.
- H\. Zhou, X\. Wan, R\. Sun, H\. Palangi, S\. Iqbal, I\. Vulić, A\. Korhonen, and S\. O\. Arik \(2026\)Multi\-agent design: optimizing agents with better prompts and topologies\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p2.1),[§1](https://arxiv.org/html/2605.16757#S1.p3.1),[§2](https://arxiv.org/html/2605.16757#S2.p3.1)\.
- M\. Zhuge, W\. Wang, L\. Kirsch, F\. Faccio, D\. Khizbullin, and J\. Schmidhuber \(2024\)GPTSwarm: language agents as optimizable graphs\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 62743–62767\.Cited by:[§1](https://arxiv.org/html/2605.16757#S1.p2.1),[§1](https://arxiv.org/html/2605.16757#S1.p3.1),[§2](https://arxiv.org/html/2605.16757#S2.p3.1),[§6\.1](https://arxiv.org/html/2605.16757#S6.SS1.SSS0.Px4.p1.1)\.

Appendix for “NeuroMAS: Multi\-Agent Systems as Neural Networks”

## Contents

- •Appendix[A](https://arxiv.org/html/2605.16757#A1): Additional Method Details
- •Appendix[B](https://arxiv.org/html/2605.16757#A2): Proofs for the Theoretical Results
- •Appendix[C](https://arxiv.org/html/2605.16757#A3): Additional Experimental Details

## Appendix AAdditional Method Details

### A\.1Policy\-Gradient Implementation

This section describes how the score\-function objective in Sections[3](https://arxiv.org/html/2605.16757#S3)and[4\.3](https://arxiv.org/html/2605.16757#S4.SS3)is implemented\. For a single language policy, the REINFORCE identity can be implemented by minimizing the sampled surrogate loss

ℒ​\(θ\)=−\(r​\(Y,Y^\)−b​\(X,Y\)\)​log⁡pθ​\(Y^∣X\),\\mathcal\{L\}\(\\theta\)=\-\\bigl\(r\(Y,\\widehat\{Y\}\)\-b\(X,Y\)\\bigr\)\\log p\_\{\\theta\}\(\\widehat\{Y\}\\mid X\),\(2\)whereb​\(X,Y\)b\(X,Y\)is a baseline that does not depend on the sampled outputY^\\widehat\{Y\}\. The baseline reduces variance without changing the expectation of the gradient estimator\.

For autoregressive language generation, the sequence log\-probability is

log⁡pθ​\(Y^∣X\)=∑t=1Slog⁡pθ​\(Y^t∣X,Y^<t\)\.\\log p\_\{\\theta\}\(\\widehat\{Y\}\\mid X\)=\\sum\_\{t=1\}^\{S\}\\log p\_\{\\theta\}\(\\widehat\{Y\}\_\{t\}\\mid X,\\widehat\{Y\}\_\{<t\}\)\.In implementation, we use the mean token log\-probability

ℓ¯=1S​∑t=1Slog⁡pθ​\(Y^t∣X,Y^<t\),\\bar\{\\ell\}=\\frac\{1\}\{S\}\\sum\_\{t=1\}^\{S\}\\log p\_\{\\theta\}\(\\widehat\{Y\}\_\{t\}\\mid X,\\widehat\{Y\}\_\{<t\}\),which reduces sensitivity to generated sequence length\.

For NeuroMAS, the same construction is applied node\-wise\. For a hidden nodevℓ,jv\_\{\\ell,j\}with input contextCℓ,jC\_\{\\ell,j\}and generated textZℓ,jZ\_\{\\ell,j\}, define

ℓ¯ℓ,j=1Sℓ,j​∑t=1Sℓ,jlog⁡πθℓ,j​\(Zℓ,j,t∣Cℓ,j,Zℓ,j,<t\)\.\\bar\{\\ell\}\_\{\\ell,j\}=\\frac\{1\}\{S\_\{\\ell,j\}\}\\sum\_\{t=1\}^\{S\_\{\\ell,j\}\}\\log\\pi\_\{\\theta\_\{\\ell,j\}\}\\left\(Z\_\{\\ell,j,t\}\\mid C\_\{\\ell,j\},Z\_\{\\ell,j,<t\}\\right\)\.The node\-level surrogate loss is

ℒℓ,j=−\(r​\(Y,Y^\)−bℓ,j\)​ℓ¯ℓ,j\.\\mathcal\{L\}\_\{\\ell,j\}=\-\\bigl\(r\(Y,\\widehat\{Y\}\)\-b\_\{\\ell,j\}\\bigr\)\\bar\{\\ell\}\_\{\\ell,j\}\.The output node is treated analogously:

ℒout=−\(r​\(Y,Y^\)−bout\)​ℓ¯out\.\\mathcal\{L\}\_\{\\mathrm\{out\}\}=\-\\bigl\(r\(Y,\\widehat\{Y\}\)\-b\_\{\\mathrm\{out\}\}\\bigr\)\\bar\{\\ell\}\_\{\\mathrm\{out\}\}\.The total sampled loss for one trajectory is

ℒ=∑ℓ=1L∑j=1nℓℒℓ,j\+ℒout\.\\mathcal\{L\}=\\sum\_\{\\ell=1\}^\{L\}\\sum\_\{j=1\}^\{n\_\{\\ell\}\}\\mathcal\{L\}\_\{\\ell,j\}\+\\mathcal\{L\}\_\{\\mathrm\{out\}\}\.\(3\)Thus, all nodes are weighted by the same terminal reward, but each node contributes the log\-probability of its own generated text\.

In our experiments, each node baseline is maintained as an exponential moving average of observed rewards,

bv←ρ​bv\+\(1−ρ\)​r​\(Y,Y^\),b\_\{v\}\\leftarrow\\rho b\_\{v\}\+\(1\-\\rho\)r\(Y,\\widehat\{Y\}\),withρ=0\.9\\rho=0\.9\. The resulting loss is optimized with AdamW\.

### A\.2Prompt Templates and Message Parsing

This section provides the fixed prompt templates and parsing rules used by NeuroMAS\. The templates are not learned\. They provide the original query, structural information, and incoming messages, but do not assign semantic roles such as planner, solver, critic, verifier, or judge\. In the experiments, the topologyτ\\taudenotes a layered fully connected feed\-forward communication graph, analogous to the layer structure of an MLP\. Nodes are organized into hidden layers, and every node in layerℓ\\ellsends a message to every node in layerℓ\+1\\ell\+1\. The width of each layer specifies the topology, such as2​\-​22\\text\{\-\}2for two hidden layers with two nodes per layer\.

#### Hidden\-node prompt\.

For a hidden nodevℓ,jv\_\{\\ell,j\}, the input contextCℓ,jC\_\{\\ell,j\}is constructed from the original queryXX, the topologyτ\\tau, the node position\(ℓ,j\)\(\\ell,j\), and the messages received from all nodes in the previous layer\. For first\-layer nodes, the incoming\-message field is omitted\. The hidden\-node prompt template is:

`Hidden\-node prompt template For first\-layer nodes, the PREVIOUS block is omitted because there are no incoming hidden\-layer messages\. The number of TO \#k fields is determined by the number of nodes in the next layer\.`

`Output\-node prompt\. The output node receives the original query XX, the topology τ\\tau, and the messages from the final hidden layer\. The output\-node prompt template is: Output\-node prompt template The task\-specific answer footer enforces the required output format\. For ARC\-Challenge, MMLU\-Abstract Algebra, MMLU\-College Physics, and MMLU\-Professional Medicine, the footer is: Multiple\-choice answer footer For BBH\-Navigate, the footer is: Yes/no answer footer For HumanEval, the footer is: HumanEval answer footer Summarizer prompt\. When a separate summarizer is used, it receives the original query XX, the topology τ\\tau, and the outputs of the relevant hidden nodes\. The summarizer prompt template is: Summarizer prompt template Message parsing\. Each non\-output node generates text Zℓ,jZ\_\{\\ell,j\} in the structured format specified by the hidden\-node prompt\. A deterministic parser extracts recipient\-specific messages: σℓ,j​\(Zℓ,j\)=\(Mℓ,j→ℓ\+1,1,…,Mℓ,j→ℓ\+1,nℓ\+1\),ℓ<L\.\\sigma\_\{\\ell,j\}\(Z\_\{\\ell,j\}\)=\\left\(M\_\{\\ell,j\\rightarrow\\ell\+1,1\},\\ldots,M\_\{\\ell,j\\rightarrow\\ell\+1,n\_\{\\ell\+1\}\}\\right\),\\qquad\\ell<L\. For final hidden\-layer nodes, the parser extracts the message sent to the output node: ML,j→out=σL,jout​\(ZL,j\)\.M\_\{L,j\\rightarrow\\mathrm\{out\}\}=\\sigma\_\{L,j\}^\{\\mathrm\{out\}\}\(Z\_\{L,j\}\)\. If a generated text cannot be parsed, the parser uses a deterministic fallback: the unparsed text is assigned to the first recipient and empty messages are assigned to the remaining recipients\. A\.3 LoRA Adapter Details All experiments use node\-specific LoRA adapters on top of the shared frozen backbone\. For each adapted pretrained weight matrix WW, node vv uses Wv′=W\+Δ​Wv,Δ​Wv=Bv​Av,W\_\{v\}^\{\\prime\}=W\+\\Delta W\_\{v\},\\qquad\\Delta W\_\{v\}=B\_\{v\}A\_\{v\}, where AvA\_\{v\} and BvB\_\{v\} are node\-specific low\-rank matrices\. The output aggregation node has its own adapter\. During progressive growth, newly added nodes are initialized with inactive adapters by setting Bv=0,B\_\{v\}=0, so that Δ​Wv=0\\Delta W\_\{v\}=0\. Thus, a newly added node initially behaves like the shared frozen backbone under its input context and specializes only through subsequent reward\-based training\. A\.4 Progressive Growth Schedule The main progressive\-growth schedule uses three topology sizes: τ3=\[1,1\],τ5=\[2,2\],τ7=\[2,2,2\],\\tau\_\{3\}=\[1,1\],\\qquad\\tau\_\{5\}=\[2,2\],\\qquad\\tau\_\{7\}=\[2,2,2\], corresponding to NeuroMAS\-3, NeuroMAS\-5, and NeuroMAS\-7, respectively\. The transition from τ3\\tau\_\{3\} to τ5\\tau\_\{5\} expands width, and the transition from τ5\\tau\_\{5\} to τ7\\tau\_\{7\} expands depth\. When a trained topology τ\\tau is expanded to a larger topology τ′\\tau^\{\\prime\}, nodes that have corresponding positions in the expanded topology inherit their adapter parameters\. Newly added nodes are initialized with inactive adapters as described in Section A\.3\. The expanded system is then trained further with the same terminal\-reward objective\. In our implementation, inherited nodes are reused when their local input\-output interface remains compatible after expansion\. If a node’s position changes in a way that changes its role in the computation, such as moving from the final hidden layer to an intermediate hidden layer, its adapter is still transferred but continues adapting during subsequent training rather than being treated as fixed\. A\.5 Algorithmic Summary Given input XX and topology τ\\tau, a NeuroMAS forward pass proceeds as follows: 1\. Construct the input context C1,jC\_\{1,j\} for each first\-layer node using the query XX, the node’s structural position, and the topology\. 2\. Sample the generated text Z1,jZ\_\{1,j\} from each first\-layer node policy\. 3\. Parse each Z1,jZ\_\{1,j\} into recipient\-specific messages for the next layer\. 4\. Repeat context construction, generation, and parsing for all subsequent hidden layers\. 5\. Construct the output\-node context CoutC\_\{\\mathrm\{out\}\} from XX, τ\\tau, and the final hidden\-layer messages\. 6\. Sample the final answer Y^\\widehat\{Y\} from the output node\. For reward\-based training, compute the terminal reward r​\(Y,Y^\)r\(Y,\\widehat\{Y\}\), assign this same reward to all sampled node outputs, and update the node\-specific trainable parameters using the sampled loss in Equation \(3\)\. For progressive growth, train the current topology, expand it by transferring inherited adapters and initializing new adapters, and then continue reward\-based training under the expanded topology\. Appendix B Proofs for the Theoretical Results This appendix provides details for the theoretical results in Section 5\. The main text treats a conditional generator abstractly as a distribution over output text\. We first connect this notation to autoregressive language models and then prove the compositional parameter\-efficiency bound and its fixed\-error equivalent\. B\.1 Autoregressive Generators and Component Errors A conditional generator π\(⋅∣X\)\\pi\(\\cdot\\mid X\) defines a distribution over finite text sequences\. When implemented by an autoregressive language model, the probability of a generated answer Y^=\(Y^1,…,Y^T^\)\\widehat\{Y\}=\(\\widehat\{Y\}\_\{1\},\\ldots,\\widehat\{Y\}\_\{\\widehat\{T\}\}\) factorizes as π​\(Y^∣X\)=∏t=1T^π​\(Y^t∣X,Y^<t\)\.\\pi\(\\widehat\{Y\}\\mid X\)=\\prod\_\{t=1\}^\{\\widehat\{T\}\}\\pi\(\\widehat\{Y\}\_\{t\}\\mid X,\\widehat\{Y\}\_\{<t\}\)\. The theoretical results do not require this factorization, but it connects the abstract generator notation used in Section 5 to standard language\-model notation\. The component\-level exact\-match error in Assumption 2 is defined analogously to the final exact\-match error\. If component ii has ideal output Zi∗Z\_\{i\}^\{\*\}, and an approximating component produces Z^i\\widehat\{Z\}\_\{i\}, then its component error is Pr⁡\(Z^i≠Zi∗\)\.\\Pr\(\\widehat\{Z\}\_\{i\}\\neq Z\_\{i\}^\{\*\}\)\. The object Zi∗Z\_\{i\}^\{\*\} may be an intermediate message, partial answer, aggregation state, or refined state\. The theorem only requires that local component errors can be controlled and that the final composition is stable with respect to those errors\. B\.2 Proof of Theorem 5\.1 Proof\. By Assumption 1, the ideal solution process can be represented by composing KK generative components ψ1∗,…,ψK∗\.\\psi\_\{1\}^\{\*\},\\ldots,\\psi\_\{K\}^\{\*\}\. Construct a modular multi\-agent generator by assigning one trainable node to approximate each component\. Allocate the total trainable parameter budget qq evenly across the KK components, so that parameters are assigned to component ii\. By Assumption 2, component ii can be approximated with error ηi​\(qi\)≤Cloc​qi−1/a=Cloc​\(qK\)−1/a=Cloc​K1/a​q−1/a\.\\eta\_\{i\}\(q\_\{i\}\)\\leq C\_\{\\mathrm\{loc\}\}q\_\{i\}^\{\-1/a\}=C\_\{\\mathrm\{loc\}\}\\left\(\\frac\{q\}\{K\}\\right\)^\{\-1/a\}=C\_\{\\mathrm\{loc\}\}K^\{1/a\}q^\{\-1/a\}\. Thus, every component can be approximated to error at most ϵ=Cloc​K1/a​q−1/a\.\\epsilon=C\_\{\\mathrm\{loc\}\}K^\{1/a\}q^\{\-1/a\}\. By Assumption 3, if the component errors are at most ϵ\\epsilon, then the final exact\-match error of the composed generator is at most Cstab​K​ϵ\.C\_\{\\mathrm\{stab\}\}K\\epsilon\. Substituting the bound on ϵ\\epsilon gives Cstab​K​ϵ=Cstab​K​\(Cloc​K1/a​q−1/a\)=Cstab​Cloc​K1\+1/a​q−1/a\.C\_\{\\mathrm\{stab\}\}K\\epsilon=C\_\{\\mathrm\{stab\}\}K\\left\(C\_\{\\mathrm\{loc\}\}K^\{1/a\}q^\{\-1/a\}\\right\)=C\_\{\\mathrm\{stab\}\}C\_\{\\mathrm\{loc\}\}K^\{1\+1/a\}q^\{\-1/a\}\. Let Cmulti=Cstab​Cloc\.C\_\{\\mathrm\{multi\}\}=C\_\{\\mathrm\{stab\}\}C\_\{\\mathrm\{loc\}\}\. The constructed modular generator therefore has final exact\-match error at most Cmulti​K1\+1/a​q−1/a\.C\_\{\\mathrm\{multi\}\}K^\{1\+1/a\}q^\{\-1/a\}\. Since εmulti,q\\varepsilon\_\{\\mathrm\{multi\},q\} is the smallest achievable error over all modular multi\-agent generators with at most qq trainable parameters, it is no larger than the error of this constructed generator\. Therefore, εmulti,q≤Cmulti​K1\+1/a​q−1/a\.\\varepsilon\_\{\\mathrm\{multi\},q\}\\leq C\_\{\\mathrm\{multi\}\}K^\{1\+1/a\}q^\{\-1/a\}\. Equivalently, εmulti,q=O​\(K1\+1/a​q−1/a\)\.\\varepsilon\_\{\\mathrm\{multi\},q\}=O\\left\(K^\{1\+1/a\}q^\{\-1/a\}\\right\)\. This proves the theorem\. ∎ B\.3 Proof of Corollary 1 Proof\. By Theorem 5\.1, there exists Cmulti\>0C\_\{\\mathrm\{multi\}\}\>0 such that εmulti,q≤Cmulti​K1\+1/a​q−1/a\.\\varepsilon\_\{\\mathrm\{multi\},q\}\\leq C\_\{\\mathrm\{multi\}\}K^\{1\+1/a\}q^\{\-1/a\}\. By Assumption 4, there exists csingle\>0c\_\{\\mathrm\{single\}\}\>0 such that, for sufficiently large qq, εsingle,q≥csingle​q−1/b\.\\varepsilon\_\{\\mathrm\{single\},q\}\\geq c\_\{\\mathrm\{single\}\}q^\{\-1/b\}\. Because b\>ab\>a, we have 1/a\>1/b1/a\>1/b, and hence q−1/aq−1/b=q−\(1/a−1/b\)→0as ​q→∞\.\\frac\{q^\{\-1/a\}\}\{q^\{\-1/b\}\}=q^\{\-\(1/a\-1/b\)\}\\rightarrow 0\\qquad\\text\{as \}q\\rightarrow\\infty\. Since KK is fixed, Cmulti​K1\+1/a​q−1/acsingle​q−1/b=Cmulti​K1\+1/acsingle​q−\(1/a−1/b\)→0\.\\frac\{C\_\{\\mathrm\{multi\}\}K^\{1\+1/a\}q^\{\-1/a\}\}\{c\_\{\\mathrm\{single\}\}q^\{\-1/b\}\}=\\frac\{C\_\{\\mathrm\{multi\}\}K^\{1\+1/a\}\}\{c\_\{\\mathrm\{single\}\}\}q^\{\-\(1/a\-1/b\)\}\\rightarrow 0\. Therefore, there exists q0\>0q\_\{0\}\>0 such that, for all q≥q0q\\geq q\_\{0\}, Cmulti​K1\+1/a​q−1/a<csingle​q−1/b\.C\_\{\\mathrm\{multi\}\}K^\{1\+1/a\}q^\{\-1/a\}<c\_\{\\mathrm\{single\}\}q^\{\-1/b\}\. Combining the upper bound on εmulti,q\\varepsilon\_\{\\mathrm\{multi\},q\} with the lower bound on εsingle,q\\varepsilon\_\{\\mathrm\{single\},q\}, we obtain εmulti,q<εsingle,q\\varepsilon\_\{\\mathrm\{multi\},q\}<\\varepsilon\_\{\\mathrm\{single\},q\} for all q≥q0q\\geq q\_\{0\}\. This proves the corollary\. ∎ B\.4 Fixed\-Error Parameter Comparison The fixed\-budget comparison in Corollary 1 can also be expressed as a fixed\-error parameter comparison\. For a target exact\-match error δ\>0\\delta\>0, define qmulti​\(δ\)=inf\{q:εmulti,q≤δ\},qsingle​\(δ\)=inf\{q:εsingle,q≤δ\}\.q\_\{\\mathrm\{multi\}\}\(\\delta\)=\\inf\\left\\\{q:\\varepsilon\_\{\\mathrm\{multi\},q\}\\leq\\delta\\right\\\},\\qquad q\_\{\\mathrm\{single\}\}\(\\delta\)=\\inf\\left\\\{q:\\varepsilon\_\{\\mathrm\{single\},q\}\\leq\\delta\\right\\\}\. From Theorem 5\.1, it is sufficient for the modular generator to choose qq such that Cmulti​K1\+1/a​q−1/a≤δ\.C\_\{\\mathrm\{multi\}\}K^\{1\+1/a\}q^\{\-1/a\}\\leq\\delta\. Solving for qq gives q≥Cmultia​K1\+a​δ−a\.q\\geq C\_\{\\mathrm\{multi\}\}^\{a\}K^\{1\+a\}\\delta^\{\-a\}\. Therefore, qmulti​\(δ\)=O​\(K1\+a​δ−a\)\.q\_\{\\mathrm\{multi\}\}\(\\delta\)=O\\left\(K^\{1\+a\}\\delta^\{\-a\}\\right\)\. For the unstructured single\-agent generator, Assumption 4 gives εsingle,q≥csingle​q−1/b\.\\varepsilon\_\{\\mathrm\{single\},q\}\\geq c\_\{\\mathrm\{single\}\}q^\{\-1/b\}\. Thus, to reach error at most δ\\delta, it is necessary that csingle​q−1/b≤δ\.c\_\{\\mathrm\{single\}\}q^\{\-1/b\}\\leq\\delta\. Solving for qq gives q≥csingleb​δ−b\.q\\geq c\_\{\\mathrm\{single\}\}^\{b\}\\delta^\{\-b\}\. Hence, qsingle​\(δ\)=Ω​\(δ−b\)\.q\_\{\\mathrm\{single\}\}\(\\delta\)=\\Omega\\left\(\\delta^\{\-b\}\\right\)\. Since b\>ab\>a, δ−b\\delta^\{\-b\} grows faster than δ−a\\delta^\{\-a\} as δ→0\\delta\\to 0\. Therefore, for sufficiently small δ\\delta, the modular class requires fewer trainable parameters than the unstructured single\-agent class, up to constants and the polynomial overhead in KK\. Appendix C Additional Experimental Details C\.1 Implementation Details We optimize with AdamW with learning rate 2×10−52\\times 10^\{\-5\}, and gradient clipping at 1\.01\.0\. Evaluation uses greedy decoding with max\_new\_tokens=200\. Accuracy is measured after canonicalizing outputs into the expected answer format\. C\.2 Datasets and Evaluation Protocol Checkpoint selection is performed on a held\-out development split that is disjoint from final evaluation\. We save checkpoints periodically during training and report the checkpoint with the best development accuracy under greedy decoding\. Final test accuracy is computed only after checkpoint selection\. Evaluation accuracy is measured after canonicalizing outputs into the expected answer format\. Multiple\-choice and yes/no tasks use canonicalized answer matching; for HumanEval, correctness is assessed by executing the generated code against the official unit tests\. C\.3 Compute Resources Experiments were conducted on NVIDIA RTX 5090 and NVIDIA H100 GPUs\.`