Codebook Agent: Amortized Topology Design for LLM Multi-Agent Systems

# Codebook Agent: Amortized Topology Design for LLM Multi-Agent Systems
Source: [https://arxiv.org/html/2609.02264](https://arxiv.org/html/2609.02264)
Yubei LiEric Hanchen JiangZhi ZhangDong LiuAffiliation:Wenxiao Zhao, Levina Li, Kai\-Wei Chang, Ying Nian Wu

###### Abstract

Adapting the communication topology of an LLM multi\-agent system to each query improves both accuracy and efficiency, yet current designers treat this as conditional graph generation: a variational, autoregressive, or diffusion decoder searches theN×NN\{\\times\}Nadjacency space, and a graph\-network proxy trained on utility and a structural cost such as edge count ranks the sampled candidates\. We argue that this formulation is misaligned with the problem\. Empirically, topologies that survive a reward filter collapse to about six distinct graphs even when the codebook capacity grows from 8 to 64; edge count is*negatively*correlated with measured token consumption \(Pearsonr≈−0\.4r\\approx\-0\.4\), so sparsifying the graph makes inference more expensive; and a message\-passing scorer over agent\-profile nodes is adjacency\-invariant whenever agents share a profile—the default configuration of published benchmarks—so it cannot rank candidates at all in that regime\. These three facts motivate*Codebook Agent*: a vector\-quantized autoencoder compresses successful topologies into a query\-independent 16\-entry codebook; a reward\-weighted MLP maps the query embedding to a distribution over codes; and an MLP proxy that reads the flattened adjacency, regressed on measured utility and per\-task normalized token cost, reranks the top decoded candidates in a single batched forward pass\. With no iterative search and no message passing at test time, Codebook Agent is the most accurate method on all six benchmarks we compare \(84\.6 average against 83\.0 for the strongest prior designer\), emits a topology in 2\.4 ms, and uses 21\.9–33\.2% fewer LLM tokens\. Our code is available here:https://github\.com/jinxiy1104/CodebookAgent\.

University of California, Los Angeles

![Refer to caption](https://arxiv.org/html/2609.02264v1/Figures/intro.png)Figure 1:Three regimes of topology design\.\(Top\) A fixed topology is reused for every query: zero design cost, but no single graph fits every task\. \(Middle\) Query\-conditioned generators adapt the graph via an iterative loop that samples and scoresKKcandidates at each ofTTsteps, typically with a proxy trained on utility and edge count\. \(Bottom\) Codebook Agent keeps per\-query adaptation and removes the loop: the design space is discretized once offline into a codebook, a query embedding selects codes, and one batched proxy call under a token\-based cost objective returns the winner\.## 1Introduction

Multi\-agent systems built from LLM agents solve reasoning, question answering, and coding tasks by exchanging intermediate outputs over a communication topology\([Wu et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib6);[Hong et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib7);[Qian et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib8);[Du et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib40)\)\. That topology is not bookkeeping: it decides which agents see which messages, and both accuracy and token consumption move with it\([Qian et al\. 2025](https://arxiv.org/html/2609.02264#bib.bib5);[Zhuge et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib2);[Zhang et al\. 2025b](https://arxiv.org/html/2609.02264#bib.bib3)\)\. The field has therefore progressed from hand\-designed structures to learned ones, and most recently to*query\-conditioned*generators that emit a fresh topology for every input\([Zhang et al\. 2025c](https://arxiv.org/html/2609.02264#bib.bib1);[Jiang et al\. 2026](https://arxiv.org/html/2609.02264#bib.bib12)\)\.

The latest designers share a natural but costly recipe\. A variational\([Zhang et al\. 2025c](https://arxiv.org/html/2609.02264#bib.bib1)\), autoregressive\([Li et al\. 2026](https://arxiv.org/html/2609.02264#bib.bib16)\), or diffusion decoder\([Jiang et al\. 2026](https://arxiv.org/html/2609.02264#bib.bib12)\)searches theN×NN\{\\times\}Nadjacency space; a graph network over agent\-profile nodes then ranks candidates with a utility head and a structural cost head, typically the edge count\|E\|\|E\|\([Zhuge et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib2);[Zhang et al\. 2025b](https://arxiv.org/html/2609.02264#bib.bib3);[Zhang et al\. 2025c](https://arxiv.org/html/2609.02264#bib.bib1);[Jiang et al\. 2026](https://arxiv.org/html/2609.02264#bib.bib12)\)\. The recipe looks principled—generate, then score—yet it quietly assumes that the useful design space is large enough to need a generative decoder, that\|E\|\|E\|tracks inference cost, and that message passing can tell two topologies apart\. Our measurements reject all three assumptions\.

Figure[1](https://arxiv.org/html/2609.02264#S0.F1)sketches the consequence\. First, the useful design space is a short list, not a manifold: topologies that solve their task fall into about six distinct codes even as codebook capacity grows from 8 to 64, and the best fixed topology stays within 1\.4 accuracy points of every generated one we measure—so capacity spent modeling adjacency is spent where the problem does not live\. Second, the structural cost surrogate is inverted: edge count correlates with measured tokens atr≈−0\.4r\\approx\-0\.4, because sparse communication yields longer completions, so minimizing\|E\|\|E\|*maximizes*the bill it was introduced to reduce\. Third, on the homogeneous teams that dominate published benchmarks, a message\-passing scorer over profile nodes is adjacency\-invariant: every candidate receives the same score, the ranking does not exist, and hundreds of guided diffusion steps reproduce a constant\.

If the problem is to choose among a few graphs under a measured cost, the matching designer is not another decoder\.*Codebook Agent*therefore amortizes topology design into three feed\-forward pieces: a vector\-quantized autoencoder\([van den Oord et al\. 2017](https://arxiv.org/html/2609.02264#bib.bib28)\)that indexes successful topologies into 16 codes, a reward\-weighted MLP that maps the query embedding to a distribution over those codes, and an MLP proxy that reads the flattened adjacency and is regressed on measured utility and per\-task normalized token cost\. At test time we decode the top codes, score them in one batched proxy call, and execute the winner—2\.4 ms, with no sampling loop and no message passing\. Our contributions:

- ❶We characterize topology design rather than assume its form: the reward\-surviving space collapses to about six graphs independently of model capacity, the common structural cost surrogate is inverted w\.r\.t\. measured tokens, and message\-passing proxies are topology\-blind on homogeneous teams\.
- ❷We propose Codebook Agent, the minimal designer these facts admit, a discrete codebook, a reward\-weighted code predictor, and one batched pass of an execution\-grounded proxy, with no iterative search and no message passing\.
- ❸On six benchmarks and two LLM backends, against single\-agent prompting, multi\-agent collaboration, and learned topology designers, Codebook Agent is the most accurate method on every benchmark of Table[1](https://arxiv.org/html/2609.02264#S4.T1), generates topologies in 2\.4 ms, and reduces token consumption by 21\.9–33\.2% \(Tables[1](https://arxiv.org/html/2609.02264#S4.T1)and[2](https://arxiv.org/html/2609.02264#S5.T2)\)\.

## 2Related Work

#### Communication topologies for LLM agents\.

Early frameworks fix the topology by hand\([Hong et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib7);[Qian et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib8);[Wu et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib6);[Chen et al\. 2024c](https://arxiv.org/html/2609.02264#bib.bib9);[Du et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib40)\)\. A second line optimizes it: GPTSwarm learns edge distributions with policy gradients\([Zhuge et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib2)\), DyLAN selects agents dynamically\([Liu et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib4)\), search\-based systems explore agentic designs offline\([Hu et al\. 2025](https://arxiv.org/html/2609.02264#bib.bib10);[Zhang et al\. 2025d](https://arxiv.org/html/2609.02264#bib.bib11)\), pruning methods sparsify a fixed structure to save tokens\([Zhang et al\. 2025b](https://arxiv.org/html/2609.02264#bib.bib3);[Wang et al\. 2025](https://arxiv.org/html/2609.02264#bib.bib13)\), and Optima trains the agents for efficiency\([Chen et al\. 2025](https://arxiv.org/html/2609.02264#bib.bib14)\)\. Closest to us are query\-conditioned generators\([Zhang et al\. 2025c](https://arxiv.org/html/2609.02264#bib.bib1);[Jiang et al\. 2026](https://arxiv.org/html/2609.02264#bib.bib12)\)\. We share their problem statement, one topology per query, and differ in what we assume about it: they model the adjacency space as something to be generated and scored by a graph network, and we show it is a short list to be selected from and scored by measured cost\.

#### Graph generation and discrete latents\.

Deep graph generators are predominantly iterative, whether autoregressive\([You et al\. 2018](https://arxiv.org/html/2609.02264#bib.bib24)\)or score\-based\([Jo et al\. 2022](https://arxiv.org/html/2609.02264#bib.bib22);[Vignac et al\. 2023](https://arxiv.org/html/2609.02264#bib.bib21);[Zhou et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib23)\), inheriting a sampling cost\([Ho et al\. 2020](https://arxiv.org/html/2609.02264#bib.bib26)\)that faster samplers reduce but do not remove\([Song et al\. 2021](https://arxiv.org/html/2609.02264#bib.bib27)\); one\-shot VAE decoders exist for small graphs\([Simonovsky and Komodakis 2018](https://arxiv.org/html/2609.02264#bib.bib25)\)\. We need one small graph per query under a latency constraint, so we discretize the output space with vector quantization\([van den Oord et al\. 2017](https://arxiv.org/html/2609.02264#bib.bib28);[Razavi et al\. 2019](https://arxiv.org/html/2609.02264#bib.bib29)\)instead of sampling from a continuous process\. VQGraph tokenizes local structure to improve GNN\-to\-MLP distillation\([Yang et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib30);[Zhang et al\. 2022](https://arxiv.org/html/2609.02264#bib.bib31);[Tian et al\. 2023](https://arxiv.org/html/2609.02264#bib.bib32);[Wu et al\. 2023](https://arxiv.org/html/2609.02264#bib.bib34);[Hinton et al\. 2015](https://arxiv.org/html/2609.02264#bib.bib33)\); we borrow its structure\-token regularizer, but our proxy has no teacher to distill from: on homogeneous teams a message\-passing teacher is constant per query, so we train the MLP directly on measured rewards\.

#### Cost of multi\-agent inference\.

Multi\-agent pipelines multiply LLM calls, and returns diminish or invert as calls grow\([Li et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib38);[Chen et al\. 2024a](https://arxiv.org/html/2609.02264#bib.bib39);[Smit et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib41)\), with many observed failures attributable to coordination rather than single\-agent ability\([Cemri et al\. 2025](https://arxiv.org/html/2609.02264#bib.bib36)\)\. Cost\-aware model selection addresses the single\-call case\([Chen et al\. 2024b](https://arxiv.org/html/2609.02264#bib.bib37)\)\. These results motivate treating measured tokens, not a structural surrogate, as the cost objective of topology design\.

![Refer to caption](https://arxiv.org/html/2609.02264v1/Figures/Codebook_main.png)Figure 2:Overview of Codebook Agent\.*Left:*offline collection executes fixed topologies on the training split and logs\(A,c,u,τ\)\(A,c,u,\\tau\)for every \(query, topology\) pair\.*Middle:*\(1\) a VQ autoencoder compresses successful topologies \(u\>0\.5u\>0\.5\) into a query\-independent codebook; \(2\) an MLP predictorpθ​\(k∣c\)p\_\{\\theta\}\(k\\mid c\)is fit to the reward\-weighted soft targets of Eq\. \([6](https://arxiv.org/html/2609.02264#S4.E6)\); \(3\) an MLP proxyfϕf\_\{\\phi\}is regressed on measured utility and per\-task normalized token cost \(Eqs\. \([8](https://arxiv.org/html/2609.02264#S4.E8)\)–\([10](https://arxiv.org/html/2609.02264#S4.E10)\)\), with an auxiliary structure\-token head used in training only\.*Right:*at test time the topMMcodes are decoded, deduplicated, and scored in one batched proxy call; the candidate maximizingu^−λ​c^\\hat\{u\}\-\\lambda\\hat\{c\}\(Eq\. \([11](https://arxiv.org/html/2609.02264#S4.E11)\)\) is executed\. Dashed arrows are training\-only; solid arrows are the frozen test\-time path \(no LLM calls inside topology generation\)\.

## 3Problem Setup and Background

#### Topology design\.

A team ofNNagents has profile textsp1,…,pNp\_\{1\},\\dots,p\_\{N\}embedded asxi=E⁡\(pi\)∈ℝdx\_\{i\}=E\(p\_\{i\}\)\\in\\mathbb\{R\}^\{d\}by a frozen sentence encoder \(d=384d=384\)\. A topology is a directed adjacency matrixA∈\{0,1\}N×NA\\in\\\{0,1\\\}^\{N\\times N\}without self\-loops, whereAi​j=1A\_\{ij\}=1shows the output of agentiito agentjj, and a decision node aggregates the final answers following GDesigner\([Zhang et al\. 2025c](https://arxiv.org/html/2609.02264#bib.bib1)\)\. Executing the team on queryqqunderAAreturns a utilityu⁡\(A,q\)∈\{0,1\}u\(A,q\)\\in\\\{0,1\\\}and a token costτ⁡\(A,q\)\\tau\(A,q\)\. Writingc=E⁡\(q\)c=E\(q\)for the query embedding, the goal is a generator mappingccto a topology maximizingR=u−λ​τ~R=u\-\\lambda\\tilde\{\\tau\}, withτ~\\tilde\{\\tau\}a normalized cost andλ=0\.1\\lambda=0\.1\. All methods in this paper are trained on the same records: per benchmark, 50 training tasks executed under 6 fixed topologies \(complete, chain, star, and three Erdos\-Renyi samples\) with real LLM agents, giving 300 tuples\(Aj,cj,uj,τj\)\(A\_\{j\},c\_\{j\},u\_\{j\},\\tau\_\{j\}\)\.

#### Two design axes\.

A query\-conditioned designer is determined by two independent choices, which we vary separately in the experiments\.*Axis A, the candidate generator*, mapsccto one or more topologies; the incumbent choice is an iterative or continuous decoder over the adjacency space, variational\([Zhang et al\. 2025c](https://arxiv.org/html/2609.02264#bib.bib1)\), autoregressive\([Li et al\. 2026](https://arxiv.org/html/2609.02264#bib.bib16)\), or a diffusion reverse process withTTsteps andKKcandidates per step\([Jiang et al\. 2026](https://arxiv.org/html/2609.02264#bib.bib12)\), which we measure at 301 to 396 ms per query\.*Axis B, the scorer*, ranks candidates; the incumbent choice, shared across the learned\-design literature\([Zhuge et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib2);[Zhang et al\. 2025b](https://arxiv.org/html/2609.02264#bib.bib3);[Zhang et al\. 2025c](https://arxiv.org/html/2609.02264#bib.bib1);[Jiang et al\. 2026](https://arxiv.org/html/2609.02264#bib.bib12)\), is a message\-passing network over a graph whose nodes carry the profile embeddingsxix\_\{i\}and whose edges are the candidateAA:

fgnn​\(A,c\)=h⁡\(pool⁡\(MPNN⁡\(A,\{xi\},c\)\)\)∈ℝ2,f\_\{\\mathrm\{gnn\}\}\(A,c\)=h\\\!\\left\(\\mathrm\{pool\}\\big\(\\mathrm\{MPNN\}\(A,\\\{x\_\{i\}\\\},c\)\\big\)\\right\)\\in\\mathbb\{R\}^\{2\},\(1\)with a two\-dimensional head regressed onto utility and a structural cost label, most commonly the edge count\|E\|\|E\|\. We implement Eq\. \([1](https://arxiv.org/html/2609.02264#S3.E1)\) as a GAT\([Veličković et al\. 2018](https://arxiv.org/html/2609.02264#bib.bib35)\)and treat it as one level of Axis B, so it can be swapped into our own pipeline with everything else held fixed\. The introduction’s three observations—a collapsed design space, an inverted cost surrogate, and adjacency\-blind message passing on homogeneous teams—determine how we instantiate both axes below\.

## 4Method

Codebook Agent instantiates Axis A as a discrete codebook with a query\-conditioned prior, and Axis B as an MLP over flattened adjacencies trained on measured utility and token cost\. Let𝒟=\{\(Aj,cj,uj,τj\)\}j=1\|𝒟\|\\mathcal\{D\}=\\\{\(A\_\{j\},c\_\{j\},u\_\{j\},\\tau\_\{j\}\)\\\}\_\{j=1\}^\{\|\\mathcal\{D\}\|\}be the execution records of Section[3](https://arxiv.org/html/2609.02264#S3)\. Define the composite reward

Rj=uj−λτ~j,τ~j=τj1\|\{j′:cj′=cj\}\|∑j′:cj′=cjτj′,R\_\{j\}\\;=\\;u\_\{j\}\-\\lambda\\,\\tilde\{\\tau\}\_\{j\},\\qquad\\tilde\{\\tau\}\_\{j\}\\;=\\;\\frac\{\\tau\_\{j\}\}\{\\displaystyle\\tfrac\{1\}\{\|\\\{j^\{\\prime\}:c\_\{j^\{\\prime\}\}=c\_\{j\}\\\}\|\}\\sum\_\{j^\{\\prime\}:c\_\{j^\{\\prime\}\}=c\_\{j\}\}\\tau\_\{j^\{\\prime\}\}\},\(2\)withλ=0\.1\\lambda=0\.1:τ~j\\tilde\{\\tau\}\_\{j\}is the token count normalized by the mean token count of the same task, so difficulty cancels and within\-task rankings remain\. Using measured tokens rather than\|E\|\|E\|is essential: on our records edge count correlates withτ\\tauatr≈−0\.4r\\approx\-0\.4, so an edge\-count head pushes the system toward more expensive graphs\. The codebook, code predictor, and reranking proxy are trained on𝒟\\mathcal\{D\}with no further LLM calls\. Figure[2](https://arxiv.org/html/2609.02264#S2.F2)shows offline collection, the three training stages, and one\-pass test\-time generation\.

### 4\.1Topology Codebook

If reward\-surviving topologies occupy only a handful of distinct graphs, the generator should index that short list rather than search the adjacency manifold\. We therefore compress observed topologies with a vector\-quantized autoencoder\. Flatten the off\-diagonal entriesa=vec⁡\(A\)∈\{0,1\}N⁡\(N−1\)a=\\mathrm\{vec\}\(A\)\\in\\\{0,1\\\}^\{N\(N\-1\)\}and encode

ze\\displaystyle z\_\{e\}=Encψ\(a\)∈ℝdz,k⋆=argmink∈\{1,…,K\}∥ze−ek∥2,\\displaystyle=\\mathrm\{Enc\}\_\{\\psi\}\(a\)\\in\\mathbb\{R\}^\{d\_\{z\}\},\\qquad k^\{\\star\}=\\arg\\min\_\{k\\in\\\{1,\\dots,K\\\}\}\\bigl\\\|z\_\{e\}\-e\_\{k\}\\bigr\\\|\_\{2\},\(3\)zq\\displaystyle z\_\{q\}=ek⋆\.\\displaystyle=e\_\{k^\{\\star\}\}\.
withdz=32d\_\{z\}=32and codebook sizeK=16K=16\. The codebook\{ek\}k=1K\\\{e\_\{k\}\\\}\_\{k=1\}^\{K\}is updated by exponential moving averages \(decay0\.990\.99\), with commitment coefficientβ=0\.25\\beta=0\.25, straight\-through gradients, and reinitialization of codes unused for 50 batches\([van den Oord et al\. 2017](https://arxiv.org/html/2609.02264#bib.bib28);[Razavi et al\. 2019](https://arxiv.org/html/2609.02264#bib.bib29)\)\. A mirror\-image decoder mapszqz\_\{q\}to edge logitsℓ^=Decω​\(zq\)∈ℝN⁡\(N−1\)\\hat\{\\ell\}=\\mathrm\{Dec\}\_\{\\omega\}\(z\_\{q\}\)\\in\\mathbb\{R\}^\{N\(N\-1\)\}; the diagonal is fixed to zero and the graph remains directed\. Writinga^=σ⁡\(ℓ^\)\\hat\{a\}=\\sigma\(\\hat\{\\ell\}\), the training objective is

ℒvq=BCEw​\(a^,a\)\+β​‖ze−sg⁡\[zq\]‖22,\\mathcal\{L\}\_\{\\mathrm\{vq\}\}\\;=\\;\\mathrm\{BCE\}\_\{w\}\\\!\\bigl\(\\hat\{a\},\\,a\\bigr\)\\;\+\\;\\beta\\,\\bigl\\\|z\_\{e\}\-\\mathrm\{sg\}\[z\_\{q\}\]\\bigr\\\|\_\{2\}^\{2\},\(4\)whereBCEw\\mathrm\{BCE\}\_\{w\}weights positive entries by the zero\-to\-one ratio of the training set to counter edge sparsity andsg\\mathrm\{sg\}is the stop\-gradient\. We fit Eq\. \([4](https://arxiv.org/html/2609.02264#S4.E4)\) only on records withuj\>0\.5u\_\{j\}\>0\.5, so the codebook stores topologies that solved their task\. Neither encoder nor decoder is conditioned on the query: after training, codekkdecodes to the binary topology

A\(k\)=1\[σ\(Decω\(ek\)\)≥1/2\],A^\{\(k\)\}\\;=\\;\\mathbf\{1\}\\bigl\[\\sigma\\bigl\(\\mathrm\{Dec\}\_\{\\omega\}\(e\_\{k\}\)\\bigr\)\\geq 1/2\\bigr\],\(5\)and all query dependence is carried by the predictor below\. We train for 200 epochs with Adam at learning rate3×10−43\{\\times\}10^\{\-4\}and batch size 16\.

### 4\.2Reward\-Weighted Code Prediction

At test time there is no target topology to encode, so we learn a conditional prior over codes\. Passing every training record through the frozen encoder and quantizer yields indiceskjk\_\{j\}\. For each distinct conditionccwe form the soft target

yk\(c\)=∑j:cj=c,kj=kexp\(γRj\)∑k′=1K∑j:cj=c,kj=k′exp\(γRj\),γ=2,y\_\{k\}\(c\)\\;=\\;\\frac\{\\sum\_\{j:\\,c\_\{j\}=c,\\,k\_\{j\}=k\}\\exp\(\\gamma R\_\{j\}\)\}\{\\sum\_\{k^\{\\prime\}=1\}^\{K\}\\sum\_\{j:\\,c\_\{j\}=c,\\,k\_\{j\}=k^\{\\prime\}\}\\exp\(\\gamma R\_\{j\}\)\},\\qquad\\gamma=2,\(6\)which concentrates mass on codes whose topologies earned high reward under Eq\. \([2](https://arxiv.org/html/2609.02264#S4.E2)\), so the prior already prefers cheap, successful graphs\. The predictorpθ​\(k∣c\)p\_\{\\theta\}\(k\\mid c\)is an MLP \(d→256→256→Kd\{\\to\}256\{\\to\}256\{\\to\}K, dropout0\.10\.1\) trained by soft cross\-entropy

ℒpred=−∑c∑k=1Kyk\(c\)logpθ\(k∣c\)\.\\mathcal\{L\}\_\{\\mathrm\{pred\}\}\\;=\\;\-\\sum\_\{c\}\\sum\_\{k=1\}^\{K\}y\_\{k\}\(c\)\\,\\log p\_\{\\theta\}\(k\\mid c\)\.\(7\)This is the only place the query enters generation, so the map from query to candidate topologies is a single matrix pipeline\.

### 4\.3Execution\-Grounded MLP Proxy

The scorer must \(i\) read the adjacency itself and \(ii\) regress measured tokens, not\|E\|\|E\|\. The first requirement is forced by the homogeneous\-team regime: whenx1=⋯=xN=xx\_\{1\}=\\dots=x\_\{N\}=x, any message\-passing layer of Eq\. \([1](https://arxiv.org/html/2609.02264#S3.E1)\) aggregates identical features over self\-loops, so every node state—and thus the pooled score—is independent ofAA\. The same holds for mean and sum aggregators; there is nothing to distill from such a teacher on anonymous teams\. We therefore score candidates with an MLP over the flattened adjacency,

fϕ​\(A,c\)=MLPϕ​\(\[vec⁡\(A\);c\]\)=\(u^,c^\)∈ℝ2,f\_\{\\phi\}\(A,c\)\\;=\\;\\mathrm\{MLP\}\_\{\\phi\}\\bigl\(\\bigl\[\\mathrm\{vec\}\(A\);\\,c\\bigr\]\\bigr\)\\;=\\;\\bigl\(\\hat\{u\},\\,\\hat\{c\}\\bigr\)\\in\\mathbb\{R\}^\{2\},\(8\)and regress directly on the measured targets,

ℒrew=1\|𝒟\|​∑j=1\|𝒟\|\(\(u^j−uj\)2\+\(c^j−τ~j\)2\)\.\\mathcal\{L\}\_\{\\mathrm\{rew\}\}\\;=\\;\\frac\{1\}\{\|\\mathcal\{D\}\|\}\\sum\_\{j=1\}^\{\|\\mathcal\{D\}\|\}\\Bigl\(\\bigl\(\\hat\{u\}\_\{j\}\-u\_\{j\}\\bigr\)^\{2\}\+\\bigl\(\\hat\{c\}\_\{j\}\-\\tilde\{\\tau\}\_\{j\}\\bigr\)^\{2\}\\Bigr\)\.\(9\)
Because\|𝒟\|\|\\mathcal\{D\}\|covers the topology space thinly, we regularize with a structure\-token objective in the spirit of VQGraph\([Yang et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib30)\)\. A separate reconstruction\-only quantizer withKtok=32K\_\{\\mathrm\{tok\}\}=32codes is fit over an augmented pool𝒜\+\\mathcal\{A\}^\{\+\}of topologies \(static families, Erdős–Rényi graphs at eight densities, single\-edge perturbations, and Bernoulli samples of smoothed edge profiles, capped at 100 per condition\)\. Writingq⁡\(k∣A\)=softmaxk​\(−‖ze​\(A\)−ek‖22\)q\(k\\mid A\)=\\mathrm\{softmax\}\_\{k\}\(\-\\\|z\_\{e\}\(A\)\-e\_\{k\}\\\|\_\{2\}^\{2\}\)for the soft code assignment of a pool graph, an auxiliary head predictsqqunder

ℒcode=−𝔼A∼𝒜\+∑k=1Ktokq\(k∣A\)logq^ϕ\(k∣A\),\\mathcal\{L\}\_\{\\mathrm\{code\}\}\\;=\\;\-\\mathbb\{E\}\_\{A\\sim\\mathcal\{A\}^\{\+\}\}\\sum\_\{k=1\}^\{K\_\{\\mathrm\{tok\}\}\}q\(k\\mid A\)\\,\\log\\hat\{q\}\_\{\\phi\}\(k\\mid A\),\(10\)and the total proxy loss isℒϕ=ℒrew\+λcode​ℒcode\\mathcal\{L\}\_\{\\phi\}=\\mathcal\{L\}\_\{\\mathrm\{rew\}\}\+\\lambda\_\{\\mathrm\{code\}\}\\mathcal\{L\}\_\{\\mathrm\{code\}\}withλcode=0\.1\\lambda\_\{\\mathrm\{code\}\}=0\.1\. The augmented pool contributes only to Eq\. \([10](https://arxiv.org/html/2609.02264#S4.E10)\); reward supervision comes exclusively from real executions\. The proxy is trained for 200 epochs with Adam at learning rate10−310^\{\-3\}and batch size 256\.

### 4\.4One\-Pass Inference

Given a query embeddingcc, let𝒞M​\(c\)=\{k1,…,kM\}\\mathcal\{C\}\_\{M\}\(c\)=\\\{k\_\{1\},\\dots,k\_\{M\}\\\}be the topM=5M=5codes underpθ\(⋅∣c\)p\_\{\\theta\}\(\\cdot\\mid c\), and let𝒜⁡\(c\)=\{A\(k\):k∈𝒞M​\(c\)\}\\mathcal\{A\}\(c\)=\\\{A^\{\(k\)\}:k\\in\\mathcal\{C\}\_\{M\}\(c\)\\\}after removing duplicates under Eq\. \([5](https://arxiv.org/html/2609.02264#S4.E5)\)\. The selected topology is

A⋆​\(c\)=arg⁡maxA∈𝒜⁡\(c\)⁡\(u^​\(A,c\)−λ​c^​\(A,c\)\),A^\{\\star\}\(c\)\\;=\\;\\arg\\max\_\{A\\in\\mathcal\{A\}\(c\)\}\\Bigl\(\\hat\{u\}\(A,c\)\-\\lambda\\,\\hat\{c\}\(A,c\)\\Bigr\),\(11\)where\(u^,c^\)=fϕ​\(A,c\)\(\\hat\{u\},\\hat\{c\}\)=f\_\{\\phi\}\(A,c\)is evaluated for all survivors in one batched forward pass\. Generation therefore costs one predictor pass, at mostMMdecoder passes, and one dense proxy pass—a fixed number of matrix products, with no sampling loop, no sparse graph construction, and no message passing on the test\-time path\.

Figure 3:Accuracy–latency Pareto on MATH\.Accuracy from Table[1](https://arxiv.org/html/2609.02264#S4.T1)versus median topology\-generation latency \(log\-scalexx\-axis\)\. Codebook Agent occupies the upper\-left frontier: highest accuracy at 2\.4 ms, against 301–396 ms for iterative generators\.Table 1:Main accuracy results \(%\)on reasoning and code\-generation benchmarks\. HumanEval and MBPP report Pass@1\. Methods are grouped into single\-agent prompting, multi\-agent collaboration, and learned agent or topology design\. Deltas are absolute accuracy points relative to Vanilla;▲\\blacktriangle/▼\\blacktriangledownmark improvements and decreases\.Boldis best in each column;underlineis second best\. All runs use gpt\-4o\-mini under the protocol of Section[5\.1](https://arxiv.org/html/2609.02264#S5.SS1)\.

## 5Experiments

We design the evaluation to stress\-test the three claims of the introduction with four questions: \(Q1\) Does amortized Codebook Agent match or beat iterative query\-conditioned designers on accuracy, or does indexing a short codebook trade quality for speed? \(Q2\) Is the useful design space truly a short list—do fixed topologies already sit within a point of generated ones, and does extra codebook capacity go unused? \(Q3\) Does an execution\-grounded MLP overvec⁡\(A\)\\mathrm\{vec\}\(A\)cut tokens where the incumbent edge\-count GNN cannot rank homogeneous teams, and is random selection enough? \(Q4\) Does one\-pass codebook generation deliver the latency and token savings of Figure[1](https://arxiv.org/html/2609.02264#S0.F1)without sacrificing transfer across LLM backends?

### 5\.1Setup

Benchmarks\.Reasoning: GSM8K\([Cobbe et al\. 2021](https://arxiv.org/html/2609.02264#bib.bib42)\), MATH\([Hendrycks et al\. 2021b](https://arxiv.org/html/2609.02264#bib.bib43)\), MultiArith\([Roy and Roth 2015](https://arxiv.org/html/2609.02264#bib.bib46)\), SVAMP\([Patel et al\. 2021](https://arxiv.org/html/2609.02264#bib.bib45)\)\. Code: MBPP\([Austin et al\. 2021](https://arxiv.org/html/2609.02264#bib.bib48)\), HumanEval\([Chen et al\. 2021](https://arxiv.org/html/2609.02264#bib.bib47)\)\(Pass@1\)\. Transfer: MMLU\([Hendrycks et al\. 2021a](https://arxiv.org/html/2609.02264#bib.bib44)\)under Qwen\-3\-8B \(Table[2](https://arxiv.org/html/2609.02264#S5.T2)\)\. Evaluation uses the fixed test splits of GSM8K \(1314\), MATH \(500\), MultiArith \(180\), SVAMP \(1000\), MBPP \(500\), HumanEval \(160\), and MMLU \(1530\); design\-axis accuracy/token numbers use 200 tasks per benchmark \(full MultiArith/HumanEval where smaller\)\.Teams\.Math\-format:4×4\{\\times\}MathSolver with FinalRefer\. Code:4×4\{\\times\}Programming Expert with FinalRefer\. MMLU:3×3\{\\times\}Knowledgeable Expert\. Homogeneous profiles are the default \(six of seven design\-axis settings\); HumanEval also runs a heterogeneous four\-role team\. Backbone gpt\-4o\-mini \(temp\.0\.70\.7, top\-pp1\.01\.0, max tokens10241024\) unless stated; embeddings are frozen all\-MiniLM\-L6\-v2 \(d=384d\{=\}384\)\.

Baselines\.Single\-agent: Vanilla, CoT\([Wei et al\. 2022](https://arxiv.org/html/2609.02264#bib.bib19)\), SC\-CoT\([Wang et al\. 2023](https://arxiv.org/html/2609.02264#bib.bib20)\)\. Multi\-agent: LLM\-Debate\([Du et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib40)\), LLM\-Blender\([Jiang et al\. 2023](https://arxiv.org/html/2609.02264#bib.bib18)\), DyLAN\([Liu et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib4)\)\. Learned design: GPTSwarm\([Zhuge et al\. 2024](https://arxiv.org/html/2609.02264#bib.bib2)\), ADAS\([Hu et al\. 2025](https://arxiv.org/html/2609.02264#bib.bib10)\), AFLOW\([Zhang et al\. 2025d](https://arxiv.org/html/2609.02264#bib.bib11)\), MaAS\([Zhang et al\. 2025a](https://arxiv.org/html/2609.02264#bib.bib15)\), AgentDropout\([Wang et al\. 2025](https://arxiv.org/html/2609.02264#bib.bib13)\), G\-Designer\([Zhang et al\. 2025c](https://arxiv.org/html/2609.02264#bib.bib1)\), ARG\-Designer\([Li et al\. 2026](https://arxiv.org/html/2609.02264#bib.bib16)\), TopoDIM\([Sun et al\. 2026](https://arxiv.org/html/2609.02264#bib.bib17)\), GTD\([Jiang et al\. 2026](https://arxiv.org/html/2609.02264#bib.bib12)\)\.Metrics\.Accuracy \(Pass@1 for code\); median topology\-generation latency \(3 warmup\+\{\+\}50/100 timed calls on one GPU\); mean LLM tokens/query; end\-to\-end wall clock\. Single evaluation run per configuration\.

### 5\.2Main Results

Table[1](https://arxiv.org/html/2609.02264#S4.T1)reports gpt\-4o\-mini accuracy; Table[2](https://arxiv.org/html/2609.02264#S5.T2)the Qwen\-3\-8B transfer; Figures[3](https://arxiv.org/html/2609.02264#S4.F3)and[4](https://arxiv.org/html/2609.02264#S5.F4)the latency and token frontiers\. We highlight four observations corresponding to Q1–Q4\.

\(Q1\) Amortized selection clears the prior designer frontier; it does not trade accuracy for a codebook\.Table[1](https://arxiv.org/html/2609.02264#S4.T1)puts Codebook Agent best on every column at 84\.62 average:\+6\.99\+6\.99over Vanilla \(77\.63\),\+4\.44\+4\.44over DyLAN \(80\.18\), and\+1\.60\+1\.60over the strongest prior topology designer GTD \(83\.02\)\. The Vanilla gap is smallest on MultiArith \(\+2\.2\+2\.2; near ceiling\) and largest on MBPP \(\+11\.1\+11\.1\)\. Against GTD the gains are spread, not concentrated:\+1\.3\+1\.3GSM8K,\+1\.0\+1\.0MATH,\+1\.2\+1\.2MultiArith,\+2\.4\+2\.4SVAMP,\+3\.1\+3\.1MBPP,\+0\.6\+0\.6HumanEval\. Reading by family: single\-agent prompting floors the high 70s; multi\-agent collaboration lifts into the low 80s under hand\-designed structure; query\-conditioned generators \(G\-Designer through GTD\) form the prior frontier at∼83\{\\sim\}83\. Codebook Agent sits above that frontier on all six benchmarks—so indexing a short list of graphs is not a speed concession; it matches or exceeds the iterative adjacency search it replaces\.

\(Q2\) The useful design space is a short list, and which fixed graph wins is not knowable a priori\.Fixed topologies need no designer, so they measure how much of the payoff lives in the choice itself\. Across seven settings the best fixed family \(fully connected / chain / star\) stays within 1\.4 accuracy points of every generated topology we measure, at comparable tokens; on MATH, fully connected is above every generated configuration\. Capacity spent modeling the adjacency manifold is therefore spent where the problem does not live—the premise of Axis A as a codebook\. The choice is nonetheless load\-bearing: which fixed graph wins changes with the setting \(fully connected on MATH, star on homogeneous HumanEval, chain on MMLU\), with gaps up to 4\.3 accuracy points and a2\.9×2\.9\{\\times\}token factor \(chain vs\. fully connected on MATH\)\. Identifying the winner for a new setting requires real LLM executions—the same cost class as our one\-time 300\-record collection—so a designer earns its keep by amortizing that selection per query\. Figure[5](https://arxiv.org/html/2609.02264#S5.F5)\(b\) closes the loop on capacity: asKKgrows from 4 to 64, accuracy moves by at most 1\.5 points \(GSM8K\) / 2\.5 \(HumanEval\), and for everyK≥8K\{\\geq\}8the encoder uses at most six codes\. Collapse is a property of reward\-surviving topologies, not of an undersized quantizer\.

\(Q3\) Measured\-token MLP scoring is where cost is won; the incumbent GNN cannot rank anonymous teams\.We hold Axis A \(codebook\) and the candidate set fixed, and swap only Axis B\. On six homogeneous settings Eq\. \([1](https://arxiv.org/html/2609.02264#S3.E1)\) is constant inAA, so every candidate in the short list receives the same score; the edge\-count head is then the only separator, and because\|E\|\|E\|correlates with tokens atr≈−0\.4r\\approx\-0\.4, it points toward expensive graphs\. Empirically this incumbent*scorer*, reranking our codebook candidates, costs 1711 tokens/query on GSM8K and 918 on homogeneous HumanEval—worse than uniform random over the same candidates \(1249 / 750\), and not comparable to the full incumbent*pipeline*of Q4\. Heterogeneous HumanEval is the exception on accuracy \(incumbent 78\.8 vs\. ours 78\.1\); our largest token saving \(33\.2%33\.2\\%\) is also there\. Replacing our proxy with random, or dropping rerank and taking the predictor’s top\-1 \(Figure[5](https://arxiv.org/html/2609.02264#S5.F5)c\), shows the ranking is load\-bearing for cost: random costs2323–35%35\\%more tokens at equal or lower accuracy; top\-1 is already cheap \(913 tokens on GSM8K\) and rerank adds0\.60\.6–1\.01\.0accuracy points\. The chain fixed topology is the extreme positive control for the inverted surrogate: sparsest connected, most token\-expensive on every math\-format benchmark\.

\(Q4\) One\-pass generation is\>120×\{\>\}120\{\\times\}faster and2222–33%33\\%cheaper in tokens, and the ordering transfers\.Figure[3](https://arxiv.org/html/2609.02264#S4.F3)places MATH accuracy against median generation latency: iterative generators sit at 301–396 ms; Codebook Agent at 2\.4–2\.5 ms \(125125–158×158\{\\times\}\)\. The gap is structural—T​K=250TK\{=\}250sparse GNN evaluations vs\. one predictor pass,≤M\{\\leq\}Mdecodes, and one batched MLP call \(Eq\. \([11](https://arxiv.org/html/2609.02264#S4.E11)\)\)—and design falls below0\.1%0\.1\\%of end\-to\-end latency\. Figure[4](https://arxiv.org/html/2609.02264#S5.F4)shows the accuracy–token frontier on MATH and MBPP; Codebook Agent occupies the lower\-right corner\. Against the full incumbent pipeline \(iterative decoder\+\{\+\}edge\-count GNN\), mean tokens drop21\.921\.9–33\.2%33\.2\\%\(e\.g\.1239→9271239\{\\to\}927GSM8K,2304→16112304\{\\to\}1611MATH,699→546699\{\\to\}546HumanEval\-homo,624→417624\{\\to\}417HumanEval\-het\), and wall clock follows \(MATH43\.7→27\.443\.7\{\\to\}27\.4min; MMLU12\.7→7\.912\.7\{\\to\}7\.9min\)\. Table[2](https://arxiv.org/html/2609.02264#S5.T2)repeats the accuracy comparison with Qwen\-3\-8B: absolute numbers fall, but Codebook Agent still leads at 74\.0 vs\. GTD 72\.7 / G\-Designer 72\.1, with MATH\+1\.7\+1\.7and MMLU\+1\.9\+1\.9over GTD—so the codebook is not an artifact of one API\.

Overall\.Q1–Q4 close the loop opened in the introduction: when reward\-surviving topologies are few, accuracy is preserved by selecting among them \(Q1–Q2\), cost is won by scoring measured tokens rather than\|E\|\|E\|on anonymous teams \(Q3\), and amortization makes that selection essentially free at test time while transferring across backends \(Q4\)\. The ablations below isolate each mechanism under controlled Axis A/B swaps\.

Figure 4:Accuracy–token trade\-off on MATH and MBPP\.Each bubble is one method:xxis accuracy,yyisln⁡\(tokens×n\)\\ln\(\\mathrm\{tokens\}\\times n\)so that Token Consump\.=ey=e^\{y\}, and area scales with mean tokens per query\. Lower\-right is better; Codebook Agent sits nearest that corner on both benchmarks\.
### 5\.3Ablations

We ablate the mechanisms behind Eqs\. \([4](https://arxiv.org/html/2609.02264#S4.E4)\)–\([11](https://arxiv.org/html/2609.02264#S4.E11)\) and test whether the main\-table ordering survives a backbone change\. Figure[5](https://arxiv.org/html/2609.02264#S5.F5)varies four controls on GSM8K and homogeneous HumanEval under a fixed training budget; Table[2](https://arxiv.org/html/2609.02264#S5.T2)repeats the accuracy comparison with Qwen\-3\-8B agents\.

![Refer to caption](https://arxiv.org/html/2609.02264v1/Figures/fig_ablation.png)Figure 5:Ablations on GSM8K and homogeneous HumanEval\.\(a\) Accuracy versus team sizeNN\(retrained perNNon the same training budget\)\. \(b\) Accuracy versus codebook sizeKK; used codes saturate near six for everyK≥8K\{\\geq\}8\. \(c\) Mean tokens per query for MLP, random, and GNN reranking under a fixed candidate set\. \(d\) Mean tokens versus the rerank cost weightλcost\\lambda\_\{\\mathrm\{cost\}\}, with default−0\.1\-0\.1\.*\(a\) Team sizeNN\.*Figure[5](https://arxiv.org/html/2609.02264#S5.F5)\(a\): retrain perN∈\{2,…,10\}N\\in\\\{2,\\dots,10\\\}on the same 50\-task budget\. GSM8K stays flat \(93\.093\.0–95\.595\.5\); HumanEval rises71\.9→87\.571\.9\{\\to\}87\.5\. Coding benefits from a wider expert pool, but the same index\-and\-select pipeline tracks that gain without architectural change—so the method is not a four\-node trick, and topology selection remains useful as the team grows\.

*\(b\) Codebook sizeKK\.*Figure[5](https://arxiv.org/html/2609.02264#S5.F5)\(b\):K∈\{4,…,64\}K\\in\\\{4,\\dots,64\\\}moves accuracy by≤1\.5\{\\leq\}1\.5\(GSM8K\) /2\.52\.5\(HumanEval\)\. For everyK≥8K\{\\geq\}8the encoder uses at most six codes—extra capacity stays idle under EMA with dead\-code reset—so the design space, not the quantizer, is what collapsed\. Searching the fullN×NN\{\\times\}Nadjacency at test time cannot be justified by needing more capacity\. The used\-code count plateaus near the same six survivors across both benchmarks, which is why enlargingKKpast 16 yields almost no accuracy return under a matched training budget\.

*\(c\) Rerank signal\.*Figure[5](https://arxiv.org/html/2609.02264#S5.F5)\(c\): same Top\-MMcandidates; only the selector changes\. Tokens on GSM8K / HumanEval are MLP927927/546546, random12491249/750750, GNN17111711/918918, with accuracy within1\.51\.5points—the ranking buys cost, not large Pass@1\. The GNN is worse than random because on homogeneous teams Eq\. \([1](https://arxiv.org/html/2609.02264#S3.E1)\) is constant inAAand its\|E\|\|E\|head favors expensive graphs \(r≈−0\.4r\\approx\-0\.4with tokens\)\. Only the MLP, readingvec⁡\(A\)\\mathrm\{vec\}\(A\)and regressing measuredτ~\\tilde\{\\tau\}, separates candidates on the true objective\. Keeping the short list fixed isolates the critic: any token gap here is attributable to Eq\. \([11](https://arxiv.org/html/2609.02264#S4.E11)\) alone\.

*\(d\) Cost weightλcost\\lambda\_\{\\mathrm\{cost\}\}\.*Figure[5](https://arxiv.org/html/2609.02264#S5.F5)\(d\):λ=0\\lambda\{=\}0is costliest \(11081108/743743tokens\) with no accuracy gain\. Tokens fall to927927\(GSM8K\) and442442\(HumanEval at−0\.4\-0\.4\); accuracy peaks at−0\.1\-0\.1/−0\.2\-0\.2\. Default−0\.1\-0\.1sits on the frontier: the proxy must both see structure and be asked to care about cost\. The prior already prefers cheap codes \(top\-1:913913on GSM8K\), and rerank adds0\.60\.6–1\.01\.0accuracy points\. Pushingλ\\lambdamore negative continues to cut tokens but begins to trade accuracy, so−0\.1\-0\.1is a deliberate operating point rather than an unconstrained minimum\-cost choice\.

*Qwen\-3\-8B transfer*\(Table[2](https://arxiv.org/html/2609.02264#S5.T2)\)\. Same teams and 300\-record protocol, backbone swapped to Qwen\-3\-8B\. Absolute numbers drop \(CoT avg\.67\.567\.5\), but Codebook Agent still leads every column \(92\.792\.7/63\.563\.5/65\.865\.8, avg\.74\.074\.0\) over GTD \(72\.772\.7\) and G\-Designer \(72\.172\.1\)\. MATH \(\+1\.7\+1\.7vs\. GTD\) and MMLU \(\+1\.9\+1\.9; three\-expert team\) show the gain is not confined to saturated arithmetic or four MathSolvers\. Design never calls the agent LLM \(Eq\. \([11](https://arxiv.org/html/2609.02264#S4.E11)\)\), so swapping the backbone changes only the rewards in𝒟\\mathcal\{D\}—the short\-list recipe is not an API artifact\. Relative ordering among adaptive designers is preserved under the weaker model, which suggests that the amortized index, not gpt\-4o\-mini idiosyncrasies, drives the main\-table gaps\.

Table 2:Qwen\-3\-8B transfer\.Accuracy \(%\) on GSM8K, MATH, and MMLU under the same agent teams and protocol as Section[5\.1](https://arxiv.org/html/2609.02264#S5.SS1), with Qwen\-3\-8B replacing gpt\-4o\-mini\. Avg\. is the three\-benchmark mean; deltas are from CoT; bold marks each column’s best\.

## 6Conclusion

We find that effective multi\-agent topologies form a short list, that denser graphs need not cost fewer tokens, and that message\-passing critics can miss structure on homogeneous teams\. Accordingly, Codebook Agent indexes a small discrete set of topologies and selects among them with a lightweight proxy \(Eqs\. \([4](https://arxiv.org/html/2609.02264#S4.E4)\)–\([11](https://arxiv.org/html/2609.02264#S4.E11)\)\), leading all six benchmarks and the Qwen\-3\-8B transfer while cutting design latency to 2\.4 ms and LLM tokens by 21\.9–33\.2%\.

## References

- Austinet al\.\(2021\)J\. Austin, A\. Odena, M\. Nye, M\. Bosma, H\. Michalewski, D\. Dohan, E\. Jiang, C\. Cai, M\. Terry, Q\. Le, and C\. SuttonProgram synthesis with large language models\.arXiv preprint arXiv:2108\.07732\.Cited by:[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p1.1)\.
- Cemriet al\.\(2025\)M\. Cemri, M\. Z\. Pan, S\. Yang, L\. A\. Agrawal, B\. Chopra, R\. Tiwari, K\. Keutzer, A\. Parameswaran, D\. Klein, K\. Ramchandran, M\. Zaharia, J\. E\. Gonzalez, and I\. StoicaWhy do multi\-agent LLM systems fail?\.InAdvances in Neural Information Processing Systems 38, Datasets and Benchmarks Track,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px3.p1.1)\.
- Chenet al\.\(2024a\)L\. Chen, J\. Q\. Davis, B\. Hanin, P\. Bailis, I\. Stoica, M\. Zaharia, and J\. ZouAre more LLM calls all you need? towards the scaling properties of compound AI systems\.InAdvances in Neural Information Processing Systems 37,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px3.p1.1)\.
- Chenet al\.\(2024b\)L\. Chen, M\. Zaharia, and J\. ZouFrugalGPT: how to use large language models while reducing cost and improving performance\.Transactions on Machine Learning Research\.Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px3.p1.1)\.
- Chenet al\.\(2021\)M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. Ponde de Oliveira Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman,et al\.Evaluating large language models trained on code\.arXiv preprint arXiv:2107\.03374\.Cited by:[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p1.1)\.
- Chenet al\.\(2024c\)W\. Chen, Y\. Su, J\. Zuo, C\. Yang, C\. Yuan, C\. Chan, H\. Yu, Y\. Lu, Y\. Hung, C\. Qian, Y\. Qin, X\. Cong, R\. Xie, Z\. Liu, M\. Sun, and J\. ZhouAgentVerse: facilitating multi\-agent collaboration and exploring emergent behaviors\.InProceedings of the 12th International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px1.p1.1)\.
- Chenet al\.\(2025\)W\. Chen, J\. Yuan, C\. Qian, C\. Yang, Z\. Liu, and M\. SunOptima: optimizing effectiveness and efficiency for LLM\-based multi\-agent system\.InFindings of the Association for Computational Linguistics: ACL 2025,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px1.p1.1)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. SchulmanTraining verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p1.1)\.
- Duet al\.\(2024\)Y\. Du, S\. Li, A\. Torralba, J\. B\. Tenenbaum, and I\. MordatchImproving factuality and reasoning in language models through multiagent debate\.InProceedings of the 41st International Conference on Machine Learning,Cited by:[§1](https://arxiv.org/html/2609.02264#S1.p1.1),[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p2.1)\.
- Hendryckset al\.\(2021a\)D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. SteinhardtMeasuring massive multitask language understanding\.InProceedings of the 9th International Conference on Learning Representations,Cited by:[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p1.1)\.
- Hendryckset al\.\(2021b\)D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. SteinhardtMeasuring mathematical problem solving with the MATH dataset\.InAdvances in Neural Information Processing Systems 34, Datasets and Benchmarks Track,Cited by:[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p1.1)\.
- Hintonet al\.\(2015\)G\. Hinton, O\. Vinyals, and J\. DeanDistilling the knowledge in a neural network\.arXiv preprint arXiv:1503\.02531\.Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px2.p1.1)\.
- Hoet al\.\(2020\)J\. Ho, A\. Jain, and P\. AbbeelDenoising diffusion probabilistic models\.InAdvances in Neural Information Processing Systems 33,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px2.p1.1)\.
- Honget al\.\(2024\)S\. Hong, M\. Zhuge, J\. Chen, X\. Zheng, Y\. Cheng, J\. Wang, C\. Zhang, Z\. Wang, S\. K\. S\. Yau, Z\. Lin, L\. Zhou, C\. Ran, L\. Xiao, C\. Wu, and J\. SchmidhuberMetaGPT: meta programming for a multi\-agent collaborative framework\.InProceedings of the 12th International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2609.02264#S1.p1.1),[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px1.p1.1)\.
- Huet al\.\(2025\)S\. Hu, C\. Lu, and J\. CluneAutomated design of agentic systems\.InProceedings of the 13th International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p2.1)\.
- Jianget al\.\(2023\)D\. Jiang, X\. Ren, and B\. Y\. LinLLM\-blender: ensembling large language models with pairwise ranking and generative fusion\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Cited by:[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p2.1)\.
- Jianget al\.\(2026\)E\. H\. Jiang, M\. Li, G\. Wan, S\. Yin, Y\. Wu, X\. Liang, X\. Li, Y\. Sun, W\. Wang, K\. Chang, and Y\. N\. WuDynamic generation of multi\-LLM agents communication topologies with graph diffusion models\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Cited by:[§1](https://arxiv.org/html/2609.02264#S1.p1.1),[§1](https://arxiv.org/html/2609.02264#S1.p2.1),[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px1.p1.1),[§3](https://arxiv.org/html/2609.02264#S3.SS0.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p2.1)\.
- Joet al\.\(2022\)J\. Jo, S\. Lee, and S\. J\. HwangScore\-based generative modeling of graphs via the system of stochastic differential equations\.InProceedings of the 39th International Conference on Machine Learning,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px2.p1.1)\.
- Liet al\.\(2024\)J\. Li, Q\. Zhang, Y\. Yu, Q\. Fu, and D\. YeMore agents is all you need\.Transactions on Machine Learning Research\.Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px3.p1.1)\.
- Liet al\.\(2026\)S\. Li, Y\. Liu, Q\. Wen, C\. Zhang, and S\. PanAssemble your crew: automatic multi\-agent communication topology design via autoregressive graph generation\.InProceedings of the 40th AAAI Conference on Artificial Intelligence,Cited by:[§1](https://arxiv.org/html/2609.02264#S1.p2.1),[§3](https://arxiv.org/html/2609.02264#S3.SS0.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p2.1)\.
- Liuet al\.\(2024\)Z\. Liu, Y\. Zhang, P\. Li, Y\. Liu, and D\. YangDynamic LLM\-agent network: an LLM\-agent collaboration framework with agent team optimization\.InProceedings of the 1st Conference on Language Modeling,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p2.1)\.
- Patelet al\.\(2021\)A\. Patel, S\. Bhattamishra, and N\. GoyalAre NLP models really able to solve simple math word problems?\.InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,Cited by:[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p1.1)\.
- Qianet al\.\(2024\)C\. Qian, W\. Liu, H\. Liu, N\. Chen, Y\. Dang, J\. Li, C\. Yang, W\. Chen, Y\. Su, X\. Cong, J\. Xu, D\. Li, Z\. Liu, and M\. SunChatDev: communicative agents for software development\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Cited by:[§1](https://arxiv.org/html/2609.02264#S1.p1.1),[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px1.p1.1)\.
- Qianet al\.\(2025\)C\. Qian, Z\. Xie, Y\. Wang, W\. Liu, K\. Zhu, H\. Xia, Y\. Dang, Z\. Du, W\. Chen, C\. Yang, Z\. Liu, and M\. SunScaling large language model\-based multi\-agent collaboration\.InProceedings of the 13th International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2609.02264#S1.p1.1)\.
- Razaviet al\.\(2019\)A\. Razavi, A\. van den Oord, and O\. VinyalsGenerating diverse high\-fidelity images with VQ\-VAE\-2\.InAdvances in Neural Information Processing Systems 32,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2609.02264#S4.SS1.p2.1)\.
- Roy and Roth \(2015\)S\. Roy and D\. RothSolving general arithmetic word problems\.InProceedings of the 2015 Conference on Empirical Methods in Natural Language Processing,Cited by:[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p1.1)\.
- Simonovsky and Komodakis \(2018\)M\. Simonovsky and N\. KomodakisGraphVAE: towards generation of small graphs using variational autoencoders\.InProceedings of the 27th International Conference on Artificial Neural Networks,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px2.p1.1)\.
- Smitet al\.\(2024\)A\. Smit, N\. Grinsztajn, P\. Duckworth, T\. D\. Barrett, and A\. PretoriusShould we be going MAD? a look at multi\-agent debate strategies for LLMs\.InProceedings of the 41st International Conference on Machine Learning,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px3.p1.1)\.
- Songet al\.\(2021\)J\. Song, C\. Meng, and S\. ErmonDenoising diffusion implicit models\.InProceedings of the 9th International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px2.p1.1)\.
- Sunet al\.\(2026\)R\. Sun, J\. Ding, C\. Gong, T\. Gu, Y\. Jiang, J\. Zhang, L\. Pan, and L\. LüTopoDIM: one\-shot topology generation of diverse interaction modes for multi\-agent systems\.InFindings of the Association for Computational Linguistics: ACL 2026,Cited by:[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p2.1)\.
- Tianet al\.\(2023\)Y\. Tian, C\. Zhang, Z\. Guo, X\. Zhang, and N\. V\. ChawlaLearning MLPs on graphs: a unified view of effectiveness, robustness, and efficiency\.InProceedings of the 11th International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px2.p1.1)\.
- van den Oordet al\.\(2017\)A\. van den Oord, O\. Vinyals, and K\. KavukcuogluNeural discrete representation learning\.InAdvances in Neural Information Processing Systems 30,Cited by:[§1](https://arxiv.org/html/2609.02264#S1.p4.1),[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2609.02264#S4.SS1.p2.1)\.
- Veličkovićet al\.\(2018\)P\. Veličković, G\. Cucurull, A\. Casanova, A\. Romero, P\. Liò, and Y\. BengioGraph attention networks\.InProceedings of the 6th International Conference on Learning Representations,Cited by:[§3](https://arxiv.org/html/2609.02264#S3.SS0.SSS0.Px2.p1.2)\.
- Vignacet al\.\(2023\)C\. Vignac, I\. Krawczuk, A\. Siraudin, B\. Wang, V\. Cevher, and P\. FrossardDiGress: discrete denoising diffusion for graph generation\.InProceedings of the 11th International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2023\)X\. Wang, J\. Wei, D\. Schuurmans, Q\. Le, E\. Chi, S\. Narang, A\. Chowdhery, and D\. ZhouSelf\-consistency improves chain of thought reasoning in language models\.InProceedings of the 11th International Conference on Learning Representations,Cited by:[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p2.1)\.
- Wanget al\.\(2025\)Z\. Wang, Y\. Wang, X\. Liu, L\. Ding, M\. Zhang, J\. Liu, and M\. ZhangAgentDropout: dynamic agent elimination for token\-efficient and high\-performance LLM\-based multi\-agent collaboration\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p2.1)\.
- Weiet al\.\(2022\)J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. Chi, Q\. Le, and D\. ZhouChain\-of\-thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems 35,Cited by:[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p2.1)\.
- Wuet al\.\(2023\)L\. Wu, H\. Lin, Y\. Huang, T\. Fan, and S\. Z\. LiExtracting low\-/high\-frequency knowledge from graph neural networks and injecting it into MLPs: an effective GNN\-to\-MLP distillation framework\.InProceedings of the 37th AAAI Conference on Artificial Intelligence,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px2.p1.1)\.
- Wuet al\.\(2024\)Q\. Wu, G\. Bansal, J\. Zhang, Y\. Wu, B\. Li, E\. Zhu, L\. Jiang, X\. Zhang, S\. Zhang, J\. Liu, A\. H\. Awadallah, R\. W\. White, D\. Burger, and C\. WangAutoGen: enabling next\-gen LLM applications via multi\-agent conversation\.InProceedings of the 1st Conference on Language Modeling,Cited by:[§1](https://arxiv.org/html/2609.02264#S1.p1.1),[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px1.p1.1)\.
- Yanget al\.\(2024\)L\. Yang, Y\. Tian, M\. Xu, Z\. Liu, S\. Hong, W\. Qu, W\. Zhang, B\. Cui, M\. Zhang, and J\. LeskovecVQGraph: rethinking graph representation space for bridging GNNs and MLPs\.InProceedings of the 12th International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px2.p1.1),[§4\.3](https://arxiv.org/html/2609.02264#S4.SS3.p2.1)\.
- Youet al\.\(2018\)J\. You, R\. Ying, X\. Ren, W\. L\. Hamilton, and J\. LeskovecGraphRNN: generating realistic graphs with deep auto\-regressive models\.InProceedings of the 35th International Conference on Machine Learning,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px2.p1.1)\.
- Zhanget al\.\(2025a\)G\. Zhang, L\. Niu, J\. Fang, K\. Wang, L\. Bai, and X\. WangMulti\-agent architecture search via agentic supernet\.InProceedings of the 42nd International Conference on Machine Learning,Cited by:[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p2.1)\.
- Zhanget al\.\(2025b\)G\. Zhang, Y\. Yue, Z\. Li, S\. Yun, G\. Wan, K\. Wang, D\. Cheng, J\. X\. Yu, and T\. ChenCut the crap: an economical communication pipeline for LLM\-based multi\-agent systems\.InProceedings of the 13th International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2609.02264#S1.p1.1),[§1](https://arxiv.org/html/2609.02264#S1.p2.1),[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px1.p1.1),[§3](https://arxiv.org/html/2609.02264#S3.SS0.SSS0.Px2.p1.1)\.
- Zhanget al\.\(2025c\)G\. Zhang, Y\. Yue, X\. Sun, G\. Wan, M\. Yu, J\. Fang, K\. Wang, T\. Chen, and D\. ChengG\-designer: architecting multi\-agent communication topologies via graph neural networks\.InProceedings of the 42nd International Conference on Machine Learning,Cited by:[§1](https://arxiv.org/html/2609.02264#S1.p1.1),[§1](https://arxiv.org/html/2609.02264#S1.p2.1),[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px1.p1.1),[§3](https://arxiv.org/html/2609.02264#S3.SS0.SSS0.Px1.p1.1),[§3](https://arxiv.org/html/2609.02264#S3.SS0.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p2.1)\.
- Zhanget al\.\(2025d\)J\. Zhang, J\. Xiang, Z\. Yu, F\. Teng, X\. Chen, J\. Chen, M\. Zhuge, X\. Cheng, S\. Hong, J\. Wang, B\. Zheng, B\. Liu, Y\. Luo, and C\. WuAFlow: automating agentic workflow generation\.InProceedings of the 13th International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p2.1)\.
- Zhanget al\.\(2022\)S\. Zhang, Y\. Liu, Y\. Sun, and N\. ShahGraph\-less neural networks: teaching old MLPs new tricks via distillation\.InProceedings of the 10th International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px2.p1.1)\.
- Zhouet al\.\(2024\)C\. Zhou, X\. Wang, and M\. ZhangUnifying generation and prediction on graphs with latent graph diffusion\.InAdvances in Neural Information Processing Systems 37,Cited by:[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px2.p1.1)\.
- Zhugeet al\.\(2024\)M\. Zhuge, W\. Wang, L\. Kirsch, F\. Faccio, D\. Khizbullin, and J\. SchmidhuberGPTSwarm: language agents as optimizable graphs\.InProceedings of the 41st International Conference on Machine Learning,Cited by:[§1](https://arxiv.org/html/2609.02264#S1.p1.1),[§1](https://arxiv.org/html/2609.02264#S1.p2.1),[§2](https://arxiv.org/html/2609.02264#S2.SS0.SSS0.Px1.p1.1),[§3](https://arxiv.org/html/2609.02264#S3.SS0.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2609.02264#S5.SS1.p2.1)\.