OrchSLM: Probing the Dynamics of Small Language Model Orchestration

# OrchSLM: Probing the Dynamics of Small Language Model Orchestration
Source: [https://arxiv.org/html/2609.13470](https://arxiv.org/html/2609.13470)
Chengxi Zhang††thanks:Equal contribution; order decided by a coin flip\.Yu Yao11footnotemark:1Affiliation:MITEmail:[yu\_yao@mit\.edu](mailto:)

###### Abstract

Although large language models \(LLMs\) have demonstrated remarkable capabilities, their reliance on cloud\-scale infrastructure poses fundamental challenges for deployment in agentic pipelines, including latency, privacy, connectivity, and substantial computational cost\. Small language models \(SLMs\) offer a compelling alternative: recent studies suggest that many repetitive and narrowly scoped subtasks in agentic workloads may be better served by specialized SLMs than by monolithic LLMs\. However, the limited capacity and context windows of SLMs can constrain long\-horizon reasoning and interaction\-heavy orchestration strategies such as iterative verification and debate\. This motivates a complementary, non\-interactive paradigm in which heterogeneous SLMs independently generate candidate solutions and a router orchestrates their cached samples without further model interaction\. To further understand the mechanisms of such orchestration, we introduce OrchSLM, a routing framework that unifies existing non\-interactive orchestration methods and exposes their underlying design choices as controllable parameters\. Using OrchSLM as a systematic probe, we reveal how orchestration behavior emerges from diverse knobs, including the task structure, model\-pool composition, and multi\-agent consensus\.

### 1Introduction

Large language models have demonstrated remarkable progress across reasoning, coding, tool use, and increasingly complex agentic tasks\([OpenAI, 2023](https://arxiv.org/html/2609.13470#bib.bib8);[Touvron et al\., 2023](https://arxiv.org/html/2609.13470#bib.bib9);[Grattafiori et al\., 2024](https://arxiv.org/html/2609.13470#bib.bib10);[DeepSeek\-AI, 2025](https://arxiv.org/html/2609.13470#bib.bib11);[Snell et al\., 2024b](https://arxiv.org/html/2609.13470#bib.bib36)\)\. Scaling model capacity and inference\-time computation has further extended their ability to solve challenging problems\([Kaplan et al\., 2020](https://arxiv.org/html/2609.13470#bib.bib3);[Hoffmann et al\., 2022](https://arxiv.org/html/2609.13470#bib.bib4);[Lightman et al\., 2024](https://arxiv.org/html/2609.13470#bib.bib5);[Brown et al\., 2024](https://arxiv.org/html/2609.13470#bib.bib6);[Snell et al\., 2024a](https://arxiv.org/html/2609.13470#bib.bib7)\)\. However, these capabilities come with substantial computational and deployment costs, particularly when language models must be invoked repeatedly as components of agentic pipelines\. As an alternative, small language models are increasingly emerging as low\-cost, deployable, and heterogeneous reasoning engines\([Wang et al\., 2024a](https://arxiv.org/html/2609.13470#bib.bib31);[Belcak et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib44)\)\. Their modest computational footprint makes it practical to query multiple models at inference time, raising the possibility of treating a collection of SLMs as a pool of complementary specialists\. However, most orchestration strategies developed for larger language models rely heavily on interaction: models exchange natural\-language messages, critique intermediate solutions, synthesize one another’s responses, or verify candidate answers before a final decision is made\([Wang et al\., 2024b](https://arxiv.org/html/2609.13470#bib.bib34);[Du et al\., 2023](https://arxiv.org/html/2609.13470#bib.bib32);[Lifshitz et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib33)\)\. For SLMs, such interaction can be costly relative to the models themselves and relies on deliberation and self\-correction capabilities that remain limited\([Taubenfeld et al\., 2024](https://arxiv.org/html/2609.13470#bib.bib47);[Huang et al\., 2024](https://arxiv.org/html/2609.13470#bib.bib48);[Liu et al\., 2023](https://arxiv.org/html/2609.13470#bib.bib49);[Fu et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib50)\)\.

This motivates a complementary,non\-interactive111We usenon\-interactiveto mean that models do not communicate with one another or condition their generations on other models’ outputs\. Each model generates independently, and cross\-model information is combined only by the router after generation, without consuming additional context tokens for inter\-model communication\.setting: each model independently generates a set of cached samples, and final answers are obtained by combining the resulting answer distributions at inference time\. Despite its simplicity, a systematic understanding of such non\-interactive orchestration still remain unknown\. Existing methods make fundamentally different implicit assumptions about what constitutes useful evidence\. Query\-level routers and cascades choose a model before generation, typically based on predicted expertise, cost, or preference\([Chen et al\., 2023](https://arxiv.org/html/2609.13470#bib.bib39);[Ong et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib30);[Chen et al\., 2024](https://arxiv.org/html/2609.13470#bib.bib46)\)\. Voting methods instead treat agreement across independently generated answers as evidence\([Wang et al\., 2023](https://arxiv.org/html/2609.13470#bib.bib13);[Li et al\., 2024](https://arxiv.org/html/2609.13470#bib.bib35)\)\. Routing\-style routing estimates within\-model consistency and favors answers from models that repeatedly support the same prediction\([Wang et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib1)\)\. These approaches expose distinct signals \(expertise,self\-consistency, andmutual agreement\) but entangle them through different routing rules\.

To understand these signals is important because each can be informative in one regime and misleading in another\. A strong expert may provide a reliable default answer, yet expert selection discards corroborating evidence from the rest of the pool\. Majority agreement can reveal shared evidence, yet a collection of weaker models may reinforce the same plausible error\([Li et al\., 2024](https://arxiv.org/html/2609.13470#bib.bib35);[Wang et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib1)\)\. High self\-consistency can indicate confidence, but it can also arise from a model that repeatedly makes the same mistake\([Wang et al\., 2023](https://arxiv.org/html/2609.13470#bib.bib13);[Wang et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib1)\)\. Even the composition of the model pool introduces a nontrivial tradeoff: adding another model may increase the probability that some model produces the correct answer while simultaneously introducing additional incorrect support that makes the answer harder to identify\. Consequently, orchestration quality is governed not by the number of available models or the amount of agreement alone, but by how expertise, agreement, and pool composition interact with the structure of the task\. This raises the central question of our work:what governs the dynamics of small language model orchestration?Rather than searching for a single universally optimal router, we seek to isolate the mechanisms that determine when heterogeneous SLMs complement one another, when collective agreement provides useful evidence, and when adding more models instead makes routing harder\.

To probe these tradeoffs systematically, we decompose non\-interactive multi\-SLM orchestration along three research axis\.\(1\) Whom should we trust?How should authority be distributed across models?\(2\) How should evidence combine?How should multiple beliefs form a collective decision?\(3\) When does agreement matter?When does consensus provide evidence beyond individual confidence? Existing routing methods implicitly make different choices along these dimensions\. Majority voting weights models equally, accumulates support across the pool, and relies on mutual agreement\. Single\-best routing places all trust in a fixed expert\. Router\-style favors the model exhibiting the strongest self\-consistency, while cross\-support treats probability mass assigned by other models to a candidate answer as question\-specific evidence\. We make these choices explicit as controllable axes, allowing existing routers and their intermediate variants to be studied within a common design space\.

We instantiate this design space as a generalized routing family in which common orchestration methods emerge as exact parameter settings, and use it as a controlled probe of orchestration dynamics\. Across MATH\-500, GSM8K, GPQA, and MMLU\-Pro with a heterogeneous model pool, our systematic analysis reveals distinct dynamics of multi\-SLM orchestration: effective routing emerges from the interplay between task structure, model expertise, pool composition, and agreement patterns, rather than from any single source of evidence\.

Contributions\.\(1\) We formulate non\-interactive multi\-SLM orchestration as a controlled design space organized around three coupled mechanisms:whom to trust,how to aggregate evidence, andwhat form of consistency should constitute confidence\. \(2\) We introduceOrchSLM, a generalized routing family that exposes these mechanisms as controllable axes and recovers majority voting, self\-consistency routing, cross\-support, expert\-weighted routing, and the single\-best limit as specific configurations within a common framework\. \(3\) We useOrchSLMto systematically probe the dynamics of multi\-SLM orchestration across tasks and model pools\. Our experiments reveal how expertise, agreement, and pool composition jointly determine routing behavior, including a persistent gap betweenoracle coverageandroutable support: more models can make the correct answer available without making it easier to identify\.

### 2Preliminaries and Methodology

Figure 1:Different routers can pick different answers from the same cached outputs\.Motivation\.Orchestration is a problem of collective decision\-making\([Arrow, 1951](https://arxiv.org/html/2609.13470#bib.bib2)\): given multiple imperfect and heterogeneous reasoners, how should their individual beliefs be transformed into a collective judgment? More voices do not necessarily imply more knowledge, agreement does not necessarily imply truth, and expertise does not necessarily imply correctness on every instance\. The challenge is therefore not merely to combine model outputs, but to determine when individual authority should prevail, when evidence should accumulate, and when agreement should be trusted\. We study this problem through three fundamental questions\.Whom should we trust?Models differ in expertise, raising a tension between democratic aggregation and expert authority\.How should evidence combine?Support may derive its strength from accumulation across many independent sources, or from the conviction of a particularly reliable one\.When does agreement matter?Repeated belief can reflect genuine confidence, but consensus can also amplify shared errors; its value therefore depends on who agrees and how their evidence is formed\. These questions expose three distinct but coupled mechanisms underlying multi\-SLM orchestration:trust,aggregation, andconsistency\. We next formalize them as controllable axes of a generalized routing family, allowing different orchestration strategies to be studied within a common framework\.

#### 2\.1Evaluation of Orchestration

How should collective evidence become a decision, and whose evidence actually shapes that decision? We evaluate orchestration through these two complementary perspectives\. The first concernsdecision: we formalize how trust, aggregation, and agreement determine which answer is selected\. The second concernsinfluence: we distinguish between models that are heard by the router and models whose presence actually improves its decision\. Together, these views characterize not only what an orchestration system decides, but also how that collective decision emerges\. Additional derivations and implementation details are provided in Appendix §[A](https://arxiv.org/html/2609.13470#A1)\.

From samples to beliefs\.For a questionxx, each modeliiindependently generatesKKsamples\. After answer extraction and normalization, these samples induce an empirical distribution over candidate answers:

Pi​a\(x\)=1K∑k=1K𝟏\[yi​k=a\]\.P\_\{ia\}\(x\)=\\frac\{1\}\{K\}\\sum\_\{k=1\}^\{K\}\\mathbf\{1\}\[y\_\{ik\}=a\]\.\(1\)Here,Pi​a​\(x\)P\_\{ia\}\(x\)represents the empirical support that modeliiassigns to answeraa\. Letai⋆=arg⁡maxa​Pi​a​\(x\)a\_\{i\}^\{\\star\}=\\arg\\max\_\{a\}P\_\{ia\}\(x\)denote modelii’s most frequent answer, with ties broken deterministically, and defineMi​a=𝟏\[a=ai⋆\]M\_\{ia\}=\\mathbf\{1\}\[a=a\_\{i\}^\{\\star\}\]\. Thus,Pi​aP\_\{ia\}captures the model’s full answer distribution, whileMi​aM\_\{ia\}identifies the answer on which its own belief concentrates most strongly\.

Table 2:Common routers mapped to\(α,κ,Agg\)\(\\alpha,\\kappa,\\mathrm\{Agg\}\)in the generalized family\.Figure 2:Existing routers as fixed corners of the\(α,κ,Agg\)\(\\alpha,\\kappa,\\mathrm\{Agg\}\)generalized family; axes are defined in Section[2\.1](https://arxiv.org/html/2609.13470#S2.SS1)\.Trust: whose evidence should count?Models need not be equally reliable\. Letacci\\mathrm\{acc\}\_\{i\}denote an expert prior for modelii, estimated from a validation split or supplied as a fixed prior score\. We translate these priors into normalized trust weights:

ri​\(κ\)=max⁡\(acci,ϵ\)κ∑jmax⁡\(accj,ϵ\)κ\.r\_\{i\}\(\\kappa\)=\\frac\{\\max\(\\mathrm\{acc\}\_\{i\},\\epsilon\)^\{\\kappa\}\}\{\\sum\_\{j\}\\max\(\\mathrm\{acc\}\_\{j\},\\epsilon\)^\{\\kappa\}\}\.\(2\)The parameterκ\\kappadetermines how concentrated authority is across the model pool\. Atκ=0\\kappa=0, every model receives equal weight, corresponding to a democratic prior over expertise\. Asκ\\kappaincreases, authority shifts toward models with stronger prior performance; in the limitκ→∞\\kappa\\rightarrow\\infty, it concentrates on the validation\-best model\. Thus,κ\\kappacontinuously interpolates between collective equality and expert authority\.

Aggregation and agreement: how should evidence combine?For each candidate answeraa, we define the routing score

W⁡\(a\)=α​Aggi​\[Mi​a​ri​\(κ\)​Pi​a​\(x\)\]\+\(1−α\)​∑iri​\(κ\)​Pi​a​\(x\),W\(a\)=\\alpha\\,\\mathrm\{Agg\}\_\{i\}\\left\[M\_\{ia\}r\_\{i\}\(\\kappa\)P\_\{ia\}\(x\)\\right\]\+\(1\-\\alpha\)\\sum\_\{i\}r\_\{i\}\(\\kappa\)P\_\{ia\}\(x\),\(3\)and selecta^=arg⁡maxa⁡W⁡\(a\)\\hat\{a\}=\\arg\\max\_\{a\}W\(a\)\. Here,Agg\\mathrm\{Agg\}controls how evidence combines across models: Sum accumulates support from multiple models, whereas Max retains the strongest supporter\. The parameterα\\alphacontrols the form of agreement, interpolating between support for each model’s preferred answer and pooled support over its full answer distribution\. Together,\(κ,Agg,α\)\(\\kappa,\\mathrm\{Agg\},\\alpha\)specify whom to trust, how to combine their evidence, and which form of agreement to use\.

Influence: being heard versus being helpful\.The routing score determines which answer is selected, but does not tell us whether each model helps make that selection correct\. We therefore measure model influence through attribution and contribution\.

For a datasetDD, leta^t\\hat\{a\}\_\{t\}be the routed answer for questiontt\. We define the attribution of modeliias

Atr⁡\(i\)=1\|D\|​∑t∈DPi,a^t\(t\)\.\\mathrm\{Atr\}\(i\)=\\frac\{1\}\{\|D\|\}\\sum\_\{t\\in D\}P^\{\(t\)\}\_\{i,\\hat\{a\}\_\{t\}\}\.\(4\)Atr measures how strongly a model supports the answers selected by the router\. A model with high Atr is frequentlyheard, but its support need not improve accuracy\.

To measure whether a model is helpful, we remove it and rerun the router\. We define its counterfactual contribution as

Ctr⁡\(i\)=Acc⁡\(full pool\)−Acc⁡\(pool without​i\)\.\\mathrm\{Ctr\}\(i\)=\\mathrm\{Acc\}\(\\text\{full pool\}\)\-\\mathrm\{Acc\}\(\\text\{pool without \}i\)\.\(5\)Equivalently, ifa^t\(−i\)\\hat\{a\}\_\{t\}^\{\(\-i\)\}is the routed answer without modelii, then

Ctr\(i\)=1\|D\|∑t∈D\(𝟏\[a^t=yt\]−𝟏\[a^t\(−i\)=yt\]\)\.\\mathrm\{Ctr\}\(i\)=\\frac\{1\}\{\|D\|\}\\sum\_\{t\\in D\}\\left\(\\mathbf\{1\}\[\\hat\{a\}\_\{t\}=y\_\{t\}\]\-\\mathbf\{1\}\[\\hat\{a\}\_\{t\}^\{\(\-i\)\}=y\_\{t\}\]\\right\)\.\(6\)

### 3Experiments

Setup\.All routers operate on the same cached generations from a seven\-model pool \(Qwen2\.5\-7B\-Instruct, Meta\-Llama\-3\.1\-8B\-Instruct, Llama\-3\.1\-Tulu\-3\-8B\-DPO, DeepSeek\-Math\-7B\-Instruct, Gemma\-2\-9B\-IT, Phi\-3\.5\-mini\-instruct, Ministral\-8B\-Instruct\-2410\) across four benchmarks: MATH\-500 \(competition math, one dominant expert\), GSM8K \(saturated arithmetic\), GPQA \(hard, balanced science\), and MMLU\-Pro \(broad multi\-topic science\)\. Implementation details are at Appendix §[A](https://arxiv.org/html/2609.13470#A1)\.

#### 3\.1Scaling up Task Structures

![Refer to caption](https://arxiv.org/html/2609.13470v1/fig3_phase_heatmaps_YlGnBu_r_nointerp.png)Figure 3:Accuracy heatmaps over\(α,κ\)\(\\alpha,\\kappa\)for Sum and Max aggregation on the full pool\. Dashed contour: cells within11pp of the best score; stars: best per task\.
Figure 4:Router generalizations\.Table 3:Optimal\-point behavior under the OrchSLM parameter scan\. The selected optima emphasize expert trust on the two math benchmarks, while the science benchmarks rely more on mixed or mutual support\.

The best routing strategy varies across tasks\.[Figure4](https://arxiv.org/html/2609.13470#S3.F4)compares four cache\-only routing strategies with the oracle upper bound\. On MATH\-500, single\-best routing is near\-optimal, indicating that routing is largely determined by the strongest expert\. On GSM8K, several strategies perform similarly\. In contrast, GPQA and MMLU\-Pro retain substantial gaps to the oracle, suggesting that the presence of a correct answer does not make it easy to identify\. These differences motivate us to study routing as a task\-dependent decision rather than fixing a single strategy across tasks\. We vary the three components of the generalized router: the trust concentrationκ\\kappa, the aggregation operatorAgg∈\{Sum,Max\}\\mathrm\{Agg\}\\in\\\{\\mathrm\{Sum\},\\mathrm\{Max\}\\\}, and the balanceα\\alphabetween top\-answer and pooled support\.[Figure3](https://arxiv.org/html/2609.13470#S3.F3)shows the resulting routing landscape, and[Table3](https://arxiv.org/html/2609.13470#S3.T3)summarizes the best regime for each task\. We next isolate the effect of each component\.

Table 4:Experts vs average models\. Rel\.Δ\\Deltais the relative change from the best equal\-weight cell \(κ=0\\kappa=0\) to the best full\-sweep cell\.Trust concentration\.Expert weighting has the largest effect on routing performance\.[Table4](https://arxiv.org/html/2609.13470#S3.T4)compares the best setting withκ\>0\\kappa\>0against the best equal\-weight setting withκ=0\\kappa=0\. Concentrating trust consistently improves routing across all four tasks\. On math, it prevents weaker models from diluting the strongest expert\. On the harder science tasks, it reduces the influence of unreliable support\. Thus, deciding which models to trust is the primary routing decision\.

Aggregation\.The choice between Sum and Max has a smaller effect\. As shown in[Table5](https://arxiv.org/html/2609.13470#S3.T5), GSM8K clearly favors Max, while the best Sum and Max settings are close on the other three tasks\. This is consistent with GSM8K being a saturated task, where selecting the strongest expert\-supported answer is sufficient\. In the remaining tasks, changing how support is aggregated matters less than changing which models receive weight\.

Table 5:Majority vs coalition support, vs strongest supporter\. Rel\.Δ\\Deltais the relative change from Sum to Max aggregation\.Agreement\.The usefulness of self\-consistency and mutual support depends on which models the router trusts\.[Table6](https://arxiv.org/html/2609.13470#S3.T6)compares the two signals by settingα=1\\alpha=1for top\-answer support andα=0\\alpha=0for pooled support under a fixedκ\\kappa\. With equal model weights, the two signals are similar and neither consistently dominates\. After expert weighting, their roles separate: math favors self\-consistent expert predictions, whereas the broad science task benefits more from pooled support\. Agreement is therefore informative only relative to the models providing it\.

#### 3\.2Scaling up Routers

Table 6:Self\-consistency vs mutual agreement\. Pooled\-only usesα=0\\alpha=0; top\-only usesα=1\\alpha=1\. Rel\.Δ\\Deltais the relative change from pooled\-only to top\-only routing\.Table 7:Oracle coverage is not routable support\.

Figure 5:Per\-bin fixes \(green\) and breaks \(red\) when switching self\-consistency to the best generalized cell, with the net rate \(line\)\.
Oracle coverage and routability\.Higher oracle coverage does not necessarily translate into higher routed accuracy\.[Table7](https://arxiv.org/html/2609.13470#S3.T7)compares the oracle upper bound with the best generalized routing result\. The gap is small on the math tasks but substantially larger on GPQA and MMLU\-Pro\. On these science benchmarks, a correct answer is often present in the model pool but cannot be reliably identified from the cached predictions\. Increasing coverage alone is therefore insufficient; the router must also distinguish useful evidence from competing support\.

Disagreement\.Disagreement creates both opportunities and risks for routing\.[Figure5](https://arxiv.org/html/2609.13470#S3.F5)groups questions by the level of disagreement among the seven models and measures how generalized routing changes the predictions of self\-consistency\. As disagreement increases, the router corrects more self\-consistency errors, but it also changes more correct predictions into incorrect ones\. The largest net benefit occurs at moderate disagreement, where alternative evidence is available without being dominated by conflicting support\.

Figure 6:Pool\-size scan\. Left: routed accuracy vs\. size \(gray: each subset; lines: oracle, single\-best, best generalized\)\. Right: best\-cell\(α,κ\)\(\\alpha,\\kappa\)shifts with pool size\.Pool composition\.Adding more models does not necessarily improve routed accuracy\. As shown in[Figure6](https://arxiv.org/html/2609.13470#S3.F6)\(left\), oracle coverage continues to increase with pool size, while routed accuracy typically peaks with three or four models\. Additional models can introduce correct answers that increase oracle coverage, but they can also introduce stable incorrect support that makes those answers harder to identify\. Consequently, the quality of the pool depends not only on the answers it contains, but also on the support patterns it creates\. Pool composition also changes the preferred routing strategy\.[Figure6](https://arxiv.org/html/2609.13470#S3.F6)\(right\) shows that the optimal\(α,κ\)\(\\alpha,\\kappa\)shifts as models are added or removed\. Smaller curated pools generally require less trust concentration because the remaining models are already more reliable\. Model selection and answer routing are therefore coupled decisions rather than independent stages\.

#### 3\.3Being Heard Is Not Being Helpful

Figure 7:Per\-model Atr \(heard\) vs\. Ctr \(helpful\) under each canonical router\.Attribution and contribution capture different aspects of model influence\. Atr measures how much probability mass a model assigns to the router’s selected answer, whereas Ctr measures how routed accuracy changes when that model is removed\. A model can therefore receive high attribution without making the router more accurate\.[Figure7](https://arxiv.org/html/2609.13470#S3.F7)shows that the two measures behave differently across tasks\. On math, the strongest expert has both high Atr and positive Ctr: the model that most strongly supports the routed answers is also the one that improves them\. On GPQA and MMLU\-Pro, however, several models receive substantial Atr while having near\-zero or negative Ctr\. Their predictions frequently agree with the router’s decisions, but removing them does not hurt accuracy and can sometimes improve it\. Pool curation reduces this mismatch\. After removing models with unhelpful support, attribution becomes more concentrated on models with positive counterfactual contribution\. This explains why expert weighting and pool selection can improve routing on the harder science tasks: effective orchestration depends not only on how much support a model provides, but also on whether that support is useful for selecting the correct answer\. Per\-model results are reported in[Table33](https://arxiv.org/html/2609.13470#A2.T33), with an accepted\-support decomposition in[Table34](https://arxiv.org/html/2609.13470#A2.T34)\.

### 4Related Work

Small Language Models\.Small language models \(SLMs\) trade capacity for lower latency, memory, and deployment cost\([Abdin and others, 2024](https://arxiv.org/html/2609.13470#bib.bib27);[Liu et al\., 2024](https://arxiv.org/html/2609.13470#bib.bib28)\)\. Recent gains at this scale come from curated and synthetic data, knowledge distillation, scale\-specific architectures, and targeted post\-training\([Wang et al\., 2024a](https://arxiv.org/html/2609.13470#bib.bib31);[Abdin and others, 2024](https://arxiv.org/html/2609.13470#bib.bib27);[Allal et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib45);[Gemma Team, 2024](https://arxiv.org/html/2609.13470#bib.bib29);[Liu et al\., 2024](https://arxiv.org/html/2609.13470#bib.bib28)\)\. This efficiency supports repeated or specialized calls on devices and in agentic systems\([Abdin and others, 2024](https://arxiv.org/html/2609.13470#bib.bib27);[Liu et al\., 2024](https://arxiv.org/html/2609.13470#bib.bib28);[Belcak et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib44);[Wang et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib1)\), while chain\-of\-thought, repeated sampling, and test\-time scaling improve reasoning at inference time\([Wei et al\., 2022](https://arxiv.org/html/2609.13470#bib.bib14);[Wang et al\., 2023](https://arxiv.org/html/2609.13470#bib.bib13);[Snell et al\., 2024b](https://arxiv.org/html/2609.13470#bib.bib36);[Muennighoff et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib37)\)\. However, weak self\-correction and long\-context use constrain interaction\-heavy orchestration\([Huang et al\., 2024](https://arxiv.org/html/2609.13470#bib.bib48);[Liu et al\., 2023](https://arxiv.org/html/2609.13470#bib.bib49)\)\. SLM\-MUX instead uses independent sampling, confidence\-based selection, and model\-subset search\([Wang et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib1)\); OrchSLM studies routing within this non\-interactive setting\.

Multi\-Agent and Compound AI Systems\.Multi\-agent and compound AI systems organize multiple model calls into a single solution or decision\. At the system level, routing and allocation determine which calls are made, while prompt and workflow search configure how they are used\([Chen et al\., 2023](https://arxiv.org/html/2609.13470#bib.bib39);[Ong et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib30);[Chen et al\., 2024](https://arxiv.org/html/2609.13470#bib.bib46);[Chen et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib25);[Poon et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib26);[Khattab et al\., 2023](https://arxiv.org/html/2609.13470#bib.bib40);[Opsahl\-Ong et al\., 2024](https://arxiv.org/html/2609.13470#bib.bib43);[Saad\-Falcon et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib42);[Zhang et al\., 2025a](https://arxiv.org/html/2609.13470#bib.bib41)\)\. Given a call configuration, systems differ in whether outputs interact during generation\. Interactive systems condition later calls on earlier outputs through debate, critique, synthesis, or verification\([Du et al\., 2023](https://arxiv.org/html/2609.13470#bib.bib32);[Wang et al\., 2024b](https://arxiv.org/html/2609.13470#bib.bib34);[Lifshitz et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib33)\)\. MoA variants refine this pattern through proposer quality and diversity, sparse communication, and residual, attention, or memory\-based flow\([Li et al\., 2025b](https://arxiv.org/html/2609.13470#bib.bib18);[Li et al\., 2025a](https://arxiv.org/html/2609.13470#bib.bib20);[Xie et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib19);[Wen et al\., 2026](https://arxiv.org/html/2609.13470#bib.bib21);[Ping et al\., 2026](https://arxiv.org/html/2609.13470#bib.bib22)\)\. Non\-interactive systems instead sample independently and aggregate only after generation\([Wang et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib1)\)\. OrchSLM focuses on this post\-generation decision\.

Ensembling and Test\-Time Inference\.Test\-time inference improves fixed models or model pools through additional sampling, search, verification, and aggregation\([Snell et al\., 2024b](https://arxiv.org/html/2609.13470#bib.bib36);[Zhang et al\., 2025b](https://arxiv.org/html/2609.13470#bib.bib38)\)\. In its non\-interactive branch, the remaining problem is how to select or combine independently generated candidates\. Existing rules range from sample voting and verifier\-based selection to confidence\-weighted consensus, multi\-generator and reward\-model ensembles, and reasoning\-structure aggregation\([Wang et al\., 2023](https://arxiv.org/html/2609.13470#bib.bib13);[Li et al\., 2024](https://arxiv.org/html/2609.13470#bib.bib35);[Lightman et al\., 2024](https://arxiv.org/html/2609.13470#bib.bib5);[Yao et al\., 2026](https://arxiv.org/html/2609.13470#bib.bib12);[Song et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib23);[Parulekar et al\., 2026](https://arxiv.org/html/2609.13470#bib.bib24);[Wang et al\., 2025](https://arxiv.org/html/2609.13470#bib.bib1)\)\. These rules can be organized along three axes: model authority \(equal or quality\-weighted\), evidence aggregation \(accumulated support or the strongest supporter\), and confidence source \(self\-consistency or mutual agreement\)\. Attribution methods add a separate question: whether a model merely supports a decision or changes it when removed\([Ribeiro et al\., 2016](https://arxiv.org/html/2609.13470#bib.bib15);[Lundberg and Lee, 2017](https://arxiv.org/html/2609.13470#bib.bib16);[Koh and Liang, 2017](https://arxiv.org/html/2609.13470#bib.bib17)\)\. OrchSLM parameterizes these choices and separates answer availability from routability and received support from counterfactual contribution\.

### 5Conclusion

We study SLM orchestration through three decisions: whom to trust, how to aggregate evidence, and which form of agreement to use\. The best choice depends on both the task and the model pool\. Expert weighting is the strongest routing factor, larger pools do not necessarily improve accuracy, and agreement is useful only when it comes from helpful models\. These results suggest that effective orchestration requires jointly selecting the models and deciding how their evidence should be combined\.

### Limitations

Our study is limited to cache\-only, non\-interactive orchestration and benchmarks with extractable answers\. Extending the framework to interactive or open\-ended settings would require additional generations and semantic equivalence mechanisms\. Expert priors also require validation data or an estimate of model quality\. Finally, while our all\-subset analysis explores the pool design space, we use split\-selected evaluation to test pool selection on held\-out data\.

A natural next step is to use routing signals to decide when additional interaction, verification, or debate is worth the compute\. The same signals could support adaptive pool construction, selecting models and routing strategies at the question or domain level\. Extending OrchSLM to open\-ended generation would further require semantic matching and partial\-credit evaluation while preserving interpretable influence diagnostics\.

### Disclosure of Generative AI Usage

We used large language models to assist with writing clarity, grammar, and experimental code development\. All final wording, technical claims, code, results, figures’ scientific content, and conclusions were reviewed, validated, and produced by the authors\.

### References

- Abdinet al\.\(2024\)M\. Abdinet al\.Phi\-3 technical report: a highly capable language model locally on your phone\.arXiv preprint arXiv:2404\.14219\.External Links:2404\.14219Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p1.1)\.
- Allalet al\.\(2025\)L\. B\. Allal, A\. Lozhkov, E\. Bakouch, G\. M\. Blázquez, G\. Penedo, L\. Tunstall, A\. Marafioti, H\. Kydlíček, A\. P\. Lajarín, V\. Srivastav,et al\.SmolLM2: when smol goes big–data\-centric training of a small language model\.arXiv preprint arXiv:2502\.02737\.Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p1.1)\.
- Arrow \(1951\)K\. J\. ArrowSocial choice and individual values\.John Wiley & Sons,New York\.Cited by:[§2](https://arxiv.org/html/2609.13470#S2.p1.1)\.
- Belcaket al\.\(2025\)P\. Belcak, G\. Heinrich, S\. Diao, Y\. Fu, X\. Dong, S\. Muralidharan, Y\. C\. Lin, and P\. MolchanovSmall language models are the future of agentic ai\.arXiv preprint arXiv:2506\.02153\.Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p1.1),[§4](https://arxiv.org/html/2609.13470#S4.p1.1)\.
- Brownet al\.\(2024\)B\. Brown, J\. Juravsky, R\. Ehrlich, R\. Clark, Q\. V\. Le, C\. Ré, and A\. MirhoseiniLarge language monkeys: scaling inference compute with repeated sampling\.arXiv preprint arXiv:2407\.21787\.Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p1.1)\.
- Chenet al\.\(2025\)L\. Chen, J\. Q\. Davis, B\. Hanin, P\. Bailis, M\. Zaharia, J\. Zou, and I\. StoicaOptimizing model selection for compound AI systems\.arXiv preprint arXiv:2502\.14815\.External Links:2502\.14815Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p2.1)\.
- Chenet al\.\(2023\)L\. Chen, M\. Zaharia, and J\. ZouFrugalGPT: how to use large language models while reducing cost and improving performance\.External Links:2305\.05176,[Link](https://arxiv.org/abs/2305.05176)Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p2.1),[§4](https://arxiv.org/html/2609.13470#S4.p2.1)\.
- Chenet al\.\(2024\)S\. Chen, W\. Jiang, B\. Lin, J\. T\. Kwok, and Y\. ZhangRouterDC: query\-based router by dual contrastive learning for assembling large language models\.External Links:2409\.19886,[Link](https://arxiv.org/abs/2409.19886)Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p2.1),[§4](https://arxiv.org/html/2609.13470#S4.p2.1)\.
- DeepSeek\-AI \(2025\)DeepSeek\-AIDeepSeek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.arXiv preprint arXiv:2501\.12948\.Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p1.1)\.
- Duet al\.\(2023\)Y\. Du, S\. Li, A\. Torralba, J\. B\. Tenenbaum, and I\. MordatchImproving factuality and reasoning in language models through multiagent debate\.External Links:2305\.14325,[Link](https://arxiv.org/abs/2305.14325)Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p1.1),[§4](https://arxiv.org/html/2609.13470#S4.p2.1)\.
- Fuet al\.\(2025\)T\. Fu, Z\. Min, H\. Zhang, J\. Yan, G\. Dai, W\. Ouyang, and Y\. WangCache\-to\-cache: direct semantic communication between large language models\.arXiv preprint arXiv:2510\.03215\.Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p1.1)\.
- Gemma Team \(2024\)Gemma TeamGemma 2: improving open language models at a practical size\.arXiv preprint arXiv:2408\.00118\.External Links:2408\.00118Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p1.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p1.1)\.
- Hoffmannet al\.\(2022\)J\. Hoffmann, S\. Borgeaud, A\. Mensch, E\. Buchatskaya, T\. Cai, E\. Rutherford, D\. de Las Casas, L\. A\. Hendricks, J\. Welbl, A\. Clark,et al\.Training compute\-optimal large language models\.Advances in Neural Information Processing Systems35,pp\. 30016–30030\.Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p1.1)\.
- Huanget al\.\(2024\)J\. Huang, X\. Chen, S\. Mishra, H\. S\. Zheng, A\. W\. Yu, X\. Song, and D\. ZhouLarge language models cannot self\-correct reasoning yet\.External Links:2310\.01798,[Link](https://arxiv.org/abs/2310.01798)Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p1.1),[§4](https://arxiv.org/html/2609.13470#S4.p1.1)\.
- Kaplanet al\.\(2020\)J\. Kaplan, S\. McCandlish, T\. Henighan, T\. B\. Brown, B\. Chess, R\. Child, S\. Gray, A\. Radford, J\. Wu, and D\. AmodeiScaling laws for neural language models\.arXiv preprint arXiv:2001\.08361\.Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p1.1)\.
- Khattabet al\.\(2023\)O\. Khattab, A\. Singhvi, P\. Maheshwari, Z\. Zhang, K\. Santhanam, S\. Vardhamanan, S\. Haq, A\. Sharma, T\. T\. Joshi, H\. Moazam, H\. Miller, M\. Zaharia, and C\. PottsDSPy: compiling declarative language model calls into self\-improving pipelines\.External Links:2310\.03714,[Link](https://arxiv.org/abs/2310.03714)Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p2.1)\.
- Koh and Liang \(2017\)P\. W\. Koh and P\. LiangUnderstanding black\-box predictions via influence functions\.InInternational Conference on Machine Learning,pp\. 1885–1894\.External Links:[Link](https://proceedings.mlr.press/v70/koh17a.html)Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p3.1)\.
- Liet al\.\(2025a\)D\. Li, Z\. Tan, P\. Qian, Y\. Li, K\. S\. Chaudhary, L\. Hu, and J\. ShenSMoA: improving multi\-agent large language models with sparse mixture\-of\-agents\.InPacific\-Asia Conference on Knowledge Discovery and Data Mining,pp\. 54–65\.Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p2.1)\.
- Liet al\.\(2024\)J\. Li, Q\. Zhang, Y\. Yu, Q\. Fu, and D\. YeMore agents is all you need\.External Links:2402\.05120,[Link](https://arxiv.org/abs/2402.05120)Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p2.1),[§1](https://arxiv.org/html/2609.13470#S1.p3.1),[§4](https://arxiv.org/html/2609.13470#S4.p3.1)\.
- Liet al\.\(2025b\)W\. Li, Y\. Lin, M\. Xia, and C\. JinRethinking mixture\-of\-agents: is mixing different large language models beneficial?\.arXiv preprint arXiv:2502\.00674\.External Links:2502\.00674Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p2.1)\.
- Lifshitzet al\.\(2025\)S\. Lifshitz, S\. A\. McIlraith, and Y\. DuMulti\-agent verification: scaling test\-time compute with multiple verifiers\.External Links:2502\.20379,[Link](https://arxiv.org/abs/2502.20379)Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p1.1),[§4](https://arxiv.org/html/2609.13470#S4.p2.1)\.
- Lightmanet al\.\(2024\)H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. CobbeLet’s verify step by step\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p1.1),[§4](https://arxiv.org/html/2609.13470#S4.p3.1)\.
- Liuet al\.\(2023\)N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. LiangLost in the middle: how language models use long contexts\.External Links:2307\.03172,[Link](https://arxiv.org/abs/2307.03172)Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p1.1),[§4](https://arxiv.org/html/2609.13470#S4.p1.1)\.
- Liuet al\.\(2024\)Z\. Liu, C\. Zhao, F\. Iandola, C\. Lai, Y\. Tian, I\. Fedorov, Y\. Xiong, E\. Chang, Y\. Shi, R\. Krishnamoorthi, L\. Lai, and V\. ChandraMobileLLM: optimizing sub\-billion parameter language models for on\-device use cases\.InProceedings of the 41st International Conference on Machine Learning,Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p1.1)\.
- Lundberg and Lee \(2017\)S\. M\. Lundberg and S\. LeeA unified approach to interpreting model predictions\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html)Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p3.1)\.
- Muennighoffet al\.\(2025\)N\. Muennighoff, Z\. Yang, W\. Shi, X\. L\. Li, L\. Fei\-Fei, H\. Hajishirzi, L\. Zettlemoyer, P\. Liang, E\. Candès, and T\. HashimotoS1: simple test\-time scaling\.External Links:2501\.19393,[Link](https://arxiv.org/abs/2501.19393)Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p1.1)\.
- Onget al\.\(2025\)I\. Ong, A\. Almahairi, V\. Wu, W\. Chiang, T\. Wu, J\. E\. Gonzalez, M\. W\. Kadous, and I\. StoicaRouteLLM: learning to route llms with preference data\.External Links:2406\.18665,[Link](https://arxiv.org/abs/2406.18665)Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p2.1),[§4](https://arxiv.org/html/2609.13470#S4.p2.1)\.
- OpenAI \(2023\)OpenAIGPT\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p1.1)\.
- Opsahl\-Onget al\.\(2024\)K\. Opsahl\-Ong, M\. J\. Ryan, J\. Purtell, D\. Broman, C\. Potts, M\. Zaharia, and O\. KhattabOptimizing instructions and demonstrations for multi\-stage language model programs\.External Links:2406\.11695,[Link](https://arxiv.org/abs/2406.11695)Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p2.1)\.
- Parulekaret al\.\(2026\)A\. Parulekar, J\. Lee, D\. Hakkani\-Tür, and H\. SundaramReasoning consensus: structural ensembling of LLM reasoning via weighted DAG aggregation\.arXiv preprint arXiv:2607\.27783\.External Links:2607\.27783Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p3.1)\.
- Pinget al\.\(2026\)H\. Ping, A\. Bhattacharjee, P\. Zhang, S\. Li, W\. Yang, A\. Jannesari, N\. Ahmed, and P\. BogdanReM\-MoA: reasoning memory sustains mixture\-of\-agents scaling\.arXiv preprint arXiv:2606\.24437\.External Links:2606\.24437Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p2.1)\.
- Poonet al\.\(2025\)M\. Poon, X\. Dai, X\. Liu, F\. Kong, J\. C\. S\. Lui, and J\. ZuoOnline multi\-LLM selection via contextual bandits under unstructured context evolution\.arXiv preprint arXiv:2506\.17670\.External Links:2506\.17670Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p2.1)\.
- Ribeiroet al\.\(2016\)M\. T\. Ribeiro, S\. Singh, and C\. Guestrin“Why should I trust you?”: explaining the predictions of any classifier\.InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,pp\. 1135–1144\.External Links:[Link](https://doi.org/10.1145/2939672.2939778)Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p3.1)\.
- Saad\-Falconet al\.\(2025\)J\. Saad\-Falcon, A\. G\. Lafuente, S\. Natarajan, N\. Maru, H\. Todorov, E\. Guha, E\. K\. Buchanan, M\. Chen, N\. Guha, C\. Ré, and A\. MirhoseiniArchon: an architecture search framework for inference\-time techniques\.External Links:2409\.15254,[Link](https://arxiv.org/abs/2409.15254)Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p2.1)\.
- Snellet al\.\(2024a\)C\. Snell, J\. Lee, K\. Xu, and A\. KumarScaling LLM test\-time compute optimally can be more effective than scaling model parameters\.arXiv preprint arXiv:2408\.03314\.Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p1.1)\.
- Snellet al\.\(2024b\)C\. Snell, J\. Lee, K\. Xu, and A\. KumarScaling llm test\-time compute optimally can be more effective than scaling model parameters\.External Links:2408\.03314,[Link](https://arxiv.org/abs/2408.03314)Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p1.1),[§4](https://arxiv.org/html/2609.13470#S4.p1.1),[§4](https://arxiv.org/html/2609.13470#S4.p3.1)\.
- Songet al\.\(2025\)Z\. Song, S\. Tang, P\. Ye, J\. Fan, and T\. ChenCTTS: collective test\-time scaling\.arXiv preprint arXiv:2508\.03333\.External Links:2508\.03333Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p3.1)\.
- Taubenfeldet al\.\(2024\)A\. Taubenfeld, Y\. Dover, R\. Reichart, and A\. GoldsteinSystematic biases in llm simulations of debates\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 251–267\.External Links:[Link](http://dx.doi.org/10.18653/v1/2024.emnlp-main.16),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.16)Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p1.1)\.
- Touvronet al\.\(2023\)H\. Touvron, L\. Martin, K\. Stone, P\. Albert, A\. Almahairi, Y\. Babaei, N\. Bashlykov, S\. Batra, P\. Bhargava, S\. Bhosale,et al\.Llama 2: open foundation and fine\-tuned chat models\.arXiv preprint arXiv:2307\.09288\.Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p1.1)\.
- Wanget al\.\(2025\)C\. Wang, Z\. Wan, H\. Kang, E\. Chen, Z\. Xie, T\. Krishna, V\. Janapa Reddi, and Y\. DuSLM\-mux: orchestrating small language models for reasoning\.CoRRabs/2510\.05077\.External Links:[Link](https://arxiv.org/abs/2510.05077)Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p2.1),[§1](https://arxiv.org/html/2609.13470#S1.p3.1),[§4](https://arxiv.org/html/2609.13470#S4.p1.1),[§4](https://arxiv.org/html/2609.13470#S4.p2.1),[§4](https://arxiv.org/html/2609.13470#S4.p3.1)\.
- Wanget al\.\(2024a\)F\. Wang, Z\. Zhang, X\. Zhang, Z\. Wu, T\. Mo, Q\. Lu, W\. Wang, R\. Li, J\. Xu, X\. Tang, Q\. He, Y\. Ma, M\. Huang, and S\. WangA comprehensive survey of small language models in the era of large language models: techniques, enhancements, applications, collaboration with llms, and trustworthiness\.External Links:2411\.03350,[Link](https://arxiv.org/abs/2411.03350)Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p1.1),[§4](https://arxiv.org/html/2609.13470#S4.p1.1)\.
- Wanget al\.\(2024b\)J\. Wang, J\. Wang, B\. Athiwaratkun, C\. Zhang, and J\. ZouMixture\-of\-agents enhances large language model capabilities\.External Links:2406\.04692,[Link](https://arxiv.org/abs/2406.04692)Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p1.1),[§4](https://arxiv.org/html/2609.13470#S4.p2.1)\.
- Wanget al\.\(2023\)X\. Wang, J\. Wei, D\. Schuurmans, Q\. Le, E\. Chi, S\. Narang, A\. Chowdhery, and D\. ZhouSelf\-consistency improves chain of thought reasoning in language models\.InInternational Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/2203.11171)Cited by:[§1](https://arxiv.org/html/2609.13470#S1.p2.1),[§1](https://arxiv.org/html/2609.13470#S1.p3.1),[§4](https://arxiv.org/html/2609.13470#S4.p1.1),[§4](https://arxiv.org/html/2609.13470#S4.p3.1)\.
- Weiet al\.\(2022\)J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. Chi, Q\. Le, and D\. ZhouChain\-of\-thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://arxiv.org/abs/2201.11903)Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p1.1)\.
- Wenet al\.\(2026\)J\. Wen, Y\. Wei, X\. Yu, C\. Xiao, and K\. ZengAttention\-MoA: enhancing mixture\-of\-agents via inter\-agent semantic attention and deep residual synthesis\.arXiv preprint arXiv:2601\.16596\.External Links:2601\.16596Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p2.1)\.
- Xieet al\.\(2025\)Z\. Xie, C\. Han, J\. Shi, W\. Cui, X\. Zhao, X\. Wu, and J\. ZhaoRMoA: optimizing mixture\-of\-agents through diversity maximization and residual compensation\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 6575–6602\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.342)Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p2.1)\.
- Yaoet al\.\(2026\)Y\. Yao, J\. Dong, Y\. Yang, J\. Li, and Y\. DuRoundtable policy: confidence\-weighted\-consensus aggregation improves multi\-agent\-system reasoning\.External Links:2509\.16839,[Link](https://arxiv.org/abs/2509.16839)Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p3.1)\.
- Zhanget al\.\(2025a\)J\. Zhang, J\. Xiang, Z\. Yu, F\. Teng, X\. Chen, J\. Chen, M\. Zhuge, X\. Cheng, S\. Hong, J\. Wang, B\. Zheng, B\. Liu, Y\. Luo, and C\. WuAFlow: automating agentic workflow generation\.External Links:2410\.10762,[Link](https://arxiv.org/abs/2410.10762)Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p2.1)\.
- Zhanget al\.\(2025b\)Q\. Zhang, F\. Lyu, Z\. Sun, L\. Wang, W\. Zhang, W\. Hua, H\. Wu, Z\. Guo, Y\. Wang, N\. Muennighoff, I\. King, X\. Liu, and C\. MaA survey on test\-time scaling in large language models: what, how, where, and how well?\.External Links:2503\.24235,[Link](https://arxiv.org/abs/2503.24235)Cited by:[§4](https://arxiv.org/html/2609.13470#S4.p3.1)\.

## Appendices

Overview\.The first sections provide protocol details, full\-pool analyses, pool\-composition results, robustness checks, and Atr/Ctr diagnostics for the seven\-model experiments\. The final sections collect additional mechanism and sensitivity tables that support the same interpretation from smaller or alternative analysis settings\.

### Appendix AMathematical Interpretation

Generalized routing score contains two sources of evidence:

Top\-answer support\.The first term,

Aggi​\[Mi​a​ri​\(κ\)​Pi​a​\(x\)\],\\mathrm\{Agg\}\_\{i\}\\left\[M\_\{ia\}\\,r\_\{i\}\(\\kappa\)\\,P\_\{ia\}\(x\)\\right\],\(7\)only counts modelii’s support for answeraaifaais that model’s own top answer\. This term measures how strongly the models that actually selectaasupport it\. The operatorAgg\\mathrm\{Agg\}determines how this support is combined across models:

- •Sum\\mathrm\{Sum\}adds support from all models whose top answer isaa: Sumi​\[zi\]=∑izi\.\\mathrm\{Sum\}\_\{i\}\[z\_\{i\}\]=\\sum\_\{i\}z\_\{i\}\.\(8\)This corresponds to coalition\-style evidence, where several models can jointly support the same answer\.
- •Max\\mathrm\{Max\}keeps only the strongest individual supporter: Maxi​\[zi\]=maxi⁡zi\.\\mathrm\{Max\}\_\{i\}\[z\_\{i\}\]=\\max\_\{i\}z\_\{i\}\.\(9\)This corresponds to strongest\-supporter evidence, where one highly confident or highly trusted model can dominate\.

Pooled answer support\.The second term,

∑iri​\(κ\)​Pi​a​\(x\),\\sum\_\{i\}r\_\{i\}\(\\kappa\)P\_\{ia\}\(x\),\(10\)counts every model’s probability mass on answeraa, even ifaais not that model’s top answer\. This term measures cross\-model or mutual support foraa\.

The interpolation parameterα∈\[0,1\]\\alpha\\in\[0,1\]controls which type of evidence dominates\. Whenα=1\\alpha=1, routing relies only on top\-answer support\. Whenα=0\\alpha=0, routing relies only on pooled support\. Thus,κ\\kappacontrols whom to trust,Agg\\mathrm\{Agg\}controls whether evidence comes from a coalition or the strongest supporter, andα\\alphacontrols whether confidence comes from top\-answer self\-consistency or pooled mutual agreement\.

### Appendix BExperimental Details

#### B\.1Stability and Robustness Check

The conclusions in experiments section persist under held\-out pool selection \([SectionB\.4](https://arxiv.org/html/2609.13470#A2.SS4),[Table20](https://arxiv.org/html/2609.13470#A2.T20)\), subsample\-replicate variability \([Table24](https://arxiv.org/html/2609.13470#A2.T24),[Table23](https://arxiv.org/html/2609.13470#A2.T23)\), and full\-cacheK=50K\{=\}50evaluation \([Tables28](https://arxiv.org/html/2609.13470#A2.T28)and[29](https://arxiv.org/html/2609.13470#A2.T29)\)\. Prior\-transform, tie\-breaking, exact cross\-support, and fix/break checks are reported in[SectionB\.5](https://arxiv.org/html/2609.13470#A2.SS5)\.

#### B\.2Protocol Details and Summary Extraction

The main experiments use K20R10: for each question and model, the evaluation samples 20 answers from a K=50 cache and repeats this deterministic subsampling protocol 10 times\. Tables report means over replicates unless otherwise noted\. The main generalized sweep uses fixed expert priors to analyze routing assumptions\. The all\-subset pool scans study how pool composition changes routed accuracy; split\-selected evaluation tests the same selection idea under held\-out evaluation\. The small main\-text summary tables extract views of the same full\-pool sweep to answer the three routing questions directly\.

Table 8:Task\-specific routing regimes from the full\-pool generalized sweep\. The table summarizes broad preferred regions; near\-best plateau statistics are reported separately\.
#### B\.3Full\-Pool Routing Decisions

This section contains the full\-pool analyses behind the main three\-decision analysis\. The tables report plateau width, single\-best gaps, no\-expert\-prior ablations, pooled\-term and top\-term ablations, Sum\-vs\-Max comparisons, and disagreement\-bin behavior\. These numbers are the table counterpart of the main heatmap and regime discussion\.

Table 9:Full\-pool plateau summary\. Broad near\-best regions support interpreting best cells as regimes with stable neighboring cells\.Table 10:Expert\-prior axis: gap between the best generalized full\-pool cell and the single\-best limit\.Table 11:No\-expert\-prior comparison on the full seven\-model pool, read from theκ=0\\kappa=0best cell in the full\-pool phase diagram\.Table 12:Top\-answer\-term removal ablation on the full seven\-model pool\. Settingα=0\\alpha=0keeps only pooled answer support\.Table 13:Pooled\-term removal ablation on the full seven\-model pool\. Settingα=1\\alpha=1removes pooled mutual support\.Table 14:Sum/Max summary\. The aggregation switch measures whether support is distributed across models or concentrated in the strongest supporter\.Table 15:Full\-pool best generalized accuracy by pair\-disagreement bin\.
#### B\.4Pool Composition and Structure

This section tests whether the model pool itself changes the routing problem\. The tables compare pools selected for absolute accuracy with pools selected for gain over the self\-consistency router, classify scanned pools by routing regime, relate structural pool metrics to routing gains, and evaluate split\-selected pool choice under held\-out evaluation\.

Table 16:Pool\-composition targets\. For each benchmark we report the pool with highest absolute accuracy and the pool with largest gain over the self\-consistency router\.Table 17:Best absolute pool from the all\-subset pool\-composition scan\. The companion figure shows how accuracy changes with pool size, while this table records the selected model names\.Table 18:Regime classification over all scanned pools\. Fractions are within benchmark\.Table 19:Structure\-predictor correlations between pool metrics and generalized\-routing gains\.Table 20:Split\-selected evaluation\. Variants differ in whether they select only scorer parameters on the full pool, choose from predefined curated pools, or choose from all subsets\.
#### B\.5Fix/Break, Robustness, and Ablations

This section stress\-tests the main conclusions\. The fix/break table separates questions corrected by generalized routing from questions newly broken by it, including comparisons against several canonical reference routers\. Corrected significance tests check whether selected\-pool gains survive multiple\-baseline correction\. Replicate\-stability tables summarize K20R10 variability\. The remaining tables vary expert priors, tie\-breaking and invalid\-answer handling, exact cross\-support bridge strength, K=50 evaluation, effective sample budget, and method\-correlation structure\.

Table 21:Fix/break summary for full and best\-absolute pools\. Counts are over all K20R10 question\-replicate rows\.Table 22:Selected\-pool paired tests with max\-stat correction across baselines within each benchmark/pool role\.Table 23:Replicate stability\. Means and standard deviations are over the 10 K20R10 subsampling replicates\.Table 24:Robustness checks\. Split\-selected results use held\-out evaluation with predefined curated pools; K=50 uses all cached samples once\. The direction of the regimes persists under both checks\.Table 25:Expert\-prior transform variants\. These checks test whether the qualitative regimes depend on a single prior transformation\.Table 26:Tie\-break sensitivity\. Tie\-breaking changes small details while preserving the main regime interpretation\.Table 27:Exact cross\-support bridge sweep\. This sweep studies how the bridge varies with cross\-support strength\.Table 28:K=50 distribution\-level phase diagram\. K=50 uses all cached samples once and serves as a robustness check for the main protocol\.Table 29:Effective sample\-budget scaling under K20R\-style subsampling\.Table 30:Method\-correlation analysis\. High correlation indicates that two methods draw on overlapping evidence\.
#### B\.6Atr/Ctr Model\-Support Analysis

This section asks which models the router listens to and whether that support is helpful\. Leave\-one\-out tables estimate the marginal effect of removing each model\. The Llama table separates pools that include Llama from comparable Llama\-free pools\. The Atr/Ctr table reports the counterfactual Ctr used in the main text\. The accepted\-support decomposition separates support on correct routed answers from support on wrong routed answers\.

Table 31:Leave\-one\-out model contribution summary\.Table 32:Llama inclusion analysis comparing the best pool containing Llama with the best pool excluding Llama\.Table 33:Per\-model Atr/Ctr diagnostics for the full pool and the selected curated pool in each benchmark\. Atr is support assigned to the routed answer; Ctr is the leave\-one\-out counterfactual contribution defined in §[2](https://arxiv.org/html/2609.13470#S2)\.BenchmarkSettingModelAtrCtrMATH\-500Fullqwen0\.82830\.1220MATH\-500Fullllama0\.50220\.0014MATH\-500Fulltulu0\.48170\.0002MATH\-500Fulldsmath0\.43920\.0002MATH\-500Fullgemma0\.49510\.0002MATH\-500Fullphi0\.4637\-0\.0004MATH\-500Fullministral0\.57590\.0012MATH\-500Curatedqwen0\.82790\.1496MATH\-500Curatedllama0\.50470\.0016MATH\-500Curateddsmath0\.44040\.0004MATH\-500Curatedgemma0\.49740\.0012GSM8KFullqwen0\.93200\.0121GSM8KFullllama0\.83400\.0037GSM8KFulltulu0\.8722\-0\.0009GSM8KFulldsmath0\.7876\-0\.0004GSM8KFullgemma0\.85770\.0003GSM8KFullphi0\.84940\.0010GSM8KFullministral0\.8542\-0\.0001GSM8KCuratedqwen0\.91790\.0228GSM8KCuratedllama0\.84820\.0177GSM8KCurateddsmath0\.81210\.0135GPQAFullqwen0\.61390\.0571GPQAFullllama0\.46280\.0020GPQAFulltulu0\.44770\.0010GPQAFulldsmath0\.2933\-0\.0015GPQAFullgemma0\.51690\.0222GPQAFullphi0\.4828\-0\.0066GPQAFullministral0\.45740\.0025GPQACuratedqwen0\.60930\.0641GPQACuratedllama0\.46820\.0081GPQACuratedgemma0\.53040\.0303GPQACuratedministral0\.45780\.0101MMLU\-ProFullqwen0\.72730\.0204MMLU\-ProFullllama0\.58980\.0094MMLU\-ProFulltulu0\.50380\.0010MMLU\-ProFulldsmath0\.33410\.0004MMLU\-ProFullgemma0\.6430\-0\.0008MMLU\-ProFullphi0\.58920\.0146MMLU\-ProFullministral0\.51560\.0024MMLU\-ProCuratedqwen0\.69740\.0624MMLU\-ProCuratedllama0\.59680\.0364MMLU\-ProCurateddsmath0\.37310\.0022MMLU\-ProCuratedphi0\.64000\.0382Table 34:Accepted\-support decomposition\. For each benchmark and pool role, we show the three models with highest Atr under best generalized routing\. Balance is correct accepted support minus wrong accepted support\.

### Appendix CSupplementary Mechanism and Sensitivity Tables

#### C\.1Six\-Model Exploratory Analyses

These tables summarize a smaller six\-model setting used to check the same routing questions under a lighter cache\. They report single\-model and baseline numbers, a fixed\-prior generalized sweep, top pools from a pool\-composition scan, and sampling\-strategy comparisons\.

Table 35:Six\-model exploratory baseline summary\.Table 36:Six\-model fixed\-prior sweep summary\.Table 37:Six\-model top pool\-composition results by best generalized accuracy\.Table 38:Sampling\-strategy comparison\. K20R10 preserves replicate variability while using the larger cache\.
#### C\.2Seven\-Model Protocol Checks

These tables record seven\-model protocol checks: comparable baselines, replicate\-aware generalized sweeps, and pool\-subset scans under the K20R10 protocol\.

Table 39:Seven\-model K20R10 baseline summary\. Standard deviations are across the 10 subsampling replicates\.Table 40:Seven\-model K20R10 fixed\-prior sweep summary\.Table 41:Seven\-model top pool\-composition results, with five best pools per benchmark by generalized accuracy\.
#### C\.3Additional Mechanism Tables

The following tables probe mechanism questions that complement the main analysis: whether cross\-support is a question\-specific signal, how it behaves under placebo tests, how sample budget affects routing signals, and where oracle coverage fails to become routable support\.

Table 42:Row\-level correctness prediction quality for isolated self\-consistencycc, cross\-supportss, andc\+sc\{\+\}s\.Table 43:Placebo tests that corrupt cross\-support while leaving self\-consistency fixed\.Table 44:Best score form in each setting from the score\-form ablation\.Table 45:Best support definition byss\-only accuracy\.Table 46:Best normalisation / calibration configuration in each setting\.Table 47:How the routing signals change with the sample budgetKK\.Table 48:Accuracy scaling with the per\-model sample budgetKKon the full 6\-model pool\.Table 49:Oracle\-gap decomposition of routing outcomes\.Table 50:Top GPQA model pairs by pairwise oracle\. All top pairs contain Gemma\.Table 51:Question\-slice analysis on GPQA by model disagreement\.Table 52:OLS coefficients for predictingΔ=Cross−SLM\\Delta=\\text\{Cross\}\-\\text\{SLM\}from subset\-level pool statistics\.Table 53:GPQA contamination analysis anchored at the curated core pool “qwen\_phi\_gemma“\.Table 54:Question\-level paired bootstrap confidence intervals and permutation tests\.Table 55:Tie\-break robustness under model\-order and validation\-accuracy protocols\.Table 56:Held\-out GPQA pool selection over 20 random 50/50 splits\.Selection frequencies:qwen\_gemma: 9, qwen\_phi\_gemma: 6, qwen\_phi\_ministral\_gemma: 4, phi\_gemma: 1\.

Table 57:Seed robustness over 20 offline subsampling seeds\.Table 58:Budget\-fair comparison on GPQA\. Curated pools with deeper sampling dominate the full 6\-model reference\.Table 59:With\- vs without\-validation comparison\. Validation\-selected single\-best is the strongest deployable baseline\.Table 60:Held\-out GPQA pool selection across four selection rules\.Table 61:Case taxonomy for the GPQA qualitative analysis\.
#### C\.4Additional Generalized\-Family Checks

These generalized\-family tables provide implementation checks, canonical\-point recovery, K interactions, validation\-aware variants, region summaries, significance checks, and budget\-fair comparisons\.

Table 62:Implementation\-level recovery of the canonical baselines as limits inside the generalized family\. All limits are recovered exactly modulo the deterministic\(W,pooled,lex\)\(W,\\text\{pooled\},\\text\{lex\}\)tie\-break specified for the family; the small MATH\-500 majority gap is from a handful of questions on which one model has fewer valid extracted answers than the others\.Table 63:Generalized two\-axis family on the full 6\-model and curated GPQA pools\. All canonical points are evaluated on the same cached generations as the rest of the paper\.α\\alphacontrols majority\-vs\-top\-answer aggregation;κ\\kappacontrols uniform\-vs\-expert weighting \(calibration\-half acci, held\-out evaluation forκ\>0\\kappa\{\>\}0\); Agg selects coalition \(Sum\) vs strongest\-supporter \(Max\) aggregation\. Best generalized point in bold per row\.Table 64:K interaction on GPQA for the generalized family\. Best validation\-freeκ=0\\kappa\{=\}0generalized point reported perKK, alongside the canonical points\. “–” marks aggregations not swept atK=2K\{=\}2andK=4K\{=\}4\. The full pool peaks atK=3K\{=\}3in theκ=0\\kappa\{=\}0region; the curated pool is monotonic inKK\.Table 65:Validation\-aware evaluation of the generalized family\. Per setting we draw a fresh 50/50 random split over questions for each of 20 seeds, fit per\-model calibration accuraciesacci\\text\{acc\}\_\{i\}on one half, and evaluate every\(α,κ,Agg\)\(\\alpha,\\kappa,\\text\{Agg\}\)cell on the held\-out half\. All numbers are held\-out accuracy averaged across the 20 seeds\. “Single\-best \(val\)” is theκ=∞\\kappa\{=\}\\inftylimit\.Table 66:Region summary\. For each \(benchmark,KK, pool\) we report the best validation\-freeκ=0\\kappa\{=\}0generalized point on each of the Sum and Max planes, alongside the canonical points and external references \(ss\-only, rank\-sum, trust\-aware top2\_strong, oracle\)\. This compressed view summarizes the best\-region map per regime\.Table 67:What the generalized family tells us about the four routing research questions on this benchmark set\. “balanced” = curated GPQA pools \(qwen\+phi\+gemma, qwen\+gemma, phi\+gemma\); “dominated” = full 6\-model pool on MATH\-500 / GSM8K \(Qwen≥10\\geq 10pts above the rest\)\.Table 68:Question\-level paired tests for the strongest validation\-free generalized point \(κ=0\\kappa\{=\}0\) against each of the SLM\-MUXans\{\}\_\{\\text\{ans\}\},ss\-only, and rank\-sum baselines\.Δ\\Deltais the per\-question mean correctness difference \(positive = generalized wins\);ppis the two\-sided paired\-permutationpp\-value withB=5000B\{=\}5000replicates; 95% CIs from paired bootstrap\. Statistically significant gains in bold \(p<0\.05p<0\.05\)\.Table 69:Budget\-fair generalized comparison on GPQA\. Each row is a fixed model×K\\times Kbudget\. “best gen” is the strongestκ=0\\kappa\{=\}0generalized point at that budget\. Curated pools with deeper sampling cleanly dominate the full6×56\{\\times\}5reference at3×3\\timesless compute\.