EntroRouter: Learning Efficient Model Routing via Entropy Regulation

# Learning Efficient Model Routing via Entropy Regulation
Source: [https://arxiv.org/html/2606.29424](https://arxiv.org/html/2606.29424)
Kaiyi Zhang1,2111Equal Contribution\., Xueliang Zhao3111Equal Contribution\., Zhuocheng Gong4 Wei Wu2222Corresponding author: Wei Wu and Yankai Lin\.,Yankai Lin1222Corresponding author: Wei Wu and Yankai Lin\. 1Gaoling School of Artificial Intelligence, Renmin University of China 2Ant International,3The University of Hong Kong,4Ant Group 🖂\{kyzhang, yankailin\}@ruc\.edu\.cn,wuwei19850318@gmail\.com

###### Abstract

Model routing balances solution accuracy and computational cost by selecting among models of varying capabilities\. While recent multi\-round frameworks interleave reasoning and planning, we identify a structural failure mode termed Trust Region Collapse\. We demonstrate that the deep coupling of reasoning and routing, exacerbated by the dominance of strong pre\-training priors under sparse supervision, leads to degenerate local optima where capable experts are systematically suppressed\. To decouple these processes, we proposeEntroRouter, a single\-round routing framework that treats entropy regulation as a core objective\. We first initialize the policy via Soft Supervision, fitting a distribution of suitable models to establish a high\-entropy prior for exploration\. Subsequently, we stabilize Reinforcement Learning using a Soft Anchor, which utilizes offline capability estimates to orchestrate controlled entropy contraction within a safe trust region\. Extensive experiments demonstrate thatEntroRouterretains 98\.3% of the strongest expert’s accuracy while reducing computational costs by 48\.25%\.

EntroRouter: Learning Efficient Model Routing via Entropy Regulation

Kaiyi Zhang1,2111Equal Contribution\., Xueliang Zhao3111Equal Contribution\., Zhuocheng Gong4Wei Wu2222Corresponding author: Wei Wu and Yankai Lin\.,Yankai Lin1222Corresponding author: Wei Wu and Yankai Lin\.1Gaoling School of Artificial Intelligence, Renmin University of China2Ant International,3The University of Hong Kong,4Ant Group🖂\{kyzhang, yankailin\}@ruc\.edu\.cn,wuwei19850318@gmail\.com

## 1Introduction

The rapid advancements in Large Language Models \(LLMs\) have introduced a critical trade\-off in deployment: while larger models \(e\.g\., Qwen3\-235B\) offer superior reasoning capabilities, they incur prohibitive computational costs and latency\. Conversely, smaller models \(e\.g\., Qwen3\-4B\) are efficient but often falter on complex tasks\. This trade\-off has necessitated the development of model routing systems, which aim to dynamically select the most cost\-effective model for a given query without compromising solution qualityChen et al\. \([2023](https://arxiv.org/html/2606.29424#bib.bib3)\); Šakota et al\. \([2024](https://arxiv.org/html/2606.29424#bib.bib17)\)\.

Current research explores two primary paradigms to solve this routing problem: iterative multi\-round agentsZhang et al\. \([2025](https://arxiv.org/html/2606.29424#bib.bib23)\)and deterministic single\-round classifiersOng et al\. \([2024](https://arxiv.org/html/2606.29424#bib.bib14)\)\. However, despite their architectural differences, we identify that both paradigms frequently suffer from a unified structural pathology:Premature Entropy Collapse\. This failure to properly manage the policy’s uncertainty, whether in the action space or the label space, prevents routers from exploring and learning the optimal cost\-accuracy frontier\.

The emerging multi\-round paradigm \(e\.g\., Router\-R1\) manifests as a collapse in the Action Space\. Specifically, the router acts as an agent that decides when to query external experts\. However, through empirical replication in the mathematical domain, we observe that the Reinforcement Learning \(RL\) process frequently drives the probability of calling external experts to zero \(as shown in Figure[1](https://arxiv.org/html/2606.29424#S1.F1)\)\. We demonstrate that this is a structural failure mode arising from the coupling of planning and execution\. When a small router fails to comprehend the Out\-Of\-Distribution reasoning traces from larger experts, dominated by its strong pre\-training priors, it effectively disregards the expert’s response\. Consequently, the RL optimizer perceives the expert call as a high\-cost, zero\-gain action, prematurely converging to a deterministic policy that refuses to route\. We term this phenomenon Trust Region Collapse\.

Parallel to this, traditional single\-round approaches suffer from a corresponding collapse in the Label Space\. This implies that in the final selection, the router is forced to prematurely commit to a rigid, deterministic choice\. Existing baselinesOng et al\. \([2024](https://arxiv.org/html/2606.29424#bib.bib14)\)rely on Hard Supervision \(e\.g\., one\-hot labels\), forcing the router to mimic a single “best” model\. This artificially suppresses the inherent ambiguity of problem difficulty\. By forcing the target probability distribution to collapse to a sharp decision boundary too early during Supervised Fine\-Tuning \(SFT\), the router loses the representational entropy required to capture nuanced trade\-offs, rendering the policy prone to overfitting\.

![Refer to caption](https://arxiv.org/html/2606.29424v1/x1.png)

Figure 1:Empirical observation of Trust Region Collapse\. Despite successful cold\-start fine\-tuning, the subsequent RL process rapidly suppresses the probability of calling external models \(Qwen3\-4B/8B\) to zero\. This represents a premature entropy collapse, effectively abandoning the expert solver\.To address these dual failures of entropy management, we proposeEntroRouter, a framework that treats entropy regulation as a first\-class objective\. We adopt the single\-round routing paradigm and introduce a two\-stage training pipeline designed to harmonize exploration and exploitation\. First, to prevent the label\-space collapse typical of hard baselines, we initialize the router viaSoft Supervision\(§[3\.2](https://arxiv.org/html/2606.29424#S3.SS2)\)\. Instead of fitting a deterministic label, we train on a distribution of viable models\. This establishes a high\-entropy prior, preserving the policy’s capacity to recognize difficulty ambiguity\. Second, during the RL phase, we regularize the optimization using a Soft Anchor derived from offline capability estimates \(§[3\.3](https://arxiv.org/html/2606.29424#S3.SS3)\)\. Crucially, this mechanism serves a dual purpose: it defines a safe trust region to prevent the policy from drifting into degeneration, while simultaneously guiding the router to rapidly minimize entropy within this region\. This allows the model to transition from the exploratory SFT state to a decisive, cost\-efficient strategy that confidently selects reliable experts\.

We extensively evaluateEntroRouteron 7 rigorous mathematical reasoning benchmarks\. Remarkably,EntroRouterretains98\.3%of the strongest fixed\-expert baseline’s accuracy while reducing computational costs by48\.25%\. Furthermore, our framework exhibits strong generalization to out\-of\-distribution tasks\. On widely recognized benchmarks,EntroRouterreduces costs by40\.5%while maintaining a relative accuracy within95\.4%of the strongest fixed\-expert baseline\.

Our main contributions are threefold\.First, we formalize a sufficient condition under which multi\-round routing suffersTrust Region Collapse, a failure regime where strong router priors suppress expert usage\.Second, we proposeEntroRouter, a framework that orchestrates acontrolled entropy contractionprocess\. It utilizes Soft Supervision to establish a high\-entropy exploration prior and employs a Soft Anchor to regularize RL optimization within a safe trust region\.Third, we verify the effectiveness of our proposed method through extensive experiments on both mathematical benchmarks and out\-of\-distribution datasets\.

## 2Preliminaries

In this section, we formalize the problem of LLM routing within the framework of a Markov Decision Process \(MDP\)\. We establish the notations used throughout the paper and review the prevailing multi\-round routing paradigm, exemplified by Router\-R1Zhang et al\. \([2025](https://arxiv.org/html/2606.29424#bib.bib23)\), which serves as the primary baseline for our analysis\.

### 2\.1Problem Formulation

We formulate the model routing task as a sequential decision\-making process where a policy model \(theRouter\) selects an inference model \(theExpert\) at each step to solve the user query\. Letxxdenote the input query, and letℳ=\{m1,m2,…,mK\}\\mathcal\{M\}=\\\{m\_\{1\},m\_\{2\},\\dots,m\_\{K\}\\\}represent the set of available candidate models that the router can choose from\. We define this process as a tuple⟨𝒮,𝒜,𝒫,ℛ⟩\\langle\\mathcal\{S\},\\mathcal\{A\},\\mathcal\{P\},\\mathcal\{R\}\\rangle\. This routing process generates a trajectoryτ=\(s0,a0,…,sL\)\\tau=\(s\_\{0\},a\_\{0\},\\dots,s\_\{L\}\)based on the policy and the selected models\.

The components of the tuple are defined as follows\. TheState Space \(𝒮\\mathcal\{S\}\)consists of statesst∈𝒮s\_\{t\}\\in\\mathcal\{S\}representing the accumulated context at time steptt, initialized with the user querys0=xs\_\{0\}=x\. TheAction Space \(𝒜\\mathcal\{A\}\)is defined as𝒜=ℳ∪\{mself\}\\mathcal\{A\}=\\mathcal\{M\}\\cup\\\{m\_\{\\text\{self\}\}\\\}, comprising both external candidate models and the router itself \(mselfm\_\{\\text\{self\}\}\)\. RegardingTransition Dynamics \(𝒫\\mathcal\{P\}\), the state update depends on the selected action: if the router performs internal reasoning \(i\.e\.,at=mselfa\_\{t\}=m\_\{\\text\{self\}\}\), it generates an outputoto\_\{t\}, updating the state asst\+1=st⊕ots\_\{t\+1\}=s\_\{t\}\\oplus o\_\{t\}, where⊕\\oplusdenotes sequence concatenation; for external delegation \(i\.e\.,at∈ℳa\_\{t\}\\in\\mathcal\{M\}\), the router formulates a queryqtq\_\{t\}, receives a responseoto\_\{t\}generated by the external modelata\_\{t\}, and updates the state to include the interaction asst\+1=st⊕qt⊕ots\_\{t\+1\}=s\_\{t\}\\oplus q\_\{t\}\\oplus o\_\{t\}\. Finally, theReward Function \(ℛ\\mathcal\{R\}\)balances solution correctness against computational cost\. We define the total return of a trajectory asR​\(τ\)=racc−λ⋅CtotalR\(\\tau\)=r\_\{\\text\{acc\}\}\-\\lambda\\cdot C\_\{\\text\{total\}\}, whereracc∈\{0,1\}r\_\{\\text\{acc\}\}\\in\\\{0,1\\\}is a sparse binary indicator for solution correctness at the terminal state, andCtotal=∑tC​\(at\)⋅\|ot\|C\_\{\\text\{total\}\}=\\sum\_\{t\}C\(a\_\{t\}\)\\cdot\|o\_\{t\}\|represents the accumulated token\-based cost, whereC​\(at\)C\(a\_\{t\}\)is the per\-token cost ofata\_\{t\}\.

The objective of the router is to learn a policyπθ​\(a∣s\)\\pi\_\{\\theta\}\(a\\mid s\)that maximizes the expected return:J​\(θ\)=𝔼τ∼πθ​\[R​\(τ\)\]J\(\\theta\)=\\mathbb\{E\}\_\{\\tau\\sim\\pi\_\{\\theta\}\}\[R\(\\tau\)\]\. In this formulation,λ\\lambdaserves as a hyperparameter controlling the trade\-off between accuracy and cost\. We operate under the premise of a positive correlation between inference cost and model capability\. We assume thatℳ\\mathcal\{M\}consists exclusively of efficient frontier models, thereby precluding cases where a candidate is both more computationally expensive and functionally inferior to another\. In the simplest scenario \(single\-step routing\), the router selects a model based solely on the initial queryxxwithout intermediate reasoning steps\.

### 2\.2The Multi\-Round Routing Paradigm

Recent advancements, such as Router\-R1Zhang et al\. \([2025](https://arxiv.org/html/2606.29424#bib.bib23)\), generalize the routing task to a sequential setting\. In this paradigm, the router operates as an agent interleaving internal reasoning steps \(at=mselfa\_\{t\}=m\_\{\\text\{self\}\}\) with external model calls \(at∈ℳa\_\{t\}\\in\\mathcal\{M\}\)\. Unlike single\-step routing where the router acts solely as a dispatcher, this framework requires the router to effectivelyinterpretandintegratethe external model’s outputoto\_\{t\}to update the state and progress toward the solution\.

Crucially, this formulation intertwines therouting decisionwith the router’s ownreasoning execution\. Consequently, the policy’s potential is bounded by the router’s ability to assimilate complex information, a process often obstructed by the dominance of its own pre\-training priors\. As demonstrated in the following section, this dependency can lead to structural failures, where the router struggles to optimize the cost\-accuracy trade\-off due to stubborn internal priors and capability mismatches under sparse outcome reward\.

## 3Methodology

![Refer to caption](https://arxiv.org/html/2606.29424v1/x2.png)\(a\)Stage I: SFT via Soft Supervision
![Refer to caption](https://arxiv.org/html/2606.29424v1/x3.png)\(b\)Stage II: Reinforcement Learning with Soft Anchor

Figure 2:Overview of theEntroRouterTraining Framework\.\(a\) Stage I: SFT via Soft Supervision\.We employ a soft supervision strategy to prevent premature policy collapse\. A weak probemprobem\_\{\\text\{probe\}\}identifies query difficulty: for complex queries \(Scenario A\), we spread the training target across all qualified experts inℳtopk\\mathcal\{M\}\_\{\\text\{topk\}\}, deliberately maintaining a high\-entropy policy to preserve exploration capacity\.\(b\) Stage II: Reinforcement Learning with Soft Anchor\.We refine the policy via GRPO using an expectation\-grounded reward\. By regularizing the update with aSoft Anchor\(πref\\pi\_\{\\text\{ref\}\}\), we constrain the router within a safe trust region while simultaneously penalizing entropy to transition from a broad exploration state to a decisive, high\-confidence routing state\.We begin by defining the theoretical failure mode observed in multi\-round routing \(§[3\.1](https://arxiv.org/html/2606.29424#S3.SS1)\)\. Motivated by this analysis, we proposeEntroRouter, a framework that treats routing as a single\-round decision process\. Specifically, our method is built upon a two\-stage training pipeline: In the first stage, we initialize the policy by distributing probability mass across multiple viable models instead of assigning a single hard label\. Thisentropy\-preservinginitialization prevents premature convergence and maintains the capacity for further exploration \(§[3\.2](https://arxiv.org/html/2606.29424#S3.SS2)\)\. Subsequently, we fine\-tune the policy via RL, regularizing updates against a reference distribution derived from offline capability estimates\. This stage implementsentropy contractionwithin a safe trust region, dynamically optimizing the trade\-off between solution correctness and computational cost while sharpening the final routing decision \(§[3\.3](https://arxiv.org/html/2606.29424#S3.SS3)\)\.

### 3\.1Policy Degeneration in Multi\-Round Routing

We identify a critical instability that can arise in the multi\-round routing paradigm under sparse outcome supervision and strong router pre\-training priors\. Despite cold\-start fine\-tuning, the subsequent RL process frequently drives the probability of utilizing the strong model to zero, effectively abandoning the expert solver\. Consequently, the policy’s exploration frontier collapses into a degenerate local optimum of “always refusing to escalate,” a phenomenon we termTrust Region Collapse\. From an information\-theoretic perspective, this manifests as a pathological and premature reduction in policy entropy: the router rapidly converges to a deterministic, low\-entropy state of refusal, thereby eliminating the exploration variance required to discover better strategies\.

We attribute this collapse to a functional conflation of planning and execution, which triggers a fatal causal chain characterized by three distinct phases\.First:When the expert model generates a complex reasoning trace in response to the router’s request, the smaller router often struggles to comprehend the logic because the trace is Out\-Of\-Distribution relative to its own training distribution\.Second:Dominated by its own pre\-training priors, the router effectively ignores the expert’s provided context and falls back to generating the solution using its own limited capabilities\.Third:Since the router has reverted to its own capabilities, the final outcome is no better than if it had acted alone\. However, under sparse outcome based rewards, the RL optimizer observes a trajectory withhigh cost\(calling the expert\) butno performance gain\. Lacking progress reward supervision to pinpoint the comprehension failure, the optimizer simply penalizes the escalation action\.

Consequently, from a value\-function perspective, the advantage of escalation turns negative\. Since calling the expert incurs a computational cost but yields no improvement in accuracy, the optimizer perceives the expert call as strictly inferior to internal reasoning, forcing the policy to collapse\.

#### Proposed Solution: Single\-Round Routing\.

The analysis above shows that the failure stems from forcing the small router tointeract with and process the reasoning outputs of external expert models\. To alleviate this issue, we adoptEntroRouter, a framework built upon the single\-round routing paradigm, as the most direct and robust solution\. UnderEntroRouter, the router decides which model to use based solely on the initial question\. This formulation shifts the router’s role: unlike the multi\-roundreasonerthat interacts with and comprehend the external solution, our router operates as ameta\-analyzer, estimating the query’s difficulty to dispatch it to the most appropriate candidate model based solely on the initial question\.

### 3\.2Stage I: SFT via Soft Supervision

Our goal is to train a router policyπθ​\(m∣x\)\\pi\_\{\\theta\}\(m\\mid x\)\*\*\*FollowingZhao et al\. \([2025](https://arxiv.org/html/2606.29424#bib.bib26)\), we equip the router with an intermediate reasoning step to explicitly assess query complexity before selection\.that analyzes the queryxxcomplexity via a reasoning trace before selecting the most suitable modelmm\. To prevent the router from overfitting to a single deterministic choice, we introduce SFT with soft supervision\. Specifically, we first identify the set of most economical and reliable experts based on offline capability estimates; then we construct a target distribution that adaptively switches between hard labels for simple queries and soft distributions for complex ones; finally, we optimize the router policy to minimize its divergence from this adaptive target\. To teach the router to prioritize economy, we first determine which models are capable of solving a given queryxx\. To do this, we estimate the ground\-truth pass rateκ​\(m,x\)\\kappa\(m,x\)for each modelmm, which quantifies the likelihood that modelmmcorrectly answers the queryxx\. This pass rate is estimated via Monte Carlo rollouts, whereκ​\(m,x\)\\kappa\(m,x\)is defined as the proportion ofNNsampled responses from modelmmthat match the ground truth\.

Based on this estimation, we formally define the optimal routing decisions\. We operate under the general assumption that model capability correlates positively with inference cost\. Under this premise, identifying the optimal expert simplifies to finding the most economical model that meets the reliability thresholdτ\\tau\. Accordingly, we define the most economical expertmbestm\_\{\\text\{best\}\}as:

mbest=arg​minm∈ℳ,κ​\(m,x\)≥τ⁡C​\(m\),m\_\{\\text\{best\}\}=\\operatorname\*\{arg\\,min\}\_\{m\\in\\mathcal\{M\},\\,\\kappa\(m,x\)\\geq\\tau\}C\(m\),\(1\)whereC​\(m\)C\(m\)denotes the per\-token inference cost of modelmm\. To construct a flexible training target, we further identify the set of the top\-kkmost cost\-effective models, denoted asℳtopk\\mathcal\{M\}\_\{\\text\{topk\}\}\. Under our assumption, this set corresponds to thekkcheapest models that effectively pass the thresholdτ\\tau:

ℳtopk=arg​minS⊆ℳ,\|S\|=k\\displaystyle\\mathcal\{M\}\_\{\\text\{topk\}\}=\\operatorname\*\{arg\\,min\}\_\{S\\subseteq\\mathcal\{M\},\\,\|S\|=k\}∑m∈SC​\(m\)\\displaystyle\\sum\_\{m\\in S\}C\(m\)\(2\)s\.t\.∀m∈S,κ​\(m,x\)≥τ\.\\displaystyle\\forall m\\in S,\\,\\kappa\(m,x\)\\geq\\tau\.
Intuitively, this selection criterion filters for the most economical experts among those sufficient for the task\. Standard SFT typically forces the router to mimic a single “best” model, which works well for simple queries but fails to capture the inherent ambiguity in more complex queries, where model suitability is not straightforward\. To explicitly model this ambiguity, we construct a target distributionQ​\(m∣x\)Q\(m\\mid x\)that preserves the natural uncertainty of the task\. We combine hard supervision \(for trivial cases\) with soft supervision \(for complex cases\) to prevent overfitting to arbitrary label noise:

Q​\(m∣x\)=g​\(x\)⋅𝕀​\[m=mbest\]⏟Hard Supervision\+\(1−g​\(x\)\)⋅𝕀​\[m∈ℳtopk\]k⏟Soft Supervision\\begin\{split\}Q\(m\\mid x\)=&\\underbrace\{g\(x\)\\cdot\\mathbb\{I\}\[m=m\_\{\\text\{best\}\}\]\}\_\{\\text\{Hard Supervision\}\}\\\\ &\+\\underbrace\{\(1\-g\(x\)\)\\cdot\\frac\{\\mathbb\{I\}\[m\\in\\mathcal\{M\}\_\{\\text\{topk\}\}\]\}\{k\}\}\_\{\\text\{Soft Supervision\}\}\\end\{split\}\(3\)where𝕀​\[⋅\]\\mathbb\{I\}\[\\cdot\]is the indicator function\. The gating termg​\(x\)g\(x\)switches the supervision mode based on a lightweight probe modelmprobem\_\{\\text\{probe\}\}\. For simple queries, whereg​\(x\)=1g\(x\)=1, if the probe demonstrates sufficient capability \(i\.e\.,κ≥τ\\kappa\\geq\\tau\), the query is deemed trivial, and the target collapses to the cheapest model inℳ\\mathcal\{M\}\. For complex queries, whereg​\(x\)=0g\(x\)=0, the query is identified as complex, and we distribute probability mass uniformly across the valid setℳtopk\\mathcal\{M\}\_\{\\text\{topk\}\}, ensuring robustness against label noise\. We optimize the router to match this target distribution\. The loss function is defined as:

ℒSFT​\(θ\)=−∑m∈ℳQ​\(m∣x\)​log⁡πθ​\(m∣x\)\.\\mathcal\{L\}\_\{\\text\{SFT\}\}\(\\theta\)=\-\\sum\_\{m\\in\\mathcal\{M\}\}Q\(m\\mid x\)\\log\\pi\_\{\\theta\}\(m\\mid x\)\.\(4\)By minimizingℒSFT\\mathcal\{L\}\_\{\\text\{SFT\}\}, the router learns to recognize simple cases for quick routing while maintaining a flexible decision boundary for complex queries\.

### 3\.3Stage II: Reinforcement Learning with Soft Anchor

While Stage I provides a reasonable initialization, the resulting policy remains static, ignoring the dynamic trade\-off between correctness and computational efficiency\. Stage II addresses this by learning a risk\-cost preference through RL\. However, unconstrained optimization risks destabilizing the policy\. To enable controlled exploration, we propose a soft anchor mechanism\. We implement this by incorporating a reference distribution derived from offline statistics directly into the optimization objective, effectively guiding the policy toward empirically safe and cost\-effective regions\.

We first construct the reference policyπref\\pi\_\{\\text\{ref\}\}to embody the ideal risk\-cost preference\. Based on the offline pass rateκ​\(m,x\)\\kappa\(m,x\)estimated in Stage I, we define a scalar utility metricμ​\(m,x\)\\mu\(m,x\)for each modelm∈ℳm\\in\\mathcal\{M\}given queryxx:

μ​\(m,x\)=κ​\(m,x\)−α⋅C​\(m\),\\mu\(m,x\)=\\kappa\(m,x\)\-\\alpha\\cdot C\(m\),\(5\)whereC​\(m\)C\(m\)is the inference cost andα\\alphagoverns the cost sensitivity\. We then convert these utilities into a probabilistic prior using a Boltzmann distribution:

πref​\(m∣x\)=exp⁡\(μ​\(m,x\)/T\)∑m′∈ℳexp⁡\(μ​\(m′,x\)/T\),\\pi\_\{\\text\{ref\}\}\(m\\mid x\)=\\frac\{\\exp\(\\mu\(m,x\)/T\)\}\{\\sum\_\{m^\{\\prime\}\\in\\mathcal\{M\}\}\\exp\(\\mu\(m^\{\\prime\},x\)/T\)\},\(6\)whereTTis a temperature hyperparameter that regulates the sharpness of the distribution\. This distribution serves as the “soft anchor”, assigning higher probability mass to models that are empirically reliable and economical\. We optimize the router policyπθ\\pi\_\{\\theta\}using Group Relative Policy Optimization \(GRPO\)Shao et al\. \([2024](https://arxiv.org/html/2606.29424#bib.bib19)\)\. Formally, our objective is to maximize the expected reward while regularizing the policy against the soft anchor:

maxπθ𝔼m∼πθ​\[r​\(m,x\)\]−β𝔻KL\[πθ\(⋅\|x\)∥πref\(⋅\|x\)\]−βℋ\(πθ\),\\begin\{split\}\\max\_\{\\pi\_\{\\theta\}\}\\,&\\mathbb\{E\}\_\{m\\sim\\pi\_\{\\theta\}\}\[r\(m,x\)\]\\\\ &\{\}\-\\beta\\mathbb\{D\}\_\{\\text\{KL\}\}\[\\pi\_\{\\theta\}\(\\cdot\|x\)\\,\\\|\\,\\pi\_\{\\text\{ref\}\}\(\\cdot\|x\)\]\-\\beta\\mathcal\{H\}\(\\pi\_\{\\theta\}\),\\end\{split\}\(7\)whereℋ​\(πθ\)\\mathcal\{H\}\(\\pi\_\{\\theta\}\)represents the entropy of the policy distribution\. The task\-performance rewardr​\(m,x\)r\(m,x\)is defined as theexpected utility:

r​\(m,x\)=κ​\(m,x\)​Rsucc\+\(1−κ​\(m,x\)\)​Rfail\.r\(m,x\)=\\kappa\(m,x\)R\_\{\\text\{succ\}\}\+\(1\-\\kappa\(m,x\)\)R\_\{\\text\{fail\}\}\.\(8\)Here,RsuccR\_\{\\text\{succ\}\}represents the cost\-aware payoff received when the selected model provides a correct answer, whileRfailR\_\{\\text\{fail\}\}is a failure penalty\. The inclusion ofRfailR\_\{\\text\{fail\}\}provides a negative gradient for high\-risk candidates, actively suppressing the selection of incompetent experts\. Our reward refines the raw rewardR​\(τ\)R\(\\tau\)defined in Section[2\.1](https://arxiv.org/html/2606.29424#S2.SS1)in two key aspects\. First, unlikeR​\(τ\)R\(\\tau\)which relies on binary outcome,r​\(m,x\)r\(m,x\)utilizes the offline capability estimateκ​\(m,x\)\\kappa\(m,x\)as a dense, expectation\-based signal to reduce training variance\. Second, instead of a simple cost subtraction \(−λ⋅Ctotal\-\\lambda\\cdot C\_\{\\text\{total\}\}\), we embed the efficiency constraint into the shape ofRsuccR\_\{\\text\{succ\}\}\. This design enforces a cost\-sensitive curriculum: for models cheaper than the optimal expertmbestm\_\{\\text\{best\}\}\(which are presumed under\-qualified under our assumption\), we clamp the reward to prevent the policy from drifting toward unreliable models; conversely, for capable but expensive models, we apply a decay penalty to discourage unnecessary computational overhead \(see Appendix[D\.1](https://arxiv.org/html/2606.29424#A4.SS1)for details\)\.

Practically, we operationalize the objective in Eq\.[7](https://arxiv.org/html/2606.29424#S3.E7)by incorporating the regularization term directly into the reward function, rather than modifying the underlying optimization algorithm\. By augmenting the extrinsic task reward with the log\-likelihood of the reference distribution, we implicitly enforce the dual regularization constraints\. As formally derived in Appendix[B](https://arxiv.org/html/2606.29424#A2), maximizing the return of this shaped reward is mathematically equivalent to the objective in Eq\.[7](https://arxiv.org/html/2606.29424#S3.E7)\. This formulation achieves a unique balance between safety and decisiveness through its regularization terms: \(1\) The−β​𝔻KL\-\\beta\\mathbb\{D\}\_\{\\text\{KL\}\}term functions as a Soft Trust Region\. Crucially, it regulates the dynamics of entropy reduction: by tethering the policy to the reference distributionπref\\pi\_\{\\text\{ref\}\}, it prevents the negative entropy term from collapsing the policy into a degenerate, overly sharp state too abruptly\. This ensures the router maintains valid exploration variance within the safe zone, preventing premature convergence to suboptimal local minima\. \(2\) The−β​ℋ​\(πθ\)\-\\beta\\mathcal\{H\}\(\\pi\_\{\\theta\}\)term encourages the router to minimize uncertainty in its final choice\. In our reward\-shaping implementation, this term is not applied as an additional standalone loss, it is algebraically absorbed into the reference\-shaped reward\. Once the reasoning process identifies a suitable model, this penalty incentivizes the router to produce a "sharp" and confident decision, preventing the policy from remaining in an indecisive probabilistic spread\.

ModelHMMT Nov 25AIME25OlympiadBenchAIME\-24AMCMATH500SMT2025Avg\.Acc\. \(↑\\uparrow\)Cost \(↓\\downarrow\)Acc\. \(↑\\uparrow\)Cost \(↓\\downarrow\)Acc\. \(↑\\uparrow\)Cost \(↓\\downarrow\)Acc\. \(↑\\uparrow\)Cost \(↓\\downarrow\)Acc\. \(↑\\uparrow\)Cost \(↓\\downarrow\)Acc\. \(↑\\uparrow\)Cost \(↓\\downarrow\)Acc\. \(↑\\uparrow\)Cost \(↓\\downarrow\)Acc\. \(↑\\uparrow\)Cost \(↓\\downarrow\)\\rowcolor\[gray\]0\.95HeuristicRandom68\.80\.278373\.30\.280980\.33\.983684\.60\.217995\.30\.421398\.01\.279076\.40\.495687\.10\.0050Qwen3\-4B59\.00\.019867\.10\.019277\.50\.289576\.90\.017393\.80\.031097\.40\.090765\.30\.032284\.50\.0004Qwen3\-30B71\.70\.248872\.10\.251980\.23\.843387\.50\.212695\.60\.389398\.21\.212376\.20\.498387\.20\.0048Qwen3\-235B81\.50\.569081\.70\.617983\.68\.781193\.30\.455397\.60\.838999\.22\.588184\.20\.918390\.10\.0105\\rowcolor\[gray\]0\.95Existing Routing ArchitecturesRouteLLM59\.00\.055967\.50\.040878\.21\.730080\.40\.043395\.80\.084897\.60\.104269\.80\.071285\.30\.0015FORC70\.40\.273175\.20\.264780\.94\.600090\.00\.343596\.20\.529598\.21\.767577\.60\.538187\.70\.0059Router\-R121\.30\.010828\.80\.011660\.30\.217441\.70\.019872\.70\.024193\.40\.001835\.40\.035970\.00\.0002xRouter28\.80\.004131\.00\.003360\.90\.066841\.70\.001872\.30\.001093\.20\.000934\.40\.006970\.40\.0001EntroRouter80\.00\.416378\.80\.292981\.85\.239790\.60\.288196\.60\.365898\.40\.523880\.40\.515588\.60\.0055

Table 1:Comparison across math benchmarks\.Avg\.aggregates performance on math tasks\.Costis measured as thetotal cost \($\\mathdollar\)for individual benchmarks, but converted toaverage per\-query costfor theAvg\.column\. The best routing method excluding the strongest fixed expert is highlighted inbold\.## 4Experiments

### 4\.1Setup

#### Benchmarks\.

We evaluate ENTROROUTER primarily on mathematical reasoning, where instances naturally span a wide range of difficulty levels and thus provide a suitable testbed for cost\-aware routing\. Our evaluation suite includesAIME 24Zhang and Math\-AI \([2024](https://arxiv.org/html/2606.29424#bib.bib24)\),AIME 25Zhang and Math\-AI \([2025](https://arxiv.org/html/2606.29424#bib.bib25)\),HMMT Nov 25Balunović et al\. \([2025](https://arxiv.org/html/2606.29424#bib.bib2)\),SMT 25,MATH500Hendrycks et al\. \([2021](https://arxiv.org/html/2606.29424#bib.bib9)\),AMCLi et al\. \([2024](https://arxiv.org/html/2606.29424#bib.bib11)\), andOlympiadBenchHe et al\. \([2024](https://arxiv.org/html/2606.29424#bib.bib7)\)\.

#### Candidate Models and Costs\.

The candidate poolℳ\\mathcal\{M\}contains three Qwen3 reasoning modelsYang et al\. \([2025](https://arxiv.org/html/2606.29424#bib.bib22)\): Qwen3\-4B\-Thinking\-2507, Qwen3\-30B\-A3B\-Thinking\-2507, and Qwen3\-235B\-A22B\-Thinking\-2507\. We abbreviate them as Qwen3\-4B, Qwen3\-30B, and Qwen3\-235B\. The largest model serves as the strongest\-expert reference\. For brevity, we also refer to it as the strongest fixed\-expert baseline\. Following prior routing work, we use standard API pricing as the cost functionC​\(m\)C\(m\)\.†††[https://novita\.ai/pricing](https://novita.ai/pricing)Router inference cost is excluded since its average overhead is only1\.7×10−51\.7\\times 10^\{\-5\}dollars per query, which is negligible relative to expert\-model costs\.

#### Training Details and Baselines\.

We use the non\-thinkingQwen3\-4Bas the router backbone\. Following the data construction pipeline in Appendix[C](https://arxiv.org/html/2606.29424#A3), we first perform SFT and then optimize the router with RL\. Full hyperparameters are provided in Appendix[D](https://arxiv.org/html/2606.29424#A4)\. We compare against heuristic strategies, including random routing and fixed model selection, as well as representative routing methods:RouteLLMOng et al\. \([2024](https://arxiv.org/html/2606.29424#bib.bib14)\),FORCŠakota et al\. \([2024](https://arxiv.org/html/2606.29424#bib.bib17)\),xRouter\(Qian et al\.,[2025](https://arxiv.org/html/2606.29424#bib.bib15)\), andRouter\-R1\(Zhang et al\.,[2025](https://arxiv.org/html/2606.29424#bib.bib23)\)\. Additional evaluation details are reported in Appendix[E](https://arxiv.org/html/2606.29424#A5)\.

### 4\.2Main Results

The results on mathematical reasoning benchmarks are summarized in Table[3\.3](https://arxiv.org/html/2606.29424#S3.SS3)\.EntroRouterachieves an average accuracy of88\.62%, outperforming all routing baselines and approaching the strongest fixed\-expert baseline while substantially reducing cost\. The results for Router\-R1 and xRouter provide empirical evidence for theTrust Region Collapsephenomenon analyzed in Section[3\.1](https://arxiv.org/html/2606.29424#S3.SS1)\. Both multi\-round architecturessuccumb tothis instability, where the policy degenerates into rarely invoking external experts and instead relying exclusively on its own limited internal capabilities\. Consequently, they yield poor performance, confirming that coupled reasoning\-routing fails to bridge the capability gap under sparse supervision\. We provide an additional intuitive explanation for this failure mode in Appendix[F](https://arxiv.org/html/2606.29424#A6)\. By treating routing as a distinct meta\-analysis task rather than an interleaved reasoning step, our framework avoids the interference of pre\-training priors\. Consequently,EntroRoutersuccessfully identifies the cost\-accuracy boundary, recovering98\.3%of the strongest expert’s accuracy while reducing computational costs by48\.25%\.

## 5Discussion

Beyond the main cost–accuracy results, we further analyzeEntroRouterfrom the perspectives of component effectiveness, routing behavior, and robustness\. We first ablate the key design choices in our training pipeline, including Soft Supervision, the gating term, the Soft Trust Region, and the performance reward \(§[5\.1](https://arxiv.org/html/2606.29424#S5.SS1)\)\. We then inspect the learned policy by visualizing routing distributions across problem\-difficulty levels \(§[5\.2](https://arxiv.org/html/2606.29424#S5.SS2)\)\. Additional studies in the appendix evaluate OOD generalization \(Appendix[H](https://arxiv.org/html/2606.29424#A8)\), sensitivity to the cost–accuracy coefficientα\\alpha\(Appendix[I](https://arxiv.org/html/2606.29424#A9)\), transfer to heterogeneous candidate pools \(Appendix[J](https://arxiv.org/html/2606.29424#A10)\), and robustness to noisy offline estimates \(Appendix[K](https://arxiv.org/html/2606.29424#A11)\)\.

MethodHMMT\-NovAIME25SMT2025Avg\.AccCostAccCostAccCostAccCostEntroRouter80\.00\.41678\.80\.29380\.40\.51679\.90\.011w/o Soft Supervision72\.10\.29472\.50\.20773\.10\.35772\.70\.008w/o Gate termg​\(⋅\)g\(\\cdot\)78\.10\.43177\.70\.42980\.20\.64579\.00\.013w/o Soft Trust Region71\.70\.28075\.80\.28076\.70\.55175\.10\.010w/o Performance Reward69\.60\.38578\.30\.36574\.80\.38474\.40\.010

Table 2:Ablation Study\.Impact of removing Soft Supervision, Gating termg​\(⋅\)g\(\\cdot\), Soft Trust Regions, and Task\-performance Reward\.### 5\.1Ablation Study

We evaluate the contribution of each component inEntroRouterby comparing it against four variants: \(1\) replacingSoft Supervisionwith hard targets; \(2\) removing theGating termg​\(⋅\)g\(\\cdot\)from the SFT target construction; \(3\) disabling theSoft Trust Regionanchor; and \(4\) substituting ourTask\-performance Rewardwith sparse outcome rewards \(See Appendix[D\.2](https://arxiv.org/html/2606.29424#A4.SS2)for details\)\. Results in Table[2](https://arxiv.org/html/2606.29424#S5.T2)indicate thatSoft Supervisionis paramount; its removal triggers a significant accuracy drop, confirming that hard supervision hinders exploration\. TheGating termg​\(⋅\)g\(\\cdot\)is essential for cost\-efficiency; by enforcing hard supervision for trivial instances during SFT, it explicitly teaches the router to assign simple queries to the cheapest expert\. Without it, average costs increase as the policy fails to learn this strict economy for easy tasks\. Finally, ablating theSoft Trust RegionorPerformance Rewarddegrades the trade\-off, validating the necessity of anchored regularization and granular feedback for stable optimization\.

### 5\.2Analysis of Routing Distribution

To verify whetherEntroRoutersuccessfully aligns model capabilities with problem complexity, we examine the routing distribution across different difficulty strata\. We stratify the evaluation set into three levels based onmbestm\_\{\\text\{best\}\}:Easy\(mbest=m\_\{\\text\{best\}\}=Qwen3\-4B\),Medium\(mbest=m\_\{\\text\{best\}\}=Qwen3\-30B\-A3B\), andHard\(remaining queries\)\. As illustrated in Figure[3](https://arxiv.org/html/2606.29424#S5.F3), the learned policy exhibits a clear adaptive strategy\. For queries classified asEasy, the router prioritizes computational efficiency, dispatching 76\.0% of instances to the lightweight Qwen3\-4B model\. Conversely, as problem complexity increases, the policy dynamically shifts probability mass towards stronger experts\. In theHardstratum, reliance on the efficient 4B model contracts to 25\.0%, while the invocation of Qwen3\-235B escalates significantly to 66\.0%\. This distribution results in a weighted average API cost that rises monotonically with difficulty, confirming thatEntroRouterhas effectively internalized the latent relationship between query hardness and necessary reasoning capability\.

This adaptive distribution empirically validates the efficacy ofEntroRouter’s entropy regulation mechanisms\. While conventional routing systems often suffer from uncontrolled entropy decay, our framework orchestrates a precise management of exploration capacity\. The Soft Supervision initializes the policy with a high\-entropy prior to ensure diversity\. Subsequently, in the RL stage, the reference\-shaped reward encourages cost\-aware decision sharpening, while the Soft Anchor constrains this contraction process\. This balanced regulation ensures the router transitions to a decisive state without suffering from precipitous trust region collapse, retaining the probabilistic capacity to recognize and respond to high\-complexity queries\.

![Refer to caption](https://arxiv.org/html/2606.29424v1/x4.png)Figure 3:Routing Distribution & Token Cost Efficiency across Difficulty Levels\.The stacked bars \(left axis\) represent the selection ratio of candidate models, while the trend line \(right axis\) tracks the weighted average API cost\.EntroRouterdemonstrates adaptive behavior by escalating model size in response to increasing problem difficulty\.

## 6Related Work

#### LLM Routing Systems

The rapid proliferation of Large Language Models \(LLMs\) has necessitated the development of sophisticated routing mechanisms to balance performance, cost, and latency\. Early approaches primarily focused on optimizing the trade\-off between cost and accuracy through heuristic or predictive means\.Chen et al\. \([2023](https://arxiv.org/html/2606.29424#bib.bib3)\)sequentially invokes models from smaller to larger scales to reduce inference costs while maintaining performance\. To standardize the evaluation of these growing routing strategies, RouterBenchHu et al\. \([2024](https://arxiv.org/html/2606.29424#bib.bib10)\)was introduced as a comprehensive benchmark\. Structurally,Feng et al\. \([2024](https://arxiv.org/html/2606.29424#bib.bib5)\)constructs interaction graphs between queries and models to capture complex dependencies that linear classifiers might miss\. More recently,Ding et al\. \([2025](https://arxiv.org/html/2606.29424#bib.bib4)\)introduced a test\-time optimal compute framework, adaptively allocating computational resources based on the instance\-level difficulty\. Inspired by the success of RL in reasoning, recent research has formalized routing as a multi\-round sequential decision process, where the router acts as an agent to iteratively query and integrate feedback from external expertsZhang et al\. \([2025](https://arxiv.org/html/2606.29424#bib.bib23)\); Qian et al\. \([2025](https://arxiv.org/html/2606.29424#bib.bib15)\)\.

#### Reinforcement Learning for Reasoning

Reinforcement Learning has emerged as a critical paradigm to stimulate model reasoning without extensive supervision\. Traditional RL approaches, such as Proximal Policy Optimization \(PPO\)Schulman et al\. \([2017](https://arxiv.org/html/2606.29424#bib.bib18)\), rely on a value function and a clipped surrogate objective to stabilize training within a trust region\. More recently, Group Relative Policy Optimization \(GRPO\)Shao et al\. \([2024](https://arxiv.org/html/2606.29424#bib.bib19)\)has gained prominence in mathematical reasoning\. By estimating the baseline from the group mean of outputs rather than a critic, GRPO significantly improves training efficiency\.

## 7Conclusion

In this work, we identifiedTrust Region Collapseas a critical failure mode in the emerging multi\-round routing paradigm, stemming from the capability mismatch between the router and expert models\. To address this, we proposedEntroRouter, a single\-round framework that decouples planning from reasoning\. By leveraging Soft Supervision and Soft Anchor regularization, our method establishes a stable optimization landscape that prevents policy degeneration\. Empirical results demonstrate thatEntroRouterachieves Pareto efficiency, retaining98\.3%of the strongest expert’s accuracy while reducing computational costs by48\.2%\. These findings suggest that small models are best utilized not merely as weaker reasoners, but as effectivemeta\-controllersfor scalable AI systems\.

## Limitations

This work focuses on establishing a robust and efficient routing framework under a single\-round decision paradigm\. Our analysis of Trust Region Collapse characterizes an important failure regime of multi\-round routing, where expert calls are costly, supervision is sparse, and the router may not effectively use out\-of\-distribution expert traces\. Future work could extend this analysis to richer multi\-step routing agents equipped with stronger process supervision, explicit verification, or more capable aggregation modules\. Similarly, whileEntroRouteruses offline capability estimatesκ​\(m,x\)\\kappa\(m,x\)to construct training targets and rewards, these estimates may contain sampling noise or verifier errors\. We have examined this issue through noisy\-estimate perturbation experiments, and a promising future direction is to develop adaptive profiling or periodic re\-estimation strategies for dynamically changing model pools\.

## References

- Agarwal et al\. \(2025\)OpenAI Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K\. Arora, Yu Bai, Bowen Baker, Hai\-Biao Bao, Boaz Barak, Ally Bennett, Tyler Bertao, N\. Archer Brett, Eugene Brevdo, Greg Brockman, Sébastien Bubeck, Cheng Chang, Kai Chen, and 105 others\. 2025\.[gpt\-oss\-120b&gpt\-oss\-20b model card](https://api.semanticscholar.org/CorpusID:280671456)\.
- Balunović et al\. \(2025\)Mislav Balunović, Jasper Dekoninck, Ivo Petrov, Nikola Jovanović, and Martin Vechev\. 2025\.[Matharena: Evaluating llms on uncontaminated math competitions](https://matharena.ai/)\.
- Chen et al\. \(2023\)Lingjiao Chen, Matei Zaharia, and James Zou\. 2023\.Frugalgpt: How to use large language models while reducing cost and improving performance\.*arXiv preprint arXiv:2305\.05176*\.
- Ding et al\. \(2025\)Dujian Ding, Ankur Mallick, Shaokun Zhang, Chi Wang, Daniel Madrigal, Mirian Del Carmen Hipolito Garcia, Menglin Xia, Laks VS Lakshmanan, Qingyun Wu, and Victor Rühle\. 2025\.Best\-route: Adaptive llm routing with test\-time optimal compute\.*arXiv preprint arXiv:2506\.22716*\.
- Feng et al\. \(2024\)Tao Feng, Yanzhen Shen, and Jiaxuan You\. 2024\.Graphrouter: A graph\-based router for llm selections\.*arXiv preprint arXiv:2410\.03834*\.
- Google \(2025\)Google\. 2025\.[A new era of intelligence with gemini 3](https://blog.google/products/gemini/gemini-3)\.
- He et al\. \(2024\)Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun\. 2024\.[OlympiadBench: A challenging benchmark for promoting AGI with olympiad\-level bilingual multimodal scientific problems](https://doi.org/10.18653/v1/2024.acl-long.211)\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 3828–3850, Bangkok, Thailand\. Association for Computational Linguistics\.
- He et al\. \(2026\)Zhiwei He, Tian Liang, Jiahao Xu, Qiuzhi Liu, Xingyu Chen, Yue Wang, Linfeng Song, Dian Yu, Zhenwen Liang, Wenxuan Wang, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu\. 2026\.[Deepmath\-103k: A large\-scale, challenging, decontaminated, and verifiable mathematical dataset for advancing reasoning](https://openreview.net/forum?id=kHB5Te5IWm)\.In*The Fourteenth International Conference on Learning Representations*\.
- Hendrycks et al\. \(2021\)Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt\. 2021\.Measuring mathematical problem solving with the math dataset\.*arXiv preprint arXiv:2103\.03874*\.
- Hu et al\. \(2024\)Qitian Jason Hu, Jacob Bieker, Xiuyu Li, Nan Jiang, Benjamin Keigwin, Gaurav Ranganath, Kurt Keutzer, and Shriyash Kaustubh Upadhyay\. 2024\.Routerbench: A benchmark for multi\-llm routing system\.*arXiv preprint arXiv:2403\.12031*\.
- Li et al\. \(2024\)Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q Jiang, Ziju Shen, and 1 others\. 2024\.Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions\.*Hugging Face repository*, 13\(9\):9\.
- McKenzie et al\. \(2023\)Ian R McKenzie, Alexander Lyzhov, Michael Pieler, Alicia Parrish, Aaron Mueller, Ameya Prabhu, Euan McLean, Aaron Kirtland, Alexis Ross, Alisa Liu, and 1 others\. 2023\.Inverse scaling: When bigger isn’t better\.*arXiv preprint arXiv:2306\.09479*\.
- Moshkov et al\. \(2025\)Ivan Moshkov, Darragh Hanley, Ivan Sorokin, Shubham Toshniwal, Christof Henkel, Benedikt Schifferer, Wei Du, and Igor Gitman\. 2025\.Aimo\-2 winning solution: Building state\-of\-the\-art mathematical reasoning models with openmathreasoning dataset\.*arXiv preprint arXiv:2504\.16891*\.
- Ong et al\. \(2024\)Isaac Ong, Amjad Almahairi, Vincent Wu, Wei\-Lin Chiang, Tianhao Wu, Joseph E Gonzalez, M Waleed Kadous, and Ion Stoica\. 2024\.Routellm: Learning to route llms with preference data\.*arXiv preprint arXiv:2406\.18665*\.
- Qian et al\. \(2025\)Cheng Qian, Zuxin Liu, Shirley Kokane, Akshara Prabhakar, Jielin Qiu, Haolin Chen, Zhiwei Liu, Heng Ji, Weiran Yao, Shelby Heinecke, Silvio Savarese, Caiming Xiong, and Huan Wang\. 2025\.[xrouter: Training cost\-aware llms orchestration system via reinforcement learning](https://arxiv.org/abs/2510.08439)\.*Preprint*, arXiv:2510\.08439\.
- Rein et al\. \(2024\)David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman\. 2024\.Gpqa: A graduate\-level google\-proof q&a benchmark\.In*First Conference on Language Modeling*\.
- Šakota et al\. \(2024\)Marija Šakota, Maxime Peyrard, and Robert West\. 2024\.Fly\-swat or cannon? cost\-effective language model choice via meta\-modeling\.In*Proceedings of the 17th ACM International Conference on Web Search and Data Mining*, pages 606–615\.
- Schulman et al\. \(2017\)John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov\. 2017\.Proximal policy optimization algorithms\.*arXiv preprint arXiv:1707\.06347*\.
- Shao et al\. \(2024\)Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y\. K\. Li, Y\. Wu, and Daya Guo\. 2024\.[Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://arxiv.org/abs/2402.03300)\.*Preprint*, arXiv:2402\.03300\.
- Sheng et al\. \(2024\)Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu\. 2024\.Hybridflow: A flexible and efficient rlhf framework\.*arXiv preprint arXiv: 2409\.19256*\.
- Wang et al\. \(2024\)Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen\. 2024\.[Mmlu\-pro: A more robust and challenging multi\-task language understanding benchmark](https://arxiv.org/abs/2406.01574)\.*Preprint*, arXiv:2406\.01574\.
- Yang et al\. \(2025\)An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others\. 2025\.[Qwen3 technical report](https://arxiv.org/abs/2505.09388)\.*Preprint*, arXiv:2505\.09388\.
- Zhang et al\. \(2025\)Haozhen Zhang, Tao Feng, and Jiaxuan You\. 2025\.Router\-r1: Teaching llms multi\-round routing and aggregation via reinforcement learning\.In*The Thirty\-ninth Annual Conference on Neural Information Processing Systems*\.
- Zhang and Math\-AI \(2024\)Yifan Zhang and Team Math\-AI\. 2024\.American invitational mathematics examination \(aime\) 2024\.
- Zhang and Math\-AI \(2025\)Yifan Zhang and Team Math\-AI\. 2025\.American invitational mathematics examination \(aime\) 2025\.
- Zhao et al\. \(2025\)Xueliang Zhao, Wei Wu, Jian Guan, and Lingpeng Kong\. 2025\.Promptcot: Synthesizing olympiad\-level problems for mathematical reasoning in large language models\.*arXiv preprint arXiv:2503\.02324*\.

## Appendix ATheoretical Analysis of Trust Region Collapse

In this appendix, we provide a formal derivation of theTrust Region Collapsephenomenon\. We first define the simplified decision paradigm and the decomposed advantage function\. We then justify our core assumption based on the inverse scaling hypothesis and provide a rigorous proof that the effective trust region collapses to an empty set under sparse supervision\.

### A\.1Problem Formulation and Advantage Decomposition

To facilitate our derivation, we abstract the router’s action space into a binary decision paradigm at steptt:

1. 1\.Internal Reasoning \(aselfa\_\{\\text\{self\}\}\):The router generates the solution using its own parameters \(at=mselfa\_\{t\}=m\_\{\\text\{self\}\}\), incurring zero marginal cost\.
2. 2\.External Escalation \(aexta\_\{\\text\{ext\}\}\):The router delegates to the model pool \(at∈ℳpoola\_\{t\}\\in\\mathcal\{M\}\_\{\\text\{pool\}\}\)\. We treat this as a unified, costly action with costC​\(aext\)=c\>0C\(a\_\{\\text\{ext\}\}\)=c\>0\.

The policy optimization is driven by the Advantage FunctionAπ​\(s,a\)A^\{\\pi\}\(s,a\)\. LetVπ​\(s\)V^\{\\pi\}\(s\)be the value function andC¯π​\(s\)\\bar\{C\}\_\{\\pi\}\(s\)the expected baseline cost\. The advantage of the costly escalation actionaexta\_\{\\text\{ext\}\}decomposes into a performance gain and a cost penalty:

Aπ​\(s,aext\)=\(𝔼​\[R∣s,aext\]−𝔼a∼π​\[R∣s,a\]\)⏟Performance Gain​\(Δ​R\)−\(c−C¯π​\(s\)\)⏟Marginal Cost​\(Δ​C\)\\begin\{split\}A^\{\\pi\}\(s,a\_\{\\text\{ext\}\}\)&=\\underbrace\{\(\\mathbb\{E\}\[R\\mid s,a\_\{\\text\{ext\}\}\]\-\\mathbb\{E\}\_\{a\\sim\\pi\}\[R\\mid s,a\]\)\}\_\{\\text\{Performance Gain \}\(\\Delta R\)\}\\\\ &\\quad\-\\underbrace\{\(c\-\\bar\{C\}\_\{\\pi\}\(s\)\)\}\_\{\\text\{Marginal Cost \}\(\\Delta C\)\}\\end\{split\}\(9\)Sinceaexta\_\{\\text\{ext\}\}is strictly more expensive thanaselfa\_\{\\text\{self\}\}, the marginal cost termΔ​C\\Delta Cremains strictly positive unless the policy has fully converged toaexta\_\{\\text\{ext\}\}\.

### A\.2Justification of Strong Prior Dominance

We posit that augmenting a weak router with a strong expert does not monotonically improve reward under sparse supervision\. We formalize this as:

###### Assumption A\.1\(Strong Prior Dominance\)\.

Letπrouter\\pi\_\{\\text\{router\}\}be initialized from a pre\-trained LM and optimized via sparse outcome rewards\. There exists a regime where the posterior update induced by an out\-of\-distribution expert traceo∼πexperto\\sim\\pi\_\{\\text\{expert\}\}is dominated by the router’s prior, such that:

𝔼o∼πexpert\(⋅∣x\)​\[R∣x⊕o\]≤𝔼​\[R∣x\]\.\\displaystyle\\mathbb\{E\}\_\{o\\sim\\pi\_\{\\text\{expert\}\}\(\\cdot\\mid x\)\}\[R\\mid x\\oplus o\]\\leq\\mathbb\{E\}\[R\\mid x\]\.\(10\)

#### Theoretical Grounding\.

This assumption is grounded in the “Inverse Scaling” taxonomyMcKenzie et al\. \([2023](https://arxiv.org/html/2606.29424#bib.bib12)\)\. Large language models rely on two information sources: pre\-training priors and in\-context prompts\. When an expert provides a complex, out\-of\-distribution \(OOD\) reasoning path, it creates a conflict\.

- •Strong Priors and Unwanted Imitation:As seen inResisting Correctiontasks, models often ignore OOD instructions \(the expert trace\) and revert to their own high\-likelihood generation manifold\. The router essentially “talks over” the expert\.
- •Distractor Task Failure:Under sparse supervision, the optimizer cannot distinguish betweenplanning failure\(wrong routing\) andexecution failure\(misinterpreting the expert\)\. The expert trace effectively acts as a “distractor,” causing the router to focus on superficial pattern matching rather than the intended logic, leading toΔ​R≤0\\Delta R\\leq 0\.

### A\.3Proof of Trust Region Collapse

We formally define theEffective Trust Region𝒯eff\\mathcal\{T\}\_\{\\text\{eff\}\}as the set of policy distributions that satisfy safety constraints, positive expected advantage, and an increased probability of utilizing the expert:

𝒯eff≜\{π′∈Δ​\(𝒜\)\|DK​L\(π′\|\|π\)≤ϵ,\(Trust Region\)𝔼a∼π′​\[Aπ​\(s,a\)\]\>0,\(Policy Improvement\)π′​\(aext\)\>π​\(aext\)\(Escalation Intent\)\}\\mathcal\{T\}\_\{\\text\{eff\}\}\\triangleq\\left\\\{\\pi^\{\\prime\}\\in\\Delta\(\\mathcal\{A\}\)\\;\\middle\|\\;\\begin\{aligned\} &D\_\{KL\}\(\\pi^\{\\prime\}\\,\|\|\\,\\pi\)\\leq\\epsilon,\\\\ &\\quad\(\\text\{Trust Region\}\)\\\\ &\\mathbb\{E\}\_\{a\\sim\\pi^\{\\prime\}\}\[A^\{\\pi\}\(s,a\)\]\>0,\\\\ &\\quad\(\\text\{Policy Improvement\}\)\\\\ &\\pi^\{\\prime\}\(a\_\{\\text\{ext\}\}\)\>\\pi\(a\_\{\\text\{ext\}\}\)\\\\ &\\quad\(\\text\{Escalation Intent\}\)\\end\{aligned\}\\right\\\}\(11\)
Based on thePerformance Difference Lemma, this region represents the viable search space where increasing reliance on the strong solver is theoretically guaranteed to improve the objective\.

###### Theorem A\.2\(Trust Region Collapse\)\.

Under Assumption[A\.1](https://arxiv.org/html/2606.29424#A1.Thmtheorem1)and strict positive costc\>0c\>0, the advantage of escalation is strictly negative \(Aπ​\(s,aext\)<0A^\{\\pi\}\(s,a\_\{\\text\{ext\}\}\)<0\)\. Consequently,𝒯eff=∅\\mathcal\{T\}\_\{\\text\{eff\}\}=\\emptyset\.

###### Proof\.

The proof proceeds by contradiction\. We first establish the sign of the advantage function and then analyze the constraints onπ′\\pi^\{\\prime\}\.

1\. Negative Advantage of Escalation\.By Assumption[A\.1](https://arxiv.org/html/2606.29424#A1.Thmtheorem1), the expected reward of escalation satisfies𝔼​\[R\|aext\]≤𝔼​\[R\|aself\]\\mathbb\{E\}\[R\|a\_\{\\text\{ext\}\}\]\\leq\\mathbb\{E\}\[R\|a\_\{\\text\{self\}\}\]\. TheQQ\-values for the actions are:

Qπ​\(s,aself\)\\displaystyle Q^\{\\pi\}\(s,a\_\{\\text\{self\}\}\)=𝔼​\[R\|aself\]\(since​C=0\)\\displaystyle=\\mathbb\{E\}\[R\|a\_\{\\text\{self\}\}\]\\quad\(\\text\{since \}C=0\)\(12\)Qπ​\(s,aext\)\\displaystyle Q^\{\\pi\}\(s,a\_\{\\text\{ext\}\}\)=𝔼​\[R\|aext\]−λ​c\\displaystyle=\\mathbb\{E\}\[R\|a\_\{\\text\{ext\}\}\]\-\\lambda c\(13\)Subtracting these yieldsQπ​\(s,aext\)−Qπ​\(s,aself\)≤−λ​cQ^\{\\pi\}\(s,a\_\{\\text\{ext\}\}\)\-Q^\{\\pi\}\(s,a\_\{\\text\{self\}\}\)\\leq\-\\lambda c\. Sinceλ,c\>0\\lambda,c\>0, we have strict dominance:Qπ​\(s,aext\)<Qπ​\(s,aself\)Q^\{\\pi\}\(s,a\_\{\\text\{ext\}\}\)<Q^\{\\pi\}\(s,a\_\{\\text\{self\}\}\)\. This implies strictly negative advantage:

Aπ​\(s,aext\)<0andAπ​\(s,aself\)\>0\.\\displaystyle A^\{\\pi\}\(s,a\_\{\\text\{ext\}\}\)<0\\quad\\text\{and\}\\quad A^\{\\pi\}\(s,a\_\{\\text\{self\}\}\)\>0\.\(14\)
2\. Constraint Contradiction\.For any candidate policyπ′∈𝒯eff\\pi^\{\\prime\}\\in\\mathcal\{T\}\_\{\\text\{eff\}\}, letΔ​p=π′​\(aext\)−π​\(aext\)\\Delta p=\\pi^\{\\prime\}\(a\_\{\\text\{ext\}\}\)\-\\pi\(a\_\{\\text\{ext\}\}\)\. TheEscalation Intentcondition requiresΔ​p\>0\\Delta p\>0\. ThePolicy Improvementcondition requires∑aπ′​\(a\)​Aπ​\(s,a\)\>0\\sum\_\{a\}\\pi^\{\\prime\}\(a\)A^\{\\pi\}\(s,a\)\>0\. Using the identity∑π​\(a\)​Aπ​\(s,a\)=0\\sum\\pi\(a\)A^\{\\pi\}\(s,a\)=0, we expand this summation:

𝔼π′​\[Aπ\]\\displaystyle\\mathbb\{E\}\_\{\\pi^\{\\prime\}\}\[A^\{\\pi\}\]=∑a\(π′​\(a\)−π​\(a\)\)​Aπ​\(s,a\)\\displaystyle=\\sum\_\{a\}\(\\pi^\{\\prime\}\(a\)\-\\pi\(a\)\)A^\{\\pi\}\(s,a\)\(15\)=Δ​p⋅Aπ​\(s,aext\)\\displaystyle=\\Delta p\\cdot A^\{\\pi\}\(s,a\_\{\\text\{ext\}\}\)\+\(−Δ​p\)⋅Aπ​\(s,aself\)\\displaystyle\\quad\+\(\-\\Delta p\)\\cdot A^\{\\pi\}\(s,a\_\{\\text\{self\}\}\)\(16\)=Δ​p​\(Aπ​\(s,aext\)−Aπ​\(s,aself\)\)⏟Strictly Negative​δ\\displaystyle=\\Delta p\\underbrace\{\\left\(A^\{\\pi\}\(s,a\_\{\\text\{ext\}\}\)\-A^\{\\pi\}\(s,a\_\{\\text\{self\}\}\)\\right\)\}\_\{\\text\{Strictly Negative \}\\delta\}\(17\)From Step 1, the termδ\\deltais strictly negative\. For the total expectation to be positive \(Δ​p⋅δ\>0\\Delta p\\cdot\\delta\>0\), we must haveΔ​p<0\\Delta p<0\.

Conclusion\.The condition for Policy Improvement \(Δ​p<0\\Delta p<0\) directly contradicts the condition for Escalation Intent \(Δ​p\>0\\Delta p\>0\)\. Thus, no policyπ′\\pi^\{\\prime\}exists that satisfies both conditions simultaneously\. The effective trust region is empty\. ∎

#### Implication\.

This result shows that, in the regime characterized by Assumption[A\.1](https://arxiv.org/html/2606.29424#A1.Thmtheorem1), collapse is not merely an optimizer accident\. It follows from the combination of sparse outcome supervision, positive escalation cost, and the router’s failure to extract reward\-improving information from the expert trace\. This necessitates theCognitive Decouplingproposed in our method, effectively separating the routing decision from the trace execution\.

#### Remark on Generalization\.

It is important to note that while our theoretical derivation assumes a binary action space \(as​e​l​fa\_\{self\}vs\.ae​x​ta\_\{ext\}\) for clarity, the conclusion generalizes to hierarchical multi\-model settings\. The decision process in a tiered candidate pool \(e\.g\., 4B→\\to30B→\\to235B\) can be decomposed into a sequence of recursive binary decisions—essentially determining whether to escalate to the next tier of capability\. Therefore, the structural validity of the Trust Region Collapse phenomenon remains applicable to the broader selection problem\.

## Appendix BTheoretical Analysis: Soft Anchor as a Trust Region

In this appendix, we formally establish the mathematical equivalence between the Soft Anchor mechanism and a constrained optimization problem solved via Lagrange multipliers\.

#### Entropy Regularization termℋ​\(πθ\)\\mathcal\{H\}\(\\pi\_\{\\theta\}\)\.

In our optimization objective, the termℋ​\(πθ\)\\mathcal\{H\}\(\\pi\_\{\\theta\}\)denotes the Shannon entropy of the stochastic policyπθ\(⋅\|x\)\\pi\_\{\\theta\}\(\\cdot\|x\)over the discrete action space of candidate modelsℳ\\mathcal\{M\}\. Formally, for a given input queryxx, it is defined as:

ℋ​\(πθ\)=−∑m∈ℳπθ​\(m\|x\)​log⁡πθ​\(m\|x\)\.\\mathcal\{H\}\(\\pi\_\{\\theta\}\)=\-\\sum\_\{m\\in\\mathcal\{M\}\}\\pi\_\{\\theta\}\(m\|x\)\\log\\pi\_\{\\theta\}\(m\|x\)\.\(18\)While entropy maximization is commonly used in Reinforcement Learning to encourage exploration, we strictly employ a negative coefficient−β\-\\betato penalize high\-entropy states\. This mechanism acts as anentropy contractionconstraint: it incentivizes the router to minimize decision uncertainty and converge towards a sharp, deterministic selection of the optimal expert, preventing the policy from remaining in an indecisive probabilistic spread at convergence\.

### B\.1Problem Formulation via Lagrange Multipliers

We formulate the router’s objective as a dual\-goal optimization problem: we seek to maximize the expected reward while simultaneously minimizing decision uncertainty \(decision sharpening\), subject to a trust\-region constraint anchored to the reference policyπref\\pi\_\{\\text\{ref\}\}\.

Formally, this is expressed as:

maxπ\\displaystyle\\max\_\{\\pi\}\\quad𝔼m∼π​\[r​\(m,x\)\]−β​ℋ​\(π\)\\displaystyle\\mathbb\{E\}\_\{m\\sim\\pi\}\[r\(m,x\)\]\-\\beta\\mathcal\{H\}\(\\pi\)\(19\)s\.t\.𝔻KL\(π\(⋅∣x\)\|\|πref\(⋅∣x\)\)≤δ\\displaystyle\\mathbb\{D\}\_\{\\text\{KL\}\}\(\\pi\(\\cdot\\mid x\)\\,\|\|\\,\\pi\_\{\\text\{ref\}\}\(\\cdot\\mid x\)\)\\leq\\deltawhereℋ​\(π\)\\mathcal\{H\}\(\\pi\)is the entropy term we wish to penalize to ensure decisiveness, andδ\\deltadefines the radius of the safe trust region\.

We solve this constrained problem using the method of Lagrange multipliers\. The Lagrangianℒ​\(π,λ\)\\mathcal\{L\}\(\\pi,\\lambda\)is:

ℒ​\(π,λ\)=𝔼π​\[r​\(m,x\)\]−β​ℋ​\(π\)−λ\(𝔻KL\(π\|\|πref\)−δ\)\.\\begin\{split\}\\mathcal\{L\}\(\\pi,\\lambda\)&=\\mathbb\{E\}\_\{\\pi\}\[r\(m,x\)\]\-\\beta\\mathcal\{H\}\(\\pi\)\\\\ &\\quad\-\\lambda\\left\(\\mathbb\{D\}\_\{\\text\{KL\}\}\(\\pi\\,\|\|\\,\\pi\_\{\\text\{ref\}\}\)\-\\delta\\right\)\.\\end\{split\}\(20\)For simplicity, and to align the regularization strength, we assume a unified coefficientλ=β\\lambda=\\beta\. The objective effectively becomes maximizing:

𝒥\(π\)=𝔼π\[r\(m,x\)\]−βℋ\(π\)−β𝔻KL\(π\|\|πref\)\.\\mathcal\{J\}\(\\pi\)=\\mathbb\{E\}\_\{\\pi\}\[r\(m,x\)\]\-\\beta\\mathcal\{H\}\(\\pi\)\-\\beta\\mathbb\{D\}\_\{\\text\{KL\}\}\(\\pi\\,\|\|\\,\\pi\_\{\\text\{ref\}\}\)\.\(21\)

### B\.2Equivalence to Reward Shaping

To demonstrate that our implementation \(Eq\.[7](https://arxiv.org/html/2606.29424#S3.E7)in the main text\) optimizes this Lagrangian, we expand the KL divergence term using the identity𝔻KL\(π\|\|πref\)=−ℋ\(π\)−𝔼π\[logπref\]\\mathbb\{D\}\_\{\\text\{KL\}\}\(\\pi\\,\|\|\\,\\pi\_\{\\text\{ref\}\}\)=\-\\mathcal\{H\}\(\\pi\)\-\\mathbb\{E\}\_\{\\pi\}\[\\log\\pi\_\{\\text\{ref\}\}\]\. Substituting this into the objective:

𝒥​\(π\)\\displaystyle\\mathcal\{J\}\(\\pi\)=𝔼π​\[r\]−β​ℋ​\(π\)\\displaystyle=\\mathbb\{E\}\_\{\\pi\}\[r\]\-\\beta\\mathcal\{H\}\(\\pi\)−β​\(−ℋ​\(π\)−𝔼π​\[log⁡πref\]\)\\displaystyle\\quad\-\\beta\\left\(\-\\mathcal\{H\}\(\\pi\)\-\\mathbb\{E\}\_\{\\pi\}\[\\log\\pi\_\{\\text\{ref\}\}\]\\right\)=𝔼π​\[r\]−β​ℋ​\(π\)\+β​ℋ​\(π\)\\displaystyle=\\mathbb\{E\}\_\{\\pi\}\[r\]\-\\beta\\mathcal\{H\}\(\\pi\)\+\\beta\\mathcal\{H\}\(\\pi\)\(22\)\+β​𝔼π​\[log⁡πref\]\\displaystyle\\quad\+\\beta\\mathbb\{E\}\_\{\\pi\}\[\\log\\pi\_\{\\text\{ref\}\}\]=𝔼π​\[r​\(m,x\)\+β​log⁡πref​\(m∣x\)\]\.\\displaystyle=\\mathbb\{E\}\_\{\\pi\}\\left\[r\(m,x\)\+\\beta\\log\\pi\_\{\\text\{ref\}\}\(m\\mid x\)\\right\]\.\(23\)
Conclusion:This derivation proves that simply augmenting the reward with the log\-likelihood of the reference policy \(i\.e\., maximizing𝔼​\[r\+β​log⁡πref\]\\mathbb\{E\}\[r\+\\beta\\log\\pi\_\{\\text\{ref\}\}\]\) is mathematically equivalent to solving the constrained optimization problem in Eq\.[19](https://arxiv.org/html/2606.29424#A2.E19)\. The implementation elegantly cancels out the intrinsic entropy bonus typically associated with KL constraints \(since−DK​L\-D\_\{KL\}includes\+ℋ\+\\mathcal\{H\}\), thereby implicitly enforcing the entropy penalty required for decision sharpening\.

## Appendix CData Construction Details

In this section, we provide a comprehensive breakdown of our data synthesis pipeline, including the criteria for difficulty assessment, the implementation of Soft Supervision, and the generation of reasoning trajectories\.

### C\.1SFT Dataset Construction

#### Data Sources and Model Pool\.

We initially curated a collection of approximately 190k raw mathematical problems from high\-quality open\-source reasoning datasets, specificallyOpenMathReasoningMoshkov et al\. \([2025](https://arxiv.org/html/2606.29424#bib.bib13)\)andDeepMathHe et al\. \([2026](https://arxiv.org/html/2606.29424#bib.bib8)\)\. Our candidate routing poolℳ\\mathcal\{M\}consists of three models with escalating scale:\{m4B,m30B,m235B\}\\\{m\_\{\\text\{4B\}\},m\_\{\\text\{30B\}\},m\_\{\\text\{235B\}\}\\\}, corresponding to theQwen3\-Thinking\-2507series \(4B, 30B\-A3B, and 235B\-A22B, respectively\)Yang et al\. \([2025](https://arxiv.org/html/2606.29424#bib.bib22)\)\. Distinct from this pool, we utilizeQwen3\-1\.7Bexclusively as aCalibration Probe\(mprobem\_\{\\text\{probe\}\}\) to identify queries at the lower bound of difficulty\. We set the thresholdτ=0\.8\\tau=0\.8\.

#### Expert Profiling and Filtering\.

For each queryxx, we estimate the ground\-truth pass rateκ​\(m,x\)\\kappa\(m,x\)viaN=5N=5Monte Carlo rollouts\. The profiling process begins with the probemprobem\_\{\\text\{probe\}\}\. Ifκ​\(mprobe,x\)≥τ\\kappa\(m\_\{\\text\{probe\}\},x\)\\geq\\tau, the gating term is set to𝒢​\(x\)=1\\mathcal\{G\}\(x\)=1, identifying the query as trivial\. Under our cost\-capability assumption, these samples are assigned tombest=m\_\{\\text\{best\}\}=Qwen3\-4B\-Thinking\-2507, the globally cheapest model inℳ\\mathcal\{M\}, to minimize inference overhead\.

For non\-trivial queries \(𝒢​\(x\)=0\\mathcal\{G\}\(x\)=0\), we profile the remaining candidates\. Themost economical expertmbestm\_\{\\text\{best\}\}is determined as the cheapest model satisfyingκ​\(m,x\)≥τ\\kappa\(m,x\)\\geq\\tau\(Eq\.[1](https://arxiv.org/html/2606.29424#S3.E1)\)\. If no model inℳ\\mathcal\{M\}meets the reliability threshold, we default tombest=m235Bm\_\{\\text\{best\}\}=m\_\{\\text\{235B\}\}to prioritize correctness\.

#### Soft Target Construction

To implement theSoft Supervisiondefined in Eq\.[3](https://arxiv.org/html/2606.29424#S3.E3), we construct the soft target support from the most economical qualified experts\. For non\-trivial instances where at least two candidate models satisfy the reliability threshold, we setℳtop2\\mathcal\{M\}\_\{\\text\{top2\}\}to the two cheapest qualified models and distribute the target probability uniformly over them\. This soft target avoids forcing the router to imitate a single arbitrary expert when multiple models are empirically viable\.

For high\-difficulty instances where the qualified support degenerates to the strongest expert, the model\-label target becomes a singleton,

𝒬​\(m\|x\)=𝕀​\[m=m235B\],\\mathcal\{Q\}\(m\|x\)=\\mathbb\{I\}\[m=m\_\{\\text\{235B\}\}\],rather than an expanded set with duplicate model labels\. To strengthen supervision for these extreme cases, we applyMulti\-path Reasoning Distillation: for the same query and the same target model labelm235Bm\_\{\\text\{235B\}\}, we synthesize two distinct reasoning trajectories\(yCoT\(1\),yCoT\(2\)\)\(y\_\{\\text\{CoT\}\}^\{\(1\)\},y\_\{\\text\{CoT\}\}^\{\(2\)\}\)\. This duplicates the training signal at the rationale level, not at the model\-label level\. By exposing the router to diverse analytical patterns associated with the strongest expert, the router learns to recognize extreme query complexity from multiple reasoning perspectives, improving the robustness of its query\-analysis behavior for the most challenging instances\.

Equivalently, this procedure can be viewed as upweighting extreme\-difficulty instances in the SFT corpus while keeping the routing target distribution𝒬​\(m\|x\)\\mathcal\{Q\}\(m\|x\)mathematically well\-defined\.

Prevention of Label Inflation\.The inclusion ofmprobem\_\{\\text\{probe\}\}for initial filtering is a critical design choice\. Sincemprobe∉ℳpoolm\_\{\\text\{probe\}\}\\notin\\mathcal\{M\}\_\{\\text\{pool\}\}, trivial queries \(solvable by a 1\.7B model\) would otherwise be assigned to Qwen3\-4B\-Thinking\-2507\. Under our setting, this would shift the target probability toward Qwen3\-30B\-A3B\-Thinking\-2507, leading to "compute waste" where simple logic is handled by mid\-to\-large tier models\. Our probe\-based filtering ensures the router focuses strictly on the appropriate reasoning complexity\.

#### Reasoning Rationale Synthesis\.

To equip the router with the intermediate reasoning capabilities described in Section[3\.2](https://arxiv.org/html/2606.29424#S3.SS2), we synthesized high\-quality reasoning rationales \(CoT\) for the valid\(x,mbest\)\(x,m\_\{\\text\{best\}\}\)pairs\. We first utilizedGemini\-3\-ProGoogle \([2025](https://arxiv.org/html/2606.29424#bib.bib6)\)to generate a reasoning exemplar\. Using this as a 1\-shot seed, we promptedGPT\-OSS\-120BAgarwal et al\. \([2025](https://arxiv.org/html/2606.29424#bib.bib1)\)to synthesize reasoning chains for the entire filtered dataset\. This distillation process ensures that the router’s analysis phase is grounded in sophisticated logical depth\. The final SFT dataset comprises approximately 270k samples, encompassing query analysis traces and their corresponding soft\-target routing labels\.

### C\.2RL Dataset Construction

For the Reinforcement Learning stage, we constructed a dataset designed to test generalization and robustness\. We sampled a total of 50k problems fromOpenMathReasoning:

- •Seen Set \(30k\):Problems re\-sampled from the SFT distribution to prevent catastrophic forgetting\.
- •Unseen Set \(20k\):Novel problems disjoint from the SFT data to encourage generalization\.

Similar to the SFT stage, we determined the difficulty reference by runningQwen3\-4B\-Thinking\-2507andQwen3\-30B\-A3B\-Thinking\-2507for 5 iterations each\. Crucially, we treat the largest expertQwen3\-235B\-A22B\-Thinking\-2507as the performance ceiling of our system, assigning it a constant pass rate ofκ​\(Qwen3\-235B\-A22B\-Thinking\-2507,x\)=1\.0\\kappa\(\\text\{Qwen3\-235B\-A22B\-Thinking\-2507\},x\)=1\.0for allxx\. The reference ground truth was assigned to the smallest model achieving a≥τ\\geq\\taupass rate\. This reference is used to compute the reward signal during GRPO training, rewarding the model for selecting the most efficient effective model\.

We observed that the initial distribution was heavily skewed toward the smallest model \(Qwen3\-4B\-Thinking\-2507\), which could bias the policy toward trivial cost\-saving behaviors\. To address this, we performed random downsampling on this class to ensure a balanced distribution\. The final curated dataset consists of approximately 35k samples\.

## Appendix DTraining Details

We detail the hyperparameters and configurations for both the Supervised Fine\-Tuning \(SFT\) and Reinforcement Learning \(RL\) stages below\.

#### SFT Configuration\.

For the SFT stage, we set the learning rate to2×10−52\\times 10^\{\-5\}with a warmup ratio of0\.10\.1\. The models were trained for55epochs\.

#### RL Configuration\.

Our reinforcement learning framework is implemented using the VeRLSheng et al\. \([2024](https://arxiv.org/html/2606.29424#bib.bib20)\), employing the standard GRPO algorithmShao et al\. \([2024](https://arxiv.org/html/2606.29424#bib.bib19)\)\. We set the training batch size to128128and the maximum sequence length to10,00010,000\. For each query, we collected1616rollouts\. Notably, we set the KL divergence coefficient to0\. We train the backbone for 40 steps\.

We also set the failure penaltyRfailR\_\{\\text\{fail\}\}to−3\.0\-3\.0\. Additionally, we calibrate the cost–performance trade\-off in Eq\.[5](https://arxiv.org/html/2606.29424#S3.E5)by setting the hyperparameterα=0\.6\\alpha=0\.6\.

### D\.1Reward Function Details

In Section[3\.3](https://arxiv.org/html/2606.29424#S3.SS3), we introduced a piecewise success rewardRsuccR\_\{\\text\{succ\}\}to balance safety and efficiency\. Here we provide the exact formulation\. Letmbestm\_\{\\text\{best\}\}be the most economical model that meets the capability thresholdτ\\tau\(as defined in Eq\.[1](https://arxiv.org/html/2606.29424#S3.E1)\)\. We distinguish two regimes based on the selected modelmm’s costC​\(m\)C\(m\):

#### Risk Zone \(C​\(m\)<C​\(mbest\)C\(m\)<C\(m\_\{\\text\{best\}\}\)\)\.

In this regime, the selected model is cheaper than the empirically optimal safe choice\. Although it may correctly answer the query \(stochastically\), it is statistically less reliable\. To avoid "reward hacking"—where the model overfits to accidental successes or noise in the verification function—we clamp the success reward to a conservative constant:

Rsucc=0\.6\.R\_\{\\text\{succ\}\}=0\.6\.\(24\)

#### Efficiency Zone \(C​\(m\)≥C​\(mbest\)C\(m\)\\geq C\(m\_\{\\text\{best\}\}\)\)\.

In this regime, the model is qualified but potentially over\-expensive\. To encourage the router to prefermbestm\_\{\\text\{best\}\}over strictly stronger but wasteful models, we introduce a linear cost penalty:

Rsucc=max⁡\(1−C​\(m\)−C​\(mbest\)δmax⋅\(1−γ\),γ\),\\displaystyle R\_\{\\text\{succ\}\}=\\max\\left\(1\-\\frac\{C\(m\)\-C\(m\_\{\\text\{best\}\}\)\}\{\\delta\_\{\\text\{max\}\}\}\\cdot\(1\-\\gamma\),\\ \\gamma\\right\),

\(25\)whereδmax\\delta\_\{\\text\{max\}\}represents the maximum cost spread in the model pool, andγ\\gamma\(set to 0\.2 in our experiments\) ensures a minimum positive signal for all correct answers\.

### D\.2Sparse Outcome Reward for Ablation

To rigorously evaluate the contribution of our continuous, expectation\-based reward shaping, we constructed a baseline using a discrete, sparse outcome reward signal\. Formally, for a given queryxx, letmbestm\_\{\\text\{best\}\}denote the optimal expert defined in Eq\. \([1](https://arxiv.org/html/2606.29424#S3.E1)\) \(i\.e\., the most economical model satisfying the capability threshold\)\. We define the sparse reward functionRsparse​\(m,x\)R\_\{\\text\{sparse\}\}\(m,x\)as follows:

Rsparse​\(m,x\)=\{1\.0,if​m=mbest,0\.5,if​C​\(m\)\>C​\(mbest\),0\.0,if​C​\(m\)<C​\(mbest\)\.R\_\{\\text\{sparse\}\}\(m,x\)=\\begin\{cases\}1\.0,&\\text\{if \}m=m\_\{\\text\{best\}\},\\\\ 0\.5,&\\text\{if \}C\(m\)\>C\(m\_\{\\text\{best\}\}\),\\\\ 0\.0,&\\text\{if \}C\(m\)<C\(m\_\{\\text\{best\}\}\)\.\\end\{cases\}\(26\)
This design imposes a strict step\-function logic:

- •Optimal Match \(1\.01\.0\):The router receives maximum reward only if it selects the exact optimal modelmbestm\_\{\\text\{best\}\}\.
- •Inefficient Success \(0\.50\.5\):If the router selects a model that is more capable but more expensive thanmbestm\_\{\\text\{best\}\}, it receives partial credit \(0\.5\)\. This acknowledges correctness but penalizes computational waste\.
- •Failure/Risk \(0\.00\.0\):If the router selects a model cheaper thanmbestm\_\{\\text\{best\}\}\(which is presumed unqualified\), it receives zero reward, effectively penalizing the risk of incorrectness\.

## Appendix EEvaluation Details

#### Infrastructure and Deployment\.

All experiments were conducted on a high\-performance computational node equipped with8×8\\timesNVIDIA B200 GPUs\. We deployed all external candidate models locally using the vLLM inference engine to ensure efficient high\-throughput serving\.

#### Generation Configuration\.

We set specific maximum generation limits to accommodate the reasoning depth of different models:

- •Candidate Models:The maximum generation length was set to 32,768 tokens for Qwen3\-4B and Qwen3\-30B, and 60,000 tokens for the Qwen3\-235B to prevent truncation of long chain\-of\-thought reasoning\.
- •Router Model:The router’s generation limit was set to 12,000 tokens\.

#### Robustness Protocol\.

To mitigate variance and ensure statistical significance on benchmarks with limited sample sizes, we adopted a multi\-run averaging strategy\. We report the average performance over 16 independent runs for smaller datasets \(HMMT, AIME 2024, and AIME 2025\) and 8 independent runs for AMC and SMT 2025\. We refer to Qwen3\-235B as the strongest\-expert reference and the strongest fixed\-expert baseline\.

#### Answer Verification\.

To evaluate correctness, we utilized math\-verify‡‡‡[https://github\.com/huggingface/Math\-Verify](https://github.com/huggingface/Math-Verify)to parse the model outputs and determine if the generated answers were consistent with the ground truth\.

## Appendix FIntuitive Analysis on the Failure of Multi\-round Baselines

While Section[3\.1](https://arxiv.org/html/2606.29424#S3.SS1)characterizes a sufficient regime for Trust Region Collapse, we here offer a complementary intuitive explanation\.

Existing multi\-round agents, such as Router\-R1Zhang et al\. \([2025](https://arxiv.org/html/2606.29424#bib.bib23)\), are largely validated on Knowledge QA tasks\. In such contexts, the primary function of the external expert is to supply missing factual information\. For a router, assimilating a specific fact \(e\.g\., an entity name\) is structurally simple and aligns well with standard token prediction\. However, the mathematical domain imposes a fundamentally higher cognitive burden: the external expert must provide a rigorous, multi\-stepreasoning chain\. Compared to factual retrieval, these complex logical traces are highly Out\-Of\-Distribution \(OOD\) for a smaller router\. The router often fails to “comprehend” this alien logic, perceiving the long expert trace as noise rather than a helpful guide, which leads it to revert to its own internal generation\.

This intuition is strongly corroborated by empirical observations from the xRouterQian et al\. \([2025](https://arxiv.org/html/2606.29424#bib.bib15)\)study\. Notably, the authors reported that they failed to successfully train the policy using the Qwen3 family, explicitly attributing this failure to the model’s “excessively strong pre\-training priors,” and were consequently forced to revert to Qwen2\.5\. This independent finding serves as a powerful validation of ourStrong Prior Dominancehypothesis: as base models become stronger \(possessing more robust internal priors\), they become paradoxically more resistant to integrating external reasoning, thereby accelerating the Trust Region Collapse\.

## Appendix GComparison with LLM\-as\-a\-Router

ModelHMMT Nov 25AIME25OlympiadBenchAIME\-24AMCMATH500SMT2025Avg\.Acc\. \(↑\\uparrow\)Cost \(↓\\downarrow\)Acc\. \(↑\\uparrow\)Cost \(↓\\downarrow\)Acc\. \(↑\\uparrow\)Cost \(↓\\downarrow\)Acc\. \(↑\\uparrow\)Cost \(↓\\downarrow\)Acc\. \(↑\\uparrow\)Cost \(↓\\downarrow\)Acc\. \(↑\\uparrow\)Cost \(↓\\downarrow\)Acc\. \(↑\\uparrow\)Cost \(↓\\downarrow\)Acc\. \(↑\\uparrow\)Cost \(↓\\downarrow\)GPT\-OSS\-120B74\.40\.297474\.60\.299680\.94\.51088\.30\.187795\.60\.228598\.20\.475778\.30\.508287\.70\.0047Qwen3\-235B\-A22B\-Instruct70\.00\.238072\.70\.262480\.03\.731287\.10\.176095\.80\.202997\.60\.662575\.70\.456586\.80\.0041EntroRouter80\.00\.416378\.80\.292981\.85\.239790\.60\.288196\.60\.365898\.40\.523880\.40\.515588\.60\.0055

Table 3:Comparison betweenEntroRouter\(ours\) and LLM\-as\-a\-Router across multiple math benchmarks\.In this section, we provide a detailed comparison between our method \(Ours\) and theLLM\-as\-a\-Routerbaseline\. TheLLM\-as\-a\-Routerapproach involves using LLMs directly as routers for selecting the best model for each query\. We specifically compare our method with two configurations of LLM\-as\-a\-Router:GPT\-OSS\-120BandQwen3\-235B\-A22B\-Instruct, both of which are high\-capability but high\-latency alternatives\.

The comparison results across these benchmarks are summarized in Table[3](https://arxiv.org/html/2606.29424#A7.T3)\. Our method consistently outperforms theLLM\-as\-a\-Routerbaseline, achieving higher accuracy with comparable cost\. WhileLLM\-as\-a\-Routerdirectly uses large LLMs to decide on the routing model for each query, our method leverages a specialized routing policy to achieve stronger accuracy under a comparable cost scale, avoiding direct reliance on large LLMs as routers\. This demonstrates the effectiveness of our optimized routing approach compared to using LLMs as routers\.

ModelGPQA\-DiamondMMLU\-ProOOD AverageAcc\. \(↑\\uparrow\)Cost↓\\downarrowAcc\. \(↑\\uparrow\)Cost↓\\downarrowAcc\. \(↑\\uparrow\)Cost↓\\downarrowGPT\-OSS\-120B71\.70\.685677\.61\.002276\.80\.0012Qwen3\-235B\-A22B\-Instruct72\.70\.314179\.40\.845978\.50\.0008EntroRouter73\.20\.901880\.62\.300279\.50\.0023

Table 4:Comparison betweenEntroRouter\(ours\) and LLM\-as\-a\-Router across OOD benchmarks\.Additionally, we evaluate both methods on OOD benchmarks, as shown in Table[4](https://arxiv.org/html/2606.29424#A7.T4)\. The results show thatEntroRoutermaintains high accuracy, confirming the practical advantage of our method in diverse real\-world scenarios\.

## Appendix HGeneralization to Out\-of\-Distribution Tasks

To evaluate the cross\-domain robustness of our approach, we report performance on two prominent Out\-of\-Distribution \(OOD\) benchmarks: GPQA\-DiamondRein et al\. \([2024](https://arxiv.org/html/2606.29424#bib.bib16)\)and MMLU\-ProWang et al\. \([2024](https://arxiv.org/html/2606.29424#bib.bib21)\)\. Due to computational constraints, we conduct our evaluation on a randomly sampled subset of 1,200 instances from each dataset\. As illustrated in Table[H](https://arxiv.org/html/2606.29424#A8),EntroRouterconsistently demonstrates superior generalization capabilities across OOD domains\. Our method maintains high competitive efficacy without domain\-specific fine\-tuning\. Specifically,EntroRouterachieves an accuracy reduction of only3\.8%3\.8\\%, while simultaneously yielding a significant cost saving of40\.5%40\.5\\%relative to the strongest fixed\-expert baseline\.

ModelGPQAMMLU\-ProAvg\.Acc\.CostAcc\.CostAcc\.Cost\\rowcolor\[gray\]0\.95HeuristicRandom71\.20\.69679\.61\.92778\.40\.002Qwen3\-4B62\.60\.05275\.00\.16873\.30\.000Qwen3\-30B69\.70\.70780\.31\.87478\.80\.000Qwen3\-235B78\.81\.45284\.03\.74983\.30\.004\\rowcolor\[gray\]0\.95Existing Routing ArchitecturesRouteLLM65\.20\.27177\.60\.94775\.80\.001FORC71\.70\.74178\.51\.64477\.50\.002Router\-R149\.50\.00149\.60\.00249\.60\.000xRouter42\.40\.00552\.10\.00350\.70\.000EntroRouter73\.20\.90280\.62\.30079\.50\.002

Table 5:Comparison across Out\-of\-Distribution \(OOD\) benchmarks\. The best performing model is highlighted in bold\.## Appendix IImpact of Cost Sensitivityα\\alpha

MethodHMMT\-NovAIME25SMT2025AverageAccCostAccCostAccCostAccCostEntroRouter\(Ablation onα\\alpha\)α=0\.2\\alpha=0\.279\.00\.47978\.50\.45482\.30\.71080\.40\.015α=0\.6\\alpha=0\.680\.00\.41678\.80\.29380\.40\.51679\.90\.011α=1\.0\\alpha=1\.072\.30\.27675\.00\.25577\.10\.44875\.30\.009

Table 6:Ablation Study on Cost\-Performance Trade\-off \(α\\alpha\)\.We evaluateEntroRouterwith varying sensitivity to computational cost\. Increasingα\\alphaeffectively reduces inference costs but may compromise accuracy if the penalty becomes too aggressive\.We further examine the impact of the hyperparameterα\\alpha\(Eq\.[5](https://arxiv.org/html/2606.29424#S3.E5)\), which governs the sensitivity of the Soft Anchor to computational cost\. As shown in Table[6](https://arxiv.org/html/2606.29424#A9.T6),α\\alphaserves as an effective control knob for navigating the accuracy\-efficiency Pareto frontier\. Specifically, while a lower setting \(α=0\.2\\alpha=0\.2\) yields the highest accuracy80\.4%80\.4\\%, increasingα\\alphato 0\.6 achieves a significant reduction in computational cost with only a slight performance drop \(from80\.4%→79\.9%80\.4\\%\\to 79\.9\\%\)\. However, an overly aggressive penalty \(α=1\.0\\alpha=1\.0\) causes the cost constraint to outweigh correctness, leading to a sharp performance decline \(75\.3%75\.3\\%\)\.

## Appendix JGeneralization to Heterogeneous Model Families

To examine whether ENTROROUTER is tied to a single model family, we further augment the original Qwen3 candidate pool with two additional experts from different model families:R1\-Distill\-Llama\-8BandGPT\-OSS\-20B\. Unlike the main experiments, where the candidate models follow a relatively clean within\-family scaling hierarchy, this setting introduces heterogeneous models with different pre\-training recipes, architectures, and reasoning behaviors\. It therefore provides a more realistic test of whether the router can generalize beyond family\-specific scaling patterns\.

Due to the additional cost of profiling the newly added experts and constructing model\-specific routing labels, we conduct this study on a subset of the training data\. We otherwise follow the same training and evaluation protocol as in the main experiments\. The results are reported in Table[7](https://arxiv.org/html/2606.29424#A10.T7)\.

MethodAIME25HMMT\-NovSMT2025AccCostAccCostAccCostRandom65\.00\.19361\.30\.19466\.30\.320R1\-Distill\-Llama\-8B31\.90\.02127\.30\.02339\.90\.032GPT\-OSS\-20B42\.10\.03333\.70\.03051\.70\.055EntroRouter73\.70\.17170\.40\.21375\.70\.345

Table 7:Generalization to an augmented heterogeneous candidate pool\. We report accuracy and total inference cost on three mathematical reasoning benchmarks\.EntroRouterconsistently improves accuracy over both fixed\-model selection and random routing\. In particular, it outperforms random routing by 8\.7, 9\.1, and 9\.4 percentage points on AIME25, HMMT\-Nov, and SMT25, respectively, while maintaining a comparable cost level\. These results indicate that the learned routing policy is not restricted to a single\-family scaling hierarchy, but can also adapt to candidate pools containing models with distinct training recipes and reasoning behaviors\.

PerturbationAIME25HMMT\-NovSMT2025AccCostAccCostAccCost0%78\.80\.29380\.00\.41680\.40\.51610%79\.60\.35780\.20\.41180\.00\.591

Table 8:Robustness to noisy offline estimates\. We inject noise into 10% of the training data by perturbing both Stage\-I target labels and Stage\-II pass\-rate estimates\.Importantly, the heterogeneous setting weakens the simple monotonic relationship between model cost and capability assumed in a clean within\-family pool\. Models from different families may exhibit different specialization patterns, and their relative performance can vary substantially across benchmarks\. Therefore, routing can no longer be reduced to selecting models according to scale or price alone\. The gains in Table[7](https://arxiv.org/html/2606.29424#A10.T7)suggest that ENTROROUTER can leverage empirical capability estimates and query\-dependent routing signals to identify effective experts even when the candidate pool does not follow a strictly monotonic cost–capability ordering\.

## Appendix KRobustness to Noisy Offline Estimates

ENTROROUTER relies on offline capability estimatesκ​\(m,x\)\\kappa\(m,x\)to construct the SFT targets and the RL reward\. Although these estimates are obtained through Monte Carlo rollouts, this profiling is a one\-time offline cost and does not affect deployment: at inference time, the router performs a single forward pass and requires no online rollouts\. This preserves the scalability of the single\-round routing framework\.

To examine the robustness of ENTROROUTER to imperfect offline estimates, we conduct a joint\-noise stress test on the training data\. Specifically, we randomly perturb 10% of the training instances in both stages: \(1\) for Stage I, we shift the target model label to an adjacent capability tier when available; and \(2\) for Stage II, we perturb the offline pass\-rate estimatesκ​\(m,x\)\\kappa\(m,x\)by±0\.2\\pm 0\.2\. This setting is intentionally stronger than an isolated Stage\-II perturbation, since it simultaneously corrupts the supervised initialization targets and the RL\-stage capability estimates\.

As shown in Table[8](https://arxiv.org/html/2606.29424#A10.T8), ENTROROUTER maintains stable accuracy under this joint\-noise perturbation\. The accuracy changes are small across all three benchmarks, indicating that the learned router is not overly sensitive to moderate corruption in the offline supervision signals\. We also observe a mild increase in cost on AIME25 and SMT25, suggesting that noisy estimates can make the router slightly more conservative by assigning more queries to stronger experts\. Overall, these results show that ENTROROUTER remains robust under imperfect offline profiling while preserving its core accuracy–cost trade\-off\.