The Shadow Price of Reasoning: Economic Perspective on Optimal Budget Allocation for LLMs

# Economic Perspective on Optimal Budget Allocation for LLMs
Source: [https://arxiv.org/html/2606.03092](https://arxiv.org/html/2606.03092)
## The Shadow Price of Reasoning: Economic Perspective on Optimal Budget Allocation for LLMs

Speed ZhuJianwei CaiGuang ChenXiming HuangWiggin ZhouMingyang Sun

###### Abstract

Inference\-time scaling has emerged as a critical avenue for enhancing Large Language Models’ performance, yet real\-world deployment is constrained by strict computational budgets\. In this work, we formulate inference budget allocation as a global constrained optimization problem governed by economic principles\. By modeling per\-query reasoning utility with a shifted\-surge function, we derive an optimal allocation policy based on a global shadow price that equilibrates marginal utility under resource scarcity\. Based on this theory, we proposeConstrainedLatent\-utilityEquilibriumAllocation forReasoning \(CLEAR\)\. It performs rational abandonment and reallocates resources from insolvent queries to solvable queries near their emergence thresholds\. Extensive experiments on several reasoning tasks with different traffic streams demonstrate thatCLEARsignificantly improves the Pareto frontier of total token cost versus mean accuracy\. In resource\-scarce regimes,CLEARachieves up to a3×\\timesimprovement in global accuracy compared to uniform allocation\.

The code is available in[Here](https://github.com/waunx/CLEAR)\.

Machine Learning, ICML

## 1Introduction

![Refer to caption](https://arxiv.org/html/2606.03092v1/Figures/s_curve_4.png)Figure 1:The S\-Shaped Compute\-Utility Curve\.We evaluateQwen2\.5\-Math\-7B\(Yanget al\.,[2024](https://arxiv.org/html/2606.03092#bib.bib27)\)on three benchmarks\. The budget\-performance relationship exhibits three distinct regions: \(1\) a pre\-thresholdStrictphase with negligible utility; \(2\) a rapidSurgephase offering high leverage; and \(3\) anAmplephase characterized by diminishing returns\.![Refer to caption](https://arxiv.org/html/2606.03092v1/Figures/modelling.png)Figure 2:Empirical Rollouts and Latent Utility\.For selectedAIME\-24problems, blue bars show the number of correct rollouts in each length bin, while red curves depict the fitted latent utility mapping induced by our shifted\-surge model\.The development of Large Language Models \(LLMs\) is undergoing a paradigm shift from training\-time scaling to inference\-time scaling\(Weiet al\.,[2022](https://arxiv.org/html/2606.03092#bib.bib1); Snellet al\.,[2024](https://arxiv.org/html/2606.03092#bib.bib2); Wuet al\.,[2024](https://arxiv.org/html/2606.03092#bib.bib10)\)\. Recent breakthroughs demonstrate that increasing test\-time compute can yield performance gains comparable to scaling model parameters by invoking “System 2” reasoning capabilities\(Weston and Sukhbaatar,[2023](https://arxiv.org/html/2606.03092#bib.bib11); Brownet al\.,[2024](https://arxiv.org/html/2606.03092#bib.bib20)\)\. A series of length\-scaling experiments has furthermore validated that by allowing models to think longer, it leads to substantial improvements in complex reasoning tasks\(Muennighoffet al\.,[2025](https://arxiv.org/html/2606.03092#bib.bib3); Aggarwal and Welleck,[2025](https://arxiv.org/html/2606.03092#bib.bib4)\)\. However, in real\-world deployment scenarios, test\-time compute is a finite and expensive commodity\. Whether serving millions of concurrent users via cloud APIs or running models on resource\-constrained edge devices, practitioners operate under strict global budget constraints\(Chenet al\.,[2023](https://arxiv.org/html/2606.03092#bib.bib15)\)\. The central challenge thus shifts from merely increasing the theoretical ceiling of model intelligence to maximizing global utility under a fixed computational cap\(Raposoet al\.,[2024](https://arxiv.org/html/2606.03092#bib.bib17)\)\.

Standard deployment practices usually impose a uniform policy, assigning the same generation limit \(e\.g\., a fixedmax\_new\_tokens\) to every query\. This implicitly assumes that all queries share a comparable compute\-utility profile, which conflicts with the large heterogeneity of reasoning tasks\. As illustrated in Figure[1](https://arxiv.org/html/2606.03092#S1.F1), reasoning utility follows an S\-shaped curve when we fix the decoding strategy and vary the token limit\. For difficult queries likeAIME, a uniform limit may leave the trajectory in theStrictphase, where compute is spent but utility remains near zero\. For easy queries likeGSM8K, the same limit may push generation into theAmplephase, where additional tokens bring only diminishing returns\. Of course, in an unconstrained\-resource setting, the inefficiency caused by pushing easy queries into theAmplephase may be less consequential\. However, this work focuses on the resource\-constrained setting, where a fixed total token budget is imposed over the entire evaluation set\. The central question is therefore how to allocate tokens across individual queries so as to maximize overall token utility under this global budget\.

We therefore treat inference budgeting as a batch\-level allocation problem: given a fixed total token supply, decide how many tokens each query should receive to maximize aggregate expected utility\. This gives rise to a global constrained optimization problem over instance\-specific utility curves\. Although the resulting objective is non\-convex, its Lagrangian form reveals a simple economic principle: at optimum, every active query should spend tokens until its marginal utility matches a commonglobal shadow price\(Boyd and Vandenberghe,[2004](https://arxiv.org/html/2606.03092#bib.bib19); Devanuret al\.,[2019](https://arxiv.org/html/2606.03092#bib.bib21)\)\. Queries whose attainable marginal gain never clears this price should receive no budget, while the remaining queries share the budget according to their utility slopes\.

Building on these findings, we proposeConstrainedLatent\-utilityEquilibriumAllocation forReasoning \(CLEAR\)\. Instead of a uniform utility for each query,CLEARmaps it into a surge\-shaped utility curve\. The system then operates as a computational market: \(1\)Threshold Modeling:We estimate the emergence threshold of each query and instantiate its latent utility curve; \(2\)Price Discovery:We employ a fast bisection search to find the unique global shadow price that clears the market, ensuring the total demand matches the available budget; \(3\)Optimal Allocation:Based on the discovered price, we apply a closed\-form policy derived from the Lambert W function to strictly determine the token limit for each query, automatically handling truncation or rational abandonment\. Crucially,CLEARrequires no retraining of the backbone LLM and operates as a plug\-and\-play inference wrapper\.

Our contributions are summarized as follows:

\(1\) We identify a three\-phase compute\-utility pattern in LLM reasoning and formalize inference\-time token allocation over non\-concave latent utility curves, where the optimal policy is governed by a global shadow price\.

\(2\) We derive a closed\-form Lambert W allocation policy and instantiate it inCLEAR, a plug\-and\-play framework that combines latent\-threshold prediction with market\-clearing price discovery\.

\(3\) We validateCLEARon mixed\-complexity mathematical reasoning benchmarks, demonstrating improved cost\-accuracy Pareto efficiency and robustness to hyperparameter choices and predictor noise\.

## 2Empirical Motivation

To better model the per\-query utility of reasoning tokens, we investigate whether longer reasoning is uniformly beneficial for each query\. If every problem has an instance\-specific favorable reasoning length, then inference utility should naturally rise after a minimum threshold but eventually saturate or decline when generation becomes excessive\.

Following the thoughtology analysis\(Marjanovićet al\.,[2025](https://arxiv.org/html/2606.03092#bib.bib23)\)inDeepSeek\-R1, we conduct a controlled experiment usingQwen2\.5\-Math\-7B\. We perform sampling with a high temperatureT=1\.0T=1\.0to induce diverse reasoning paths of varying lengths\. We generateN=50N=50responses forAIME\-24andN=4N=4for theGSM8K,MATH\-500benchmarks, and group the resulting trajectories into length bins to compute the conditionalPass@1accuracy\.

As visualized in Figure[2](https://arxiv.org/html/2606.03092#S1.F2), reasoning utility exhibits non\-linear dynamics rather than simple scaling\. We identify three recurring regimes: aStrictphase where short trajectories fail consistently, implying a minimum solvable threshold; aSurgephase where performance rises sharply past this threshold; and anAmplephase where additional generation yields diminishing returns and may eventually degrade solution quality\.

## 3Problem Formulation

Based on the empirical observations above, we first model the per\-query reasoning utility with a function that captures theStrict–Surge–Amplestructure\. Under this utility model, the problem of assigning tokens subject to a fixed total budget naturally becomes a global constrained optimization problem with an instance\-dependent, non\-concave utility landscape\.

### 3\.1Modeling the Physics of Reasoning

While the observed outcome of a reasoning task is binary \(correct or incorrect\), we posit that this outcome is governed by a continuous, unobservable variable: the reasoning utilityϕ​\(t\)\\phi\(t\)\. For a querysis\_\{i\}, we model the accumulated potential after generatingtttokens follows a Shifted Surge Function:

ϕi​\(t\)=\{00≤t<τiαi​\(t−τi\)⋅e−βi​\(t−τi\)t≥τi\\phi\_\{i\}\(t\)=\\begin\{cases\}0&0\\leq t<\\tau\_\{i\}\\\\ \\alpha\_\{i\}\(t\-\\tau\_\{i\}\)\\cdot e^\{\-\\beta\_\{i\}\(t\-\\tau\_\{i\}\)\}&t\\geq\\tau\_\{i\}\\end\{cases\}\(1\)This parameterization aligns the latent utility model with the three empirical regimes observed above:

Strict:Utility remains zero until the generation length crosses the latent emergence thresholdτi\\tau\_\{i\}\.

Surge:Once the threshold is crossed, reasoning potential rises with initial velocityαi\\alpha\_\{i\}, capturing the rapid accumulation of valid reasoning\.

Ample:When generation extends excessively, the exponential decay terme−βi​Δ​te^\{\-\\beta\_\{i\}\\Delta t\}dominates, reflecting diminishing returns\.

In Figure[2](https://arxiv.org/html/2606.03092#S1.F2), the red curve visualizes the fitted latent reasoning potential, which provides a continuous utility mapping from the empirical rollout outcomes observed at different generation lengths\.

### 3\.2The Global Optimization Objective

We adopt a pure token\-based cost model where the costCi​\(t\)=tC\_\{i\}\(t\)=t\. The system’s goal is to allocate a vector of tokens𝐭=\[t1,…,tN\]\\mathbf\{t\}=\[t\_\{1\},\\dots,t\_\{N\}\]forNNqueries to maximize aggregate reasoning potential under a global token budget limitBtotalB\_\{\\text\{total\}\}:

max𝐭∈ℝ≥0N\\displaystyle\\max\_\{\\mathbf\{t\}\\in\\mathbb\{R\}\_\{\\geq 0\}^\{N\}\}∑i=1Nϕi​\(ti\)\\displaystyle\\sum\_\{i=1\}^\{N\}\\phi\_\{i\}\(t\_\{i\}\)\(2\)s\.t\.∑i=1Nti≤Btotal\.\\displaystyle\\sum\_\{i=1\}^\{N\}t\_\{i\}\\leq B\_\{\\text\{total\}\}\.

## 4Theoretical Analysis

To solve the non\-convex optimization problem in Eq\. \([2](https://arxiv.org/html/2606.03092#S3.E2)\), we apply the method of Lagrange multipliers\. By relaxing the global budget constraint, we establish an economic principle governing optimal inference: the equalization of marginal reasoning potential\.

### 4\.1Shadow Price Parity

We construct the Lagrangianℒ​\(𝐭,λ\)=∑iϕi​\(ti\)−λ​\(∑iti−Btotal\)\\mathcal\{L\}\(\\mathbf\{t\},\\lambda\)=\\sum\_\{i\}\\phi\_\{i\}\(t\_\{i\}\)\-\\lambda\(\\sum\_\{i\}t\_\{i\}\-B\_\{\\text\{total\}\}\), whereλ≥0\\lambda\\geq 0is the Lagrange multiplier\. In economic terms,λ\\lambdarepresents theglobal shadow priceof computation—the marginal gain in total potential obtained by relaxing the budget by one token\.

The Karush\-Kuhn\-Tucker \(KKT\) conditions imply that for any globally optimal allocation𝐭∗\\mathbf\{t\}^\{\*\}, the marginal gain of each active task must align with this global price\. This parity determines whether a query should be lifted out ofStrict, placed near its high\-leverageSurge, or capped before wasting budget inAmple\. We formalize this as:

###### Definition 4\.1\(Shadow Price Parity\)\.

A strictly positive budget allocationti∗\>0t\_\{i\}^\{\*\}\>0is a local maximum if it satisfies the first\-order stationarity condition, equalizing the marginal reasoning potential to the global shadow price:

∂ϕi​\(ti∗\)∂ti=λ\.\\frac\{\\partial\\phi\_\{i\}\(t\_\{i\}^\{\*\}\)\}\{\\partial t\_\{i\}\}=\\lambda\.\(3\)The global optimal policy is then determined by comparing the utility surplus at this stationary point against the abandonment option \(ti=0t\_\{i\}=0\)\. We term this theRational Abandonment Condition, where tasks whose maximum net surplus falls below zero are deemed economically insolvent and assigned zero budget\.

### 4\.2Individual Optimal Allocation Policy

For the surge potentialϕi​\(t\)=αi​Δ​t​e−βi​Δ​t\\phi\_\{i\}\(t\)=\\alpha\_\{i\}\\Delta te^\{\-\\beta\_\{i\}\\Delta t\}, whereΔ​t=t−τi\\Delta t=t\-\\tau\_\{i\}, the shadow price parity condition yields the differential equation:

d​ϕid​t=αi​e−βi​Δ​t​\(1−βi​Δ​t\)=λ\.\\frac\{d\\phi\_\{i\}\}\{dt\}=\\alpha\_\{i\}e^\{\-\\beta\_\{i\}\\Delta t\}\(1\-\\beta\_\{i\}\\Delta t\)=\\lambda\.\(4\)While standard algebraic methods cannot solve forttin this transcendental equation, we prove that the exact solution is given by the Lambert W functionW​\(z\)W\(z\), defined as the inverse off​\(w\)=w​ewf\(w\)=we^\{w\}\.

###### Theorem 4\.2\(Individual Optimal Allocation Policy\)\.

Under the shadow price parity condition, the optimal token allocationti∗t\_\{i\}^\{\*\}for a givenλ\\lambdafollows a closed\-form solution:

ti∗​\(λ\)=τi\+1βi​\[1−W0​\(λ​eαi\)\],t\_\{i\}^\{\*\}\(\\lambda\)=\\tau\_\{i\}\+\\frac\{1\}\{\\beta\_\{i\}\}\\left\[1\-W\_\{0\}\\left\(\\frac\{\\lambda e\}\{\\alpha\_\{i\}\}\\right\)\\right\],\(5\)subject to the solvency constraintϕi​\(ti∗\)\>λ​ti∗\\phi\_\{i\}\(t\_\{i\}^\{\*\}\)\>\\lambda t\_\{i\}^\{\*\}\. Here,W0​\(⋅\)W\_\{0\}\(\\cdot\)denotes the principal branch of the Lambert W function, andeeis Euler’s number\.

###### Proof\.

The objective for each taskiiis to maximize the net economic surplusJi​\(t\)J\_\{i\}\(t\), defined as the difference between the latent reasoning potential and the opportunity cost of tokens:

Ji​\(t\)=ϕi​\(t\)−λ​t=αi​\(t−τi\)​e−βi​\(t−τi\)−λ​t\.J\_\{i\}\(t\)=\\phi\_\{i\}\(t\)\-\\lambda t=\\alpha\_\{i\}\(t\-\\tau\_\{i\}\)e^\{\-\\beta\_\{i\}\(t\-\\tau\_\{i\}\)\}\-\\lambda t\.\(6\)We assume the task is in the active reasoning phase \(t≥τit\\geq\\tau\_\{i\}\)\. LetΔ​t=t−τi\\Delta t=t\-\\tau\_\{i\}denote the effective generation length\. The first\-order necessary condition for optimality requires the marginal potential to equal the shadow price:

d​ϕid​t=λ\.\\frac\{d\\phi\_\{i\}\}\{dt\}=\\lambda\.\(7\)Applying the product rule to differentiate the surge function with respect tott:

dd​t​\[αi​Δ​t​e−βi​Δ​t\]\\displaystyle\\frac\{d\}\{dt\}\\left\[\\alpha\_\{i\}\\Delta te^\{\-\\beta\_\{i\}\\Delta t\}\\right\]=αi​\(1⋅e−βi​Δ​t\+Δ​t⋅\(−βi\)​e−βi​Δ​t\)\\displaystyle=\\alpha\_\{i\}\\left\(1\\cdot e^\{\-\\beta\_\{i\}\\Delta t\}\+\\Delta t\\cdot\(\-\\beta\_\{i\}\)e^\{\-\\beta\_\{i\}\\Delta t\}\\right\)\(8\)=αi​e−βi​Δ​t​\(1−βi​Δ​t\)\.\\displaystyle=\\alpha\_\{i\}e^\{\-\\beta\_\{i\}\\Delta t\}\(1\-\\beta\_\{i\}\\Delta t\)\.Equating this toλ\\lambda, we obtain the transcendental equation:

αi​e−βi​Δ​t​\(1−βi​Δ​t\)=λ\.\\alpha\_\{i\}e^\{\-\\beta\_\{i\}\\Delta t\}\(1\-\\beta\_\{i\}\\Delta t\)=\\lambda\.\(9\)To solve forΔ​t\\Delta t, we transform this equation into the canonical form of the Lambert W function,w​ew=zwe^\{w\}=z\. We introduce a change of variable:

u=1−βi​Δ​t\.u=1\-\\beta\_\{i\}\\Delta t\.\(10\)This impliesβi​Δ​t=1−u\\beta\_\{i\}\\Delta t=1\-u, and consequently, the exponential term becomese−βi​Δ​t=e−\(1−u\)=eu−1e^\{\-\\beta\_\{i\}\\Delta t\}=e^\{\-\(1\-u\)\}=e^\{u\-1\}\. Substituting these into the optimality condition:

αi⋅eu−1⋅u\\displaystyle\\alpha\_\{i\}\\cdot e^\{u\-1\}\\cdot u=λ\\displaystyle=\\lambda\(11\)u​eu​e−1\\displaystyle ue^\{u\}e^\{\-1\}=λαi\\displaystyle=\\frac\{\\lambda\}\{\\alpha\_\{i\}\}u​eu\\displaystyle ue^\{u\}=λ​eαi\.\\displaystyle=\\frac\{\\lambda e\}\{\\alpha\_\{i\}\}\.By the definition of the Lambert W function, the solution foruuis given by the principal branchW0W\_\{0\}, as we require the solution corresponding to the physical expansion phase:

u=W0​\(λ​eαi\)\.u=W\_\{0\}\\left\(\\frac\{\\lambda e\}\{\\alpha\_\{i\}\}\\right\)\.\(12\)Finally, we substituteuuback to solve for the optimal allocationti∗t\_\{i\}^\{\*\}:

1−βi​\(ti∗−τi\)\\displaystyle 1\-\\beta\_\{i\}\(t\_\{i\}^\{\*\}\-\\tau\_\{i\}\)=W0​\(λ​eαi\)\\displaystyle=W\_\{0\}\\left\(\\frac\{\\lambda e\}\{\\alpha\_\{i\}\}\\right\)\(13\)βi​\(ti∗−τi\)\\displaystyle\\beta\_\{i\}\(t\_\{i\}^\{\*\}\-\\tau\_\{i\}\)=1−W0​\(λ​eαi\)\\displaystyle=1\-W\_\{0\}\\left\(\\frac\{\\lambda e\}\{\\alpha\_\{i\}\}\\right\)ti∗\\displaystyle t\_\{i\}^\{\*\}=τi\+1βi​\[1−W0​\(λ​eαi\)\]\.\\displaystyle=\\tau\_\{i\}\+\\frac\{1\}\{\\beta\_\{i\}\}\\left\[1\-W\_\{0\}\\left\(\\frac\{\\lambda e\}\{\\alpha\_\{i\}\}\\right\)\\right\]\.This interior solution is valid if and only if the resulting net surplus is positive \(Ji​\(ti∗\)\>0J\_\{i\}\(t\_\{i\}^\{\*\}\)\>0\)\. If the surplus is non\-positive, the global optimum defaults to the boundary solutionti∗=0t\_\{i\}^\{\*\}=0\. ∎

### 4\.3Global Equilibrium and Phase Transitions

The Lambert W policy derived in Eq\. \([5](https://arxiv.org/html/2606.03092#S4.E5)\) describes the local optimum for a fixed priceλ\\lambda\. However, the inference system operates under a hard global constraint\. Theglobal optimal policyis realized by finding the unique market\-clearing priceλ∗\\lambda^\{\*\}that satisfies the resource budget:

∑i=1Nti∗​\(λ∗\)=Btotal\.\\sum\_\{i=1\}^\{N\}t\_\{i\}^\{\*\}\(\\lambda^\{\*\}\)=B\_\{\\text\{total\}\}\.\(14\)Sinceti∗​\(λ\)t\_\{i\}^\{\*\}\(\\lambda\)is strictly monotonically decreasing with respect toλ\\lambda, a uniqueλ∗\\lambda^\{\*\}is guaranteed to exist\. The magnitude of the global budgetBtotalB\_\{\\text\{total\}\}inversely determinesλ∗\\lambda^\{\*\}\. This induces three system\-level allocation regimes, which correspond to different operating points on the per\-queryStrict–Surge–Ampleutility curves:

1\. Abundant\-Budget Regime\.WhenBtotalB\_\{\\text\{total\}\}is large, the shadow priceλ∗\\lambda^\{\*\}approaches zero\. In this limit, the Lambert W term vanishes:W0​\(0\)=0W\_\{0\}\(0\)=0, and the allocation converges toti→τi\+1/βit\_\{i\}\\to\\tau\_\{i\}\+1/\\beta\_\{i\}\. Active tasks are therefore allowed to approach the peak of their utility curves, near the transition fromSurgetoAmple\.

2\. Scarce\-Budget Regime\.AsBtotalB\_\{\\text\{total\}\}tightens,λ∗\\lambda^\{\*\}rises\. The termW0​\(λ​eαi\)W\_\{0\}\(\\frac\{\\lambda e\}\{\\alpha\_\{i\}\}\)increases, compressing the allocationti∗t\_\{i\}^\{\*\}toward the thresholdτi\\tau\_\{i\}\. The system thus prioritizes queries whose allocated tokens can still land in the high\-marginal\-returnSurgesegment\.

3\. Abandonment Regime\.WhenBtotalB\_\{\\text\{total\}\}is critically low,λ∗\\lambda^\{\*\}exceeds the initial velocityαi\\alpha\_\{i\}for difficult tasks \(λ∗\>αi\\lambda^\{\*\}\>\\alpha\_\{i\}\)\. No positive allocation can generate enough marginal surplus to justify crossing the threshold, so such tasks are abandoned and remain before theirStrictthreshold with indicator function𝕀​\(⋅\)=0\\mathbb\{I\}\(\\cdot\)=0\.

## 5Methodology

In this section, we translate the theoretical optimal policy derived in Theorem[4\.2](https://arxiv.org/html/2606.03092#S4.Thmtheorem2)intoCLEAR, a practical inference\-time control algorithm\. Unlike heuristic methods that rely on manual rules for truncation or abandonment,CLEARnumerically solves for the shadow priceλ∗\\lambda^\{\*\}that equilibrates the aggregate demand of the batch with the global supply of tokens\.

### 5\.1Threshold Prediction and Parametric Scaling

The core input to our Lambert W policy is the emergence thresholdτi\\tau\_\{i\}, which captures the minimum computation required for querysis\_\{i\}to enter the productive reasoning regime\. We employ a lightweight predictorfθ:𝒮→ℝ\+f\_\{\\theta\}:\\mathcal\{S\}\\to\\mathbb\{R\}\_\{\+\}based onDeBERTa\-v3\-baseto estimate this value:

τ^i=exp⁡\(fθ​\(si\)\)\.\\hat\{\\tau\}\_\{i\}=\\exp\(f\_\{\\theta\}\(s\_\{i\}\)\)\.\(15\)To map these predictions into actionable utility curves,CLEARrelies on two critical parameters that govern the system’s behavior:

Initial Velocityα\\alpha:This parameter defines the initial magnitude of the utility function’s derivative at the predicted thresholdτ^\\hat\{\\tau\}\. It functions as the system’sreservation price: any query where the global market\-clearing priceλ∗\\lambda^\{\*\}exceedsα\\alphayields a negative net surplus and is assigned zero tokens\.

Decay Rateβ\\beta:This parameter governs the exponential rate at which marginal utility diminishes beyond the predicted lengthτ^\\hat\{\\tau\}\. It is inversely proportional to the characteristic length of the token allocation window\. To accommodate varying traffic conditions, we implement an adaptive mechanism whereβ\\betais derived dynamically from the aggregate budget surplus:

β=1max⁡\(ϵ,B¯−τ¯\)\.\\beta=\\frac\{1\}\{\\max\(\\epsilon,\\bar\{B\}\-\\bar\{\\tau\}\)\}\.\(16\)Here,B¯\\bar\{B\}is the designated budget constraint andτ¯\\bar\{\\tau\}is the mean predicted budget\.

Remarks\.While Theorem[4\.2](https://arxiv.org/html/2606.03092#S4.Thmtheorem2)provides a general solution for task\-specific shaping parameters\{αi,βi\}\\\{\\alpha\_\{i\},\\beta\_\{i\}\\\}, predictingβi\\beta\_\{i\}of an unseen query is computationally intractable and prone to high variance\. Therefore, we adopt a model\-intrinsic assumption: we treatα\\alphaandβ\\betaas global hyperparameters that characterize the average reasoning dynamics of the LLM backbone itself, rather than the specific query\. We assume task heterogeneity is primarily captured by the threshold parameterτi\\tau\_\{i\}, which we predict dynamically\.

### 5\.2Global Shadow Price Optimization

The core ofCLEARlies in identifying the preciseλ∗\\lambda^\{\*\}that saturates the global budget constraint\. Leveraging the monotonicity established in Section[4\.3](https://arxiv.org/html/2606.03092#S4.SS3), where the individual optimal allocationti∗​\(λ\)t\_\{i\}^\{\*\}\(\\lambda\)strictly decreases with respect to the shadow price, we guarantee that the aggregate consumption functionC​\(λ\)=∑ti∗​\(λ\)C\(\\lambda\)=\\sum t\_\{i\}^\{\*\}\(\\lambda\)is invertible\. Therefore, we solveλ∗\\lambda^\{\*\}using the bisection method, proceeding in three distinct steps:

- •Search Space Initialization:We define the search interval\[λmin,λmax\]\[\\lambda\_\{\\min\},\\lambda\_\{\\max\}\]\. The lower boundλmin=0\\lambda\_\{\\min\}=0represents the saturation regime \(infinite budget\)\. The upper boundλmax=α\\lambda\_\{\\max\}=\\alphacorresponds to the total abandonment regime\. Crucially,α\\alphaserves as the theoretical ceiling; any priceλ\>α\\lambda\>\\alphayields negative surplus for all tasks, resulting in zero allocation\.
- •Candidate Evaluation:In each iteration, we propose a candidate priceλmid\\lambda\_\{\\text\{mid\}\}and compute the optimal allocation for every query using the Lambert W Policy \(Eq\.[5](https://arxiv.org/html/2606.03092#S4.E5)\)\. This involves three substeps: first, computing the unconstrained surge length via the principal branchW0W\_\{0\}using the adaptiveβ\\beta; second, enforcing the solvency condition by zeroing out allocations where the costλ⋅t\\lambda\\cdot texceeds the utility \(Rational Abandonment\); and third, applying the physical context limitTmaxT\_\{\\text\{max\}\}\.
- •Market Clearing Update:We aggregate the individual allocations to determine the total consumptionCtotalC\_\{\\text\{total\}\}\. IfCtotalC\_\{\\text\{total\}\}exceeds the global budgetBtotalB\_\{\\text\{total\}\}, indicating excess demand, we raise the price by settingλmin←λmid\\lambda\_\{\\min\}\\leftarrow\\lambda\_\{\\text\{mid\}\}\. Conversely, if there is a supply surplus, we lower the price by settingλmax←λmid\\lambda\_\{\\max\}\\leftarrow\\lambda\_\{\\text\{mid\}\}\.

The full procedure is summarized in Algorithm[1](https://arxiv.org/html/2606.03092#alg1)\. Note that the Lambert W functionW0​\(z\)W\_\{0\}\(z\)is efficiently computed via standard numerical libraries, thus introducing negligible latency compared to the autoregressive generation process\.

## 6Experimental Settings

##### Models and Training Data

We employQwen2\.5\-Math\-7B\-InstructandQwen3\-30B\-A3B\-Instruct\(Yanget al\.,[2025](https://arxiv.org/html/2606.03092#bib.bib22)\)as the frozen backbones for all reasoning tasks\. To estimate the latent emergence thresholdτ​\(s\)\\tau\(s\), we fine\-tune aDeBERTa\-v3\-baseencoder as the threshold predictor\. The predictor is trained to regress the logarithmic token length of the model’s generated solutions, which serves as a proxy for the emergence threshold\. The training data is drawn exclusively from splits ofGSM8KandMATH\.

##### Evaluation Datasets and Streams

As illustrated in Figure[3](https://arxiv.org/html/2606.03092#S6.F3), we evaluate on a mixed oracle pool spanningMATH\-500,AMC\-23,AIME\-24,AIME\-25,Minerva, andOlympiadBench, and construct four synthetic inference streams:Balanced,Mostly\-Easy,Mostly\-Hard, andU\-Shaped\. The detailed training and evaluation setting is shown in Table[6](https://arxiv.org/html/2606.03092#A3.T6)and[7](https://arxiv.org/html/2606.03092#A3.T7)\.

![Refer to caption](https://arxiv.org/html/2606.03092v1/Figures/traffic_stream.png)Figure 3:Oracle\-Length Distribution across Evaluation Traffic Streams\.Each panel shows one synthetic traffic stream \(Balanced,Mostly\-Easy,Mostly\-Hard, andU\-Shaped\), withn=500n\{=\}500queries sampled from the 7B oracle pool\.
##### Baselines and Allocation Policies

We benchmarkCLEAR\(Lambert\)111We useCLEAR\(Lambert\) to denote the fullCLEARpipeline proposed above, distinguishing it from the internalCLEARvariants used for ablation\.against several allocation strategies\.Uniformevenly distributes the global budget across all queries, whilePredictorassigns tokens proportional to the predicted thresholdτ^i\\hat\{\\tau\}\_\{i\}but does not support abandonment\.TALE\-EP\(Hanet al\.,[2025](https://arxiv.org/html/2606.03092#bib.bib7)\)serves as an external length\-prediction baseline that estimates per\-query token demand and renormalizes allocations to the same global budget\. We also compare against two internal ablations:CLEAR\(Heuristic\), which applies an affine allocation with a median\-based rejection cutoff, andCLEAR\(Auction\), which greedily admits queries by predicted return on investment under the budget constraint\. Finally,Oracleprovides an upper bound using ground\-truth solution lengths\. Detailed formulations are provided in Appendix[A\.2](https://arxiv.org/html/2606.03092#A1.SS2)\.

## 7Results and Analysis

### 7\.1Latent Threshold Modeling

Accurately predicting the exact token consumption for reasoning tasks is inherently challenging\. However, for our resource allocation purpose, the global shadow priceλ\\lambdanaturally acts as a normalizer, absorbing absolute prediction errors\. Thus, the allocation primarily relies on the predictor’s ability to correctly rank tasks by latent threshold\. Figure[4](https://arxiv.org/html/2606.03092#S7.F4)demonstrates the log\-scale regression performance of our predictor\.

![Refer to caption](https://arxiv.org/html/2606.03092v1/Figures/ranking_analysis.png)Figure 4:Predictor Performance Analysis\.\(Left\)Log\-log scatter plot of predicted vs\. actual token lengths\. High rank correlation coefficients \(Spearman’sρ\\rho, Kendall’sτ\\tau\) indicate the model effectively captures relative threshold\.\(Right\)Stratified comparison of token count distributions across threshold tiers\.Table 1:Comprehensive Performance Comparison across Evaluation Streams\.We compareCLEAR\(Lambert\) against various baselines includingUniformallocation, ProportionalPredictor, and twoCLEARvariants\. TheOraclerepresents the theoretical upper bound\.Boldindicates the best practical performance among non\-oracle methods\. The bottom row highlights theAbsolute Accuracy GainofCLEAR\(Lambert\) over theUniform\.![Refer to caption](https://arxiv.org/html/2606.03092v1/Figures/phase_transition_lambert_vs_uniform_3.png)Figure 5:Phase Transition Analysis\.Dual\-axis plots showing Accuracy in blue and Abandonment Rate in red as the global budget increases\.CLEARsignificantly outperforms the gray dashed Uniform curve in the low\-budget regime by maintaining a high abandonment rate\. As the budget increases, abandonment drops to zero and the policies converge\.![Refer to caption](https://arxiv.org/html/2606.03092v1/Figures/exp2_allocation_real_MostlyHard_Lambert2.png)Figure 6:Visualization of Allocation Policies\.Each point represents a query from the Balanced stream\. The top row shows the scarcity regime with a budget of 256, while the bottom row shows the abundance regime with a budget of 1024\. Blue dots represent allocated tasks, and red crosses indicate abandoned tasks\.![Refer to caption](https://arxiv.org/html/2606.03092v1/Figures/sensitivity_beta_static_vs_adaptive2.png)Figure 7:Sensitivity Analysis of the Decay Rateβ\\beta\.Thestar markers\(⋆\\star\) indicate the operating points automatically selected by our adaptiveβ\\betamechanism\.![Refer to caption](https://arxiv.org/html/2606.03092v1/Figures/alpha_mechanism_dual_axis2.png)Figure 8:Scale Invariance of the Initial Velocityα\\alpha\.The left axis \(grey\) shows that accuracy remains effectively constant\. The right axis \(blue dashed\) reveals the optimized shadow priceλ∗\\lambda^\{\*\}scales linearly withα\\alpha\(log⁡λ∗∝log⁡α\\log\\lambda^\{\*\}\\propto\\log\\alpha\)\.
### 7\.2Main Performance Comparison

##### Comparison on Mathematical Reasoning\.

Figure[6](https://arxiv.org/html/2606.03092#S7.F6)and Table[1](https://arxiv.org/html/2606.03092#S7.T1)summarizes the accuracy ofCLEARcompared to the Uniform, Predictor\-based, and Heuristic and Auction\-based variants\. The largest improvements occur under the most constrained budget of 256 tokens per query\. In theBalancedstream,CLEARimproves over the Uniform policy by\+11\.6accuracy points\. In theMostly\-Easystream,CLEARachieves an even larger gain of\+24\.0accuracy points, indicating that uniform allocation wastes scarce tokens on queries that cannot produce sufficient utility under the global budget\. In theMostly\-HardandU\-Shapedstreams,CLEARfurther improves over Uniform by\+5\.2and\+14\.2points, respectively\. As the budget increases, the absolute gains naturally shrink because most methods receive enough tokens to cross the latent emergence threshold\. This trend confirms thatCLEARis most beneficial in resource\-scarce regimes, where shadow\-price allocation and rational abandonment prevent the system from spreading compute too thinly across the batch\.

##### Generalization to Code Generation

To examine whether the proposed allocation principle extends beyond mathematical reasoning, we evaluateCLEARon code\-generation tasks\. We useQwen2\.5\-Coder\-7Band retrain the length regressor in\-domain using completion lengths fromHumanEvalandMBPP, while keeping the allocation algorithm unchanged\. Results are reported under a best\-of\-KKprotocol withK=4K=4: each trajectory has a cap ofB=1024B=1024, and each query receives the same total completion\-token budget of4​B4Bacross methods\.

Table 2:Code Generation Results\.Accuracy comparison under aligned best\-of\-4 completion\-token budgets\.As shown in Table[2](https://arxiv.org/html/2606.03092#S7.T2),CLEARconsistently improves over uniform allocation across all three code benchmarks\. This suggests that global budget clearing and strategic abandonment are not limited to the math\-focused setting, but can also benefit stochastic code generation under strict compute constraints\.

##### Visualization of Token Allocation

To understand the mechanism behind the performance ofCLEAR, we visualize the token allocation in Figure[6](https://arxiv.org/html/2606.03092#S7.F6)\.

Under the scarcity regime of 256 tokens, theUniformandPredictor\-based policies distribute the budget continuously but insufficiently across all queries\. Consequently, the majority of tasks receive fewer tokens than their ground\-truth reasoning length, leading to systemic failure\. In contrast,CLEARexplicitly abandons tasks where the computational cost outweighs the potential gain\. Crucially, the resources conserved from these discarded tasks are redistributed to the admitted queries\. Unlike binary cut\-off or auction, the Lambert policy dynamically adjusts the extra allowance based on the decay rateβ\\beta, ensuring valid tasks receive ample resources to handle uncertainty\.

### 7\.3Robustness and Sensitivity

We further analyze the robustness ofCLEARacross three dimensions: phase transitions, predictor noise, and hyperparameter sensitivity\.

Robustness to Predictor Noise\.Figure[11](https://arxiv.org/html/2606.03092#A2.F11)shows the accuracy degradation as we inject log\-normal noise into the predictor\. While performance naturally declines with increasing noise,CLEARmaintains a significant advantage over Uniform even at high noise levels\.

Sensitivity to Hyperparameters\.We examine the system’s sensitivity to its two core economic parameters: the decay rateβ\\betaand the initial velocityα\\alpha\.

Figure[7](https://arxiv.org/html/2606.03092#S7.F7)evaluates the impact of staticβ\\betavalues\. We observe a distinct regime shift: low\-budget scenarios require a strict, highβ\\betato minimize waste and ensure solvency for easier tasks, whereas high\-budget scenarios benefit from a lenient, lowβ\\betathat allows for extended reasoning surges\. Therefore, staticβ\\betacannot simultaneously satisfy these conflicting demands\. Our adaptive mechanism dynamically calibratesβ\\betabased on the aggregate surplus, consistently landing on the Pareto\-optimal frontier across all regimes without manual tuning\. Unlikeβ\\beta, the thresholdα\\alphaexhibits a unique scale\-invariant property\. As shown in Figure[8](https://arxiv.org/html/2606.03092#S7.F8), varyingα\\alpharesults in negligible fluctuations in accuracy\. Meanwhile, the market\-clearing shadow priceλ∗\\lambda^\{\*\}scales linearly withα\\alpha\. Since the Lambert W allocation depends on the ratioλ/α\\lambda/\\alpha, the optimizer automatically adjustsλ∗\\lambda^\{\*\}to offset any changes inα\\alpha\.

### 7\.4Structural Utility Variants

To test whether the gains ofCLEARdepend on the exact shifted\-surge parameterization, we evaluate two alternative utility structures that preserve the same threshold\-aware allocation principle but replace the latent utility shape\. TheTriangularvariant uses a tent\-shaped utility that rises linearly after the predicted threshold and then decays linearly after its peak\. TheQuadraticvariant uses a concave quadratic peak over the post\-threshold extension\. In both cases, the system still solves for a global budget\-clearing allocation and applies rational abandonment under tight budgets\. LetΔ​t=t−τi\\Delta t=t\-\\tau\_\{i\}denote the effective post\-threshold generation length,cic\_\{i\}denote the maximum supported post\-threshold extension,pi∈\(0,ci\)p\_\{i\}\\in\(0,c\_\{i\}\)denote the triangular peak location, anduimaxu\_\{i\}^\{\\max\}denote the peak utility\. TheTriangularutility is defined as:

ϕiTri​\(t\)=\{00≤t<τiuimax⋅Δ​tpi0≤Δ​t≤piuimax⋅ci−Δ​tci−pipi<Δ​t≤ci0Δ​t\>ci\.\\phi\_\{i\}^\{\\texttt\{Tri\}\}\(t\)=\\begin\{cases\}0&0\\leq t<\\tau\_\{i\}\\\\ u\_\{i\}^\{\\max\}\\cdot\\frac\{\\Delta t\}\{p\_\{i\}\}&0\\leq\\Delta t\\leq p\_\{i\}\\\\ u\_\{i\}^\{\\max\}\\cdot\\frac\{c\_\{i\}\-\\Delta t\}\{c\_\{i\}\-p\_\{i\}\}&p\_\{i\}<\\Delta t\\leq c\_\{i\}\\\\ 0&\\Delta t\>c\_\{i\}\.\\end\{cases\}\(17\)
TheQuadraticutility uses a smooth concave peak over the same post\-threshold support:

ϕiQuad​\(t\)=\{00≤t<τiuimax⋅4​Δ​t​\(ci−Δ​t\)ci20≤Δ​t≤ci0Δ​t\>ci\.\\phi\_\{i\}^\{\\texttt\{Quad\}\}\(t\)=\\begin\{cases\}0&0\\leq t<\\tau\_\{i\}\\\\ u\_\{i\}^\{\\max\}\\cdot\\frac\{4\\Delta t\(c\_\{i\}\-\\Delta t\)\}\{c\_\{i\}^\{2\}\}&0\\leq\\Delta t\\leq c\_\{i\}\\\\ 0&\\Delta t\>c\_\{i\}\.\\end\{cases\}\(18\)
Table 3:Structural Utility Variant Results\.Accuracy comparison under the tight\-budget setting \(B¯=256\\bar\{B\}=256\)\.![Refer to caption](https://arxiv.org/html/2606.03092v1/Figures/structural_ablation.png)Figure 9:Visualization of Structural Utility Variants\.Table[3](https://arxiv.org/html/2606.03092#S7.T3)shows that both structural variants remain substantially above uniform allocation in the most resource\-constrained regime\. This suggests that the main benefit comes from global budget clearing and abandonment\-aware allocation, rather than from a single fragile choice of utility curve\.

## 8Related Work

##### Resource Allocation for LLM Inference\.

LLM deployment is increasingly studied as a resource allocation problem, where computation, latency, and tokens are scarce assets to be assigned under global constraints\. Systems such as FrugalGPT route queries through model cascades to reduce cost\(Chenet al\.,[2023](https://arxiv.org/html/2606.03092#bib.bib15)\), while RouteLLM learns preference\-based routing policies across models\(Onget al\.,[2024](https://arxiv.org/html/2606.03092#bib.bib24)\)\. Other work formulates model selection or query allocation as contextual bandits, knapsack optimization, or cascade scheduling, including LLM\-Bandit and TREACLE\(Zhanget al\.,[2024](https://arxiv.org/html/2606.03092#bib.bib28)\)\. Complementary infrastructure analyses characterize the hardware\-level Pareto frontier of throughput, latency, and energy\(Kwonet al\.,[2023](https://arxiv.org/html/2606.03092#bib.bib13); Shenget al\.,[2023](https://arxiv.org/html/2606.03092#bib.bib14); Wilhelmet al\.,[2025](https://arxiv.org/html/2606.03092#bib.bib26)\)\.

##### Efficient CoT\.

CoT prompting and inference\-time scaling show that allocating more computation at test time can substantially improve reasoning performance\(Weiet al\.,[2022](https://arxiv.org/html/2606.03092#bib.bib1); Snellet al\.,[2024](https://arxiv.org/html/2606.03092#bib.bib2); Wuet al\.,[2024](https://arxiv.org/html/2606.03092#bib.bib10); Wanet al\.,[2025](https://arxiv.org/html/2606.03092#bib.bib8)\)\. A related line of work improves token efficiency by adapting the reasoning procedure itself\. DSC\(Wanget al\.,[2025](https://arxiv.org/html/2606.03092#bib.bib5)\)allocates different numbers of sampled reasoning paths according to question difficulty and posterior confidence\. SelfBudgeter\(Liet al\.,[2025](https://arxiv.org/html/2606.03092#bib.bib6)\)trains the model to estimate and obey instance\-specific reasoning budgets, reducing unnecessary overthinking\. TALE\(Hanet al\.,[2025](https://arxiv.org/html/2606.03092#bib.bib7)\)estimates a token budget for each problem and injects it into the prompt to guide shorter CoT generation\. These methods focus on controlling or aggregating reasoning traces within a query\. In contrast,CLEARaddresses a batch\-level allocation problem: given fixed decoding behavior and a fixed global token budget, it decides how much budget each query should receive and when a query should be rationally abandoned\.

## 9Conclusion

In this work, we study how to allocate a fixed inference\-time token budget across heterogeneous reasoning queries\. We show that uniform budgeting can waste tokens on hard queries that cannot be solved within the available budget, while overlooking queries that could be solved with a more targeted allocation\. To address this mismatch, we formulate batch\-level token allocation as a constrained optimization problem and introduceCLEAR\.CLEARassigns zero budget to queries with negative expected surplus and reallocates tokens to queries where additional computation is more likely to improve utility\. Extensive experiments show thatCLEARimproves the cost\-accuracy Pareto frontier, with the largest gains appearing in resource\-scarce regimes where uniform allocation spreads computation too thinly\. Future work will focus on narrowing the gap toward oracle\-level allocation by improving the robustness of utility modelling\.

## Impact Statement

This paper presents work whose goal is to advance the field of LLM and machine learning\. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here\.

## Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant 72571007 and Grant 72595830/72595831\.

## References

- P\. Aggarwal and S\. Welleck \(2025\)L1: controlling how long a reasoning model thinks with reinforcement learning\.arXiv preprint arXiv:2503\.04697\.Cited by:[§1](https://arxiv.org/html/2606.03092#S1.p1.1)\.
- S\. Boyd and L\. Vandenberghe \(2004\)Convex optimization\.Cambridge university press\.Cited by:[§1](https://arxiv.org/html/2606.03092#S1.p3.1)\.
- B\. Brown, J\. Juravsky, R\. Ehrlich, R\. Clark, Q\. V\. Le, C\. Ré, and A\. Mirhoseini \(2024\)Large language monkeys: scaling inference compute with repeated sampling\.arXiv preprint arXiv:2407\.21787\.Cited by:[§1](https://arxiv.org/html/2606.03092#S1.p1.1)\.
- L\. Chen, M\. Zaharia, and J\. Zou \(2023\)Frugalgpt: how to use large language models while reducing cost and improving performance\.arXiv preprint arXiv:2305\.05176\.Cited by:[§1](https://arxiv.org/html/2606.03092#S1.p1.1),[§8](https://arxiv.org/html/2606.03092#S8.SS0.SSS0.Px1.p1.1)\.
- N\. R\. Devanur, K\. Jain, B\. Sivan, and C\. A\. Wilkens \(2019\)Near optimal online algorithms and fast approximation algorithms for resource allocation problems\.Journal of the ACM \(JACM\)66\(1\),pp\. 1–41\.Cited by:[§1](https://arxiv.org/html/2606.03092#S1.p3.1)\.
- T\. Han, Z\. Wang, C\. Fang, S\. Zhao, S\. Ma, and Z\. Chen \(2025\)Token\-budget\-aware llm reasoning\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 24842–24855\.Cited by:[§6](https://arxiv.org/html/2606.03092#S6.SS0.SSS0.Px3.p1.1),[§8](https://arxiv.org/html/2606.03092#S8.SS0.SSS0.Px2.p1.1)\.
- W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. Gonzalez, H\. Zhang, and I\. Stoica \(2023\)Efficient memory management for large language model serving with pagedattention\.InProceedings of the 29th symposium on operating systems principles,pp\. 611–626\.Cited by:[§8](https://arxiv.org/html/2606.03092#S8.SS0.SSS0.Px1.p1.1)\.
- Z\. Li, Q\. Dong, J\. Ma, D\. Zhang, K\. Jia, and Z\. Sui \(2025\)Selfbudgeter: adaptive token allocation for efficient llm reasoning\.arXiv preprint arXiv:2505\.11274\.Cited by:[§8](https://arxiv.org/html/2606.03092#S8.SS0.SSS0.Px2.p1.1)\.
- S\. V\. Marjanović, A\. Patel, V\. Adlakha, M\. Aghajohari, P\. BehnamGhader, M\. Bhatia, A\. Khandelwal, A\. Kraft, B\. Krojer, X\. H\. Lù,et al\.\(2025\)DeepSeek\-r1 thoughtology: let’s think about llm reasoning\.arXiv preprint arXiv:2504\.07128\.Cited by:[§2](https://arxiv.org/html/2606.03092#S2.p2.3)\.
- N\. Muennighoff, Z\. Yang, W\. Shi, X\. L\. Li, L\. Fei\-Fei, H\. Hajishirzi, L\. Zettlemoyer, P\. Liang, E\. Candès, and T\. B\. Hashimoto \(2025\)S1: simple test\-time scaling\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 20286–20332\.Cited by:[§1](https://arxiv.org/html/2606.03092#S1.p1.1)\.
- I\. Ong, A\. Almahairi, V\. Wu, W\. Chiang, T\. Wu, J\. E\. Gonzalez, M\. W\. Kadous, and I\. Stoica \(2024\)Routellm: learning to route llms with preference data\.arXiv preprint arXiv:2406\.18665\.Cited by:[§8](https://arxiv.org/html/2606.03092#S8.SS0.SSS0.Px1.p1.1)\.
- D\. Raposo, S\. Ritter, B\. Richards, T\. Lillicrap, P\. C\. Humphreys, and A\. Santoro \(2024\)Mixture\-of\-depths: dynamically allocating compute in transformer\-based language models\.arXiv preprint arXiv:2404\.02258\.Cited by:[§1](https://arxiv.org/html/2606.03092#S1.p1.1)\.
- Y\. Sheng, L\. Zheng, B\. Yuan, Z\. Li, M\. Ryabinin, B\. Chen, P\. Liang, C\. Ré, I\. Stoica, and C\. Zhang \(2023\)Flexgen: high\-throughput generative inference of large language models with a single gpu\.InInternational Conference on Machine Learning,pp\. 31094–31116\.Cited by:[§8](https://arxiv.org/html/2606.03092#S8.SS0.SSS0.Px1.p1.1)\.
- C\. Snell, J\. Lee, K\. Xu, and A\. Kumar \(2024\)Scaling llm test\-time compute optimally can be more effective than scaling model parameters\.arXiv preprint arXiv:2408\.03314\.Cited by:[§1](https://arxiv.org/html/2606.03092#S1.p1.1),[§8](https://arxiv.org/html/2606.03092#S8.SS0.SSS0.Px2.p1.1)\.
- X\. Wan, W\. Wang, W\. Xu, W\. Yin, J\. Song, and M\. Sun \(2025\)Adapthink: adaptive thinking preferences for reasoning language model\.arXiv preprint arXiv:2506\.18237\.Cited by:[§8](https://arxiv.org/html/2606.03092#S8.SS0.SSS0.Px2.p1.1)\.
- X\. Wang, S\. Feng, Y\. Li, P\. Yuan, Y\. Zhang, C\. Tan, B\. Pan, Y\. Hu, and K\. Li \(2025\)Make every penny count: difficulty\-adaptive self\-consistency for cost\-efficient reasoning\.InFindings of the Association for Computational Linguistics: NAACL 2025,pp\. 6904–6917\.Cited by:[§8](https://arxiv.org/html/2606.03092#S8.SS0.SSS0.Px2.p1.1)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou,et al\.\(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.Advances in neural information processing systems35,pp\. 24824–24837\.Cited by:[§1](https://arxiv.org/html/2606.03092#S1.p1.1),[§8](https://arxiv.org/html/2606.03092#S8.SS0.SSS0.Px2.p1.1)\.
- J\. Weston and S\. Sukhbaatar \(2023\)System 2 attention \(is something you might need too\)\.arXiv preprint arXiv:2311\.11829\.Cited by:[§1](https://arxiv.org/html/2606.03092#S1.p1.1)\.
- P\. Wilhelm, T\. Wittkopp, and O\. Kao \(2025\)Beyond test\-time compute strategies: advocating energy\-per\-token in llm inference\.InProceedings of the 5th Workshop on Machine Learning and Systems,pp\. 208–215\.Cited by:[§8](https://arxiv.org/html/2606.03092#S8.SS0.SSS0.Px1.p1.1)\.
- Y\. Wu, Z\. Sun, S\. Li, S\. Welleck, and Y\. Yang \(2024\)Inference scaling laws: an empirical analysis of compute\-optimal inference for problem\-solving with language models\.arXiv preprint arXiv:2408\.00724\.Cited by:[§1](https://arxiv.org/html/2606.03092#S1.p1.1),[§8](https://arxiv.org/html/2606.03092#S8.SS0.SSS0.Px2.p1.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§6](https://arxiv.org/html/2606.03092#S6.SS0.SSS0.Px1.p1.1)\.
- A\. Yang, B\. Zhang, B\. Hui, B\. Gao, B\. Yu, C\. Li, D\. Liu, J\. Tu, J\. Zhou, J\. Lin,et al\.\(2024\)Qwen2\. 5\-math technical report: toward mathematical expert model via self\-improvement\.arXiv preprint arXiv:2409\.12122\.Cited by:[Figure 1](https://arxiv.org/html/2606.03092#S1.F1),[Figure 1](https://arxiv.org/html/2606.03092#S1.F1.8.2.1)\.
- X\. Zhang, Z\. Huang, E\. O\. Taga, C\. Joe\-Wong, S\. Oymak, and J\. Chen \(2024\)Efficient contextual llm cascades through budget\-constrained policy learning\.Advances in Neural Information Processing Systems37,pp\. 91691–91722\.Cited by:[§8](https://arxiv.org/html/2606.03092#S8.SS0.SSS0.Px1.p1.1)\.

## Appendix AAppendix: Algorithm Details

Algorithm 1CLEAR\(Lambert\)1:Input:Batch

𝒮=\{s1,…,sN\}\\mathcal\{S\}=\\\{s\_\{1\},\\dots,s\_\{N\}\\\}, total budget

BtotalB\_\{\\text\{total\}\}, token cap

TmaxT\_\{\\max\}
2:Params:Initial velocity

α\\alpha, tolerance

ϵ\\epsilon
3:Output:Allocation vector

𝐭∗\\mathbf\{t\}^\{\*\}
4:

𝝉^←PredictThreshold​\(𝒮\)\\hat\{\\boldsymbol\{\\tau\}\}\\leftarrow\\text\{PredictThreshold\}\(\\mathcal\{S\}\)
5:

B¯←Btotal/N,τ¯←1N​∑i=1Nτ^i\\bar\{B\}\\leftarrow B\_\{\\text\{total\}\}/N,\\quad\\bar\{\\tau\}\\leftarrow\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\hat\{\\tau\}\_\{i\}
6:

β←1/max⁡\(ϵ,B¯−τ¯\)\\beta\\leftarrow 1/\\max\(\\epsilon,\\bar\{B\}\-\\bar\{\\tau\}\)\{batch\-adaptive decay rate\}

7:

𝐭best←𝟎\\mathbf\{t\}^\{\\text\{best\}\}\\leftarrow\\mathbf\{0\}
8:// Bisection search for the shadow priceλ∗\\lambda^\{\*\}

9:

λmin←0,λmax←α−10−6\\lambda\_\{\\min\}\\leftarrow 0,\\quad\\lambda\_\{\\max\}\\leftarrow\\alpha\-10^\{\-6\}
10:while

\(λmax−λmin\)\>ϵ\(\\lambda\_\{\\max\}\-\\lambda\_\{\\min\}\)\>\\epsilondo

11:

λ←\(λmin\+λmax\)/2\\lambda\\leftarrow\(\\lambda\_\{\\min\}\+\\lambda\_\{\\max\}\)/2
12:

Ccurrent←0C\_\{\\text\{current\}\}\\leftarrow 0
13:for

i=1i=1to

NNdo

14:if

λ≥α\\lambda\\geq\\alphathen

15:

ti←0t\_\{i\}\\leftarrow 0
16:else

17:

z←\(λ​e\)/αz\\leftarrow\(\\lambda e\)/\\alpha
18:

u←W0​\(z\)u\\leftarrow W\_\{0\}\(z\)
19:

Δ​ti←\(1−u\)/β\\Delta t\_\{i\}\\leftarrow\(1\-u\)/\\beta
20:if

Δ​ti≤0\\Delta t\_\{i\}\\leq 0then

21:

ti←0t\_\{i\}\\leftarrow 0
22:else

23:

tiunc←τ^i\+Δ​tit\_\{i\}^\{\\text\{unc\}\}\\leftarrow\\hat\{\\tau\}\_\{i\}\+\\Delta t\_\{i\}
24:

ϕi←α​Δ​ti​exp⁡\(−β​Δ​ti\)\\phi\_\{i\}\\leftarrow\\alpha\\Delta t\_\{i\}\\exp\(\-\\beta\\Delta t\_\{i\}\)
25:if

ϕi\>λ​tiunc\\phi\_\{i\}\>\\lambda t\_\{i\}^\{\\text\{unc\}\}then

26:

ti←min⁡\(tiunc,Tmax\)t\_\{i\}\\leftarrow\\min\(t\_\{i\}^\{\\text\{unc\}\},T\_\{\\max\}\)
27:else

28:

ti←0t\_\{i\}\\leftarrow 0
29:endif

30:endif

31:endif

32:

Ccurrent←Ccurrent\+tiC\_\{\\text\{current\}\}\\leftarrow C\_\{\\text\{current\}\}\+t\_\{i\}
33:endfor

34:if

Ccurrent\>BtotalC\_\{\\text\{current\}\}\>B\_\{\\text\{total\}\}then

35:

λmin←λ\\lambda\_\{\\min\}\\leftarrow\\lambda
36:else

37:

λmax←λ\\lambda\_\{\\max\}\\leftarrow\\lambda
38:

𝐭best←𝐭\\mathbf\{t\}^\{\\text\{best\}\}\\leftarrow\\mathbf\{t\}
39:endif

40:endwhile

41:

𝐭∗←⌊𝐭best⌋\\mathbf\{t\}^\{\*\}\\leftarrow\\lfloor\\mathbf\{t\}^\{\\text\{best\}\}\\rfloor
42:Clip each

ti∗t\_\{i\}^\{\*\}to

\[0,Tmax\]\[0,T\_\{\\max\}\]and optionally correct residual tokens\.

43:return

𝐭∗\\mathbf\{t\}^\{\*\}

### A\.1Pseudocode

We present the pseudocode for theCLEARin AlgorithmLABEL:alg:clear\_lambert, which computes the optimal resource allocation vector under a given total budget constraint\.

### A\.2Allocation Policy Details

In this section, we formally define the token allocationtit\_\{i\}for a batch of queries𝒮=\{s1,…,sN\}\\mathcal\{S\}=\\\{s\_\{1\},\\dots,s\_\{N\}\\\}under a total budget constraintBtotalB\_\{\\text\{total\}\}\. Letτ^i\\hat\{\\tau\}\_\{i\}denote the predicted emergence threshold used by practical policies, and letdid\_\{i\}denote the ground\-truth solution length used only by theOraclepolicy\. The average budget per query isB¯=Btotal/N\\bar\{B\}=B\_\{\\text\{total\}\}/N\.

#### A\.2\.1UniformPolicy

This is the standard baseline where resources are distributed equally, agnostic to task complexity\.

tiunif=⌊B¯⌋\.t\_\{i\}^\{\\text\{unif\}\}=\\lfloor\\bar\{B\}\\rfloor\.\(19\)

#### A\.2\.2Predictor\-Based Proportional Policy

This policy assumes a linear relationship between the latent threshold and required tokens, scaling the budget proportionally to the predicted length without abandonment\.

tiprop=τ^i∑j=1Nτ^j⋅Btotal\.t\_\{i\}^\{\\text\{prop\}\}=\\frac\{\\hat\{\\tau\}\_\{i\}\}\{\\sum\_\{j=1\}^\{N\}\\hat\{\\tau\}\_\{j\}\}\\cdot B\_\{\\text\{total\}\}\.\(20\)In practice, we clip the allocation to a minimum \(e\.g\., 32 tokens\) and maximum context length\.

#### A\.2\.3TALE\-EPPolicy

TALE\-EPfirst uses a stronger teacher model to estimate per\-query completion needs, then enforces both a soft prompt\-level budget constraint and a hard decoding cap during regeneration\. Letτ~i\\tilde\{\\tau\}\_\{i\}denote the raw budget estimate from the teacher model \(e\.g\.,Qwen3\-30B\-Instruct\)\.

1. 1\.Teacher\-based budget estimation:For each querysis\_\{i\}, the teacher predicts a raw token needτ~i\\tilde\{\\tau\}\_\{i\}\.
2. 2\.Renormalized proportional allocation:The raw estimates are converted into per\-query caps under the same total budget: t^itale=τ~i∑j=1Nτ~j⋅Btotal\.\\hat\{t\}\_\{i\}^\{\\text\{tale\}\}=\\frac\{\\tilde\{\\tau\}\_\{i\}\}\{\\sum\_\{j=1\}^\{N\}\\tilde\{\\tau\}\_\{j\}\}\\cdot B\_\{\\text\{total\}\}\.\(21\)In practice, we apply clipping and integerization: titale=clip\(⌊t^itale⌉,tmin,tmax\),t\_\{i\}^\{\\text\{tale\}\}=\\mathrm\{clip\}\\\!\\left\(\\left\\lfloor\\hat\{t\}\_\{i\}^\{\\text\{tale\}\}\\right\\rceil,\\,t\_\{\\min\},\\,t\_\{\\max\}\\right\),\(22\)followed by a residual correction step to ensure exact budget conservation: ∑i=1Ntitale=Btotal\.\\sum\_\{i=1\}^\{N\}t\_\{i\}^\{\\text\{tale\}\}=B\_\{\\text\{total\}\}\.\(23\)
3. 3\.Budget\-conditioned regeneration:For each querysis\_\{i\}, we construct a budget\-aware prompt with a soft instruction \(e\.g\., “keep the entire response within at mosttitalet\_\{i\}^\{\\text\{tale\}\}completion tokens”\), and run decoding with a hard cap: yi∼pθ\(⋅∣si,titale\),max\_tokens=titale\.y\_\{i\}\\sim p\_\{\\theta\}\(\\cdot\\mid s\_\{i\},\\;t\_\{i\}^\{\\text\{tale\}\}\),\\qquad\\texttt\{max\\\_tokens\}=t\_\{i\}^\{\\text\{tale\}\}\.\(24\)We use greedy decoding in our setup \(temperature=0\\texttt\{temperature\}=0,top\_p=1\\texttt\{top\\\_p\}=1\)\.

Final accuracy is computed by applyingmath\_verifyto regenerated outputs\{yi\}i=1N\\\{y\_\{i\}\\\}\_\{i=1\}^\{N\}\.

#### A\.2\.4CLEAR\(Heuristic\) Policy

This baseline uses a fixed cutoff rule based on the predicted thresholds\. When the average budget is too small relative to the average predicted threshold, it drops the harder half of the batch and reallocates the budget to the remaining queries\.

1. 1\.Check budget scarcity:Let𝒯^=\{τ^1,…,τ^N\}\\hat\{\\mathcal\{T\}\}=\\\{\\hat\{\\tau\}\_\{1\},\\dots,\\hat\{\\tau\}\_\{N\}\\\}and defineη=B¯/𝔼​\[𝒯^\]\\eta=\\bar\{B\}/\\mathbb\{E\}\[\\hat\{\\mathcal\{T\}\}\]\. Ifη<0\.8\\eta<0\.8, the cutoff rule is activated\.
2. 2\.Select easier queries:Queries withτ^i\>Median​\(𝒯^\)\\hat\{\\tau\}\_\{i\}\>\\text\{Median\}\(\\hat\{\\mathcal\{T\}\}\)receive zero budget\. The remaining queries share the total budget using: tiheur=\{μsur\+κ⋅\(τ^i−μsur\)if​τ^i≤Median​\(𝒯^\)0otherwise,t\_\{i\}^\{\\text\{heur\}\}=\\begin\{cases\}\\mu\_\{\\text\{sur\}\}\+\\kappa\\cdot\(\\hat\{\\tau\}\_\{i\}\-\\mu\_\{\\text\{sur\}\}\)&\\text\{if \}\\hat\{\\tau\}\_\{i\}\\leq\\text\{Median\}\(\\hat\{\\mathcal\{T\}\}\)\\\\ 0&\\text\{otherwise,\}\\end\{cases\}\(25\)whereμsur\\mu\_\{\\text\{sur\}\}is the mean predicted threshold among selected queries, andκ\\kappais chosen so that the selected queries use exactlyBtotalB\_\{\\text\{total\}\}tokens in total\.

#### A\.2\.5CLEAR\(Auction\) Policy

This ablation uses a simple greedy selection rule\. It first keeps the queries with smaller predicted thresholds, since they are more likely to be completed under a limited budget, and then distributes the available tokens among the selected queries\.

1. 1\.Sort by predicted length:Sort the query indices by increasing predicted threshold, yielding a permutation\(p1,…,pN\)\(p\_\{1\},\\dots,p\_\{N\}\)such thatτ^p1≤⋯≤τ^pN\\hat\{\\tau\}\_\{p\_\{1\}\}\\leq\\dots\\leq\\hat\{\\tau\}\_\{p\_\{N\}\}\.
2. 2\.Select survivors:Keep the largest prefix of this sorted list whose predicted thresholds fit within the total budget: m∗=max⁡\{m∈\[1,N\]∣∑j=1mτ^pj≤Btotal\}\.m^\{\*\}=\\max\\left\\\{m\\in\[1,N\]\\mid\\sum\_\{j=1\}^\{m\}\\hat\{\\tau\}\_\{p\_\{j\}\}\\leq B\_\{\\text\{total\}\}\\right\\\}\.\(26\)
3. 3\.Allocate to survivors:Assign zero budget to all non\-selected queries\. The selected queries then share the full budget using the same affine allocation rule asCLEAR\(Heuristic\), rescaled so that∑iti=Btotal\\sum\_\{i\}t\_\{i\}=B\_\{\\text\{total\}\}\.

#### A\.2\.6OraclePolicy

This policy is an upper\-bound baseline that uses ground\-truth solution lengthsdid\_\{i\}, which are unavailable at test time\. It spends tokens on the shortest solvable queries first, maximizing the number of completed tasks under the total budget\.

1. 1\.Sort by true length:Sort indices into a permutation\(o1,…,oN\)\(o\_\{1\},\\dots,o\_\{N\}\)such thatdo1≤⋯≤doNd\_\{o\_\{1\}\}\\leq\\dots\\leq d\_\{o\_\{N\}\}\.
2. 2\.Fill the budget greedily:Allocate exactlydojd\_\{o\_\{j\}\}tokens to each query in sorted order until the next query would exceedBtotalB\_\{\\text\{total\}\}: tojoracle=\{dojif​∑l=1jdol≤Btotal0otherwise\.t\_\{o\_\{j\}\}^\{\\text\{oracle\}\}=\\begin\{cases\}d\_\{o\_\{j\}\}&\\text\{if \}\\sum\_\{l=1\}^\{j\}d\_\{o\_\{l\}\}\\leq B\_\{\\text\{total\}\}\\\\ 0&\\text\{otherwise\.\}\\end\{cases\}\(27\)

## Appendix BAppendix: More Results

To demonstrate the generalizability of our framework to larger\-scale reasoning models, we present additional experimental results using Qwen3\-30B\-A3B\-Instruct\.

Figure[10](https://arxiv.org/html/2606.03092#A2.F10)validates the threshold predictor’s accuracy on this larger model, while Table[4](https://arxiv.org/html/2606.03092#A2.T4)details the downstream allocation performance across varying supply\-demand scenarios\. In summary, CLEAR consistently outperforms the uniform baseline across all tested conditions, achieving up to\+2\.4accuracy gains in strictly constrained budget environments where efficient resource distribution is most critical\.

![Refer to caption](https://arxiv.org/html/2606.03092v1/Figures/ranking_analysis_30b.png)Figure 10:Predictor Performance Analysis on Qwen3\-30B\-A3B\-Instruct\.Table 4:Performance Comparison across Difficulty Scenarios and Budgets on Qwen3\-30B\-A3B\-Instruct\.We compare our proposedCLEARagainst Uniform, Direct Predictor, and heuristic CLEAR variants\. Results are reported as accuracy \(%\)\.Boldindicates the best performance among non\-oracle methods\. The bottom row shows the absolute accuracy gain of CLEAR over the Uniform baseline\.![Refer to caption](https://arxiv.org/html/2606.03092v1/Figures/sensitivity_noise_4streams.png)Figure 11:Robustness to Predictor Noise\.Performance of CLEAR versus Uniform under increasing predictor noiseσ\\sigma\. CLEAR maintains a performance advantage over the baseline even under significant noise levels\.Table 5:Detailed Experimental Configuration\.This table summarizes the hyperparameters and settings used for data generation, predictor training, and the CLEAR allocation mechanism across all experiments\.ComponentParameter / SettingValue / Description1\. Data Generation \(Oracle\)Backbone ModelsQwen2\.5\-Math\-7B, Qwen3\-30B\-A3B\-InstructMax New Tokens16,384 \(for 30B\), 8,192 \(for 7B\)Decoding StrategyGreedy Decoding \(Temperature = 0\)2\. Threshold PredictorArchitectureDeBERTa\-v3\-base \(86M parameters\)Training SourcesGSM8K \(Train\), MATH \(Train\)Input TokenizationLeft\-Truncation \(Retain last 512 tokens\)Max Sequence Length512 tokensTraining ObjectiveMean Squared Error \(MSE\) on Log\-LengthOptimizationAdamW \(LR=2e\-5, Weight Decay=0\.01, Batch=32\)Training Schedule10 Epochs3\. CLEAR Allocation MechanismGlobal Parametersα=2\.0\\alpha=2\.0Optimization MethodBisection Search \(40 iterations,ϵ=1​e−6\\epsilon=1e\-6\)4\. Evaluation ScenariosSample SizeN=5,00N=5,00queries per simulation streamEvaluation SourcesMATH\-500, OlympiadBench, AIME \(24/25\), AMC\-23, Minerva
## Appendix CAppendix: Data Composition and Statistics

##### Training and Test Data

Table[6](https://arxiv.org/html/2606.03092#A3.T6)and Table[7](https://arxiv.org/html/2606.03092#A3.T7)provide a detailed breakdown of the datasets used for training the threshold predictor and evaluating the budget allocation policies\. To ensure the robustness of ourCLEARframework, the evaluation suite is intentionally designed to be Out\-of\-Distribution \(OOD\) relative to the predictor’s training set\.

Table 6:Detailed Statistics of Training and Evaluation Datasets for Qwen\-2\.5\-math\-7B\-Instructunder greedy decoding and4Knew token constraints\.Use CaseDatasetSplitTotalCorrectPass@1Avg\. LenThreshold TierPredictor TrainingGSM8KTrain1,3191,02578\.5%77\.71EasyMATHTrain12,5009,47775\.8%658ModeratePerformance Eval\.MATH 500Test50035470\.8%1,074ModerateOlympiadBenchTest6758813\.0%1,059HardAMC\-23Test401947\.5%1,490HardAIME 2024Test30826\.7%1,369Very HardAIME 2025Test3026\.7%1,886Very HardMinervaTest2723512\.9%2,274Very HardTable 7:Detailed Statistics of Training and Evaluation Datasets for Qwen3\-30B\-A3B\-Instructunder greedy decoding and16Knew token constraints\. The threshold tiers are assigned based on the average reasoning length of these solutions\.Use CaseDatasetSplitTotalCorrectPass@1Avg\. LenThreshold TierPredictor TrainingGSM8KTrain1,05593588\.6%1,254EasyMATHTrain12,5008,24666\.0%3,763ModeratePerformance Eval\.MATH 500Test50028857\.6%9,887ModerateOlympiadBenchTest67517926\.5%9,450HardAMC\-23Test401435\.0%13,958HardAIME 2024Test30310\.0%14,428Very HardAIME 2025Test301446\.7%14,425Very HardMinervaTest2723713\.6%10,771Very Hard