SCX Router: Streaming Zero-Shot Model Selection with a Decoder-KV Classifier and a Real-World Task Ontology

# SCX Router: Streaming Zero-Shot Model Selection with a Decoder-KV Classifier and a Real-World Task Ontology
Source: [https://arxiv.org/html/2609.02292](https://arxiv.org/html/2609.02292)
Aleksandr SmechovAffiliation:SCX\.ai Holdings LimitedMykhailo ShtopkoAffiliation:KnowledgatorDmytro VodianytskyiAffiliation:KnowledgatorOleksandr LukashovAffiliation:Knowledgator

###### Abstract

The rapid proliferation of large language models \(LLMs\) and the growing diversity of their applications presents a unique optimization opportunity: selecting the right model for the task, while optimizing for speed, cost, and quality at a per\-task level\. However, inference endpoints can vary widely in quality, price, latency, context support, tool use, domain expertise, and reasoning behavior\. This heterogeneity makes manual heuristics difficult to maintain and unlikely to achieve consistently favorable speed–cost–quality trade\-offs on their own\. We introduceSCX Router, a lightweight GLiClass\-based router that assigns a suitability score to each inference\-time model label without autoregressive generation\. The released 0\.6B\-parameter checkpoint111Checkpoint:[https://huggingface\.co/scx\-admin/scx\-router\-v0\.1](https://huggingface.co/scx-admin/scx-router-v0.1)\. Code:[https://github\.com/Knowledgator/GLiClass\-Model\-Router](https://github.com/Knowledgator/GLiClass-Model-Router)\.\(Apache 2\.0\) combines a Qwen3 decoder with a shallow bidirectional scorer\. Its decoder\-KV execution path preserves a text\-only key–value cache across a session, encodes only new dialogue turns, and evaluates transient candidate\-label tokens without adding them to the persistent cache\. The same checkpoint also predicts task type, difficulty, reasoning mode, and expected output length, and supports custom zero\-shot labels\. For task generation, we construct a task ontology with 23 families, 115 task types, 345 routable subtypes, 1,173 synthetic examples, and an orthogonal axis of 30 domains\. Using this structure, we generate 150,000 verifier\-scored tasks and 15,000 open\-ended tasks\. We then train the Qwen3 decoder on these tasks, while explicitly separating learned request prediction from per\-task policies for attributes such as eligibility, cost, cache reuse, safety, and sovereignty\. Across six LiveBench subsets, the router outperforms the mean candidate; on the selected 1,000\-task subset, it achieves an aggregate top\-1 score of 0\.707 versus 0\.696 for the strongest fixed model, with benchmark\-dependent gains\.

Keywords:large language model routing; model selection; GLiClass; zero\-shot classification; decoder KV cache; task ontology; agent evaluation; cost\-aware inference; multi\-turn routing\.

## 1Introduction

Closed\-source, frontier LLMs continue to fight for dominance on general intelligence leaderboards222[https://arcprize\.org/leaderboard](https://arcprize.org/leaderboard)\. But open\-weights model families vary widely in their ability to reason, code, follow instructions, cover multiple languages, and more, while also ranging in price, latency, context length, tool access, and operational constraints\. This gives users the advantage of choice from a large pool of intelligence, but also presents a dillema: how do you choose the right model for the task? Prior routing systems learn a binary strong\-versus\-weak decision, estimate answer quality, or cascade from inexpensive to expensive models\[[Chen et al\., 2023](https://arxiv.org/html/2609.02292#bib.bib7),[Aggarwal et al\., 2023](https://arxiv.org/html/2609.02292#bib.bib8),[Ong et al\., 2024](https://arxiv.org/html/2609.02292#bib.bib5),[Dekoninck et al\., 2024](https://arxiv.org/html/2609.02292#bib.bib9)\]\. RouterBench formalizes the broader multi\-model setting and demonstrates the value—and difficulty—of evaluation on a shared outcome matrix\[[Hu et al\., 2024](https://arxiv.org/html/2609.02292#bib.bib6)\]\. In production, the problem is harder: candidate rosters change, dialogue histories grow, cache reuse changes the incremental price of switching, and some requests require hard policy constraints that cannot be reduced to a scalar preference\.

SCX Routeris designed for such settings\. It adapts GLiClass\[[Stepanov et al\., 2025](https://arxiv.org/html/2609.02292#bib.bib1)\], a generalist label–text classifier derived from GLiNER\[[Zaratiana et al\., 2023](https://arxiv.org/html/2609.02292#bib.bib2)\], to a causal decoder with a persistent KV cache\. Instead of prompting another LLM to emit a route token, the router jointly processes a request and natural\-language candidate labels and returns one logit per label in a non\-generative forward pass\. Because labels are inputs rather than fixed output neurons, the same interface supports model names, task taxonomies, difficulty levels, reasoning modes, output\-length buckets, safety signals, and deployment\-specific labels\.

The architectural choice matters most in dialogue\. A stateless encoder must process the full conversation whenever a new message arrives\. A generative router may reuse its prefix cache but must still decode and parse route tokens\. The decoder\-KV path inSCX Routerkeeps only request\-context keys and values between turns\. New request tokens extend that cache; candidate labels are then scored against the cached context and their KV tensors are discarded\. This yields a streaming classifier rather than a second conversational generator\.

Model scores alone are not a routing policy\. We therefore keep the learned predictor separate from a deterministic decision layer\. The latter can exclude endpoints that violate context, tool, modality, residency, privacy, or safety requirements, then trade predicted performance against incremental token cost, average latency, and cache state\. This separation also makes empirical model\-performance profiles replaceable without retraining the semantic classifier\.

### 1\.1Contributions

We make the following contributions\.

1. 1\.A streaming zero\-shot router\.We present a decoder\-KV GLiClass architecture that combines inference\-time label semantics, a chat\-aligned causal backbone, a non\-generative classification head, and text\-only session\-cache reuse\.
2. 2\.A multi\-signal routing interface\.One checkpoint estimates model suitability, task type, difficulty, reasoning mode, expected output length, and arbitrary custom labels\. Learned prediction remains separate from hard eligibility and business policy\.
3. 3\.A task ontology and synthetic data suite\.We construct an ontology containing 23 task families, 115 task types, 345 routable subtypes, 1,173 synthetic examples, and an orthogonal axis of 30 domains\. It structures 150,000 verifier\-scored and 15,000gpt5\.6\-sol\-judged synthetic tasks with real\-world workflow structure\.
4. 4\.A taxonomy of model\-routing patterns\.We introduce direct endpoint routing, attribute\-mediated performance routing, hybrid constrained routing, and hierarchical planner–worker routing for agentic workflows, while distinguishing released paths, implemented components, and proposed compositions\.
5. 5\.An evaluation of routing quality\.We evaluate the released classification heads and distinguish mean\-candidate from fixed\-model end\-to\-end baselines\. For the expanded 11\-endpoint collection, observation masks preserve unequal coverage and prevent missing outcomes from becoming negative labels\.

## 2Related Work

### 2\.1Routing and Cascading Across LLMs

FrugalGPT learns cascades under budget constraints and shows that heterogeneous API models can improve the cost–quality frontier\[[Chen et al\., 2023](https://arxiv.org/html/2609.02292#bib.bib7)\]\. AutoMix similarly escalates based on self\-verification and a partially observable decision process\[[Aggarwal et al\., 2023](https://arxiv.org/html/2609.02292#bib.bib8)\]\. RouteLLM learns strong\-versus\-weak routing from preference data and studies transfer when the underlying pair changes\[[Ong et al\., 2024](https://arxiv.org/html/2609.02292#bib.bib5)\]\. More recent work unifies routing and cascading and identifies quality estimation as the central statistical problem\[[Dekoninck et al\., 2024](https://arxiv.org/html/2609.02292#bib.bib9)\]\. RouterBench supplies more than 405,000 inference outcomes and emphasizes common task coverage for trustworthy comparisons\[[Hu et al\., 2024](https://arxiv.org/html/2609.02292#bib.bib6)\]\.

SCX Routerdiffers along three axes\. First, it performs multi\-label suitability prediction over a configurable roster rather than a fixed binary gate\. Second, natural\-language labels allow semantic transfer to new candidates and auxiliary taxonomies\. Third, the persistent state belongs to a classifier built on a causal decoder: it can reuse dialogue tokens without generating a route\. Semantic zero\-shot transfer does not eliminate the need for outcome data, however; a new endpoint name may be accepted mechanically while remaining poorly calibrated empirically\.

### 2\.2Efficient Zero\-Shot Classification

GLiNER jointly represents text and entity labels for generalist named\-entity recognition\[[Zaratiana et al\., 2023](https://arxiv.org/html/2609.02292#bib.bib2)\]\. GLiClass extends this paradigm to sequence classification, supporting zero\- and few\-shot classification while avoiding one text–label forward pass per class\[[Stepanov et al\., 2025](https://arxiv.org/html/2609.02292#bib.bib1)\]\. The original GLiClass family primarily uses bidirectional encoders\.SCX Routerretains its label\-conditioned scorer but places it after a Qwen3 causal backbone\[[Qwen Team, 2025](https://arxiv.org/html/2609.02292#bib.bib3)\]\. A shallow DeBERTa\-style encoder\[[He et al\., 2020](https://arxiv.org/html/2609.02292#bib.bib4)\]restores bidirectional interaction over the transient label suffix before classification\.

Embedding and bi\-encoder methods precompute cheap label representations but expose limited token\-level label–text interaction\. Pairwise cross\-encoders offer stronger interaction at a cost proportional to the number of labels\. Prompted generative routers are flexible and chat\-native, but add decoding latency and output\-format variance\. The decoder\-KV design occupies a middle point: labels remain dynamic, the backbone is causal and cacheable, and the output is discriminative\.

### 2\.3Real\-World and Agentic Evaluation

Static question answering is an incomplete proxy for routing real applications\. GAIA emphasizes questions that combine reasoning, browsing, multimodality, and tools\[[Mialon et al\., 2023](https://arxiv.org/html/2609.02292#bib.bib16)\]; AgentBench evaluates LLMs across interactive environments\[[Liu et al\., 2023](https://arxiv.org/html/2609.02292#bib.bib17)\]; SWE\-bench requires repository\-scale code modification and test execution\[[Jimenez et al\., 2023](https://arxiv.org/html/2609.02292#bib.bib18)\]; and OSWorld pairs realistic tasks with initial\-state configurations, files, and execution\-based evaluators\[[Xie et al\., 2024](https://arxiv.org/html/2609.02292#bib.bib19)\]\. These benchmarks motivate our task packages: the prompt alone is insufficient for many workloads, so generation includes the context, artifacts, criteria, and checks needed to evaluate an outcome\.

Open\-ended evaluation often relies on LLM judges\. MT\-Bench documents both their scalability and systematic position, verbosity, and self\-enhancement biases\[[Zheng et al\., 2023](https://arxiv.org/html/2609.02292#bib.bib14)\]\. Panels of heterogeneous judges can reduce single\-model bias\[[Verga et al\., 2024](https://arxiv.org/html/2609.02292#bib.bib15)\]\. We therefore treat deterministic verifiers as preferable when semantics permit and require judge identity, prompt, rubric, and coverage metadata for subjective tasks\.

## 3Task Formulation

Letxtx\_\{t\}be the new request content at turntt,h<th\_\{<t\}the preceding conversation, andℳt=\{m1,…,mK\}\\mathcal\{M\}\_\{t\}=\\\{m\_\{1\},\\ldots,m\_\{K\}\\\}the available candidate\-label vocabulary before product constraints\. The classifier computes a logitat,ka\_\{t,k\}and probability\-like score

pt,k=σ⁡\(at,k\)p\_\{t,k\}=\\sigma\(a\_\{t,k\}\)\(1\)for each candidate in the multi\-label view\. A candidate is emitted whenpt,k≥τp\_\{t,k\}\\geq\\tau, with a default threshold of0\.50\.5in the released pipeline\. For a single\-label auxiliary task, softmax normalization and an argmax decision are used instead\.

Routing supervision is derived from observed downstream outcomes\. For taskiiand modelmm, letsi,m∈\[0,1\]s\_\{i,m\}\\in\[0,1\]denote a benchmark\-normalized score andoi,m∈\{0,1\}o\_\{i,m\}\\in\\\{0,1\\\}denote whether the outcome was actually observed\. Our dataset builder defines the positive set as models tied for the highest observed score, optionally within a toleranceϵ\\epsilon:

Yi=\{m∈ℳi:si,m≥maxj∈ℳi⁡si,j−ϵ\}\.Y\_\{i\}=\\left\\\{m\\in\\mathcal\{M\}\_\{i\}:s\_\{i,m\}\\geq\\max\_\{j\\in\\mathcal\{M\}\_\{i\}\}s\_\{i,j\}\-\\epsilon\\right\\\}\.\(2\)Equation[2](https://arxiv.org/html/2609.02292#S3.E2)operationalizes success from the best observed outcomes and avoids imposing one universal threshold across heterogeneous benchmark evaluators\.

If every candidate is evaluated on every task,YiY\_\{i\}supports a paired routing objective\. When coverage differs,oi,m=0o\_\{i,m\}=0is missing data, not a failed response\. A valid loss must mask unobserved pairs,

ℒmasked=−1∑i,moi,m∑i,moi,m\[yi,mlogpi,m\+\(1−yi,m\)log\(1−pi,m\)\],\\mathcal\{L\}\_\{\\mathrm\{masked\}\}=\-\\frac\{1\}\{\\sum\_\{i,m\}o\_\{i,m\}\}\\sum\_\{i,m\}o\_\{i,m\}\\left\[y\_\{i,m\}\\log p\_\{i,m\}\+\(1\-y\_\{i,m\}\)\\log\(1\-p\_\{i,m\}\)\\right\],\(3\)or restrict training and evaluation to a paired intersection\. The eleven\-endpoint collection has unequal endpoint counts, so this paper does not infer unobserved failures or report 11\-way comparative accuracy\.

The final route is a policy decision\. Letℰt⊆ℳt\\mathcal\{E\}\_\{t\}\\subseteq\\mathcal\{M\}\_\{t\}be the candidates remaining after context, tool, modality, region, privacy, and safety constraints\. A policy chooses

mt∗=arg⁡maxm∈ℰt​Ut​\(m\),m\_\{t\}^\{\*\}=\\arg\\max\_\{m\\in\\mathcal\{E\}\_\{t\}\}U\_\{t\}\(m\),\(4\)whereUtU\_\{t\}can combine request\-specific scores, empirical performance profiles, expected incremental cost, latency, and cache reuse\. Section[9](https://arxiv.org/html/2609.02292#S9)gives the implemented cost\-aware form\.

## 4Decoder\-KV Router

The released checkpoint is an approximately 0\.6B\-parameter GLiClass model with a Qwen3\-0\.6B causal backbone\[[Qwen Team, 2025](https://arxiv.org/html/2609.02292#bib.bib3)\]\. The backbone has 28 decoder layers, hidden size 1,024, 16 query heads and 8 KV heads; training uses sequences up to 4,096 tokens\. Three special tokens organize the classification sequence:<<LABEL\>\>,<<SEP\>\>, and<<EXAMPLE\>\>\. The scorer is a two\-layer DeBERTa\-v2 encoder without its own embedding layer, followed by projections and a shared MLP of widths2​h→1024→512→12h\\rightarrow 1024\\rightarrow 512\\rightarrow 1\.

Figure[1](https://arxiv.org/html/2609.02292#S4.F1)summarizes the separation between learned request signals and the deployment policy that selects an eligible endpoint\.

![Refer to caption](https://arxiv.org/html/2609.02292v1/figures/system_overview.png)Figure 1:System overview\. The classifier returns request signals; a separate application policy applies hard eligibility constraints and selects an endpoint\. The displayed signal families use the same label\-conditioned inference interface\.### 4\.1Sequence and Scoring

The canonical implementation formats the persistent context and transient labels as

c≤t\\displaystyle c\_\{\\leq t\}=format⁡\(prompt,examples,h<t,xt\),\\displaystyle=\\operatorname\{format\}\(\\text\{prompt\},\\text\{examples\},h\_\{<t\},x\_\{t\}\),\(5\)q⁡\(ℒ\)\\displaystyle q\(\\mathcal\{L\}\)=<<SEP\>\>ℓ1<<LABEL\>\>⋯ℓK<<LABEL\>\><<SEP\>\>\.\\displaystyle=\\texttt\{<<SEP\>\>\}\\;\\ell\_\{1\}\\;\\texttt\{<<LABEL\>\>\}\\;\\cdots\\;\\ell\_\{K\}\\;\\texttt\{<<LABEL\>\>\}\\;\\texttt\{<<SEP\>\>\}\.\(6\)The label text therefore precedes its<<LABEL\>\>marker, matching the data\-processing and streaming implementation\.

For a stateless call, the causal decoder processes\[c≤t;q⁡\(ℒ\)\]\[c\_\{\\leq t\};q\(\\mathcal\{L\}\)\]\. For a session, letΔ​ct\\Delta c\_\{t\}denote only the persistent\-context tokens appended since the last cached call\. Those new tokens update a persistent text\-only cache,

\(Htx,Kt,Vt\)=Dθ​\(Δ​ct,Kt−1,Vt−1\),\(H\_\{t\}^\{x\},K\_\{t\},V\_\{t\}\)=D\_\{\\theta\}\(\\Delta c\_\{t\};K\_\{t\-1\},V\_\{t\-1\}\),\(7\)and label tokens are evaluated conditionally,

Htℓ=Dθ​\(q⁡\(ℒ\),Kt,Vt\)\.H\_\{t\}^\{\\ell\}=D\_\{\\theta\}\(q\(\\mathcal\{L\}\);K\_\{t\},V\_\{t\}\)\.\(8\)The label\-stage keys and values are not written back to\(Kt,Vt\)\(K\_\{t\},V\_\{t\}\)\. Thus repeated classification can change the label roster without contaminating dialogue state\.

The transient hidden states pass through a bidirectional scorer encoderBϕB\_\{\\phi\}:

Z=Bϕ​\(Htℓ\)\.Z=B\_\{\\phi\}\(H\_\{t\}^\{\\ell\}\)\.\(9\)The scorer extracts a joint representationztz\_\{t\}from the final separator and label representationszkz\_\{k\}from the<<LABEL\>\>positions\. BecauseBϕB\_\{\\phi\}is bidirectional, the final separator summarizes the full transient label section as well as decoder\-conditioned request context; it is not simply the last request\-token state\. Projected request and label representations are concatenated and scored:

z~t\\displaystyle\\tilde\{z\}\_\{t\}=Wt​zt,\\displaystyle=W\_\{t\}z\_\{t\},z~k\\displaystyle\\tilde\{z\}\_\{k\}=Wℓ​zk,\\displaystyle=W\_\{\\ell\}z\_\{k\},\(10\)at,k\\displaystyle a\_\{t,k\}=MLPψ⁡\(\[z~t;z~k\]\)\.\\displaystyle=\\operatorname\{MLP\}\_\{\\psi\}\(\[\\tilde\{z\}\_\{t\};\\tilde\{z\}\_\{k\}\]\)\.\(11\)
Two properties follow from this persistent–transient split\. First, cache reuse amortizes request\-side representation, not the entire routing decision\. If a turn contributesΔ​n\\Delta ncontext tokens and the current roster occupiesLLlabel tokens, the cached path extends the state withΔ​n\\Delta ntokens and, when triggered, executes the transientLL\-token suffix and scorer\. The suffix still attends to retained history, so latency can depend on both cache length and roster size; reuse avoids recomputing earlier hidden states but does not make routing cost independent of history\. Second, labels are in\-band inputs rather than fixed output neurons: the same projections and MLP score every marker\. The head therefore admits a variable roster and independent suitability scores for multiple acceptable endpoints\. Label shuffling during training discourages position\-specific shortcuts, but a new label is only semantically scorable—not automatically calibrated—until supported by outcome evidence\.

Figure[2](https://arxiv.org/html/2609.02292#S4.F2)visualizes the complete text–label scoring path\.

![Refer to caption](https://arxiv.org/html/2609.02292v1/figures/gliclass_decoder_architecture.png)Figure 2:GLiClass decoder architecture\. A causal decoder processes the text and appended class labels in one sequence\. The final text\-token state and label\-marker states are extracted, fused through cross\-attention, and scored by a shared per\-label MLP\.
### 4\.2Streaming Execution

Each session owns a cache and a classification trigger\. The implementation supports classification on every chunk, afternntokens, on a delimiter, never, or over a sliding window\. Sessions of unequal cache length are left\-padded for batched decoder calls, with masks and explicit positions preserving logical token locations\. When physical cache truncation is enabled, absolute rotary positions continue increasing rather than reusing cropped positions\. Optional CPU offload bounds GPU cache memory\. At the service layer, TTL, LRU capacity, maximum cache length, explicit reset, and deletion govern cache lifetime\.

Cache update and classification are deliberately decoupled\. A nonempty chunk can advance persistent state even when the trigger suppresses scoring; the next triggered call then sees that accumulated context\. Trigger timing and context selection are also separate: a periodic or delimiter trigger can be composed with a bounded recent window\. This makes the deployment trade\-off explicit\. End\-of\-turn triggers reduce redundant intermediate decisions, while per\-chunk triggers provide earlier route changes; windows and physical truncation bound active context at the cost of discarding older evidence\.

The router cache is distinct from the prompt cache of the selected generative endpoint\. It amortizes repeated classification only; after a route change, the destination may still need to replay the conversation\. We therefore expose downstream cache state to the policy instead of treating router\-side reuse as evidence that switching is free\.

Table[1](https://arxiv.org/html/2609.02292#S4.T1)contrasts this design with common alternatives\.

Table 1:Qualitative comparison of router architectures\. “Dynamic labels” means that candidate label text can change at inference time\.ApproachDynamic labelsNew dialogue turnGenerationPrimary trade\-offEmbedding / bi\-encodernativeusually re\-embed full contextnocheapest interaction, weaker cross\-text reasoningEncoder classifier / NLIfixed head or pairwisere\-encode full contextnostrong discrimination, no token\-level session cacheGenerative LLM routerprompt\-nativeprefix cache may be reusedyesflexible and explainable, but decoding and parsing requiredSCX Routerdecoder\-KVnativeencode only new text; re\-score labelsnocompact discriminative path with persistent dialogue state

## 5Task Ontology

We construct the task ontology as a three\-level hierarchy of intent: family, task type, and routable subtype\. It contains 23 families, 115 task types, and 345 subtypes\. Every family contains five task types and 15 subtypes\. We additionally generate 1,173 synthetic example specifications to define coverage and seed downstream task generation; these examples are not benchmark prompts\. An orthogonal domain axis and cross\-cutting routing dimensions allow the same intent hierarchy to describe different application settings without multiplying the stable task labels\. Figure[3](https://arxiv.org/html/2609.02292#S5.F3)depicts these axes and their composition\.

![Refer to caption](https://arxiv.org/html/2609.02292v1/figures/ontology_overview.png)Figure 3:Ontology structure\. Task intent is modeled independently from domain and routing dimensions\. Composing 345 subtypes with 30 domains defines 10,350 initial design cells, but domain\-specific examples are instantiated only when needed\.### 5\.1Intent, Domain, and Cross\-Cutting Dimensions

We separate semantic intent from context to avoid an unstable Cartesian label vocabulary\. “Summarize a clinical handoff” and “summarize a deployment incident” share an operation but differ in domain, risk, artifacts, and likely endpoint requirements\. The domain axis contains eight sectors and 30 domains\. We bind a subtype to a domain scenario only when constructing an example or downstream record\.

We define eight cross\-cutting dimensions: reasoning level \(1–5\), risk \(low through critical\), freshness, interaction mode, input modality, output mode, route kind, and instruction complexity\. These dimensions need not be mutually exclusive task classes\. In particular, multi\-turn and instruction\-following behavior can apply to any semantic family, so we represent them as interaction and constraint dimensions rather than forcing them into the top\-level intent axis\.

We also define boundaries between historically overlapping labels\. Analysis interprets evidence, reasoning derives conclusions, and problem solving proposes and validates a remedy\. Comparison enumerates differences, evaluation scores against criteria, critique identifies weaknesses and improvements, and verification checks truth or acceptance evidence\. Retrieval returns sources, question answering returns an answer, and fact checking returns an evidence\-backed claim status\. These distinctions target the weakest classes in the released 28\-way task\-type evaluation, where conceptually overlapping labels exhibit the largest confusion\.

Table[2](https://arxiv.org/html/2609.02292#S5.T2)summarizes the ontology dimensions and their roles\.

Table 2:Summary of the task ontology\.AxisCountRoleTask families23broad semantic operation, such as retrieval, reasoning, programming, or agentic executionTask types115operational subdivisions; five per familyRoutable subtypes345stable fine\-grained intents; three per task typeSynthetic example specifications1,173generated coverage examples and task\-generation seedsDomain sectors8broad economic or social contextsDomains30reusable contexts composed with task subtypesSubtype–domain design cells10,350possible initial single\-turn generation jobsCross\-cutting dimensions8reasoning, risk, freshness, interaction, modality, output, route kind, and instruction complexity

## 6Data Sources and Supervision

We use two sources of routing supervision\. The initial corpus is benchmark\-derived: candidate models are evaluated on training subsets of existing benchmarks\. The second corpus is generated synthetically from the task ontology\. Here*real\-world*describes task structure—dialogue state, tools, files, repository context, and acceptance criteria—rather than production user\-log provenance\. Table[3](https://arxiv.org/html/2609.02292#S6.T3)separates prompt origin from label meaning\.

Table 3:Data inventory by origin and supervision\. Broad and focused counts are training records; synthetic counts are generated task instances\.Dataset or signalOriginScaleSupervision and meaningBenchmark model routingExisting benchmark train subsets evaluated with candidate models∼\\sim104\.5k broad; 22,303 focusedObserved outcomes identify the best\-performing model or tied models\.Task typeBenchmark prompts classified by LLMs∼\\sim105\.3k broad; 15,015 focusedSemantic operation requested by the task, independent of model identity\.DifficultyCandidate outcomes on the same tasks∼\\sim104\.6k broad; 17,318 focusedFive\-level, roster\-relative solvability: tasks solved by fewer models are harder\.Synthetic, verifier\-scoredOntology\-conditioned synthetic task packages150,000 tasksDeterministic, executable, or environment\-based checks score candidate results\.Synthetic, judge\-scoredOntology\-conditioned synthetic open\-ended tasks15,000 tasksgpt5\.6\-solscores outputs against explicit task criteria\.### 6\.1Benchmark\-Derived Routing Data

The released routing data begins with prompts from the training partitions of existing benchmarks spanning classification, question answering, reasoning, code, summarization, translation, instruction following, multi\-turn interaction\. We execute the candidate models on each prompt and retain the score and observation indicator for every attempted endpoint\. Equation[2](https://arxiv.org/html/2609.02292#S3.E2)converts these observed outcomes into model\-routing targets; endpoint names or descriptions do not determine the target\.

Task type and difficulty have distinct semantics\. LLM annotators classify each benchmark prompt into the task\-type vocabulary according to the requested operation, rather than its source benchmark\. Difficulty is assigned after model evaluation from the relative ability of the candidate roster to solve the task\. It is therefore roster\- and evaluator\-dependent: tasks solved by many models are easier, whereas tasks solved by few models are harder\. A change in candidate roster or success rule can change the difficulty label\.

The released checkpoint uses a broad 524,035\-record GLiClass mixture followed by a focused 65,099\-record routing mixture\. The focused stage contains 22,303 model\-routing, 17,318 difficulty, 15,015 task\-type, 1,797 expected\-output\-length, 890 reasoning\-mode, and approximately 7,700 propositional\-logic records\. These are classification\-record counts, not counts of unique benchmark prompts\. General topic, hallucination, guardrail, sentiment, emotion, NLI, toxicity, safety, and QA\-domain data in the broad mixture preserve zero\-shot classification ability\.

### 6\.2Ontology\-Driven Synthetic Data

The task ontology supplies task family, task type, subtype, domain, and interaction constraints to the generator\. Every task instantiated through this path is synthetic\. Depending on the environment, a package contains a prompt, dialogue or execution context, files, tool interfaces, repository state, assessment criteria, and an evaluator\. This covers single\- and multi\-turn text, tool use, coding, repository\-level work, agentic workflows, and multi\-agent tasks\. The initial generated difficulty is metadata only; the training difficulty label is recomputed from relative model outcomes\.

We generated 150,000 tasks whose results can be verified with deterministic, executable, or environment\-based checks\. We generated a further 15,000 open\-ended tasks for which deterministic verification is unsuitable;gpt5\.6\-solevaluates their candidate outputs against the generated criteria\. The judge identity, criteria, task package, candidate response, and score are retained as evaluation provenance\. Together these pools contain 165,000 synthetic tasks\. Agentic packages follow the reproducibility principle of OSWorld and SWE\-bench: success depends on initial state and executable or rubric\-based evaluation, not on an isolated prompt\[[Xie et al\., 2024](https://arxiv.org/html/2609.02292#bib.bib19),[Jimenez et al\., 2023](https://arxiv.org/html/2609.02292#bib.bib18)\]\.

### 6\.3Outcome Coverage

The expanded evaluation uses eleven endpoint identifiers, but endpoints were evaluated on different numbers of tasks\. The 165,000\-task total therefore does not imply a complete165,000×11165\{,\}000\\times 11outcome matrix\. Every task–model record retains the observation flagoi,mo\_\{i,m\}, and only observed outcomes contribute to routing labels, difficulty estimates, or performance profiles\. Missing evaluations are not failures\. Raw model averages over different task mixtures are not directly comparable, so an 11\-endpoint leaderboard requires a paired task intersection or an explicit missing\-outcome estimator\.

## 7Training

The released decoder\-KV model is trained for multi\-label classification with shuffled label order\. For each row, only labels inall\_labelsare tokenized into the suffix, and binary targets indicate the selected winners\. The implementation supports standard binary cross entropy when focal parameters are disabled, as well as configurable focal modulation\[[Lin et al\., 2017](https://arxiv.org/html/2609.02292#bib.bib20)\]\. The exact focal arguments, optimizer state, and complete launch command are not available, so we leave those fields unspecified\.

Table[4](https://arxiv.org/html/2609.02292#S7.T4)summarizes the available training configuration\.

Table 4:Training configuration for the released checkpoint\. Unavailable fields are marked accordingly\.HyperparameterValueBackboneQwen3\-0\.6BArchitecturedecoder\-KV GLiClassProblem typemulti\-label classificationMaximum sequence length4,096Scorer encoder layers / heads2 / 16Scorer MLP hidden size1,024Dropout / scorer attention dropout0\.1 / 0\.1Stage\-1 epochs / steps3 / 18,300Stage\-1 per\-device batch size4Stage\-1 schedulelinear with warmupStage\-2 decoder / scorer learning rate10−610^\{\-6\}/10−610^\{\-6\}Precisionbfloat16Gradient checkpointingenabledTrain/evaluation split90/10Seed / label shuffling42 / enabledTraining hardware, global batch, wall timenot availableBroad pre\-training reaches flattened weighted binary accuracy 0\.9273, F1 0\.9241, precision 0\.9238, recall 0\.9273, and loss 0\.218\. These are label\-decision metrics rather than example\-level exact\-match routing accuracy; many negative model labels can dominate them\. We therefore use the routing\-family and downstream results in Section[10](https://arxiv.org/html/2609.02292#S10)as the more relevant evidence\.

## 8Model\-Routing Patterns

The classifier supports four routing patterns, chosen according to endpoint stability, outcome\-data coverage, and whether execution is a single call or a workflow\. All patterns first restrict candidates to an eligible setℰt⊆ℳt\\mathcal\{E\}\_\{t\}\\subseteq\\mathcal\{M\}\_\{t\}; context, modality, tools, privacy, residency, and safety remain hard constraints\. Figure[4](https://arxiv.org/html/2609.02292#S8.F4)summarizes the designs\.

![Refer to caption](https://arxiv.org/html/2609.02292v1/figures/routing_patterns.png)Figure 4:Major model\-routing patterns\. \(A\) Direct routing scores endpoint labels from the request\. \(B\) Attribute\-mediated routing predicts task, difficulty, and domain, then consults performance profiles\. \(C\) Hybrid routing combines direct and profile evidence before deployment policy\. \(D\) Hierarchical routing selects a planner and routes each typed task node with verification feedback\. Only the direct path has released end\-to\-end evidence; hard top\-1 profile aggregation is implemented without end\-to\-end results, while posterior, hybrid, and agentic variants are proposed\. Unequal endpoint coverage does not establish their relative merit\.### 8\.1Direct Endpoint Routing

The direct pattern presents endpoint descriptions as classifier labels and produces one suitability score for each eligible endpoint:

Qdir​\(m∣xt,h<t\)=σ⁡\(sm​\(xt,h<t,ℳt\)\),m∈ℰt,Q\_\{\\mathrm\{dir\}\}\(m\\mid x\_\{t\},h\_\{<t\}\)=\\sigma\\\!\\left\(s\_\{m\}\(x\_\{t\},h\_\{<t\};\\mathcal\{M\}\_\{t\}\)\\right\),\\qquad m\\in\\mathcal\{E\}\_\{t\},\(12\)wheresms\_\{m\}is the decoder\-KV classifier logit\. A quality\-only policy takes the highest score, while production passes the scores to Section[9](https://arxiv.org/html/2609.02292#S9)\. Dynamic labels permit endpoint changes without modifying the head, but unseen endpoint names remain uncalibrated without measured outcomes\. The released eight\-endpoint checkpoint implements and evaluates this path\.

### 8\.2Attribute\-Mediated Performance Routing

When endpoints change faster than router training, the classifier can predict stable attributes and use a replaceable performance table\. Let𝒟=\{task,difficulty,domain\}\\mathcal\{D\}=\\\{\\text\{task\},\\text\{difficulty\},\\text\{domain\}\\\}, let𝒵d\\mathcal\{Z\}\_\{d\}denote the categories of dimensiondd, and letμm,d,z\\mu\_\{m,d,z\}be modelmm’s historical mean outcome for categoryzz\. The current optimizer selects the top category in each dimension,

z^d=arg⁡maxz∈𝒵d​πd​\(z∣xt,h<t\),\\widehat\{z\}\_\{d\}=\\arg\\max\_\{z\\in\\mathcal\{Z\}\_\{d\}\}\\pi\_\{d\}\(z\\mid x\_\{t\},h\_\{<t\}\),\(13\)and computes

Qprofhard​\(m∣xt,h<t\)=1W​∑d∈𝒟wd​μm,d,z^d,W=∑d∈𝒟wd\.Q\_\{\\mathrm\{prof\}\}^\{\\mathrm\{hard\}\}\(m\\mid x\_\{t\},h\_\{<t\}\)=\\frac\{1\}\{W\}\\sum\_\{d\\in\\mathcal\{D\}\}w\_\{d\}\\mu\_\{m,d,\\widehat\{z\}\_\{d\}\},\\qquad W=\\sum\_\{d\\in\\mathcal\{D\}\}w\_\{d\}\.\(14\)Missing cells fall back to the model’s overall mean\. This decouples semantic prediction from the model–task table, but hard boundaries and sparse cells can mislead\. The path is implemented but lacks end\-to\-end results\.

A proposed probabilistic extension instead marginalizes attribute uncertainty:

Qprofpost​\(m∣xt,h<t\)=1W​∑d∈𝒟wd​∑z∈𝒵dπd​\(z∣xt,h<t\)​μ~m,d,z\.Q\_\{\\mathrm\{prof\}\}^\{\\mathrm\{post\}\}\(m\\mid x\_\{t\},h\_\{<t\}\)=\\frac\{1\}\{W\}\\sum\_\{d\\in\\mathcal\{D\}\}w\_\{d\}\\sum\_\{z\\in\\mathcal\{Z\}\_\{d\}\}\\pi\_\{d\}\(z\\mid x\_\{t\},h\_\{<t\}\)\\widetilde\{\\mu\}\_\{m,d,z\}\.\(15\)Hereμ~\\widetilde\{\\mu\}should shrink sparse cells toward a model or task prior\. Unequal endpoint coverage additionally requires confidence intervals, common\-task comparisons, or missing\-outcome modeling\. Equation[15](https://arxiv.org/html/2609.02292#S8.E15)is therefore proposed rather than evaluated\.

### 8\.3Hybrid and Adaptive Routing

Direct scores capture request–endpoint interactions, while profiles incorporate new outcome data without retraining\. After calibration, a hybrid combines both:

Qhyb​\(m∣xt,h<t\)=γt​norm⁡\(Qdir​\(m\)\)\+\(1−γt\)​norm⁡\(Qprofpost​\(m\)\),Q\_\{\\mathrm\{hyb\}\}\(m\\mid x\_\{t\},h\_\{<t\}\)=\\gamma\_\{t\}\\,\\operatorname\{norm\}\\\!\\left\(Q\_\{\\mathrm\{dir\}\}\(m\)\\right\)\+\(1\-\\gamma\_\{t\}\)\\,\\operatorname\{norm\}\\\!\\left\(Q\_\{\\mathrm\{prof\}\}^\{\\mathrm\{post\}\}\(m\)\\right\),\(16\)The held\-out parameterγt∈\[0,1\]\\gamma\_\{t\}\\in\[0,1\]may depend on score margin, entropy, profile support, drift, or endpoint novelty, allowing reweighting or abstention\. Fusion follows eligibility filtering and precedes cost, latency, cache, and capacity policy\.

A confidence\-gated cascade invokes an economical model and escalates when a verifier predicts failure, following FrugalGPT and AutoMix\[[Chen et al\., 2023](https://arxiv.org/html/2609.02292#bib.bib7),[Aggarwal et al\., 2023](https://arxiv.org/html/2609.02292#bib.bib8)\]\. A parallel portfolio instead executes top\-kkcandidates and judges or synthesizes their outputs\. Both require extra resources and a deployable verifier; therouter@​k\\text\{router@\}kresults in Section[10](https://arxiv.org/html/2609.02292#S10)are oracle diagnostics, not portfolio evidence\. Contextual\-bandit adaptation is another proposed option, requiring controlled exploration, delayed\-feedback handling, and safety boundaries\.

### 8\.4Hierarchical Agentic Routing

An agentic system adds decisions beyond the initial prompt\. A first gate chooses direct execution or decomposition; for the latter, the router selects a planner that creates a typed task graph\. Each node contextcvc\_\{v\}contains the objective, local instruction, predecessor outputs, files and tools, acceptance criteria, risk, and remaining budget\. A role\-aware policy chooses

mv∗=arg⁡maxm∈ℰv⁡U⁡\(m∣cv,rv\),rv∈\{plan,execute,verify,synthesize\}\.m\_\{v\}^\{\*\}=\\arg\\max\_\{m\\in\\mathcal\{E\}\_\{v\}\}U\(m\\mid c\_\{v\},r\_\{v\}\),\\qquad r\_\{v\}\\in\\\{\\text\{plan\},\\text\{execute\},\\text\{verify\},\\text\{synthesize\}\\\}\.\(17\)Per\-node routing permits different specialists for planning, execution, verification, and synthesis\. Verification may accept, retry, switch models, escalate compute, or replan, subject to explicit budgets and loop limits\. The 165,000 synthetic tasks in Section[6](https://arxiv.org/html/2609.02292#S6)provide context, artifacts, and criteria for future evaluation, but hierarchical routing has not been evaluated end to end\.

Table[5](https://arxiv.org/html/2609.02292#S8.T5)summarizes the evidence available for each routing pattern\.

Table 5:Routing patterns and their evidence status in this work\. “Proposed” denotes a design, not an empirical result\.PatternDecision signalExecutionEvidence statusDirect endpointPrompt\-conditioned endpoint\-label scoresOne selected endpointReleased and evaluated for eight endpointsHard attribute profileTop\-1 task, difficulty, and domain; historical tableOne selected endpointComponents implemented; no end\-to\-end resultPosterior profileAttribute posteriors and support\-aware tableOne selected endpointProposed extensionHybrid constrainedCalibrated direct and profile scores plus policy stateOne selected endpointProposed compositionAdaptive cascadeInitial route plus confidence or verifier feedbackSequential escalationProposed compositionParallel portfolioTop\-kkcandidates plus a deployable judgeParallel calls and selectionProposed; evaluatedrouter@​k\\text\{router@\}kis oracle\-basedPlanner–workerDecomposition, role, node state, and verifier feedbackRouted task graphProposed; agentic data is an evaluation substrate

## 9Cache\-Aware Routing Policy

Each single\-endpoint pattern in Section[8](https://arxiv.org/html/2609.02292#S8)supplies a performance signal, not an immutable product policy\. After applying hard eligibility constraints, a deployment can combine that signal with incremental monetary cost and cache state\. The same policy can also score planner and worker choices in Equation[17](https://arxiv.org/html/2609.02292#S8.E17)\.

LetNcN\_\{c\}be cached conversation tokens,NnN\_\{n\}new input tokens, andN^o\\widehat\{N\}\_\{o\}expected output tokens\. Each modelmmhas full input pricepin​\(m\)p\_\{\\mathrm\{in\}\}\(m\), cached\-input pricepcache​\(m\)p\_\{\\mathrm\{cache\}\}\(m\), and output pricepout​\(m\)p\_\{\\mathrm\{out\}\}\(m\)per million tokens\. Ifmcm\_\{c\}is the current generation model andqqis the probability that its cache is reusable, the effective history rate is

rh​\(m\)=\{q​pcache​\(m\)\+\(1−q\)​pin​\(m\),m=mc,pin​\(m\),m≠mc\.r\_\{h\}\(m\)=\\begin\{cases\}q\\,p\_\{\\mathrm\{cache\}\}\(m\)\+\(1\-q\)p\_\{\\mathrm\{in\}\}\(m\),&m=m\_\{c\},\\\\ p\_\{\\mathrm\{in\}\}\(m\),&m\\neq m\_\{c\}\.\\end\{cases\}\(18\)The next\-request cost is

Ct​\(m\)=rh​\(m\)​Nc\+pin​\(m\)​Nn\+pout​\(m\)​N^o106\.C\_\{t\}\(m\)=\\frac\{r\_\{h\}\(m\)N\_\{c\}\+p\_\{\\mathrm\{in\}\}\(m\)N\_\{n\}\+p\_\{\\mathrm\{out\}\}\(m\)\\widehat\{N\}\_\{o\}\}\{10^\{6\}\}\.\(19\)Previously billed tokens are sunk; Equation[19](https://arxiv.org/html/2609.02292#S9.E19)prices only the next action\. Switching models replays history at the new candidate’s full input rate, making long cache\-warm conversations naturally sticky\.

The selector normalizes predicted performance and the natural logarithm of cost to\[0,1\]\[0,1\], inverts cost so larger is better, and computes

Ut​\(m\)=α​Uperf​\(m\)\+\(1−α\)​Ucost​\(m\)\.U\_\{t\}\(m\)=\\alpha U\_\{\\mathrm\{perf\}\}\(m\)\+\(1\-\\alpha\)U\_\{\\mathrm\{cost\}\}\(m\)\.\(20\)Normalization uses historical mean±3​σ\\pm 3\\sigmawhen available, configured bounds otherwise, and candidate min–max as a fallback\. Degenerate ranges yield neutral utilities\. Ties break by higher raw performance, lower cost, and then model identifier, making the decision deterministic\. One configurable multi\-signal example assigns weights 0\.60 to suitability, 0\.15 to task fit, 0\.15 to difficulty fit, and 0\.10 to reasoning fit\. These weights are illustrative, not learned constants or universal defaults\.

Safety, privacy, data residency, tool permissions, and context feasibility should remain outside Equation[20](https://arxiv.org/html/2609.02292#S9.E20)as hard filters\. A high utility must never compensate for ineligibility\.

## 10Evaluation

### 10\.1Metrics

We evaluate the released model at two levels: per\-family classification and end\-to\-end fixed\-versus\-router comparisons\. The former measures the prediction heads; the latter measures realized downstream task quality\.

For multi\-label model routing, we compute per\-candidate precision, recall, and F1 at threshold 0\.5 and take their unweighted macro average\. For single\-label auxiliary families, macro F1 is averaged across classes\. A separate 3,192\-record evaluation uses a different aggregation and obtains model\-routing F1 0\.805, whereas the explicit per\-candidate macro average is 0\.759\. Because the protocols differ, we keep these values separate and do not interpret the difference as a temporal regression\.

Figure[5](https://arxiv.org/html/2609.02292#S10.F5)and Table[6](https://arxiv.org/html/2609.02292#S10.T6)summarize the released\-checkpoint classification families\.

![Refer to caption](https://arxiv.org/html/2609.02292v1/figures/head_metrics.png)Figure 5:Macro F1 by released\-checkpoint classification family\. The values summarize distinct label families and are not averaged into a single system score\.Table 6:Released\-checkpoint metrics\. The hallucination score is a directional response signal\.FamilyClassesMacro F1Decision viewModel suitability80\.759multi\-label, threshold 0\.5Task type280\.837single\-labelDifficulty50\.789single\-label, ordinalReasoning mode20\.897single\-labelExpected output length70\.788single\-label, ordinal bucketsHallucination2∼\\sim0\.65directional response signal
### 10\.2Model\-Routing Classification

Table[7](https://arxiv.org/html/2609.02292#S10.T7)shows the released eight\-candidate model\-routing result\. Precision is comparatively stable \(0\.739–0\.840\), whereas recall ranges from 0\.650 to 0\.950\. This matters operationally: a high\-recall generalist appears in more positive sets, while a specialist can have high precision but be missed on tasks it could solve\. Threshold tuning should therefore be candidate\- and objective\-aware rather than assumed universal\.

Table 7:Per\-candidate routing precision, recall, and F1\. Endpoint names are shortened only for display\.CandidatePrecisionRecallF1Gemma 4 31B0\.80640\.95010\.8723Llama 4 Maverick0\.78920\.75290\.7706GPT\-OSS 120B0\.83960\.70630\.7672Coder0\.81400\.70000\.7527Llama 3\.3 70B0\.76380\.72390\.7433DeepSeek V3\.10\.74750\.72200\.7346MAGPiE0\.73940\.72340\.7313Qwen3 32B0\.75050\.64990\.6966Macro mean0\.76880\.74110\.7586The 28\-way task taxonomy is near\-perfect on semantically distinct classes such as analysis, code, generation, information extraction, QA, reasoning, summarization, and translation\. The principal failure is problem solving \(F1 0\.127\), followed by math\-and\-reasoning \(0\.537\), decision support \(0\.599\), information retrieval \(0\.626\), and evaluation \(0\.642\)\. These failures motivate the ontology’s explicit boundary notes and its decision to move interaction properties out of the intent hierarchy\.

Difficulty is ordinal\. The released\-checkpoint evaluation obtains macro F1 0\.789, with most errors between adjacent hard and extra\-hard levels\. The separate 3,192\-record evaluation gives top\-1, top\-2, and top\-3 hit rates of 0\.507, 0\.748, and 0\.873, respectively\. Because the protocols differ, we do not merge the values; both motivate distance\-aware ordinal error alongside exact class F1\.

### 10\.3End\-to\-End Routing

We use two end\-to\-end baselines\. The first compares the routed top\-1 score with the arithmetic mean of eight candidate scores on six LiveBench subsets\[[White et al\., 2024](https://arxiv.org/html/2609.02292#bib.bib10)\]\. This measures improvement over a uniformly sampled candidate, not over a strong fixed policy\. Table[8](https://arxiv.org/html/2609.02292#S10.T8)and Figure[6](https://arxiv.org/html/2609.02292#S10.F6)give the results\.

![Refer to caption](https://arxiv.org/html/2609.02292v1/figures/livebench_gains.png)Figure 6:LiveBench gain over the mean of eight candidate scores\. This experiment evaluates an earlier system checkpoint and is not a final\-checkpoint claim\.Table 8:Prior LiveBench routed score versus the mean candidate score, 100 tasks per subset\.SubsetRouterMean of 8GainLanguage0\.7790\.517\+0\.262Math0\.7380\.555\+0\.183Instruction following0\.8630\.768\+0\.095Reasoning0\.6010\.551\+0\.050Coding0\.5000\.456\+0\.044Data analysis0\.5400\.539\+0\.001The second baseline is stricter\.fixed@kselects the globally top\-kkcandidates by mean score on the same evaluation records;router@kforms a per\-task top\-kkshortlist and takes the best*realized*outcome within that shortlist\. Atk=1k=1, both are deployable single\-model policies\. Fork\>1k\>1,router@kis an oracle\-within\-shortlist diagnostic unless an online selector is specified\. The globally fixed ranking is selected in\-sample and can therefore be optimistic\.

The full evaluation contains 1,500 tasks across 13 benchmarks\. The dataset\-level analysis uses a selected 1,000\-task subset comprising LiveBench and non\-LiveBench datasets with a positive router gain at one or more depths\. On this subset, fixed/router scores are 0\.696/0\.707 atk=1k=1, 0\.788/0\.794 atk=2k=2, and 0\.837/0\.824 atk=3k=3, giving gains of\+0\.012\+0\.012,\+0\.007\+0\.007, and−0\.013\-0\.013\. Table[9](https://arxiv.org/html/2609.02292#S10.T9)gives the top\-1 dataset values\.

Table 9:Top\-1 fixed\-versus\-router diagnostic\. “All shown” is sample\-weighted over the selected 1,000\-task subset, not the full 1,500\-task evaluation\.DatasetFixed@1Router@1GainBBEH\[[Kazemi et al\., 2025](https://arxiv.org/html/2609.02292#bib.bib11)\]0\.8300\.830\+0\.000MuSR\[[Sprague et al\., 2023](https://arxiv.org/html/2609.02292#bib.bib12)\]0\.6770\.683\+0\.007LiveBench coding0\.4700\.500\+0\.030LiveBench data analysis0\.5600\.550−0\.010\-0\.010LiveBench instruction following\[[Zhou et al\., 2023](https://arxiv.org/html/2609.02292#bib.bib13)\]0\.8340\.854\+0\.020LiveBench language0\.6590\.726\+0\.067LiveBench math0\.7380\.738\+0\.000LiveBench reasoning0\.8340\.824−0\.010\-0\.010All shown0\.6960\.707\+0\.012These results support a narrower conclusion than “routing always wins\.” The router is most useful where candidate outcomes disagree and request semantics reveal that disagreement; it adds little where candidates cluster and can underperform a fixed baseline\. Because the 1,000\-task analysis subset omits five benchmarks and lacks uncertainty intervals, repeated seeds, endpoint versions, and held\-out fixed\-policy selection, it does not support a global significance claim\. We do not report downstream comparisons for the unequally covered eleven\-endpoint collection\.

## 11Discussion

SCX Routerdemonstrates a practical benefit of treating model selection as compact, dynamic\-label classification rather than as another generation task\. A small router can score a changing endpoint roster, reuse conversational context through the decoder\-KV path, and expose task, difficulty, reasoning, and output\-length signals to an explicit deployment policy\. This separation makes eligibility, safety, price, latency, and cache reuse controllable rather than implicit in a single model prediction\. The task ontology and 165,000 synthetic tasks further extend this interface from conventional prompts to tool use, repository work, and agentic workflows\.

The empirical advantage is nevertheless conditional\. Routing helps most when candidate models disagree and the request contains signals that predict those differences; a strong fixed model remains competitive when outcomes are similar\. Current end\-to\-end evidence covers direct routing on a selected subset, while unequal coverage prevents a balanced comparison across all eleven expanded endpoints\. Synthetic generation, verifier design, andgpt5\.6\-soljudging can introduce author\-model, evaluation\-shortcut, and judge biases\[[Zheng et al\., 2023](https://arxiv.org/html/2609.02292#bib.bib14),[Verga et al\., 2024](https://arxiv.org/html/2609.02292#bib.bib15)\]\. The full task corpus and outcome matrix are also not yet public\. The router should therefore be treated as a performance signal inside a constrained policy, with safety, privacy, residency, and tool authorization enforced independently\.

Future work should add more model families, sizes, modalities, context lengths, tool\-use capabilities, and price–latency tiers, with every endpoint evaluated on shared, versioned task strata\. The four routing patterns should then be compared on the same outcome matrix: direct endpoint prediction, attribute\-mediated performance profiles, hybrid scoring, and hierarchical planner–worker routing for agentic tasks\. Evaluation should measure realized quality, cost, latency, calibration, endpoint\-drift robustness, and regret to an oracle\. Logged propensities, controlled exploration, and shadow evaluation are also needed to improve the router without reinforcing its own selection bias\. This program would reveal where each routing pattern is useful and turn the ontology expansion into a reproducible benchmark\.

## 12Conclusion

SCX Routerframes model selection as lightweight, dynamic\-label classification over an evolving endpoint roster\. The approximately 0\.6B\-parameter checkpoint combines request\-conditioned model scoring with task, difficulty, reasoning, and output\-length predictions, while its decoder\-KV path supports repeated routing over conversational state\. On the selected 1,000\-task evaluation subset, direct routing reaches a top\-1 score of 0\.707 compared with 0\.696 for the strongest fixed endpoint\. Together with the per\-head classification results, this shows that a compact router can exploit predictable model specialization without requiring an additional generative decision step\. The mixed per\-benchmark gains also clarify that this advantage depends on meaningful disagreement among candidate models\.

The task ontology, 150,000 verifier\-scored tasks, and 15,000gpt5\.6\-sol\-judged tasks broaden the routing target toward realistic applications and agentic workflows\. The routing\-pattern framework connects this data to direct, attribute\-mediated, hybrid, and hierarchical planner–worker strategies while keeping deployment constraints separate from learned predictions\. This work therefore provides a foundation rather than a final universal leaderboard\. A paired and versioned outcome matrix across more models, followed by realized quality–cost–latency evaluation of each routing pattern, is the central requirement for establishing when routing delivers reliable operational value\.

## Acknowledgements

This work was developed jointly by SCX\.ai Holdings Limited and Knowledgator\.

## References

- Aggarwalet al\.\(2023\)P\. Aggarwal, A\. Madaan, A\. Anand, S\. P\. Potharaju, S\. Mishra, P\. Zhou, A\. Gupta, D\. Rajagopal, K\. Kappaganthu, Y\. Yang, S\. Upadhyay, M\. Faruqui, and MausamAutoMix: automatically mixing language models\.arXiv preprint arXiv:2310\.12963\.External Links:[Link](https://arxiv.org/abs/2310.12963)Cited by:[§1](https://arxiv.org/html/2609.02292#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.02292#S2.SS1.p1.1),[§8\.3](https://arxiv.org/html/2609.02292#S8.SS3.p2.1)\.
- Chenet al\.\(2023\)L\. Chen, M\. Zaharia, and J\. ZouFrugalGPT: how to use large language models while reducing cost and improving performance\.arXiv preprint arXiv:2305\.05176\.External Links:[Link](https://arxiv.org/abs/2305.05176)Cited by:[§1](https://arxiv.org/html/2609.02292#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.02292#S2.SS1.p1.1),[§8\.3](https://arxiv.org/html/2609.02292#S8.SS3.p2.1)\.
- Dekonincket al\.\(2024\)J\. Dekoninck, M\. Baader, and M\. VechevA unified approach to routing and cascading for LLMs\.arXiv preprint arXiv:2410\.10347\.External Links:[Link](https://arxiv.org/abs/2410.10347)Cited by:[§1](https://arxiv.org/html/2609.02292#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.02292#S2.SS1.p1.1)\.
- Heet al\.\(2020\)P\. He, X\. Liu, J\. Gao, and W\. ChenDeBERTa: decoding\-enhanced BERT with disentangled attention\.arXiv preprint arXiv:2006\.03654\.External Links:[Link](https://arxiv.org/abs/2006.03654)Cited by:[§2\.2](https://arxiv.org/html/2609.02292#S2.SS2.p1.1)\.
- Huet al\.\(2024\)Q\. J\. Hu, J\. Bieker, X\. Li, N\. Jiang, B\. Keigwin, G\. Ranganath, K\. Keutzer, and S\. K\. UpadhyayRouterBench: a benchmark for multi\-LLM routing system\.arXiv preprint arXiv:2403\.12031\.External Links:[Link](https://arxiv.org/abs/2403.12031)Cited by:[§1](https://arxiv.org/html/2609.02292#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.02292#S2.SS1.p1.1)\.
- Jimenezet al\.\(2023\)C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. NarasimhanSWE\-bench: can language models resolve real\-world GitHub issues?\.arXiv preprint arXiv:2310\.06770\.External Links:[Link](https://arxiv.org/abs/2310.06770)Cited by:[§2\.3](https://arxiv.org/html/2609.02292#S2.SS3.p1.1),[§6\.2](https://arxiv.org/html/2609.02292#S6.SS2.p2.1)\.
- Kazemiet al\.\(2025\)M\. Kazemi, B\. Fatemi, H\. Bansal, J\. Palowitch, C\. Anastasiou, S\. V\. Mehta, L\. K\. Jain, V\. Aglietti, D\. Jindal, P\. Chen, N\. Dikkala, G\. Tyen, X\. Liu, U\. Shalit, S\. Chiappa, K\. Olszewska, Y\. Tay, V\. Q\. Tran, Q\. V\. Le, and O\. FiratBIG\-Bench Extra Hard\.arXiv preprint arXiv:2502\.19187\.External Links:[Link](https://arxiv.org/abs/2502.19187)Cited by:[Table 9](https://arxiv.org/html/2609.02292#S10.T9.5.2.1.1)\.
- Linet al\.\(2017\)T\. Lin, P\. Goyal, R\. Girshick, K\. He, and P\. DollárFocal loss for dense object detection\.InProceedings of the IEEE International Conference on Computer Vision,External Links:[Link](https://arxiv.org/abs/1708.02002)Cited by:[§7](https://arxiv.org/html/2609.02292#S7.p1.1)\.
- Liuet al\.\(2023\)X\. Liu, H\. Yu, H\. Zhang, Y\. Xu, X\. Lei, H\. Lai, Y\. Gu, H\. Ding, K\. Men, K\. Yang,et al\.AgentBench: evaluating LLMs as agents\.arXiv preprint arXiv:2308\.03688\.External Links:[Link](https://arxiv.org/abs/2308.03688)Cited by:[§2\.3](https://arxiv.org/html/2609.02292#S2.SS3.p1.1)\.
- Mialonet al\.\(2023\)G\. Mialon, C\. Fourrier, C\. Swift, T\. Wolf, Y\. LeCun, and T\. ScialomGAIA: a benchmark for general AI assistants\.arXiv preprint arXiv:2311\.12983\.External Links:[Link](https://arxiv.org/abs/2311.12983)Cited by:[§2\.3](https://arxiv.org/html/2609.02292#S2.SS3.p1.1)\.
- Onget al\.\(2024\)I\. Ong, A\. Almahairi, V\. Wu, W\. Chiang, T\. Wu, J\. E\. Gonzalez, M\. W\. Kadous, and I\. StoicaRouteLLM: learning to route LLMs with preference data\.arXiv preprint arXiv:2406\.18665\.External Links:[Link](https://arxiv.org/abs/2406.18665)Cited by:[§1](https://arxiv.org/html/2609.02292#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.02292#S2.SS1.p1.1)\.
- Qwen Team \(2025\)Qwen TeamQwen3 technical report\.arXiv preprint arXiv:2505\.09388\.External Links:[Link](https://arxiv.org/abs/2505.09388)Cited by:[§2\.2](https://arxiv.org/html/2609.02292#S2.SS2.p1.1),[§4](https://arxiv.org/html/2609.02292#S4.p1.1)\.
- Spragueet al\.\(2023\)Z\. Sprague, X\. Ye, K\. Bostrom, S\. Chaudhuri, and G\. DurrettMuSR: testing the limits of chain\-of\-thought with multistep soft reasoning\.arXiv preprint arXiv:2310\.16049\.External Links:[Link](https://arxiv.org/abs/2310.16049)Cited by:[Table 9](https://arxiv.org/html/2609.02292#S10.T9.5.3.1.1)\.
- Stepanovet al\.\(2025\)I\. Stepanov, M\. Shtopko, D\. Vodianytskyi, O\. Lukashov, A\. Yavorskyi, and M\. YaroshenkoGLiClass: generalist lightweight model for sequence classification tasks\.arXiv preprint arXiv:2508\.07662\.External Links:[Link](https://arxiv.org/abs/2508.07662)Cited by:[§1](https://arxiv.org/html/2609.02292#S1.p2.1),[§2\.2](https://arxiv.org/html/2609.02292#S2.SS2.p1.1)\.
- Vergaet al\.\(2024\)P\. Verga, S\. Hofstatter, S\. Althammer, Y\. Su, A\. Piktus, A\. Arkhangorodsky, M\. Xu, N\. White, and P\. LewisReplacing judges with juries: evaluating LLM generations with a panel of diverse models\.arXiv preprint arXiv:2404\.18796\.External Links:[Link](https://arxiv.org/abs/2404.18796)Cited by:[§11](https://arxiv.org/html/2609.02292#S11.p2.1),[§2\.3](https://arxiv.org/html/2609.02292#S2.SS3.p2.1)\.
- Whiteet al\.\(2024\)C\. White, S\. Dooley, M\. Roberts, A\. Pal, B\. Feuer, S\. Jain, R\. Shwartz\-Ziv, N\. Jain, K\. Saifullah, S\. Naidu, C\. Hegde, Y\. LeCun, T\. Goldstein, W\. Neiswanger, and M\. GoldblumLiveBench: a challenging, contamination\-free LLM benchmark\.arXiv preprint arXiv:2406\.19314\.External Links:[Link](https://arxiv.org/abs/2406.19314)Cited by:[§10\.3](https://arxiv.org/html/2609.02292#S10.SS3.p1.1)\.
- Xieet al\.\(2024\)T\. Xie, D\. Zhang, J\. Chen, X\. Li, S\. Zhao, R\. Cao, T\. J\. Hua, Z\. Cheng, D\. Shin, F\. Lei,et al\.OSWorld: benchmarking multimodal agents for open\-ended tasks in real computer environments\.arXiv preprint arXiv:2404\.07972\.External Links:[Link](https://arxiv.org/abs/2404.07972)Cited by:[§2\.3](https://arxiv.org/html/2609.02292#S2.SS3.p1.1),[§6\.2](https://arxiv.org/html/2609.02292#S6.SS2.p2.1)\.
- Zaratianaet al\.\(2023\)U\. Zaratiana, N\. Tomeh, P\. Holat, and T\. CharnoisGLiNER: generalist model for named entity recognition using bidirectional transformer\.arXiv preprint arXiv:2311\.08526\.External Links:[Link](https://arxiv.org/abs/2311.08526)Cited by:[§1](https://arxiv.org/html/2609.02292#S1.p2.1),[§2\.2](https://arxiv.org/html/2609.02292#S2.SS2.p1.1)\.
- Zhenget al\.\(2023\)L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. StoicaJudging LLM\-as\-a\-judge with MT\-Bench and Chatbot Arena\.arXiv preprint arXiv:2306\.05685\.External Links:[Link](https://arxiv.org/abs/2306.05685)Cited by:[§11](https://arxiv.org/html/2609.02292#S11.p2.1),[§2\.3](https://arxiv.org/html/2609.02292#S2.SS3.p2.1)\.
- Zhouet al\.\(2023\)J\. Zhou, T\. Lu, S\. Mishra, S\. Brahma, S\. Basu, Y\. Luan, D\. Zhou, and L\. HouInstruction\-following evaluation for large language models\.arXiv preprint arXiv:2311\.07911\.External Links:[Link](https://arxiv.org/abs/2311.07911)Cited by:[Table 9](https://arxiv.org/html/2609.02292#S10.T9.5.6.1.1)\.

## Appendix AOntology Families

Table[10](https://arxiv.org/html/2609.02292#A1.T10)lists the complete top\-level ontology\. Its deliberately regular construction assigns five task types, 15 subtypes, and 51 synthetic example specifications to each family\.

Table 10:Complete task\-family inventory\.IDFamilyTypesSubtypesTasksF01Information Retrieval and Discovery51551F02Question Answering and Explanation51551F03Summarization and Synthesis51551F04Information Extraction and Structuring51551F05Classification and Tagging51551F06Clustering and Organization51551F07Analysis and Insight51551F08Comparison and Benchmarking51551F09Evaluation and Scoring51551F10Critique and Review51551F11Fact\-Checking and Verification51551F12Logical, Mathematical, and Causal Reasoning51551F13Problem Solving and Troubleshooting51551F14Forecasting and Estimation51551F15Decision Support and Recommendation51551F16Planning, Scheduling, and Optimization51551F17Writing, Rewriting, and Translation51551F18Content Generation and Visualization51551F19Programming and Software Engineering51551F20Data, Quantitative, and Scientific Computing51551F21Agentic Tool Use and Workflow Execution51551F22Conversation and Customer Support51551F23Safety, Risk, and Compliance51551Total1153451,173The 30 domains are grouped into Public, Social, and Education Services \(3\); Health and Life Sciences \(2\); Financial, Legal, and People Services \(5\); Digital Technology and Communications \(6\); Industry, Infrastructure, and Property \(5\); Commerce, Logistics, and Hospitality \(4\); Environment and Primary Production \(2\); and Research, Media, and Sport \(3\)\.