SWRouter: Similarity-Contractive Window Routing for Multi-Turn Large Language Model Conversations

# SWRouter: Similarity-Contractive Window Routing for Multi-Turn Large Language Model Conversations
Source: [https://arxiv.org/html/2609.11414](https://arxiv.org/html/2609.11414)
CCS:Information systems Retrieval models and rankingCCS:Computing methodologies Natural language generationCCS:Computing methodologies Machine learning approachesYu Wang[https://orcid.org/0009-0002-9928-1246](https://orcid.org/0009-0002-9928-1246)email:[wangyv123@sjtu\.edu\.cn](mailto:[email protected])Affiliation:Department of Big Data Management and Application, Shanghai Jiao Tong University,Shanghai,ChinaYuchen Li[https://orcid.org/0000-0002-3869-7881](https://orcid.org/0000-0002-3869-7881)email:[yuchenli1230@gmail\.com](mailto:[email protected])Affiliation:School of Computer Science, Shanghai Jiao Tong University,Shanghai,ChinaAffiliation:Baidu Inc\.,Beijing,China,Rui Kong[https://orcid.org/0009-0003-2889-2266](https://orcid.org/0009-0003-2889-2266)email:[monster119120@gmail\.com](mailto:[email protected])Affiliation:Baidu Inc\.,Beijing,China,Xinran Chen[https://orcid.org/0009-0009-9071-1933](https://orcid.org/0009-0009-9071-1933)email:[fantastique0910@gmail\.com](mailto:[email protected])Affiliation:Baidu Inc\.,Beijing,China,Jiamin Chen[https://orcid.org/0009-0003-6674-7764](https://orcid.org/0009-0003-6674-7764)email:[jmchen26\-c@my\.cityu\.edu\.hk](mailto:[email protected])Affiliation:Baidu Inc\.,Beijing,China,Hengyi Cai[https://orcid.org/0000-0002-7147-5666](https://orcid.org/0000-0002-7147-5666)email:[hengyi1995@gmail\.com](mailto:[email protected])Affiliation:Baidu Inc\.,Beijing,China,Shuaiqiang Wang[https://orcid.org/0000-0002-9212-1947](https://orcid.org/0000-0002-9212-1947)email:[shqiang\.wang@gmail\.com](mailto:[email protected])Affiliation:Baidu Inc\.,Beijing,China,Jiashu Zhao[https://orcid.org/0000-0002-1241-0686](https://orcid.org/0000-0002-1241-0686)email:[jzhao@wlu\.ca](mailto:[email protected])Affiliation:Wilfrid Laurier University,Waterloo,Canada,Yulun Zhang[https://orcid.org/0000-0002-2288-5079](https://orcid.org/0000-0002-2288-5079)email:[yulun100@gmail\.com](mailto:[email protected])Affiliation:School of Computer Science, Shanghai Jiao Tong University,Shanghai,China,Zhonghao Lyu[https://orcid.org/0000-0002-0980-1395](https://orcid.org/0000-0002-0980-1395)email:[lzhon@kth\.se](mailto:[email protected])Affiliation:Department of Computer Science, The Hang Seng University of Hong Kong,Hong Kong SAR,China,Haoyi Xiong[https://orcid.org/0000-0002-5451-3253](https://orcid.org/0000-0002-5451-3253)email:[haoyi\.xiong\.fr@ieee\.org](mailto:[email protected])Affiliation:Baidu Inc\.,Beijing,China,Linghe Kong[https://orcid.org/0000-0001-9266-3044](https://orcid.org/0000-0001-9266-3044)email:[linghe\.kong@sjtu\.edu\.cn](mailto:[email protected])Affiliation:School of Computer Science, Shanghai Jiao Tong University,Shanghai,China,Jimmy Xiangji Huang[https://orcid.org/0000-0003-1292-1491](https://orcid.org/0000-0003-1292-1491)email:[jhuang@yorku\.ca](mailto:[email protected])Affiliation:York University,Toronto,CanadaandDawei Yin[https://orcid.org/0000-0002-0684-6205](https://orcid.org/0000-0002-0684-6205)email:[yindawei@acm\.org](mailto:[email protected])Affiliation:Baidu Inc\.,Beijing,China

© , 2025

###### Abstract\.

Large language models exhibit complementary strengths, motivating routing methods that dispatch each query to the most suitable model\. Although existing routers are effective in single\-turn settings, they do not directly transfer to multi\-turn dialogue, where routing performance critically depends on how historical context is segmented, retained, and incorporated into the current prompt\. This introduces two fundamental challenges: preventing information loss and information confusion during context construction, and evaluating routing quality without conflating model selection with prompt construction quality\. In this paper, we proposeSWRouter, a Similarity\-Contractive Window Router for multi\-turn large language model routing\. SWRouter combines a similarity\-based context segmentation mechanism for prompt construction with a dual\-metric evaluation framework that decouples construction accuracy from router performance\. Experiments on multi\-turn dialogue benchmarks demonstrate thatSWRouterconsistently surpasses strong baselines, achieving a16\.26%improvement in evaluation accuracy over the best individual large language model and an additional8\.22%gain over the Conv\-ID Context baseline\. Our results highlight that multi\-turn large language model routing requires a joint design of context construction and evaluation, rather than a direct extension of single\-turn routing methods\.

###### Keywords:

Large Language Models, Multi\-turn Dialogue, LLM Routing, Context Segmentation, Evaluation Framework, Model Selection

## 1\.Introduction

Large Language Models \(LLMs\) have become core infrastructure for a wide range of intelligent applications, including conversational assistants, code generation, content creation, and increasingly, AI search and agentic question answering\([Li et al\., 2025](https://arxiv.org/html/2609.11414#bib.bib45);[Chen et al\., 2025](https://arxiv.org/html/2609.11414#bib.bib48);[Li et al\., 2026a](https://arxiv.org/html/2609.11414#bib.bib46)\)\. Their rapid adoption has also stimulated growing efforts to improve the efficiency of long\-context processing and inference\([Li et al\., 2026b](https://arxiv.org/html/2609.11414#bib.bib47);[Li et al\., 2026c](https://arxiv.org/html/2609.11414#bib.bib49)\)\. Meanwhile, many organizations have developed their own LLM families, such as OpenAI’s ChatGPT\([OpenAI, 2022](https://arxiv.org/html/2609.11414#bib.bib8)\), DeepSeek’s DeepSeek\([DeepSeek\-AI, 2024](https://arxiv.org/html/2609.11414#bib.bib9)\), Meta’s LLaMA\([Touvron et al\., 2023a](https://arxiv.org/html/2609.11414#bib.bib26)\), Google’s Gemini\([Gemini Team, 2023](https://arxiv.org/html/2609.11414#bib.bib10)\), and Anthropic’s Claude\([Anthropic, 2024](https://arxiv.org/html/2609.11414#bib.bib11)\)\. However, no single model consistently performs best across all tasks and applications\. For example, OpenAI’s GPT series is known for strong text generation ability\([OpenAI, 2022](https://arxiv.org/html/2609.11414#bib.bib8)\), DeepSeek demonstrates strength in mathematical reasoning\([Shao et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib13)\), and Claude is particularly effective in code generation and understanding\([Anthropic, 2024](https://arxiv.org/html/2609.11414#bib.bib11)\)\. Such heterogeneity has motivated a growing line of work on*LLM routing*, which dynamically dispatches each query to the most suitable model rather than relying on a fixed model for all inputs\.

Existing routing methods have achieved promising results in single\-turn settings, where the routing decision is made from a single prompt in isolation\. Representative examples include routers that select among models of similar scale, such as RouterDC\([Chen et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib24)\), as well as methods that decide between small and large models, such as C2MAB\-V\([Dai et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib25)\)\. However, directly applying these routing strategies to multi\-turn dialogue is fundamentally difficult, because model selection in conversational systems depends not only on the current user query, but also on how the dialogue history is segmented, retained, and incorporated into the prompt\.

![A two-part schematic showing context segmentation failures and attribution bias in multi-turn LLM routing evaluation.](https://arxiv.org/html/2609.11414v1/Wang-fig1.png)Figure 1\.Illustration of the two challenges in multi\-turn large language model routing\.\(a\)The context\-construction stage receives a dialogue history and the current user turn, then decides which historical turns should be retained before routing\. When this step drops necessary history, the routed model receives an under\-specified prompt and suffers frominformation loss; when it carries obsolete turns across a topic shift, the prompt contains misleading context and causesinformation confusion\.\(b\)The evaluation stage observes only the final routed response score, but that score is jointly determined by prompt construction and model selection\. A high\-quality window can raise the absolute scores of all candidate LLMs, whereas a poor window can depress them even if the router still chooses the relatively best model, producing attribution bias unless construction quality and router performance are measured separately\.A two\-part schematic showing context segmentation failures and attribution bias in multi\-turn LLM routing evaluation\.In multi\-turn dialogue, naive use of historical context introduces two fundamental challenges\.

- •First, there is alack of context segmentation mechanism\. When historical information is not properly segmented and integrated, the router may fail to distinguish relevant from irrelevant past context\. This leads to two concrete failure modes:information loss, where critical historical information required for the current response is omitted, andinformation confusion, where outdated context is mistakenly carried into a new topic\. As illustrated in Figure[1](https://arxiv.org/html/2609.11414#acmlabel1), missing prior context can cause failure on rule\-dependent queries, while incorrect reuse of obsolete context can degrade performance after topic shifts\.
- •Second, there is anattribution bias in multi\-turn router evaluation\. In multi\-turn settings, final response quality is jointly determined by two factors: the quality of context construction and the selected model’s performance under the constructed prompt\. A poorly constructed prompt may substantially degrade the responses of all candidate models, regardless of which model is selected\. Conversely, even with well\-constructed context, poor model selection can still lead to unsatisfactory responses\.

As illustrated in Figure[1](https://arxiv.org/html/2609.11414#acmlabel1), a well\-combined prompt and a poorly combined prompt may lead to dramatically different response quality across all candidate models \(e\.g\., 0\.9/0\.7/0\.3 vs\. 0\.09/0\.07/0\.03\)\. However, the observed inference accuracy alone does not indicate whether such performance variation comes from the context construction stage or from errors in the trained router\. This conflates construction quality with routing quality, making faithful evaluation of multi\-turn routing systems difficult\.

To address these issues, we proposeSWRouter, a multi\-turn LLM routingframeworkthat jointly addresses context construction, router training, and evaluation\. SWRouter consists of three tightly coupled components: \(1\) asimilarity\-based context segmentationmechanism that selectively incorporates semantically relevant historical turns when constructing the prompt, thereby reducing both information loss and information confusion; \(2\) apluggable router backbonetrained on the constructed prompts with contrastive objectives to select the most suitable model; and \(3\) adecoupled evaluation frameworkthat separatesconstruction accuracyfromrouter performance, enabling faithful assessment of each component independently\. We evaluate SWRouter on multi\-turn dialogue benchmarks and compare it against representative routing baselines and strong individual LLMs\. Empirically,SWRouterconsistently matches or surpasses the Conv\-ID Context baseline, improves average accuracy by16\.26%over the best individual LLM, and further achieves an8\.22%gain over the Conv\-ID Context baseline on multi\-turn dialogue benchmarks\.

Our contributions are summarized as follows:

- •We identify that the lack of context segmentation in multi\-turn dialogue routing leads to two concrete failure modes—information lossandinformation confusion\. To address this issue, we propose a similarity\-based prompt construction mechanism that integrates historical and current information\.
- •We reveal a classification bias in existing router evaluation protocols and introduce a decoupled evaluation framework with three complementary metrics:evaluation accuracy,construction accuracy, androuter performance\.
- •We construct a strong Conv\-ID Context baseline and conduct extensive in\-distribution and out\-of\-distribution evaluations\.SWRouterachieves16\.26%and8\.22%gains over the best single LLM and Conv\-ID Context on multi\-turn benchmarks, and outperforms Conv\-ID Context by2\.68%on OOD tasks\. Decoupled metric analysis further confirms that the gains stem from improvedconstruction accuracy\(w¯\\bar\{w\}\), whilerouter performance\(Prouter\>1P\_\{\\text\{router\}\}\>1\) validates effective model selection across all settings\.

## 2\.Background and Motivation

### 2\.1\.Problem Formulation

In conventional single\-turn LLM routing, the routing decision is made from the current prompt alone: given a user query, the router selects the candidate model that is expected to produce the best response\. In multi\-turn dialogue, however, this problem becomes fundamentally more challenging\. The router must not only determine*which*model to select, but also decide*what historical context should be preserved*and*how that context should be incorporated*into the current prompt\.

Formally, letℳ=\{Mt\}t=1T\\mathcal\{M\}=\\\{M\_\{t\}\\\}\_\{t=1\}^\{T\}denote a pool ofTTcandidate LLMs, and let𝒟train=\{\(xi,yi\)\}i=1n\\mathcal\{D\}\_\{\\text\{train\}\}=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{n\}denote the training set, wherexix\_\{i\}is the current user input andyiy\_\{i\}is the corresponding target answer\. In a single\-turn setting, routing is performed directly onxix\_\{i\}\. In a multi\-turn setting, by contrast, the current turn should be interpreted together with relevant dialogue history\. Therefore, the effective routing input is no longer the raw queryxix\_\{i\}, but a*context\-enhanced query*cic\_\{i\}, which is constructed by combining the current turn with selected historical information\.

More specifically, for each current queryxix\_\{i\}, we construct a combined querycic\_\{i\}by integrating it with semantically relevant historical turns\. This yields a transformed training set

𝒞train=\{\(ci,yi\)\}i=1n,\\mathcal\{C\}\_\{\\text\{train\}\}=\\\{\(c\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{n\},wherecic\_\{i\}captures the multi\-turn context on which routing should be based\. The router then produces a probability distribution over the candidate model poolℳ\\mathcal\{M\}and selects the model that is most suitable for answering under the constructed prompt\.

Frequently used symbols are summarized in Table[1](https://arxiv.org/html/2609.11414#S2.T1)\.

Table 1\.Symbols and Definitions\.This formulation highlights a key distinction between single\-turn and multi\-turn routing: in multi\-turn dialogue, routing quality depends not only on model selection, but also on prompt construction\. If relevant history is omitted, the routed model may fail because of*information loss*; if irrelevant or outdated history is retained, the routed model may instead suffer from*information confusion*\. As illustrated in Figure[1](https://arxiv.org/html/2609.11414#acmlabel1), these two failure modes arise before the routing decision itself and directly affect the quality of downstream responses\.

### 2\.2\.Single\-turn LLM Routing and Its Limitation

Among existing single\-turn routing methods, RouterDC\([Chen et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib24)\)is a representative approach\. It learns a router that maps an input query to a probability distribution over candidate LLMs, and optimizes the routing process with dual contrastive objectives\. This design is effective in single\-turn scenarios because the router only needs to reason over one prompt in isolation\([Chen et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib24)\)\.

However, this assumption does not hold in multi\-turn dialogue\. A single\-turn router such as RouterDC\([Chen et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib24)\)takes the current prompt as the routing unit, while ignoring the fact that the prompt itself may be under\-specified or even misleading if dialogue history is not properly segmented and incorporated\. As a consequence, even when the router selects the relatively best model for the observed input, the end\-to\-end system can still fail because the routing input is poorly constructed\.

This limitation is particularly important in our setting for two reasons\. First, multi\-turn dialogue introduces a*context selection problem*: the system must decide which historical turns are relevant to the current request before routing can be performed reliably\. Second, conventional router evaluation is largely based on relative ranking signals, which are suitable for stable optimization but insufficient for measuring end\-to\-end usefulness when prompt construction quality varies\. As discussed in Section[4\.4](https://arxiv.org/html/2609.11414#S4.SS4), two prompts can induce the same relative ranking over candidate models while yielding drastically different true scores\. Therefore, directly extending a single\-turn router to multi\-turn dialogue is insufficient\.

These observations motivate a routing framework that jointly addresses*context construction*and*routing evaluation*\. In the next section, we introduce SWRouter , which augments router learning with similarity\-based window partitioning for prompt construction and a decoupled evaluation framework for faithful assessment in multi\-turn settings\.

## 3\.Preliminaries and Problem Setup

This section formalizes the multi\-turn routing setting studied in this paper\. The goal is to make explicit the objects that are optimized by the router and the objects that are produced by the context\-construction module\. Unless otherwise specified, the notation follows Table[1](https://arxiv.org/html/2609.11414#S2.T1)\.

### 3\.1\.Multi\-turn Dialogue Instances

We consider a collection of multi\-turn dialogues\. For a routing instance indexed byii, let

𝒵i=\{\(ui,1,ai,1\),\(ui,2,ai,2\),…,\(ui,mi−1,ai,mi−1\),ui,mi\}\\mathcal\{Z\}\_\{i\}=\\\{\(u\_\{i,1\},a\_\{i,1\}\),\(u\_\{i,2\},a\_\{i,2\}\),\\ldots,\(u\_\{i,m\_\{i\}\-1\},a\_\{i,m\_\{i\}\-1\}\),u\_\{i,m\_\{i\}\}\\\}denote the dialogue observed before producing the next assistant response\. Here,ui,ju\_\{i,j\}is thejj\-th user turn,ai,ja\_\{i,j\}is the corresponding assistant turn, andui,miu\_\{i,m\_\{i\}\}is the current user request\. For consistency with the rest of the paper, we writexi=ui,mix\_\{i\}=u\_\{i,m\_\{i\}\}for the current query\. The raw dialogue history beforexix\_\{i\}is denoted by

ℋiraw=\{\(ui,1,ai,1\),…,\(ui,mi−1,ai,mi−1\)\}\.\\mathcal\{H\}^\{\\mathrm\{raw\}\}\_\{i\}=\\\{\(u\_\{i,1\},a\_\{i,1\}\),\\ldots,\(u\_\{i,m\_\{i\}\-1\},a\_\{i,m\_\{i\}\-1\}\)\\\}\.The router does not directly operate onℋiraw\\mathcal\{H\}^\{\\mathrm\{raw\}\}\_\{i\}\. Instead, a context\-construction functiong⁡\(⋅\)g\(\\cdot\)first selects and organizes the relevant historical information, producing a retained semantic windowHiH\_\{i\}and a context\-enhanced prompt

\(1\)ci=g⁡\(ℋiraw,xi\)\.c\_\{i\}=g\(\\mathcal\{H\}^\{\\mathrm\{raw\}\}\_\{i\},x\_\{i\}\)\.In this paper,g⁡\(⋅\)g\(\\cdot\)is instantiated by similarity\-based window construction, but the problem setup allows other context selectors such as full\-history concatenation, current\-turn\-only prompting, or retrieval\-based context selection\.

This distinction is important becausexix\_\{i\}alone may be under\-specified\. For example, the current request may contain pronouns, omitted constraints, or references to entities introduced in previous turns\. At the same time, the entire raw history may contain outdated goals, corrected assumptions, or off\-topic turns\. Therefore, a valid multi\-turn routing instance is not merely a user query; it is the pair consisting of the current request and the constructed prompt that determines what the candidate LLMs actually observe\.

### 3\.2\.Candidate Models and Router Objective

Letℳ=\{Mt\}t=1T\\mathcal\{M\}=\\\{M\_\{t\}\\\}\_\{t=1\}^\{T\}be a fixed pool ofTTcandidate LLMs\. Given a context\-enhanced promptcic\_\{i\}, a router with parametersθ\\thetaoutputs a probability distribution

R⁡\(ci,θ\)=\[R1​\(ci,θ\),R2​\(ci,θ\),…,RT​\(ci,θ\)\],R\(c\_\{i\};\\theta\)=\\left\[R\_\{1\}\(c\_\{i\};\\theta\),R\_\{2\}\(c\_\{i\};\\theta\),\\ldots,R\_\{T\}\(c\_\{i\};\\theta\)\\right\],whereRt​\(ci,θ\)R\_\{t\}\(c\_\{i\};\\theta\)is the probability of selecting candidate modelMtM\_\{t\}\. At inference time, the selected model index is

\(2\)t^i=arg⁡maxt∈\{1,…,T\}​Rt​\(ci,θ\),\\hat\{t\}\_\{i\}=\\arg\\max\_\{t\\in\\\{1,\\ldots,T\\\}\}R\_\{t\}\(c\_\{i\};\\theta\),and the final response is generated byMt^i​\(ci\)M\_\{\\hat\{t\}\_\{i\}\}\(c\_\{i\}\)\.

The ideal routing decision depends on the response quality that each candidate model would obtain under the same constructed prompt\. Letwi,t∈\[0,1\]w\_\{i,t\}\\in\[0,1\]denote the normalized quality score of modelMtM\_\{t\}on promptcic\_\{i\}\. The oracle candidate for this prompt is

\(3\)ti⋆=arg⁡maxt∈\{1,…,T\}⁡wi,t\.t\_\{i\}^\{\\star\}=\\arg\\max\_\{t\\in\\\{1,\\ldots,T\\\}\}w\_\{i,t\}\.The routing objective is therefore to learn a model selector that approaches the oracle candidate while using only the constructed prompt at inference time\. Equivalently, the end\-to\-end objective can be written as

\(4\)maxθ,g⁡1n​∑i=1nwi,t^i,t^i=arg⁡maxt​Rt​\(g⁡\(ℋiraw,xi\),θ\)\.\\max\_\{\\theta,g\}\\;\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}w\_\{i,\\hat\{t\}\_\{i\}\},\\quad\\hat\{t\}\_\{i\}=\\arg\\max\_\{t\}R\_\{t\}\(g\(\\mathcal\{H\}^\{\\mathrm\{raw\}\}\_\{i\},x\_\{i\}\);\\theta\)\.This expression makes explicit that multi\-turn routing has two coupled components: the context\-construction functiong⁡\(⋅\)g\(\\cdot\)and the routerR⁡\(⋅,θ\)R\(\\cdot;\\theta\)\. Ifg⁡\(⋅\)g\(\\cdot\)removes necessary history, even a strong router may select a model using an incomplete prompt\. Ifg⁡\(⋅\)g\(\\cdot\)introduces irrelevant history, the candidate models may all produce lower\-quality responses, making the routing decision less meaningful\.

### 3\.3\.Training Supervision and Evaluation Scores

During training and evaluation, each constructed promptcic\_\{i\}is paired with responses from all candidate models\. A judge model then assigns an integer score to each response, which is normalized to obtainwi,tw\_\{i,t\}\. These scores serve two roles\. First, they provide relative supervision for router learning by identifying strong and weak candidate models for each prompt\. For example, the top\-scoring models form the positive setIi\+I\_\{i\}^\{\+\}, while the bottom\-scoring models form the negative setIi−I\_\{i\}^\{\-\}\. Second, they provide absolute response\-quality measurements for decoupled evaluation\.

The distinction between relative and absolute signals is central to our setting\. Relative supervision is useful for training because it tells the router which candidate models should be preferred for a particular prompt\. However, relative ranking alone cannot determine whether the constructed prompt itself is good\. If all candidate models receive low absolute scores because the prompt omits necessary history, the best candidate model may still be easy to identify, but the final user\-facing answer remains poor\. For this reason, Section[4\.4](https://arxiv.org/html/2609.11414#S4.SS4)separately measures evaluation accuracy, construction accuracy, and router performance\.

### 3\.4\.Scope

We focus on routing among a fixed pool of already available LLMs\. The router does not modify the candidate models, and the context\-construction module operates before generation\. This scope matches practical deployments in which different models have complementary strengths and costs, but invoking all models for every user request is undesirable\. The problem studied here is therefore not how to train a stronger LLM, but how to construct an informative multi\-turn prompt and select an appropriate model for that prompt\.

## 4\.Methodology

### 4\.1\.Overview

SWRouter is designed to address the two central challenges of multi\-turn LLM routing introduced in Section[1](https://arxiv.org/html/2609.11414#S1): \(1\) how to construct an effective routing prompt from dialogue history, and \(2\) how to evaluate routing quality without conflating it with prompt construction quality\. To this end, SWRouter consists of three stages: \(1\)similarity\-based window partitioningto construct combined prompts from multi\-turn dialogue history \(Section[4\.2](https://arxiv.org/html/2609.11414#S4.SS2)\); \(2\)model routing based on the constructed prompts, optimized with contrastive training objectives that rely on relative, ranking\-based scores \(Section[4\.3](https://arxiv.org/html/2609.11414#S4.SS3)\); and \(3\)decoupled evaluationthat distinguishesconstruction accuracyfromrouter performanceusing absolute true scores after prompt construction \(Section[4\.4](https://arxiv.org/html/2609.11414#S4.SS4)\)\.

In the first stage, SWRouter organizes multi\-turn dialogue history into semantically coherent windows\. Instead of directly concatenating the entire dialogue history, it measures the semantic similarity between adjacent user turns and dynamically decides whether the current turn should be merged into the existing window or begin a new one\. This design allows SWRouter to preserve relevant historical information while reducing the risk of introducing irrelevant or outdated context\.

In the second stage, the constructed context\-enhanced prompt is fed into a pluggable router backbone that selects the most suitable model from the candidate LLM pool\. Concretely, the router backbone encodes the combined prompt into a dense representation and computes its similarity with a set of learnable model embeddings, yielding a probability distribution over candidate LLMs\. The router backbone is optimized with contrastive objectives that encourage it to distinguish strong candidate models from weak ones under multi\-turn settings\.

In the third stage, SWRouter adopts a decoupled evaluation framework tailored to multi\-turn routing\. Since existing routing evaluations are typically based on relative ranking signals, they are suitable for optimization but insufficient for measuring end\-to\-end usefulness when prompt construction quality varies\. Therefore, SWRouter separately reportsconstruction accuracy, which reflects the average quality of all candidate models under the constructed prompt, androuter performance, which measures how much the router improves over random selection\.

Overall, these three stages form a unified framework for multi\-turn LLM routing\. The similarity\-based window partitioning stage determines what historical context should be preserved, the routing stage determines which model should be selected under the constructed prompt, and the decoupled evaluation stage determines how the resulting system should be assessed\.

Figure 2\.Overview of SWRouter \. Given a multi\-turn dialogue, SWRouter first encodes adjacent user turns and computes their semantic similarity to decide whether the current turn should extend the existing window or start a new one\. The retained window is then merged into a context\-enhanced prompt and passed to a pluggable router, which represents the prompt with an encoder, compares it with learnable candidate\-model embeddings, and selects the LLM with the highest routing score\. During training, the router is optimized with sample\-LLM and sample\-sample contrastive objectives so that high\-quality models and semantically related prompts are pulled closer in representation space\. During evaluation, the generated responses are scored by judge models and decomposed into construction accuracy, evaluation accuracy, and router performance, allowing prompt construction quality and model\-selection ability to be analyzed separately\.Pipeline diagram of SWRouter showing similarity\-based window partitioning, router training and inference, and decoupled evaluation\.
### 4\.2\.Similarity\-based Window Partitioning

A core challenge in multi\-turn LLM routing is that the current user query should not be interpreted in isolation\. Instead, the router should determine which historical turns remain semantically relevant to the current request and should therefore be retained in the routing prompt\. To address this issue, SWRouter employs a similarity\-based window partitioning algorithm that organizes dialogue history into semantically coherent windows\. This formulation follows the broader view of sequential structure analysis and segmentation in formal\-language and string\-processing algorithms\([Aho and Ullman, 1972](https://arxiv.org/html/2609.11414#bib.bib1);[Gusfield, 1997](https://arxiv.org/html/2609.11414#bib.bib5);[Chandra et al\., 1981](https://arxiv.org/html/2609.11414#bib.bib3)\), while using neural semantic similarity rather than symbolic parsing rules\.

LetE⁡\(x,w\)E\(x;w\)denote an encoder that maps an input utterancexxinto an embedding inℝp\\mathbb\{R\}^\{p\}\. Given the current user queryxix\_\{i\}, we first compute its embedding

ei=E⁡\(xi,w\)\.e\_\{i\}=E\(x\_\{i\};w\)\.We then measure the similarity between the current query and the previous query:

\(5\)similarity=sim⁡\(ei,ei−1\),\\text\{similarity\}=\\mathrm\{sim\}\(e\_\{i\},e\_\{i\-1\}\),wheresim⁡\(⋅,⋅\)\\mathrm\{sim\}\(\\cdot,\\cdot\)denotes cosine similarity,eie\_\{i\}is the embedding of the current queryxix\_\{i\}, andei−1e\_\{i\-1\}is the embedding of the previous queryxi−1x\_\{i\-1\}\.

Based on this similarity score, SWRouter updates the historical windowHiH\_\{i\}as follows:

\(6\)Hi=\{\{xi\},if​sim​\(ei,ei−1\)<τ,Hi−1∪\{xi\},if​sim​\(ei,ei−1\)≥τ,H\_\{i\}=\\begin\{cases\}\\\{x\_\{i\}\\\},&\\text\{if \}\\mathrm\{sim\}\(e\_\{i\},e\_\{i\-1\}\)<\\tau,\\\\ H\_\{i\-1\}\\cup\\\{x\_\{i\}\\\},&\\text\{if \}\\mathrm\{sim\}\(e\_\{i\},e\_\{i\-1\}\)\\geq\\tau,\\end\{cases\}whereτ\\tauis a similarity threshold\. Intuitively, if the current turn is insufficiently similar to the previous one, SWRouter starts a new window; otherwise, it appends the current turn to the existing window\. In this way, the dialogue history is partitioned into locally coherent semantic segments rather than being naively concatenated into a single undifferentiated context\.

After the historical window is determined, SWRouter constructs a context\-enhanced promptcic\_\{i\}for routing by combining the current query with the selected dialogue history inHiH\_\{i\}\. This design enables the router to make decisions based on relevant multi\-turn context, rather than relying solely on the current turn\. Compared with naive history concatenation, the similarity\-based windowing mechanism offers two advantages\. First, it preserves relevant context needed for rule\-dependent or context\-dependent requests, thereby reducing*information loss*\. Second, it avoids carrying obsolete or off\-topic history into the new prompt, thereby reducing*information confusion*\. These two effects correspond exactly to the two failure modes illustrated in Figure[1](https://arxiv.org/html/2609.11414#acmlabel1)\.

The similarity thresholdτ\\taucontrols the granularity of context segmentation\. A smallerτ\\tautends to merge more turns into the same window, which may preserve more context but also increases the risk of introducing irrelevant information\. A largerτ\\tauproduces more aggressive segmentation, which can better isolate topic shifts but may also discard useful dependencies\. Therefore,τ\\taugoverns the trade\-off between context retention and context purity, and its impact will be analyzed in Section[6\.5](https://arxiv.org/html/2609.11414#S6.SS5)\.

Algorithm[1](https://arxiv.org/html/2609.11414#algorithm1)summarizes the similarity\-window construction procedure\. The procedure is applied independently to each dialogue\. It processes user turns in chronological order, compares each user turn with the immediately preceding user turn, and updates the active semantic window according to the thresholdτ\\tau\. Assistant turns that lie inside the retained window are preserved when constructing the final prompt, so that the generated prompt keeps both the user’s constraints and the assistant\-side facts that may be needed for resolving follow\-up references\.

Algorithm 1Similarity\-based window construction for multi\-turn routing\. The algorithm scans a dialogue from left to right, starts a new semantic window when adjacent user turns are weakly related, and otherwise extends the current window\. For each current request, it outputs a context\-enhanced prompt that contains only the active semantic window\.Input:Dialogue𝒵i=\{\(ui,1,ai,1\),…,\(ui,mi−1,ai,mi−1\),ui,mi\}\\mathcal\{Z\}\_\{i\}=\\\{\(u\_\{i,1\},a\_\{i,1\}\),\\ldots,\(u\_\{i,m\_\{i\}\-1\},a\_\{i,m\_\{i\}\-1\}\),u\_\{i,m\_\{i\}\}\\\}; encoderE⁡\(⋅,w\)E\(\\cdot;w\); thresholdτ\\tau

Output:Context\-enhanced prompt

cic\_\{i\}and retained window

HiH\_\{i\}for the current request

xi=ui,mix\_\{i\}=u\_\{i,m\_\{i\}\}
1

H←\{ui,1,ai,1\}H\\leftarrow\\\{u\_\{i,1\},a\_\{i,1\}\\\};

2

eprev←E⁡\(ui,1,w\)e\_\{\\mathrm\{prev\}\}\\leftarrow E\(u\_\{i,1\};w\);

3for*j=2j=2tomim\_\{i\}*do

4

ej←E⁡\(ui,j,w\)e\_\{j\}\\leftarrow E\(u\_\{i,j\};w\);

5

ρj←sim⁡\(ej,eprev\)\\rho\_\{j\}\\leftarrow\\mathrm\{sim\}\(e\_\{j\},e\_\{\\mathrm\{prev\}\}\);

6if*ρj<τ\\rho\_\{j\}<\\tau*then

7

H←\{ui,j\}H\\leftarrow\\\{u\_\{i,j\}\\\};//topic shift; reset the semantic window

8else

9

H←H∪\{ui,j\}H\\leftarrow H\\cup\\\{u\_\{i,j\}\\\};//same local context; extend the window

10if*j<mij<m\_\{i\}andui,j∈Hu\_\{i,j\}\\in H*then

11

H←H∪\{ai,j\}H\\leftarrow H\\cup\\\{a\_\{i,j\}\\\};//retain assistant\-side context inside the window

12

eprev←eje\_\{\\mathrm\{prev\}\}\\leftarrow e\_\{j\};

13

Hi←HH\_\{i\}\\leftarrow H;

14

ci←FormatPrompt⁡\(Hi,xi\)c\_\{i\}\\leftarrow\\mathrm\{FormatPrompt\}\(H\_\{i\},x\_\{i\}\);

15return*ci,Hic\_\{i\},H\_\{i\}*;

### 4\.3\.Routing and Training

For a queryxix\_\{i\}, after being transformed into a combined querycic\_\{i\}using the similarity\-based window partitioning algorithm, SWRouter generates a selection probability distribution overTTcandidate LLMs:

\(7\)R⁡\(ci,θ\)=softmax⁡\(\[sim⁡\(E⁡\(ci,w\),k1\)sim⁡\(E⁡\(ci,w\),k2\)sim⁡\(E⁡\(ci,w\),kT\)\]\),R\(c\_\{i\};\\theta\)=\\mathrm\{softmax\}\\left\(\\begin\{bmatrix\}\\mathrm\{sim\}\(E\(c\_\{i\};w\),k\_\{1\}\)\\\\ \\mathrm\{sim\}\(E\(c\_\{i\};w\),k\_\{2\}\)\\\\ \\vdots\\\\ \\mathrm\{sim\}\(E\(c\_\{i\};w\),k\_\{T\}\)\\end\{bmatrix\}\\right\),whereθ≡\{w,k1,k2,…,kT\}\\theta\\equiv\\\{w,k\_\{1\},k\_\{2\},\\ldots,k\_\{T\}\\\}denotes the parameters of SWRouter , andsim⁡\(⋅,⋅\)\\mathrm\{sim\}\(\\cdot,\\cdot\)denotes cosine similarity\.

A straightforward way to train the router is to align its output distribution with a score distribution:

\(8\)min⁡∑\(ci,yi\)∈𝒞trainθ⁡KL⁡\(R⁡\(ci,θ\),softmax⁡\[si\(1\),…,si\(T\)\]\),\\min\_\{\\theta\}\\sum\_\{\(c\_\{i\},y\_\{i\}\)\\in\\mathcal\{C\}\_\{\\text\{train\}\}\}\\mathrm\{KL\}\\Big\(R\(c\_\{i\};\\theta\),\\;\\mathrm\{softmax\}\[s\_\{i\}^\{\(1\)\},\\ldots,s\_\{i\}^\{\(T\)\}\]\\Big\),whereKL⁡\(⋅,⋅\)\\mathrm\{KL\}\(\\cdot,\\cdot\)denotes the Kullback–Leibler divergence\([Kullback and Leibler, 1951](https://arxiv.org/html/2609.11414#bib.bib43)\)\. This KL objective has recently been used in LLM routing\([Lu et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib20)\)\. However, we argue that it is not an ideal proxy for router training in our setting, because the goal of routing is to assign queries to top\-performing LLMs rather than to fit the full score distribution, especially for bottom\-performing models\. Following the contrastive learning spirit of RouterDC\([Chen et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib24)\), SWRouter adopts contrastive training objectives over the context\-enhanced query\.

#### 4\.3\.1\.Sample\-LLM Contrastive Loss

For each combined querycic\_\{i\}, SWRouter constructs positive and negative LLM index setsIi\+I\_\{i\}^\{\+\}andIi−I\_\{i\}^\{\-\}\. The sample\-LLM contrastive loss is defined as

\(9\)Lsample\-LLM​\(ci,yi,θ\)=−log⁡Pi\+Pi\+\+Pi−,L\_\{\\text\{sample\-LLM\}\}\(c\_\{i\},y\_\{i\};\\theta\)=\-\\log\\frac\{P\_\{i\}^\{\+\}\}\{P\_\{i\}^\{\+\}\+P\_\{i\}^\{\-\}\},where

\(10\)Pi\+=∑t∈Ii\+esim⁡\(E⁡\(ci,w\),kt\),P\_\{i\}^\{\+\}=\\sum\_\{t\\in I\_\{i\}^\{\+\}\}e^\{\\mathrm\{sim\}\(E\(c\_\{i\};w\),k\_\{t\}\)\},is the sum of the exponentiated similarities between the query embeddingE⁡\(ci,w\)E\(c\_\{i\};w\)and the top\-K\+K\_\{\+\}model embeddings, and

\(11\)Pi−=∑t∈Ii−esim⁡\(E⁡\(ci,w\),kt\),P\_\{i\}^\{\-\}=\\sum\_\{t\\in I\_\{i\}^\{\-\}\}e^\{\\mathrm\{sim\}\(E\(c\_\{i\};w\),k\_\{t\}\)\},is the corresponding sum over the bottom\-K−K\_\{\-\}model embeddings\. Here,Ii\+I\_\{i\}^\{\+\}andIi−I\_\{i\}^\{\-\}denote the index sets of the top\-K\+K\_\{\+\}and bottom\-K−K\_\{\-\}LLMs, respectively\.

#### 4\.3\.2\.Sample\-Sample Contrastive Loss

In addition to contrasting candidate models for each query, SWRouter leverages unsupervised clustering to group semantically related queries and constructs a sample\-sample contrastive loss: This representation\-learning design is related to scalable log\-linear optimization and predictive structure learning across related tasks\([Andrew and Gao, 2007](https://arxiv.org/html/2609.11414#bib.bib4);[Ando and Zhang, 2005](https://arxiv.org/html/2609.11414#bib.bib7)\)\.

\(12\)Lsample\-sample​\(ci,θ\)=−log⁡Qi\+Qi\+\+Qi−,L\_\{\\text\{sample\-sample\}\}\(c\_\{i\};\\theta\)=\-\\log\\frac\{Q\_\{i\}^\{\+\}\}\{Q\_\{i\}^\{\+\}\+Q\_\{i\}^\{\-\}\},where

\(13\)Qi\+=esim⁡\(E⁡\(ci,w\),E⁡\(ci\+,w\)\)Q\_\{i\}^\{\+\}=e^\{\\mathrm\{sim\}\(E\(c\_\{i\};w\),E\(c\_\{i\}^\{\+\};w\)\)\}represents the similarity between the embedding of querycic\_\{i\}and that of a randomly chosen in\-group queryci\+c\_\{i\}^\{\+\}, and

\(14\)Qi−=∑ci−∈Xi−esim⁡\(E⁡\(ci,w\),E⁡\(ci−,w\)\)Q\_\{i\}^\{\-\}=\\sum\_\{c\_\{i\}^\{\-\}\\in X\_\{i\}^\{\-\}\}e^\{\\mathrm\{sim\}\(E\(c\_\{i\};w\),E\(c\_\{i\}^\{\-\};w\)\)\}denotes the summed similarities betweencic\_\{i\}and a set of out\-group queriesXi−X\_\{i\}^\{\-\}\.

The overall training objective combines the above two contrastive losses:

\(15\)min⁡∑θ⁡\(α​Lsample\-LLM\+β​Lsample\-sample\),\\min\_\{\\theta\}\\sum\\left\(\\alpha L\_\{\\text\{sample\-LLM\}\}\+\\beta L\_\{\\text\{sample\-sample\}\}\\right\),whereα\\alphaandβ\\betaare hyperparameters that balance the two loss terms\.

Algorithm[2](https://arxiv.org/html/2609.11414#algorithm2)summarizes the training and inference pipeline of SWRouter \. The algorithm uses Algorithm[1](https://arxiv.org/html/2609.11414#algorithm1)as a preprocessing step to transform raw multi\-turn dialogues into context\-enhanced prompts\. Training then relies on model\-level and sample\-level contrastive supervision, while inference only requires a single router forward pass before calling the selected candidate LLM\.

Algorithm 2Router training and inference for SWRouter \. The procedure first converts raw dialogues into context\-enhanced prompts using Algorithm[1](https://arxiv.org/html/2609.11414#algorithm1), then builds contrastive supervision from candidate\-model scores, optimizes the router with sample\-LLM and sample\-sample losses, and finally selects one candidate LLM for each test prompt\.Input:Training dialogues𝒟train\\mathcal\{D\}\_\{\\text\{train\}\}, test dialogues𝒟test\\mathcal\{D\}\_\{\\text\{test\}\}, LLM poolℳ=\{Mt\}t=1T\\mathcal\{M\}=\\\{M\_\{t\}\\\}\_\{t=1\}^\{T\}, encoderE⁡\(⋅,w\)E\(\\cdot;w\), model embeddings\{kt\}t=1T\\\{k\_\{t\}\\\}\_\{t=1\}^\{T\}, thresholdτ\\tau, hyperparametersK\+K\_\{\+\},K−K\_\{\-\},NN,λ\\lambda,bb,η\\eta

Output:Trained router

R⁡\(⋅,θ\)R\(\\cdot;\\theta\)and routed responses on the test set

1Data preprocessing:

2for*𝒵i∈𝒟train\\mathcal\{Z\}\_\{i\}\\in\\mathcal\{D\}\_\{\\text\{train\}\}*do

3

\(ci,Hi\)←WindowConstruct⁡\(𝒵i,E,τ\)\(c\_\{i\},H\_\{i\}\)\\leftarrow\\mathrm\{WindowConstruct\}\(\\mathcal\{Z\}\_\{i\},E,\\tau\)using Algorithm[1](https://arxiv.org/html/2609.11414#algorithm1);

4for*t=1t=1toTT*do

5Generate response

ri,t←Mt​\(ci\)r\_\{i,t\}\\leftarrow M\_\{t\}\(c\_\{i\}\);

6Score

ri,tr\_\{i,t\}with the judge model to obtain

wi,tw\_\{i,t\};

7

𝒞train←\{\(ci,\{wi,t\}t=1T\)\}\\mathcal\{C\}\_\{\\text\{train\}\}\\leftarrow\\\{\(c\_\{i\},\\\{w\_\{i,t\}\\\}\_\{t=1\}^\{T\}\)\\\};

8Training:

9Cluster queries

\{ci\}i=1n\\\{c\_\{i\}\\\}\_\{i=1\}^\{n\}into

NNgroups

10repeat

11Sample mini\-batch

BBfrom

𝒞train\\mathcal\{C\}\_\{\\text\{train\}\}
12for*\(ci,yi\)∈B\(c\_\{i\},y\_\{i\}\)\\in B*do

13Construct

Ii\+I\_\{i\}^\{\+\}and

Ii−I\_\{i\}^\{\-\}using

wi,tw\_\{i,t\}
14Compute

Lsample\-LLM​\(ci,yi,θ\)L\_\{\\text\{sample\-LLM\}\}\(c\_\{i\},y\_\{i\};\\theta\)
15Sample in\-group query

ci\+c\_\{i\}^\{\+\}and out\-group queries

\{ci−\}\\\{c\_\{i\}^\{\-\}\\\}
16Compute

Lsample\-sample​\(ci,θ\)L\_\{\\text\{sample\-sample\}\}\(c\_\{i\};\\theta\)
17

L⁡\(B,θ\)←∑\(ci,yi\)∈B\[Lsample\-LLM\+λ​Lsample\-sample\]L\(B;\\theta\)\\leftarrow\\sum\_\{\(c\_\{i\},y\_\{i\}\)\\in B\}\\big\[L\_\{\\text\{sample\-LLM\}\}\+\\lambda L\_\{\\text\{sample\-sample\}\}\\big\]
18

θ←θ−η​∇θL​\(B,θ\)\\theta\\leftarrow\\theta\-\\eta\\nabla\_\{\\theta\}L\(B;\\theta\)
19until*converged*;

20Inference:

21for*𝒵i∈𝒟test\\mathcal\{Z\}\_\{i\}\\in\\mathcal\{D\}\_\{\\text\{test\}\}*do

22

\(ci,Hi\)←WindowConstruct⁡\(𝒵i,E,τ\)\(c\_\{i\},H\_\{i\}\)\\leftarrow\\mathrm\{WindowConstruct\}\(\\mathcal\{Z\}\_\{i\},E,\\tau\);

23

t^i←arg⁡maxt​Rt​\(ci,θ\)\\hat\{t\}\_\{i\}\\leftarrow\\arg\\max\_\{t\}R\_\{t\}\(c\_\{i\};\\theta\);

24

y^i←Mt^i​\(ci\)\\hat\{y\}\_\{i\}\\leftarrow M\_\{\\hat\{t\}\_\{i\}\}\(c\_\{i\}\);

25return*R⁡\(⋅,θ\)R\(\\cdot;\\theta\)and\{y^i\}\\\{\\hat\{y\}\_\{i\}\\\}*;

### 4\.4\.Decoupled Evaluation Metrics

For each combined querycic\_\{i\}and each candidate modelℳt\\mathcal\{M\}\_\{t\}\(t=1,…,Tt=1,\\ldots,T\), we obtain a*true score*0≤wi,t≤10\\leq w\_\{i,t\}\\leq 1from judge models, which reflects the absolute response quality ofℳt\\mathcal\{M\}\_\{t\}oncic\_\{i\}\.

##### Evaluation Accuracy \(absolute quality\)\.

Given the router’s selection indicatorsi\(t\)∈\{0,1\}s\_\{i\}^\{\(t\)\}\\in\\\{0,1\\\}, wheresi\(t\)=1s\_\{i\}^\{\(t\)\}=1if the router selectsℳt\\mathcal\{M\}\_\{t\}forcic\_\{i\}and00otherwise, the score of the selected model is

\(16\)si=∑t=1Twi,t⋅si\(t\)\.s\_\{i\}=\\sum\_\{t=1\}^\{T\}w\_\{i,t\}\\cdot s\_\{i\}^\{\(t\)\}\.We defineevaluation accuracyas the average true score achieved by the routing system:

\(17\)s​i¯=1n​∑i=1nsi,\\overline\{si\}=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}s\_\{i\},which reflects the absolute usefulness of the routed responses under the constructed prompts\.

##### Construction Accuracy \(prompt quality\)\.

To characterize the overall quality of the constructed prompt across all candidate models, we define the average true score

\(18\)w¯=1n​T​∑i=1n∑t=1Twi,t,\\bar\{w\}=\\frac\{1\}\{nT\}\\sum\_\{i=1\}^\{n\}\\sum\_\{t=1\}^\{T\}w\_\{i,t\},which reflects the average capability of the candidate model pool under a given prompt construction\. A higherw¯\\bar\{w\}indicates that the constructed prompt enables better responses across all candidate models\.

##### Router Performance \(relative improvement\)\.

Based ons​i¯\\overline\{si\}andw¯\\bar\{w\}, we define router performance as

\(19\)Prouter=s​i¯w¯\.P\_\{\\text\{router\}\}=\\frac\{\\overline\{si\}\}\{\\bar\{w\}\}\.Here,w¯\\bar\{w\}reflects the average capability of the candidate model pool under a given prompt construction, whileProuterP\_\{\\text\{router\}\}measures how much the router improves over this average by selecting stronger models\. A value greater than 1 indicates that the router outperforms random model selection\.

Figure 3\.Illustration of the three complementary metrics used for decoupled evaluation\. For each constructed prompt, all candidate LLMs are first evaluated with absolute true scores\. Construction accuracyw¯\\bar\{w\}averages these scores across the whole candidate pool and therefore measures whether the prompt construction stage preserves useful context for models in general\. Evaluation accuracys​i¯\\overline\{si\}then records the true score of the single model selected by the router, representing the end\-to\-end quality observed by users\. Router performanceProuter=s​i¯/w¯P\_\{\\text\{router\}\}=\\overline\{si\}/\\bar\{w\}normalizes the selected\-model score by the candidate average, showing whether the router improves over random model selection under the same constructed prompts\.Diagram comparing construction accuracy, evaluation accuracy, and router performance for multi\-turn routing\.
##### Discussion: three complementary metrics\.

For evaluation in multi\-turn dialogue, we adopt three complementary metrics to characterize the system:evaluation accuracy\(s​i¯\\overline\{si\}\),construction accuracy\(w¯\\bar\{w\}\), androuter performance\(ProuterP\_\{\\text\{router\}\}\)\. Figure[3](https://arxiv.org/html/2609.11414#acmlabel3)illustrates why these metrics are complementary\. Under good prompt construction, the candidate models obtain scores of\{0\.9,0\.7,0\.3\}\\\{0\.9,0\.7,0\.3\\\}, yielding an average construction accuracy ofw¯=0\.633\\bar\{w\}=0\.633\. If the router selects the best model, thens​i¯=0\.9\\overline\{si\}=0\.9andProuter=0\.9/0\.633=1\.42P\_\{\\text\{router\}\}=0\.9/0\.633=1\.42, indicating both strong construction quality and effective routing\. In contrast, under poor prompt construction, information loss uniformly degrades the candidate scores to\{0\.09,0\.07,0\.03\}\\\{0\.09,0\.07,0\.03\\\}, reducing construction accuracy tow¯=0\.0633\\bar\{w\}=0\.0633\. Even if the router still selects the best model, the final evaluation accuracy drops tos​i¯=0\.09\\overline\{si\}=0\.09, while the router performance remains unchanged:Prouter=0\.09/0\.0633=1\.42P\_\{\\text\{router\}\}=0\.09/0\.0633=1\.42\. This example shows that high router performance does not necessarily imply high end\-to\-end response quality\. Instead,s​i¯\\overline\{si\}reflects the final usefulness of the routed response,w¯\\bar\{w\}measures the intrinsic quality of prompt construction, andProuterP\_\{\\text\{router\}\}quantifies the router’s ability to select better\-than\-average candidates\. Together, these metrics enable a decoupled analysis of construction quality and routing effectiveness\.

## 5\.Implementation

We implement SWRouter following the standard router\-training and evaluation pipeline for multi\-turn LLM routing\. Our implementation consists of candidate model construction, multi\-turn dataset preprocessing, similarity\-window segmentation, response scoring, and router optimization\.

Candidate LLMs\.We evaluate SWRouter on seven open\-source LLMs from HuggingFace: Mistral\-7B\([Jiang et al\., 2023a](https://arxiv.org/html/2609.11414#bib.bib30)\), MetaMath\-Mistral\-7B\([Yu et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib31)\), zephyr\-7b\-beta\([Tunstall et al\., 2023](https://arxiv.org/html/2609.11414#bib.bib32)\), Chinese\-Mistral\-7B\([Zhou and Bai, 2024](https://arxiv.org/html/2609.11414#bib.bib34)\), dolphin\-2\.6\-mistral\-7b\([Cognitive Computations, 2024a](https://arxiv.org/html/2609.11414#bib.bib35)\), Llama\-3\-8B\([Llama Team, 2024](https://arxiv.org/html/2609.11414#bib.bib36)\), and dolphin\-2\.9\-llama3\-8b\([Cognitive Computations, 2024b](https://arxiv.org/html/2609.11414#bib.bib37)\)\. The first five models are Mistral\-based, while the last two are Llama\-3\-based\. These models cover general\-purpose, instruction\-tuned, domain\-tuned, and language\-adapted variants, enabling us to evaluate whether SWRouter can select among LLMs with heterogeneous capabilities\. The Llama\-family candidates build on the broader open\-foundation\-model line represented by LLaMA and LLaMA 2\([Touvron et al\., 2023a](https://arxiv.org/html/2609.11414#bib.bib26);[Touvron et al\., 2023b](https://arxiv.org/html/2609.11414#bib.bib44)\), and zephyr\-7b\-beta follows preference\-optimization techniques such as direct preference optimization\([Rafailov et al\., 2023](https://arxiv.org/html/2609.11414#bib.bib33)\)\.

Datasets\.We conduct experiments on two multi\-turn dialogue datasets, MTBench\([Bai et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib38)\)and ShareGPT\([Schmid, 2023](https://arxiv.org/html/2609.11414#bib.bib39)\)\. MTBench contains multi\-turn conversations across diverse task categories, while ShareGPT consists of real user\-AI interactions\. We preprocess ShareGPT by removing noisy samples and filtering out non\-multi\-turn dialogues\. For both datasets, we randomly split the data into 70% for training and 30% for testing\. All training samples are combined as𝒞train\\mathcal\{C\}\_\{\\text\{train\}\}for router training\.

Baselines\.We compare SWRouter with three groups of baselines\. First,Single Modeldirectly uses one candidate LLM to answer each query without routing\. This group includes all seven candidate LLMs listed above\. Second,Conv\-ID Contextpreserves the original dialogue segmentation according to conversation IDs and performs routing with the full original multi\-turn context\. This baseline serves as an ID\-based static context reference\. Third,ZOOTER\([Lu et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib20)\)is used as a strong single\-turn router backbone\. To evaluate the generality of the proposed similarity\-window mechanism, we combine ZOOTER with the same similarity\-window segmentation and compare it with SWRouter \.

Evaluation protocol\.We use the Language Model Evaluation Harness\([Gao et al\., 2023](https://arxiv.org/html/2609.11414#bib.bib40)\)for model evaluation, following common benchmark practice from MMLU\-style multitask evaluation\([Hendrycks et al\., 2021](https://arxiv.org/html/2609.11414#bib.bib29)\)\. For open\-ended generation, we generateM=10M=10candidate responses using stochastic beam search, where beam expansion is sampled with temperature 0\.2\. The generated responses are then scored by judge models, and the router is evaluated using testing weighted accuracy\.

Similarity\-window configurations\.We use mDeBERTaV3\-base\([He et al\., 2021](https://arxiv.org/html/2609.11414#bib.bib41)\)as the encoderE⁡\(x,w\)E\(x;w\), which contains approximately 86M backbone parameters\. This choice follows the use of efficient neural encoders for structured text analysis\([Rasooli and Tetreault, 2015](https://arxiv.org/html/2609.11414#bib.bib6)\)\. The similarity threshold is set toτ=0\.91\\tau=0\.91for constructing similarity windows\. The LLM embedding dimension is 768, and the number of clusters is set toN=5N=5\.

Router training configurations\.The router is trained for 1,000 steps using AdamW\([Loshchilov and Hutter, 2019](https://arxiv.org/html/2609.11414#bib.bib42)\)\. The learning rate is5×10−55\\times 10^\{\-5\}, the weight decay is 0\.01, and the batch size is 16\. All experiments are conducted on NVIDIA A100 80GB GPUs\.

## 6\.Evaluation

We evaluate SWRouter to answer the following questions:

1. \(1\)Can SWRouter improve routing performance on multi\-turn dialogue tasks?
2. \(2\)How well does SWRouter generalize to out\-of\-distribution \(OOD\) scenarios?
3. \(3\)How much does each component contribute to the effectiveness of SWRouter ?
4. \(4\)How sensitive is SWRouter to key hyperparameters?
5. \(5\)What is the computational overhead introduced by SWRouter ?

### 6\.1\.Evaluation Setup

#### 6\.1\.1\.Prompts Used for Evaluation

To evaluate the quality of routed responses in multi\-turn dialogue, we employ a judge model with the following evaluation prompt: The prompt is written as a structured evaluation rubric following standard academic reporting conventions\([American Psychological Association, 1983](https://arxiv.org/html/2609.11414#bib.bib2)\)\.

> You are given a multi\-turn conversation, including the previous dialogue history, the current user request, and an assistant response generated by a candidate LLM\. Please evaluate the quality of the assistant response with a single integer score from 1 to 10\. Judge the response semantically rather than by exact wording: paraphrases, different ordering, or different surface expressions should be treated as acceptable when they preserve the same meaning and satisfy the user’s request\. Do not reward verbosity by itself; longer answers should receive higher scores only when the additional content improves correctness, usefulness, or coverage\. Use the following criteria when assigning the score\.Intent satisfactionevaluates whether the response identifies the user’s actual goal in the current turn and provides a useful answer or action for that goal\.Dialogue context usageevaluates whether the response correctly incorporates relevant prior turns, carries over established constraints, resolves references such as “it” or “that” appropriately, and avoids being distracted by obsolete or unrelated context\.Instruction followingevaluates whether the response obeys explicit requirements about format, language, style, scope, tools, refusal behavior, or output length\.Factuality and reasoningevaluates whether the response is factually correct, logically consistent, and supported by the information available in the conversation or by reliable reasoning\.Completenessevaluates whether all important parts of the request are addressed, including edge cases, caveats, or requested explanations\.Clarity and usefulnessevaluates whether the response is easy to understand, well organized, appropriately concise, and practically helpful to the user\. Apply the following scoring guideline\. A score of10should be reserved for a response that fully satisfies the user’s intent, uses all necessary dialogue context correctly, follows every instruction, is factually sound, complete, and clear\. A score of9indicates an excellent response with only negligible wording or presentation issues\. Scores of7–8indicate a mostly correct and useful response that satisfies the main request but has minor omissions, mild ambiguity, limited explanation, or only partial use of relevant context\. Scores of5–6indicate a partially useful response that addresses the general topic but misses important constraints, overlooks meaningful dialogue history, contains noticeable but non\-fatal factual or reasoning errors, or leaves substantial parts of the task incomplete\. Scores of3–4indicate a weak response that is only loosely related to the request, substantially incomplete, poorly grounded in the dialogue, or affected by major factual, reasoning, or instruction\-following errors\. Scores of1–2indicate a response that is irrelevant, contradicts the conversation, refuses without justification, is unsafe when safety is required, fails to answer the user, or is dominated by hallucinated content\. When several issues are present, choose the score that best reflects the overall usefulness of the response to the user\. Penalize strongly for errors that would cause the user to take a wrong action, violate an explicit constraint, or misunderstand the task outcome\. A response with a major factual error should not receive a score above 6, even if it is fluent\. A response that ignores the current user request or relies on the wrong dialogue context should not receive a score above 4\. A response that is correct but unnecessarily verbose, poorly organized, or missing minor details may still receive a high score if the user’s intent is satisfied\. Return only the final score\.

The judge model returns an integer rating from 1 to 10, which we normalize to the range\[0,1\]\[0,1\]as the true scorewi,tw\_\{i,t\}for evaluation metrics computation\.

### 6\.2\.Overall Performance

Figure 4\.Overallevaluation accuracy\(%\) on ShareGPT and MTBench\. Horizontal bars rank all candidate single LLMs and routing methods by average accuracy, while markers show the corresponding ShareGPT and MTBench scores\. The dashed line marks the strongest single LLM, making the routing gain of SWRouter visually explicit\.Ranked horizontal bar chart showing average evaluation accuracy for all single LLMs and routing methods, with markers for ShareGPT and MTBench scores\.As shown in Figure[4](https://arxiv.org/html/2609.11414#acmlabel4), SWRouter achieves the best overall performance across the two multi\-turn dialogue datasets\. Compared with the strongest single LLM, dolphin\-2\.9\-llama3\-8b, SWRouter improves the averageevaluation accuracyfrom 24\.63% to 40\.89%, corresponding to an absolute gain of 16\.26%\. The improvement is consistent across both datasets, with gains of 14\.29% on ShareGPT and 18\.22% on MTBench\.

Compared with Conv\-ID Context, which preserves the original dialogue segmentation by conversation IDs, SWRouter further improves the averageevaluation accuracyby 8\.22%\. This result suggests that exact ID\-based partitioning is not necessarily the optimal context construction strategy for multi\-turn routing\. Instead, dynamically constructing semantic windows provides more useful routing inputs\.

Moreover, ZOOTER equipped with the similarity\-window mechanism achieves 39\.21% averageevaluation accuracy, substantially outperforming Conv\-ID Context\. This indicates that the proposed similarity\-window design is general and can improve different routing backbones by constructing better multi\-turn contexts\.

### 6\.3\.OOD Generalization

Figure 5\.Evaluation accuracy\(%\) of SWRouter and Conv\-ID Context on out\-of\-distribution datasets\. PreAlgebra, MBPP, and C\-EVAL cover mathematical reasoning, code generation, and Chinese knowledge\-intensive evaluation, respectively\. The connected markers compare SWRouter with Conv\-ID Context on each task, and the right\-side annotations report the absolute gain or loss\. This visualization emphasizes that SWRouter improves the average OOD result despite not winning on every individual task\.Dumbbell chart comparing SWRouter and Conv\-ID Context on PreAlgebra, MBPP, C\-EVAL, and their average, with gain annotations\.We further evaluate the generalization ability of SWRouter under out\-of\-distribution scenarios, includingPreAlgebra,MBPP, andC\-EVAL\. These datasets cover mathematical reasoning, code generation, and Chinese knowledge\-intensive evaluation, respectively, and are related to established mathematical and Chinese multitask evaluation settings\([Cobbe et al\., 2021](https://arxiv.org/html/2609.11414#bib.bib28);[Li et al\., 2024a](https://arxiv.org/html/2609.11414#bib.bib27)\)\.

As shown in Figure[5](https://arxiv.org/html/2609.11414#acmlabel5), SWRouter achieves the best average OOD performance, reaching 43\.99% and outperforming Conv\-ID Context by 2\.68%\. In particular, SWRouter improves Conv\-ID Context by 10\.10% on PreAlgebra and 4\.28% on MBPP\. Although Conv\-ID Context performs better on C\-EVAL, SWRouter exhibits stronger average robustness across heterogeneous OOD tasks\.

This result indicates that SWRouter does not simply fit ID\-based routing labels\. Instead, the learned selection strategy captures transferable routing patterns that generalize beyond the original dialogue distribution\.

### 6\.4\.Performance Breakdown

Figure 6\.Ablation study results measured byevaluation accuracy\(%\) of SWRouter \. Each bar reports the accuracy drop relative to the full SWRouter model after removing one component\. The full model obtains 27\.98% on ShareGPT and 53\.79% on MTBench\. This drop\-oriented view highlights that similarity\-window construction is the dominant contributor, especially on MTBench, while both contrastive losses also provide consistent gains\.Bar chart showing accuracy drops from full SWRouter after removing similarity\-window construction, sample\-LLM loss, or sample\-sample loss\.We conduct ablation studies to quantify the contribution of each component in SWRouter \. As shown in Figure[6](https://arxiv.org/html/2609.11414#acmlabel6), removing any component consistently degrades performance, showing that all components are necessary for effective routing\.

Similarity window\.Removing the similarity window causes the largest performance drop\.Evaluation Accuracydecreases by 11\.33% on ShareGPT and 46\.57% on MTBench\. This confirms that adaptive context construction is the most important component of SWRouter \.

Sample\-LLM loss\.RemovingLsample\-LLML\_\{\\text\{sample\-LLM\}\}reduces performance to 25\.70% on ShareGPT and 38\.17% on MTBench\. This indicates that modeling the relationship between dialogue samples and candidate LLMs is critical for accurate routing\.

Sample\-sample loss\.RemovingLsample\-sampleL\_\{\\text\{sample\-sample\}\}also significantly hurts performance, especially on MTBench\. This shows that sample\-level representation alignment helps construct a more structured routing space\.

Actual score\.We further observe that proxy routing scores can substantially overestimate the final end\-to\-end performance if prompt construction is ignored\. This highlights the necessity of evaluating actual generated responses rather than relying only on intermediate routing objectives\.

### 6\.5\.Sensitivity Analysis

\(a\)Router performanceProuterP\_\{\\text\{router\}\}\.\(b\)Average value ofs​i¯\\overline\{si\}\.
Figure 7\.Effect of the similarity thresholdτ\\tauon router performanceProuterP\_\{\\text\{router\}\}and the average value ofs​i¯\\overline\{si\}\. The left subfigure shows how strongly the router improves over the average candidate\-model score under each threshold, while the right subfigure reports the resulting end\-to\-end routed response quality\. Together, the curves show that threshold selection affects both model\-selection effectiveness and absolute response quality, with the best operating region concentrated aroundτ=0\.91\\tau=0\.91\.Two line charts showing how the similarity threshold affects router performance and average evaluation accuracy\.\(a\)MT\-Bench\.\(b\)ShareGPT\.
Figure 8\.Effect of the similarity thresholdτ\\tauon the average value ofw¯\\bar\{w\}for MT\-Bench and ShareGPT\. Sincew¯\\bar\{w\}averages the true scores of all candidate LLMs under the constructed prompts, these curves isolate the prompt\-construction quality from the router’s model\-selection behavior\. The comparison shows how stricter or looser window partitioning changes the amount of useful context retained for the candidate model pool\.Two line charts showing how the similarity threshold affects construction accuracy on MT\-Bench and ShareGPT\.We analyze the sensitivity of SWRouter to the similarity thresholdτ\\tau\.

Effect of Similarity Thresholdτ\\tau\.We conduct a sensitivity study on the similarity thresholdτ\\tauwith respect to three key performance metrics: the average value ofs​i¯\\overline\{si\}, the average value ofw¯\\bar\{w\}, and router performanceProuterP\_\{\\text\{router\}\}\.

Average Value ofs​i¯\\overline\{si\}\(s​i¯=1n​∑i=1nsi\\overline\{si\}=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}s\_\{i\}\)\. Figure[7](https://arxiv.org/html/2609.11414#acmlabel7)\(b\) shows the effect ofτ\\tauons​i¯\\overline\{si\}\. The similarity\-window construction achieves its best performance atτ=0\.91\\tau=0\.91, reaching 40\.89% accuracy and outperforming the Conv\-ID Context baseline of 32\.67%\. Whenτ\\taumoves away from this value, the performance decreases noticeably\. For example, the accuracy drops to 25\.6% atτ=0\.85\\tau=0\.85, 22\.2% atτ=0\.90\\tau=0\.90, 20\.2% atτ=0\.92\\tau=0\.92, and 18\.3% atτ=0\.95\\tau=0\.95\. These results indicate thats​i¯\\overline\{si\}is highly sensitive to the similarity threshold, and thatτ=0\.91\\tau=0\.91provides the most effective similarity window\.

Average Value ofw¯\\bar\{w\}\.Figure[8](https://arxiv.org/html/2609.11414#acmlabel8)presents the average value ofw¯\\bar\{w\}under different similarity thresholdsτ\\tauon MT\-Bench and ShareGPT\. The results show thatw¯\\bar\{w\}is strongly affected by the similarity threshold\. On MT\-Bench,w¯\\bar\{w\}increases from 9\.60% atτ=0\.85\\tau=0\.85to 12\.10% atτ=0\.87\\tau=0\.87, and further reaches the maximum value of 23\.70% atτ=0\.91\\tau=0\.91\. However, when the threshold is increased toτ=0\.95\\tau=0\.95,w¯\\bar\{w\}drops sharply to 13\.70%, suggesting that an overly strict similarity constraint can substantially reduce the retained high\-quality samples\.

A similar trend can also be observed on ShareGPT\. The average value ofw¯\\bar\{w\}is 6\.00% atτ=0\.85\\tau=0\.85, increases significantly to 14\.90% atτ=0\.87\\tau=0\.87, and remains at a comparable level of 14\.70% whenτ=0\.91\\tau=0\.91\. It then decreases to 12\.80% atτ=0\.95\\tau=0\.95\. These results suggest that ShareGPT is slightly more stable than MT\-Bench around the optimal region, but its performance is still affected by threshold selection\.

Overall, the high\-quality region is mainly concentrated aroundτ=0\.91\\tau=0\.91\. MT\-Bench achieves its best result atτ=0\.91\\tau=0\.91, while ShareGPT maintains competitive performance in the range ofτ∈\[0\.87,0\.91\]\\tau\\in\[0\.87,0\.91\]\. Considering both datasets,τ=0\.91\\tau=0\.91provides a stable trade\-off between sample quality and coverage, and we therefore adopt it as the default threshold\.

Router Performance\(ProuterP\_\{\\text\{router\}\}\)\. Figure[7](https://arxiv.org/html/2609.11414#acmlabel7)\(a\) illustrates the effect of the similarity thresholdτ\\tauon router performanceProuterP\_\{\\text\{router\}\}\. Overall,ProuterP\_\{\\text\{router\}\}is influenced by the choice ofτ\\tau, although its variation is less pronounced than that ofw¯\\bar\{w\}\. Specifically,ProuterP\_\{\\text\{router\}\}is 15\.4% atτ=0\.87\\tau=0\.87, increases to its peak of 21\.5% atτ=0\.91\\tau=0\.91, and then decreases to 14\.7% atτ=0\.92\\tau=0\.92and 12\.1% atτ=0\.95\\tau=0\.95\.

These results indicate that router performance still depends on the similarity threshold, but the overall fluctuation is relatively moderate\. Compared withw¯\\bar\{w\},ProuterP\_\{\\text\{router\}\}is less sensitive to changes inτ\\tau\. Nevertheless, the best router performance is still achieved atτ=0\.91\\tau=0\.91, further supporting our choice ofτ=0\.91\\tau=0\.91as the default threshold\.

### 6\.6\.Robustness Analysis

#### 6\.6\.1\.Extension: Top\-K Retrieval for Long\-Range Dependency Recovery

As an extension to the core similarity\-based windowing approach, we investigate whether augmenting it with a lightweight Top\-K retrieval mechanism can further recover semantically related queries that are separated by unrelated turns\.

We simulate a scenario where a user issues a querya, then several unrelated queries, and finally a related queryb\. For eachb, similarity\-based Top\-K retrieval is performed over the dialogue history to test whetheracan be recalled\.

Table 2\.Recall performance of similarity\-based retrieval under different levels of semantic interference\. Inserted queries denote unrelated turns placed between two semantically related user requests; Top\-KKrecall measures whether the retrieval module can recover the earlier related turn from the dialogue history\.We also test a more challenging case where 20 unrelated queries are inserted betweenaandb\. The results are shown below\.

Table 3\.Retrieval robustness under the high\-difficulty setting with 20 distractor turns\. This setting stresses long\-range dependency recovery by separating two related requests with many unrelated turns, and the reported recall indicates whether Top\-KKretrieval can still recover the relevant earlier query\.As shown in Tables[2](https://arxiv.org/html/2609.11414#S6.T2)and[3](https://arxiv.org/html/2609.11414#S6.T3), similarity\-based retrieval substantially improves robustness under semantic drift\. Top\-3 retrieval provides an effective trade\-off between recall and efficiency\. Even under high\-difficulty conditions with 20 distractors, Top\-5 recall remains above 66%, indicating strong resilience\. These results demonstrate that augmenting local similarity\-based segmentation with retrieval mechanisms can further enhance context reconstruction in realistic multi\-turn dialogue scenarios\.

#### 6\.6\.2\.Semantic Coherence in Long Dialogues

To further analyze the potential semantic drift as the dialogue length increases, we conducted a similarity analysis over multi\-turn dialogues on theShareGPTandMT\-Benchdatasets\. Specifically, we computed the cosine similarity between the first and last user turns within each dialogue window under different minimum round sizes \(20, 25, and 30\)\.

Table 4\.Cosine similarity between the first and last user turns across dialogues of varying lengths\. For each dataset, we group dialogues by minimum round size and report the proportion of long\-range turn pairs above or below the default similarity thresholdτ=0\.91\\tau=0\.91, together with the maximum, minimum, and average similarity\. The table characterizes how quickly semantic drift appears as dialogue length increases\.As shown in Table[4](https://arxiv.org/html/2609.11414#S6.T4), inShareGPT, over 70% of long\-range pairs maintain a cosine similarity above 0\.91, and the average similarity remains above 0\.92, suggesting strong semantic consistency even across extended dialogues\. InMT\-Bench, the average similarity is slightly lower but still indicates limited semantic drift\. Extremely low similarity cases \(<<0\.80\) are rare, implying that our similarity\-based segmentation remains semantically coherent over long dialogue windows\.

### 6\.7\.Detailed Overhead Analysis

We conduct a comprehensive analysis of the computational overhead introduced by SWRouter from three aspects: inference latency, training cost, and token generation cost\.

#### 6\.7\.1\.Latency Analysis

Table 5\.Time consumption of SWRouter during inference\. The table decomposes latency into similarity calculation for window construction, router selection, and downstream LLM inference, showing that the additional cost introduced by similarity\-window routing is small relative to candidate\-model generation\.As shown in Table[5](https://arxiv.org/html/2609.11414#S6.T5), the similarity calculation in SWRouter accounts for less than 1% \(approximately 2\.2‰\) of the total inference time, indicating high computational efficiency\. The dominant cost comes from LLM inference, which is consistent with all routing\-based systems\.

#### 6\.7\.2\.Training Cost Analysis

The training process of SWRouter consists of four steps:

1. \(1\)Dataset window partitioning: We use a lightweight encoder \(microsoft/mdeberta\-v3\-base\) with approximately 86M backbone parameters, which is much smaller than the 7B/8B candidate LLMs, making this step’s cost negligible\.
2. \(2\)Candidate response generation: We use the seven small models mentioned in the paper \(Mistral\-7B, MetaMath\-Mistral\-7B, zephyr\-7b\-beta, Chinese\-Mistral\-7B, dolphin\-2\.6\-mistral\-7b, Meta\-Llama\-3\-8B, dolphin\-2\.9\-llama3\-8b\) to generate candidate answers for each query\.
3. \(3\)Judge\-model scoring: We use GPT for scoring\. Since the output is just a single numerical score, this step does not significantly increase memory usage\.
4. \(4\)Router training: The similarity\-window encoder introduces limited overhead: microsoft/mdeberta\-v3\-base has approximately 86M backbone parameters, which is much smaller than the 7B/8B candidate LLMs\. Moreover, the encoder is used only for lightweight similarity computation, while the dominant cost remains candidate LLM inference\. Router training is performed only once before deployment, making it a one\-time overhead\.

#### 6\.7\.3\.Token Generation Cost

To evaluate the efficiency of model selection, we compare the token\-level costs between SWRouter and the Conv\-ID Context method\. Due to the unavailability of precise pricing information for small\-sized models such as 7B and 8B, we adopt token count as a unified proxy for cost estimation\.

Table 6\.Overall evaluation accuracy and token cost comparison\. Evaluation accuracy measures response quality after routing, while token cost counts the generated tokens used by each routing strategy as a model\-agnostic proxy for serving cost\. The comparison highlights the effectiveness–cost tradeoff between Conv\-ID Context, SimWindow \+ ZOOTER, and SWRouter \.As shown in Table[6](https://arxiv.org/html/2609.11414#S6.T6), SWRouter incurs approximately1\.23×1\.23\\timesthe token cost of Conv\-ID Context while achieving an8\.22%8\.22\\%absolute improvement in evaluation accuracy\. Compared to ZOOTER, SWRouter requires only1\.02×1\.02\\timesthe token cost, yet delivers a further1\.69%1\.69\\%accuracy gain\. This result suggests that SWRouter achieves a reasonable trade\-off between performance and efficiency, despite lacking access to ground\-truth optimal selections\.

#### 6\.7\.4\.Additional Analysis

We further analyze the experimental results from the following four aspects:

\(1\) Prompt Construction Impact\.The similar performance patterns across models at different similarity thresholds suggest that prompt construction, rather than model architecture, is the primary performance driver\.τ=0\.91\\tau=0\.91performs best among tested thresholds; performance is relatively strong in the neighborhood around 0\.91\.

\(2\) Router\-Score Correlation\.Analysis reveals a weak negative correlation \(Pearson R = \-0\.2301\) between router performance and true scores, indicating that higher true scores do not necessarily translate to better routing performance\. This underscores the importance of decoupled evaluation metrics\.

\(3\) Prompt Construction Dominance\.τ=0\.91\\tau=0\.91performs best among tested thresholds; performance is relatively strong in the neighborhood around 0\.91, emphasizing that prompt construction quality outweighs router performance in multi\-turn dialogue systems\.

\(4\) Exact Partitioning is Not Necessarily Optimal\.This can be compared from three aspects: \(i\) OnEvaluation Accuracy\(s​i¯\\overline\{si\}\), whenτ=0\.91\\tau=0\.91with 40\.89%, it is 8\.22% higher than Conv\-ID Context \(32\.67%\), proving that the router model trained with exact partitioning does not perform as well as SWRouter \. \(ii\) OnConstruction Accuracy\(w¯\\bar\{w\}\), when the similarity thresholdτ\\tauis 0\.90, the average score of 7 individual models significantly outperforms the prompt constructed by Conv\-ID Context, indicating that in multi\-turn dialogue scenarios, strictly accurate context partitioning cannot be considered the optimal context segmentation method\. \(iii\) OnRouter Performance\(ProuterP\_\{\\text\{router\}\}\), the average performance for bothτ=0\.85\\tau=0\.85andτ=0\.91\\tau=0\.91surpass Conv\-ID Context, meaning that the prompt constructed by Conv\-ID Context also fails to train the optimal router performance\.

Overall, SWRouter introduces limited additional computational overhead while substantially improving routing performance, demonstrating a favorable effectiveness–efficiency tradeoff for multi\-turn LLM routing\.

## 7\.Related Work

Large language models have become increasingly heterogeneous in capability, cost, latency, and domain specialization\. This heterogeneity has motivated systems that use multiple LLMs rather than committing to a single model for all inputs\. Existing work can be broadly grouped into LLM ensembling and cascading, single\-turn LLM routing, and context\-aware routing for multi\-turn dialogue\.

### 7\.1\.LLM Ensembling and Cascading

LLM ensembling aims to improve response quality by consulting multiple models or multiple generations before producing the final answer\. A simple and widely used strategy is voting or self\-consistency, where multiple reasoning paths or model outputs are aggregated to reduce variance and improve reliability\([Li et al\., 2024b](https://arxiv.org/html/2609.11414#bib.bib12);[Wang et al\., 2023](https://arxiv.org/html/2609.11414#bib.bib14)\)\. More structured ensemble methods go beyond voting: LLM\-Blender\([Jiang et al\., 2023b](https://arxiv.org/html/2609.11414#bib.bib15)\)first ranks candidate outputs with PairRanker and then synthesizes a final response with GenFuser, showing that complementary model outputs can be combined into a stronger answer\.

Another line of work focuses on cascades, where models are queried sequentially according to estimated difficulty, confidence, or cost\. FrugalGPT\([Chen et al\., 2023](https://arxiv.org/html/2609.11414#bib.bib16)\)studies cost\-effective model selection through cascaded calls, while language\-model cascades further explore uncertainty\-aware and token\-level routing policies\([Gupta et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib17);[Yue et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib21)\)\. Online cascade learning extends this idea to streaming or adaptive inference settings\([Nie et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib18)\)\. These methods reduce cost compared with invoking all available models, but they may still require multiple model calls for a single user request\. In contrast, SWRouter follows the routing paradigm: it aims to choose one suitable candidate LLM after constructing the multi\-turn prompt, thereby avoiding repeated generation from many models at inference time\.

### 7\.2\.Single\-turn LLM Routing

LLM routing selects the most suitable model for a query without necessarily calling all candidate LLMs\. Early routing studies often formulate the problem as correctness prediction or reward prediction for each candidate model\. Shnitzer et al\.\([Shnitzer et al\., 2023](https://arxiv.org/html/2609.11414#bib.bib19)\)construct benchmark datasets for LLM routing and train model\-specific binary classifiers to estimate whether each model will answer correctly\. ZOOTER\([Lu et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib20)\)aligns a router with reward\-model supervision and shows that learned routers can outperform static model selection\. Cost\-aware routing methods such as C2MAB\-V\([Dai et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib25)\)further incorporate online decision making and reward signals to balance model quality and inference cost\.

Recent work also studies representation\-based routing\. LoraRetriever\([Zhao et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib22)\)routes inputs by predicting task identity and retrieving suitable LoRA modules, while Srivatsa et al\.\([Srivatsa et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib23)\)analyze classifier\-based and clustering\-based routing strategies\. RouterDC\([Chen et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib24)\)introduces a dual contrastive learning objective that models both query\-model relations and query\-query relations, making it a strong single\-turn router backbone\. These methods provide useful supervision and representation\-learning tools for model selection, and SWRouter builds on this direction\. However, they generally assume that each routing instance is already a complete prompt\. This assumption is reasonable for single\-turn benchmarks, but it becomes fragile in multi\-turn dialogue because the routing input itself depends on how historical context is selected and assembled\.

### 7\.3\.Context Construction for Multi\-turn Dialogue

Multi\-turn dialogue introduces a context\-construction problem before routing can even begin\. A conversational request may depend on previous constraints, definitions, or preferences, but the full dialogue history may also contain obsolete or off\-topic content\. Directly concatenating all previous turns can therefore increase prompt length and introduce irrelevant information, whereas using only the current turn can omit essential context\. Long\-context benchmarks such as MT\-Bench\-101\([Bai et al\., 2024](https://arxiv.org/html/2609.11414#bib.bib38)\)and real dialogue data such as ShareGPT\([Schmid, 2023](https://arxiv.org/html/2609.11414#bib.bib39)\)highlight that multi\-turn tasks often require tracking user intent across turns rather than treating each prompt independently\.

Several adjacent research areas offer partial solutions\. Retrieval\-based context construction can recover long\-range dependencies by searching dialogue history for semantically related turns\. Sequence segmentation and similarity\-based partitioning, rooted in broader sequence analysis and string\-processing ideas\([Aho and Ullman, 1972](https://arxiv.org/html/2609.11414#bib.bib1);[Gusfield, 1997](https://arxiv.org/html/2609.11414#bib.bib5);[Chandra et al\., 1981](https://arxiv.org/html/2609.11414#bib.bib3)\), provide a way to divide a conversation into locally coherent windows\. Neural encoders such as DeBERTa\([He et al\., 2021](https://arxiv.org/html/2609.11414#bib.bib41)\)can then map turns into dense representations for semantic matching\. Nevertheless, these techniques are usually studied as prompt\-construction or retrieval components rather than as part of a routing evaluation framework\. SWRouter integrates similarity\-based window construction with router learning so that the selected model is conditioned on a prompt that is explicitly designed for multi\-turn relevance\.

### 7\.4\.Evaluation of Routing Under Constructed Prompts

Most routing evaluations focus on whether the router selects a high\-scoring model for a given input\. This is appropriate when the input prompt is fixed and complete, but it can be misleading when prompt construction varies\. In multi\-turn dialogue, a poorly constructed prompt can lower the absolute scores of all candidate LLMs even if their relative ranking remains unchanged\. Conversely, a better prompt can improve all candidate responses while leaving router\-selection difficulty similar\. Standard aggregate metrics such as benchmark accuracy\([Hendrycks et al\., 2021](https://arxiv.org/html/2609.11414#bib.bib29);[Li et al\., 2024a](https://arxiv.org/html/2609.11414#bib.bib27);[Cobbe et al\., 2021](https://arxiv.org/html/2609.11414#bib.bib28)\)or reward\-model score therefore conflate two factors: whether the constructed prompt preserves enough information, and whether the router chooses the best model under that prompt\.

The key difference between SWRouter and prior routing work is this explicit separation\. Instead of evaluating only the selected model’s final score, SWRouter reports evaluation accuracy, construction accuracy, and router performance\. This decoupled view makes it possible to determine whether improvements come from better prompt construction, better model selection, or both\. As a result, SWRouter extends single\-turn LLM routing to a setting where context segmentation is a first\-class part of the routing problem rather than a preprocessing detail\.

## 8\.Conclusion

This paper presents SWRouter , a similarity\-contractive window router that addresses two fundamental challenges in multi\-turn LLM routing: information loss and information confusion caused by improper context segmentation, and attribution bias in router evaluation that conflates prompt construction quality with routing effectiveness\.

To tackle these challenges, SWRouter introduces a similarity\-based window partitioning mechanism that dynamically constructs semantically coherent prompts from dialogue history, and a decoupled evaluation framework comprising three complementary metrics: evaluation accuracy \(s​i¯\\overline\{si\}\), construction accuracy \(w¯\\bar\{w\}\), and router performance \(ProuterP\_\{\\text\{router\}\}\)\. Extensive experiments on ShareGPT and MTBench demonstrate that SWRouter achieves16\.26%and8\.22%gains over the best individual LLM and the Conv\-ID Context baseline, respectively\. On out\-of\-distribution tasks \(PreAlgebra, MBPP, and C\-EVAL\), SWRouter further outperforms Conv\-ID Context by2\.68%on average, demonstrating strong generalization\. Decoupled metric analysis confirms that the gains are primarily driven by improved construction accuracy, whileProuter\>1P\_\{\\text\{router\}\}\>1consistently validates effective model selection\. These results highlight that multi\-turn LLM routing requires a joint design of context construction and evaluation, rather than a direct extension of single\-turn routing methods\.

Despite these advances, the current framework is evaluated on 7B/8B scale models; its behavior under larger or proprietary LLMs remains to be studied\. Future work may extend the routing framework to heterogeneous model pools spanning different scales and explore its applicability to proprietary LLMs\.

## References

- Aho and Ullman \(1972\)A\. V\. Aho and J\. D\. UllmanThe theory of parsing, translation, and compiling\.Vol\.1,Prentice\-Hall,Englewood Cliffs, NJ\.Cited by:[§4\.2](https://arxiv.org/html/2609.11414#S4.SS2.p1.1),[§7\.3](https://arxiv.org/html/2609.11414#S7.SS3.p2.1)\.
- American Psychological Association \(1983\)American Psychological AssociationPublications manual\.American Psychological Association,Washington, DC\.Cited by:[§6\.1\.1](https://arxiv.org/html/2609.11414#S6.SS1.SSS1.p1.1)\.
- Ando and Zhang \(2005\)R\. K\. Ando and T\. ZhangA framework for learning predictive structures from multiple tasks and unlabeled data\.Journal of Machine Learning Research6,pp\. 1817–1853\.External Links:[Link](http://jmlr.org/papers/v6/ando05a.html)Cited by:[§4\.3\.2](https://arxiv.org/html/2609.11414#S4.SS3.SSS2.p1.1)\.
- Andrew and Gao \(2007\)G\. Andrew and J\. GaoScalable training of L1\-regularized log\-linear models\.InProceedings of the 24th International Conference on Machine Learning,Corvallis, OR, USA,pp\. 33–40\.External Links:[Document](https://dx.doi.org/10.1145/1273496.1273501)Cited by:[§4\.3\.2](https://arxiv.org/html/2609.11414#S4.SS3.SSS2.p1.1)\.
- Anthropic \(2024\)AnthropicClaude\.Note:[https://docs\.anthropic\.com/en/docs/welcome](https://docs.anthropic.com/en/docs/welcome)Cited by:[§1](https://arxiv.org/html/2609.11414#S1.p1.1)\.
- Baiet al\.\(2024\)G\. Bai, J\. Liu, X\. Bu, Y\. He, J\. Liu, Z\. Zhou, Z\. Lin, W\. Su, T\. Ge, B\. Zheng, and W\. OuyangMT\-bench\-101: A fine\-grained benchmark for evaluating large language models in multi\-turn dialogues\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics,Bangkok, Thailand,pp\. 7421–7454\.External Links:[Document](https://dx.doi.org/10.18653/V1/2024.ACL-LONG.401),[Link](https://aclanthology.org/2024.acl-long.401)Cited by:[§5](https://arxiv.org/html/2609.11414#S5.p3.1),[§7\.3](https://arxiv.org/html/2609.11414#S7.SS3.p1.1)\.
- Chandraet al\.\(1981\)A\. K\. Chandra, D\. Kozen, and L\. J\. StockmeyerAlternation\.Journal of the ACM28\(1\),pp\. 114–133\.External Links:[Document](https://dx.doi.org/10.1145/322234.322243)Cited by:[§4\.2](https://arxiv.org/html/2609.11414#S4.SS2.p1.1),[§7\.3](https://arxiv.org/html/2609.11414#S7.SS3.p2.1)\.
- Chenet al\.\(2023\)L\. Chen, M\. Zaharia, and J\. ZouFrugalGPT: how to use large language models while reducing cost and improving performance\.CoRRabs/2305\.05176\.External Links:[Link](https://doi.org/10.48550/arXiv.2305.05176),[Document](https://dx.doi.org/10.48550/ARXIV.2305.05176),2305\.05176Cited by:[§7\.1](https://arxiv.org/html/2609.11414#S7.SS1.p2.1)\.
- Chenet al\.\(2024\)S\. Chen, W\. Jiang, B\. Lin, J\. T\. Kwok, and Y\. ZhangRouterDC: query\-based router by dual contrastive learning for assembling large language models\.InAdvances in Neural Information Processing Systems,Vol\.37,Vancouver, BC, Canada,pp\. 66305–66328\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/7a641b8ec86162fc875fb9f6456a542f-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2609.11414#S1.p2.1),[§2\.2](https://arxiv.org/html/2609.11414#S2.SS2.p1.1),[§2\.2](https://arxiv.org/html/2609.11414#S2.SS2.p2.1),[§4\.3](https://arxiv.org/html/2609.11414#S4.SS3.p2.2),[§7\.2](https://arxiv.org/html/2609.11414#S7.SS2.p2.1)\.
- Chenet al\.\(2025\)X\. Chen, Y\. Li, H\. Cai, Z\. Ma, X\. Chen, H\. Xiong, S\. Wang, B\. He, L\. Sun, and D\. YinMulti\-agent proactive information seeking with adaptive llm orchestration for non\-factoid question answering\.InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V\. 2,pp\. 4341–4352\.Cited by:[§1](https://arxiv.org/html/2609.11414#S1.p1.1)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. SchulmanTraining verifiers to solve math word problems\.CoRRabs/2110\.14168\.External Links:[Link](https://arxiv.org/abs/2110.14168),2110\.14168Cited by:[§6\.3](https://arxiv.org/html/2609.11414#S6.SS3.p1.1),[§7\.4](https://arxiv.org/html/2609.11414#S7.SS4.p1.1)\.
- Cognitive Computations \(2024a\)Cognitive ComputationsCognitivecomputations/dolphin\-2\.6\-mistral\-7b\.Note:[https://huggingface\.co/cognitivecomputations/dolphin\-2\.6\-mistral\-7b](https://huggingface.co/cognitivecomputations/dolphin-2.6-mistral-7b)Cited by:[§5](https://arxiv.org/html/2609.11414#S5.p2.1)\.
- Cognitive Computations \(2024b\)Cognitive ComputationsCognitivecomputations/dolphin\-2\.9\-llama3\-8b\.Note:[https://huggingface\.co/cognitivecomputations/dolphin\-2\.9\-llama3\-8b](https://huggingface.co/cognitivecomputations/dolphin-2.9-llama3-8b)Cited by:[§5](https://arxiv.org/html/2609.11414#S5.p2.1)\.
- Daiet al\.\(2024\)X\. Dai, J\. Li, X\. Liu, A\. Yu, and J\. C\. S\. LuiCost\-effective online multi\-LLM selection with versatile reward models\.CoRRabs/2405\.16587\.External Links:[Link](https://doi.org/10.48550/arXiv.2405.16587),[Document](https://dx.doi.org/10.48550/ARXIV.2405.16587),2405\.16587Cited by:[§1](https://arxiv.org/html/2609.11414#S1.p2.1),[§7\.2](https://arxiv.org/html/2609.11414#S7.SS2.p1.1)\.
- DeepSeek\-AI \(2024\)DeepSeek\-AIDeepSeek\-V3 technical report\.CoRRabs/2412\.19437\.External Links:[Link](https://doi.org/10.48550/arXiv.2412.19437),[Document](https://dx.doi.org/10.48550/ARXIV.2412.19437),2412\.19437Cited by:[§1](https://arxiv.org/html/2609.11414#S1.p1.1)\.
- Gaoet al\.\(2023\)L\. Gao, J\. Tow, B\. Abbasi, S\. Biderman, S\. Black, A\. DiPofi, C\. Foster, L\. Golding, J\. Hsu, A\. L\. Noac’h, H\. Li, K\. McDonell, N\. Muennighoff, C\. Ociepa, J\. Phang, L\. Reynolds, H\. Schoelkopf, A\. Skowron, L\. Sutawika, E\. Tang, A\. Thite, B\. Wang, K\. Wang, and A\. ZouA framework for few\-shot language model evaluation\.Note:[https://zenodo\.org/records/10256836](https://zenodo.org/records/10256836)External Links:[Document](https://dx.doi.org/10.5281/zenodo.10256836)Cited by:[§5](https://arxiv.org/html/2609.11414#S5.p5.1)\.
- Gemini Team \(2023\)Gemini TeamGemini: A family of highly capable multimodal models\.CoRRabs/2312\.11805\.External Links:[Link](https://doi.org/10.48550/arXiv.2312.11805),[Document](https://dx.doi.org/10.48550/ARXIV.2312.11805),2312\.11805Cited by:[§1](https://arxiv.org/html/2609.11414#S1.p1.1)\.
- Guptaet al\.\(2024\)N\. Gupta, H\. Narasimhan, W\. Jitkrittum, A\. S\. Rawat, A\. K\. Menon, and S\. KumarLanguage model cascades: token\-level uncertainty and beyond\.CoRRabs/2404\.10136\.External Links:[Link](https://doi.org/10.48550/arXiv.2404.10136),[Document](https://dx.doi.org/10.48550/ARXIV.2404.10136),2404\.10136Cited by:[§7\.1](https://arxiv.org/html/2609.11414#S7.SS1.p2.1)\.
- Gusfield \(1997\)D\. GusfieldAlgorithms on strings, trees, and sequences: computer science and computational biology\.Cambridge University Press,Cambridge, UK\.External Links:[Document](https://dx.doi.org/10.1017/CBO9780511574931)Cited by:[§4\.2](https://arxiv.org/html/2609.11414#S4.SS2.p1.1),[§7\.3](https://arxiv.org/html/2609.11414#S7.SS3.p2.1)\.
- Heet al\.\(2021\)P\. He, X\. Liu, J\. Gao, and W\. ChenDeBERTa: decoding\-enhanced BERT with disentangled attention\.InThe Ninth International Conference on Learning Representations,Virtual Event, Austria\.External Links:[Link](https://openreview.net/forum?id=XPZIaotutsD)Cited by:[§5](https://arxiv.org/html/2609.11414#S5.p6.1),[§7\.3](https://arxiv.org/html/2609.11414#S7.SS3.p2.1)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. SteinhardtMeasuring massive multitask language understanding\.InThe Ninth International Conference on Learning Representations,Virtual Event, Austria\.External Links:[Link](https://openreview.net/forum?id=d7KBjmI3GmQ)Cited by:[§5](https://arxiv.org/html/2609.11414#S5.p5.1),[§7\.4](https://arxiv.org/html/2609.11414#S7.SS4.p1.1)\.
- Jianget al\.\(2023a\)A\. Q\. Jiang, A\. Sablayrolles, A\. Mensch, C\. Bamford, D\. S\. Chaplot, D\. de Las Casas, F\. Bressand, G\. Lengyel, G\. Lample, L\. Saulnier, L\. R\. Lavaud, M\. Lachaux, P\. Stock, T\. L\. Scao, T\. Lavril, T\. Wang, T\. Lacroix, and W\. E\. SayedMistral 7b\.CoRRabs/2310\.06825\.External Links:[Link](https://doi.org/10.48550/arXiv.2310.06825),[Document](https://dx.doi.org/10.48550/ARXIV.2310.06825),2310\.06825Cited by:[§5](https://arxiv.org/html/2609.11414#S5.p2.1)\.
- Jianget al\.\(2023b\)D\. Jiang, X\. Ren, and B\. Y\. LinLLM\-Blender: ensembling large language models with pairwise ranking and generative fusion\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics,Toronto, Canada,pp\. 14165–14178\.External Links:[Document](https://dx.doi.org/10.18653/V1/2023.ACL-LONG.792),[Link](https://aclanthology.org/2023.acl-long.792)Cited by:[§7\.1](https://arxiv.org/html/2609.11414#S7.SS1.p1.1)\.
- Kullback and Leibler \(1951\)S\. Kullback and R\. A\. LeiblerOn information and sufficiency\.The Annals of Mathematical Statistics22\(1\),pp\. 79–86\.External Links:[Document](https://dx.doi.org/10.1214/aoms/1177729694)Cited by:[§4\.3](https://arxiv.org/html/2609.11414#S4.SS3.p2.2)\.
- Liet al\.\(2024a\)H\. Li, Y\. Zhang, F\. Koto, Y\. Yang, H\. Zhao, Y\. Gong, N\. Duan, and T\. BaldwinCMMLU: measuring massive multitask language understanding in chinese\.InFindings of the Association for Computational Linguistics: ACL 2024,Bangkok, Thailand,pp\. 11260–11285\.External Links:[Document](https://dx.doi.org/10.18653/V1/2024.FINDINGS-ACL.671),[Link](https://aclanthology.org/2024.findings-acl.671)Cited by:[§6\.3](https://arxiv.org/html/2609.11414#S6.SS3.p1.1),[§7\.4](https://arxiv.org/html/2609.11414#S7.SS4.p1.1)\.
- Liet al\.\(2024b\)J\. Li, Q\. Zhang, Y\. Yu, Q\. Fu, and D\. YeMore agents is all you need\.CoRRabs/2402\.05120\.External Links:[Link](https://doi.org/10.48550/arXiv.2402.05120),[Document](https://dx.doi.org/10.48550/ARXIV.2402.05120),2402\.05120Cited by:[§7\.1](https://arxiv.org/html/2609.11414#S7.SS1.p1.1)\.
- Liet al\.\(2025\)Y\. Li, H\. Cai, R\. Kong, X\. Chen, J\. Chen, J\. Yang, H\. Zhang, J\. Li, J\. Wu, Y\. Chen,et al\.Towards ai search paradigm\.arXiv preprint arXiv:2506\.17188\.Cited by:[§1](https://arxiv.org/html/2609.11414#S1.p1.1)\.
- Liet al\.\(2026a\)Y\. Li, J\. Chen, X\. Chen, Z\. Li, H\. Zhang, R\. Kong, J\. Li, X\. Ma, H\. Cai, L\. Su,et al\.Retain to refine: adaptive online question answering via query routing and long\-short memory\.InProceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V\. 1,pp\. 2312–2322\.Cited by:[§1](https://arxiv.org/html/2609.11414#S1.p1.1)\.
- Liet al\.\(2026b\)Y\. Li, R\. Kong, X\. Chen, C\. Zhang, J\. Chen, C\. Deng, X\. Ma, H\. Zhang, T\. Peng, H\. Cai,et al\.Probe\-and\-fetch: dynamic kv cache pruning for accelerated long\-context inference in web\-scale ai search\.InProceedings of the ACM Web Conference 2026,pp\. 8127–8137\.Cited by:[§1](https://arxiv.org/html/2609.11414#S1.p1.1)\.
- Liet al\.\(2026c\)Y\. Li, R\. Kong, Z\. Lyu, Q\. Li, X\. Chen, H\. Cai, L\. Yan, S\. Wang, J\. Zhao, G\. Zhu,et al\.Flexspec: frozen drafts meet evolving targets in edge\-cloud collaborative llm speculative decoding\.IEEE Transactions on Mobile Computing\.Cited by:[§1](https://arxiv.org/html/2609.11414#S1.p1.1)\.
- Llama Team \(2024\)Llama TeamThe llama 3 herd of models\.CoRRabs/2407\.21783\.External Links:[Link](https://doi.org/10.48550/arXiv.2407.21783),[Document](https://dx.doi.org/10.48550/ARXIV.2407.21783),2407\.21783Cited by:[§5](https://arxiv.org/html/2609.11414#S5.p2.1)\.
- Loshchilov and Hutter \(2019\)I\. Loshchilov and F\. HutterDecoupled weight decay regularization\.InInternational Conference on Learning Representations,New Orleans, LA, USA\.External Links:[Link](https://openreview.net/forum?id=Bkg6RiCqY7)Cited by:[§5](https://arxiv.org/html/2609.11414#S5.p7.1)\.
- Luet al\.\(2024\)K\. Lu, H\. Yuan, R\. Lin, J\. Lin, Z\. Yuan, C\. Zhou, and J\. ZhouRouting to the expert: efficient reward\-guided ensemble of large language models\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,Mexico City, Mexico,pp\. 1964–1974\.External Links:[Document](https://dx.doi.org/10.18653/V1/2024.NAACL-LONG.109),[Link](https://aclanthology.org/2024.naacl-long.109)Cited by:[§4\.3](https://arxiv.org/html/2609.11414#S4.SS3.p2.2),[§5](https://arxiv.org/html/2609.11414#S5.p4.1),[§7\.2](https://arxiv.org/html/2609.11414#S7.SS2.p1.1)\.
- Nieet al\.\(2024\)L\. Nie, Z\. Ding, E\. Hu, C\. M\. Jermaine, and S\. ChaudhuriOnline cascade learning for efficient inference over streams\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,Vienna, Austria,pp\. 38071–38090\.External Links:[Link](https://proceedings.mlr.press/v235/nie24a.html)Cited by:[§7\.1](https://arxiv.org/html/2609.11414#S7.SS1.p2.1)\.
- OpenAI \(2022\)OpenAIChatGPT: optimizing language models for dialogue\.Note:[https://openai\.com/blog/chatgpt](https://openai.com/blog/chatgpt)Cited by:[§1](https://arxiv.org/html/2609.11414#S1.p1.1)\.
- Rafailovet al\.\(2023\)R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. FinnDirect preference optimization: your language model is secretly a reward model\.InAdvances in Neural Information Processing Systems,Vol\.36,New Orleans, LA, USA,pp\. 53728–53741\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/a85b405ed65c6477a4fe8302b5e06ce7-Abstract-Conference.html)Cited by:[§5](https://arxiv.org/html/2609.11414#S5.p2.1)\.
- Rasooli and Tetreault \(2015\)M\. S\. Rasooli and J\. R\. TetreaultYara parser: A fast and accurate dependency parser\.CoRRabs/1503\.06733\.External Links:[Link](http://arxiv.org/abs/1503.06733),1503\.06733Cited by:[§5](https://arxiv.org/html/2609.11414#S5.p6.1)\.
- Schmid \(2023\)P\. SchmidShareGPT 90k raw dataset\.Note:[https://huggingface\.co/datasets/philschmid/sharegpt\-raw/tree/main/sharegpt\_90k\_raw\_dataset](https://huggingface.co/datasets/philschmid/sharegpt-raw/tree/main/sharegpt_90k_raw_dataset)Cited by:[§5](https://arxiv.org/html/2609.11414#S5.p3.1),[§7\.3](https://arxiv.org/html/2609.11414#S7.SS3.p1.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. GuoDeepSeekMath: pushing the limits of mathematical reasoning in open language models\.CoRRabs/2402\.03300\.External Links:[Link](https://doi.org/10.48550/arXiv.2402.03300),[Document](https://dx.doi.org/10.48550/ARXIV.2402.03300),2402\.03300Cited by:[§1](https://arxiv.org/html/2609.11414#S1.p1.1)\.
- Shnitzeret al\.\(2023\)T\. Shnitzer, A\. Ou, M\. Silva, K\. Soule, Y\. Sun, J\. Solomon, N\. Thompson, and M\. YurochkinLarge language model routing with benchmark datasets\.CoRRabs/2309\.15789\.External Links:[Link](https://doi.org/10.48550/arXiv.2309.15789),[Document](https://dx.doi.org/10.48550/ARXIV.2309.15789),2309\.15789Cited by:[§7\.2](https://arxiv.org/html/2609.11414#S7.SS2.p1.1)\.
- Srivatsaet al\.\(2024\)K\. A\. Srivatsa, K\. K\. Maurya, and E\. KochmarHarnessing the power of multiple minds: lessons learned from LLM routing\.CoRRabs/2405\.00467\.External Links:[Link](https://doi.org/10.48550/arXiv.2405.00467),[Document](https://dx.doi.org/10.48550/ARXIV.2405.00467),2405\.00467Cited by:[§7\.2](https://arxiv.org/html/2609.11414#S7.SS2.p2.1)\.
- Touvronet al\.\(2023a\)H\. Touvron, T\. Lavril, G\. Izacard, X\. Martinet, M\. Lachaux, T\. Lacroix, B\. Rozière, N\. Goyal, E\. Hambro, F\. Azhar, A\. Rodriguez, A\. Joulin, E\. Grave, and G\. LampleLLaMA: open and efficient foundation language models\.CoRRabs/2302\.13971\.External Links:[Link](https://doi.org/10.48550/arXiv.2302.13971),[Document](https://dx.doi.org/10.48550/ARXIV.2302.13971),2302\.13971Cited by:[§1](https://arxiv.org/html/2609.11414#S1.p1.1),[§5](https://arxiv.org/html/2609.11414#S5.p2.1)\.
- Touvronet al\.\(2023b\)H\. Touvron, L\. Martin, K\. Stone, P\. Albert, A\. Almahairi, Y\. Babaei, N\. Bashlykov, S\. Batra, P\. Bhargava, S\. Bhosale, D\. Bikel, L\. Blecher, C\. Canton\-Ferrer, M\. Chen, G\. Cucurull, D\. Esiobu, J\. Fernandes, J\. Fu, W\. Fu, B\. Fuller, C\. Gao, V\. Goswami, N\. Goyal, A\. Hartshorn, S\. Hosseini, R\. Hou, H\. Inan, M\. Kardas, V\. Kerkez, M\. Khabsa, I\. Kloumann, A\. Korenev, P\. S\. Koura, M\. Lachaux, T\. Lavril, J\. Lee, D\. Liskovich, Y\. Lu, Y\. Mao, X\. Martinet, T\. Mihaylov, P\. Mishra, I\. Molybog, Y\. Nie, A\. Poulton, J\. Reizenstein, R\. Rungta, K\. Saladi, A\. Schelten, R\. Silva, E\. M\. Smith, R\. Subramanian, X\. E\. Tan, B\. Tang, R\. Taylor, A\. Williams, J\. X\. Kuan, P\. Xu, Z\. Yan, I\. Zarov, Y\. Zhang, A\. Fan, M\. Kambadur, S\. Narang, A\. Rodriguez, R\. Stojnic, S\. Edunov, and T\. ScialomLlama 2: open foundation and fine\-tuned chat models\.CoRRabs/2307\.09288\.External Links:[Link](https://doi.org/10.48550/arXiv.2307.09288),[Document](https://dx.doi.org/10.48550/ARXIV.2307.09288),2307\.09288Cited by:[§5](https://arxiv.org/html/2609.11414#S5.p2.1)\.
- Tunstallet al\.\(2023\)L\. Tunstall, E\. Beeching, N\. Lambert, N\. Rajani, K\. Rasul, Y\. Belkada, S\. Huang, L\. von Werra, C\. Fourrier, N\. Habib, N\. Sarrazin, O\. Sanseviero, A\. M\. Rush, and T\. WolfZephyr: direct distillation of LM alignment\.CoRRabs/2310\.16944\.External Links:[Link](https://doi.org/10.48550/arXiv.2310.16944),[Document](https://dx.doi.org/10.48550/ARXIV.2310.16944),2310\.16944Cited by:[§5](https://arxiv.org/html/2609.11414#S5.p2.1)\.
- Wanget al\.\(2023\)X\. Wang, J\. Wei, D\. Schuurmans, Q\. V\. Le, E\. H\. Chi, S\. Narang, A\. Chowdhery, and D\. ZhouSelf\-consistency improves chain of thought reasoning in language models\.InThe Eleventh International Conference on Learning Representations,Kigali, Rwanda\.External Links:[Link](https://openreview.net/forum?id=1PL1NIMMrw)Cited by:[§7\.1](https://arxiv.org/html/2609.11414#S7.SS1.p1.1)\.
- Yuet al\.\(2024\)L\. Yu, W\. Jiang, H\. Shi, J\. Yu, Z\. Liu, Y\. Zhang, J\. T\. Kwok, Z\. Li, A\. Weller, and W\. LiuMetaMath: bootstrap your own mathematical questions for large language models\.InThe Twelfth International Conference on Learning Representations,Vienna, Austria\.External Links:[Link](https://openreview.net/forum?id=N8N0hgNDRt)Cited by:[§5](https://arxiv.org/html/2609.11414#S5.p2.1)\.
- Yueet al\.\(2024\)M\. Yue, J\. Zhao, M\. Zhang, L\. Du, and Z\. YaoLarge language model cascades with mixture of thought representations for cost\-efficient reasoning\.InThe Twelfth International Conference on Learning Representations,Vienna, Austria\.External Links:[Link](https://openreview.net/forum?id=6okaSfANzh)Cited by:[§7\.1](https://arxiv.org/html/2609.11414#S7.SS1.p2.1)\.
- Zhaoet al\.\(2024\)Z\. Zhao, L\. Gan, G\. Wang, W\. Zhou, H\. Yang, K\. Kuang, and F\. WuLoraRetriever: input\-aware LoRA retrieval and composition for mixed tasks in the wild\.InFindings of the Association for Computational Linguistics: ACL 2024,Bangkok, Thailand,pp\. 4447–4462\.External Links:[Document](https://dx.doi.org/10.18653/V1/2024.FINDINGS-ACL.263),[Link](https://aclanthology.org/2024.findings-acl.263)Cited by:[§7\.2](https://arxiv.org/html/2609.11414#S7.SS2.p2.1)\.
- Zhou and Bai \(2024\)C\. Zhou and Y\. BaiChinese\-Mistral: an efficient and effective chinese large language model\.Note:[https://github\.com/THU\-ESIS/Chinese\-Mistral](https://github.com/THU-ESIS/Chinese-Mistral)Cited by:[§5](https://arxiv.org/html/2609.11414#S5.p2.1)\.