IR3DE: A Linear Router for Large Language Models
Summary
IR3DE is a ridge regression-based router that selects domain-expert LLMs for different tasks, achieving competitive performance while enabling dynamic addition or removal of experts without retraining.
View Cached Full Text
Cached at: 06/10/26, 09:43 AM
Paper page - IR3DE: A Linear Router for Large Language Models
Source: https://huggingface.co/papers/2606.06098
Abstract
A ridge regression-based routing method achieves competitive performance in selecting domain-expert LLMs for different tasks while enabling dynamic addition/removal of experts without retraining.
Foundational Large Language Models (LLMs) demonstrate proficiency on a wide range of general tasks, and achieve remarkable results on various specialized tasks viadomain-expert LLMs. With the ever-growing list of available LLMs,inference routersare being proposed to select the most appropriate LLM for each prompt. However, existing routing methods either optimize cost across weak-to-strong generalist LLMs or require substantial training to support domain-expertise routing. In this paper, we propose IR3DE, aRidge Regression-based Router for Domain Experts that provides cheap and fast routing decisions for each prompt. We evaluate IR3DE in twoCausal Language Modeling(CLM) settings where the tasks arenext-token predictionfor all domains, and one reasoning setting where each domain has its own distinct reasoning task. Despite being alinear router, IR3DE achieves performance comparable to the other baselines in both CLM settings, and surpassing them in the reasoning setting, with a normalized performance of 98.4%. Moreover, IR3DE enables the addition or removal of new domain experts without requiring the router to be retrained from scratch, allowing a dynamic set of LLMs to be served with minimal disruption to the router itself. Our code is available at: github.com/gensyn-ai/IR3DE.
View arXiv pageView PDFProject pageGitHub2Add to collection
Get this paper in your agent:
hf papers read 2606\.06098
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.06098 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.06098 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.06098 in a Space README.md to link it from this page.
Collections including this paper1
Similar Articles
$R^2$-dLLM: Accelerating Diffusion Large Language Models via Spatio-Temporal Redundancy Reduction
R²-dLLM introduces spatio-temporal redundancy reduction techniques that cut diffusion LLM decoding steps by up to 75% while preserving generation quality, addressing a key deployment bottleneck.
Dynamic Latent Routing
Dynamic Latent Routing (DLR) lets LLMs learn their own inner monologue by composing sub-policies via search, inspired by language compositionality. In low-data fine-tuning, DLR matches or outperforms standard supervised fine-tuning.
Redesign Mixture-of-Experts Routers with Manifold Power Iteration
Researchers propose a novel router redesign for Mixture-of-Experts models that aligns router rows with principal singular directions using Manifold Power Iteration, improving model effectiveness.
Routing-Aware Expert Calibration for Machine Unlearning in Mixture-of-Experts Language Models
The paper proposes TRACE, a method for machine unlearning in Mixture-of-Experts language models that calibrates retain regularization by reweighting token-level retain losses to address forget-retain routing mismatch. Experiments show improved forget-utility trade-off across multiple MoE LLMs.
RouteProfile: Elucidating the Design Space of LLM Profiles for Routing
This paper introduces RouteProfile, a design space for LLM profiles in routing systems, demonstrating that structured profiles and query-level signals improve routing performance and generalization to new models.