LLMRouter open-sources 16+ router library with xRouteBench
Summary
A new paper on arXiv introduces an open-source library called LLMRouter with over 16 router implementations and a benchmark xRouteBench, demonstrating that learned routers can outperform fixed-model baselines by 14.6%.
Similar Articles
LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers
LLMRouter presents a unified formulation of LLM routing as a sequential decision process, along with an open-source infrastructure and benchmark (xRouteBench) for developing, evaluating, and deploying LLM routers. Empirical results show learned routers achieve 14.6% relative improvement over the strongest fixed-model baseline.
Task- and Session-Level Model Routing: A Common-Interface Hybrid Evaluation of Four Open-Source Routers Across Four Benchmarks
This paper evaluates four open-source LLM routers using a common protocol across four benchmarks, finding that performance gains are more closely tied to model tier composition than task-specific targeting.
@ClementDelangue: Super excited about open-source router systems and routing models like @vllm_project semantic router: https://huggingfa…
Clement Delangue highlights vLLM's new semantic router, an open-source system for routing LLM queries to the most appropriate model, aiming to shift value from expensive frontier models to a diverse ecosystem of open-source models.
llm-openrouter 0.7
Release of llm-openrouter version 0.7, updating compatibility with LLM 0.32 and adding new server-side tools like Shell, WebFetch, and WebSearch for improved use with reasoning LLMs via OpenRouter.
Arch-Router: Aligning LLM Routing with Human Preferences
Arch-Router is a compact 1.5B model that aligns LLM routing with human preferences by mapping queries to user-defined domains and action types, outperforming proprietary models in subjective evaluations.