Tag
Nace AI launched Drex Router, a 12B Gemma-based model that selects the best LLM/SLM and reasoning effort per agent step in 57–87 ms, scoring 87.6% on 1,400 exam questions versus rival Jev Router's 86.4% at a fraction of the cost, and returns per-candidate probabilities with explanations.
FinVeriQA is an open-source multi-agent financial research prototype over SEC filings and live market data that shows a compact typed router beats LLMs, Python beats LLM arithmetic, and provenance-aware verification independent of writer evidence is essential — shared-evidence checks catch far fewer errors than checks against original passages.
Edgee's Agent Gateway frames mid-conversation LLM switching as a ski-rental problem, combining a difficulty tier map with a switching policy that accounts for prompt-cache rebuild costs, and learns both via weekly replays of past traffic. The post explains why routing must optimize not just which model handles a turn, but when a session should switch models without losing its warm cache.
Prompt caches live on the backend that served the last request, so route hops re-bill long prefixes as fresh input — e.g. a 700k Kimi K3 context costs ~$2 per call on a hop vs ~$0.20 with a warm cache. The post discusses pinning providers, single-host multi-model setups, and error-only fallbacks to keep caches warm.
A new Meta paper shows AI agents learn better from past attempts when experiences are routed into per-problem-type memory clusters rather than a single shared history, boosting performance by 15-30% of the max score while cutting API costs by 65-72%.
FAIR at Meta 提出 Routed Continual Learning:将过往轨迹按簇划分、每个簇搭配固定记忆库中的一种记忆系统,并由 LLM 路由器决定复用或新建簇,在交错任务流上相对全局 ICL 历史提升 15–30% 归一化奖励并降低 65–72% API 成本。
文章清晰解释了前缀感知的 LLM 路由机制:通过按块哈希提示词并维护块到 worker 的索引,让请求路由到持有共享前缀 KV 缓存的 GPU,实测在 8 个 vLLM Pod(16 张 H100)上将 p90 首 token 延迟从 92.5 秒降至 0.54 秒。
The paper introduces SkillEval, a framework that combines LLM-discovered interpretable skills with cognitive diagnosis models to produce fine-grained ability profiles of LLMs across benchmarks, improving on average-score evaluation and enabling LLM routing and adaptive testing.
Notion Auto がリニューアルし、エージェントハーネスが再構築されて 24 時間以上のタスク実行が可能に。タスクごとに最適なモデルへ自動ルーティングし、Opus 5.5 相当の性能を約 90% 低コストで提供するほか、ChatGPT プラン接続、Notion MCP の刷新、新しいクレジット制を導入した。
该论文提出一种基于约束上下文老虎机的个性化 LLM 路由框架,通过 primal-dual GLM-UCB 算法在线学习用户偏好,在最小化推理成本的同时以高概率保证响应质量满意度。
Ramp 宣布将内部构建的 AI 工具 Glass 对外开放给其他公司使用。Glass 源于 Ramp 一周内烧掉 150 万美元 AI token 的经历,旨在将企业专属信息转化为全组织可用的工具,并通过 Ramp Router 自动选择合适模型,使账单下降 30%。
Navige 是一款在 AI agent 运行时降低成本的工具,通过 Sufficiency Intelligence 将请求智能路由到更小的模型、缓存重复读取并精简工具输出,同时提供预算设置、失控循环检测和敏感操作审批功能。
清华大学等团队发布了 TokenRouter,一个支持小模型与大模型在单次回答中进行 token 级协作的推理服务引擎,通过减少模型切换的等待开销,在双卡 A100 上将 8 路并发的总吞吐从约 135 tokens/s 提升到 372 tokens/s(2.76 倍)。代码与论文均已开源公开。
Cisco researchers evaluate semantic entropy as an escalation signal for routing queries from small to large LLMs across three benchmarks, showing it works on GSM8K but can be misleadingly matched by naive difficulty heuristics. The paper's main contribution is a checklist of evaluation pitfalls—question-only baselines, ambiguous success metrics, benchmark saturation, and hidden sampling costs—for assessing routing signals.
TokenRouter 是一个面向 token 级 LLM 路由的高效推理服务系统,采用「请求中心编程、模型中心执行」架构与延迟批调度策略,实现较现有系统 2.01–64.15 倍的解码吞吐提升。
Liquid Inference 是一款 LLM 路由器,让多家模型提供商针对每次 prompt 动态竞价,从而为用户提供最低推理成本,兼容 Claude Code、Cursor 等主流 agentic 编码工具,支持数百个开源与闭源模型。
Together AI launches Together Link, a developer tool that connects coding agents to open models on Together AI, allowing developers to keep their existing workflows while reducing model costs.
The paper analyzes six commercial dynamic LLM routers across 14 settings and finds none outperforms a router that randomly selects between two well-chosen models at matched cost, attributing failures to difficulty blindness, length reversal, semantic matching, and roster suboptimality — patterns the standard evaluation objective actually rewards.
FlexRouter 提出一种显式建模模型互补性的 LLM 路由框架,将路由建模为基于覆盖的子集选择问题,并用 Determinantal Point Processes 对模型能力与冗余进行联合建模,同时通过边缘化失败集合的目标函数直接优化答案覆盖率。在 RouterEval 基准上,该方法在域内与域外任务上以更低冗余实现了更高的覆盖率,且推理成本灵活可控。
The paper introduces Collective Bias Mitigation (CBM), a framework that reduces LLM bias by fine-tuned model behavior and knowledge sharing among diverse LLMs, using Debating and Committee routing topologies to balance bias reduction and inference cost.