model-routing

The article argues that token price is an inadequate cost metric for AI agents, proposing that effective cost should be measured by successful runs with validation, and discusses routing and failure strategies.

Factory.ai is recommended as an agent-native software development tool for optimizing model routing in enterprise AI tasks, enhancing accuracy and cost-efficiency.

分享一个通过TeamoRouter服务免费使用DeepSeek V4和Codex的入口，该服务提供API路由和高达90%的折扣。

The article argues that many AI agent workflows waste money by routing every task to frontier models, and suggests using cheaper model tiers for simple, structured tasks while escalating harder ones. It provides a cost comparison showing up to 75% savings with a tiered approach.

MERA introduces a multi-cycle adaptation approach for LLM agents that improves small models by distilling execution-verified demonstrations into a SkillBook and fine-tuning LoRA adapters, with router-based deployment and verifier-backed fallback. Experiments show Qwen2.5-Coder-1.5B improves from 28.7% to 49.7% pass on HumanEval++MBPP while retaining most quality at lower cost.

Nvidia released Nemotron 3.5 Lightning, a 30B open mixture-of-experts model, and NeMo Switchyard, an open-source routing library that dynamically assigns each step of an AI agent workflow to the most suitable model. Nvidia claims the combination can cut agent task costs to about a third while maintaining frontier-level performance.

LangChain tested NVIDIA's open-source router Switchyard with Deep Agents, showing that routing between models can cut costs by ~70% while retaining ~90% accuracy, and introduced new middleware integration for NVIDIA models.

This paper demonstrates that replay-based static evaluation of model switching in LLM agents is fundamentally flawed: when swapping models mid-trajectory, the environment and subsequent actions diverge dramatically from logged trajectories, invalidating most benchmark results. The authors propose branching rollouts as a more faithful evaluation method and release their harness and trajectories.

LangChain's Max Agency podcast features Cognition president Russell Kaplan discussing why developers now prioritize efficiency over the best model, and how agents like Devin are being deployed at scale.

LLMRouter presents a unified formulation of LLM routing as a sequential decision process, along with an open-source infrastructure and benchmark (xRouteBench) for developing, evaluating, and deploying LLM routers. Empirical results show learned routers achieve 14.6% relative improvement over the strongest fixed-model baseline.

Cursor explains how its Router uses a data-driven approach with the Compass complexity predictor to route developer tasks to the most appropriate AI model, achieving significant cost reductions while maintaining or improving user satisfaction.

The author argues that AI coding harnesses and model routing are as important as the model itself, sharing tests with Oh-My-Pi and OpenCode that cut token usage and errors, and recommending tiered model subscriptions for high-volume lightweight tasks.

The article argues that the biggest AI shift is not larger models but better systems around them, such as context, model routing, caching, agent workflows, and evaluation, making the model the engine and the system the product.

Google Cloud API Gateway now offers model routing in Public Preview, providing a serverless ingress layer that accepts OpenAI-compatible requests and dynamically routes them to Gemini, Claude, or OpenAI models.

Not Diamond announces Not Diamond Code, an intelligent model router for coding agents that selects the best model and reasoning effort per step, claiming 20%+ cost savings and Pareto-optimal performance on coding benchmarks.

Not Diamond Code is announced as an intelligent model router for long-horizon coding agents, selecting the best model and reasoning effort per step to reduce costs by 20-65%.

Google Cloud API Gateway announces public preview of model routing, letting developers access Gemini, Claude, and OSS models via a single endpoint using OpenAPI specs.

The article discusses how most AI traffic consists of simple, repeatable tasks like classification and extraction, yet frontier models are often used for everything. It questions whether routing tasks to smaller specialized models will become standard practice to reduce cost and latency.

The article argues that model routing isn't the real problem to solve—token efficiency is. It advocates for a holistic closed-loop approach combining cheaper defaults, preference-aware routing, and better caching (e.g., Coinbase's 5%→60% cache hit improvement) to maximize useful intelligence per dollar.

Compares API list prices of 18 LLMs from major providers, highlighting a 100x cost difference for the same workload and recommending model routing for cost efficiency.