Tag
A paper proposing a latency-aware LLM query router that jointly optimizes latency, accuracy, and cost using a lightweight latency estimator, achieving up to 40% improvement in accuracy–cost utility while maintaining comparable latencies.
Google DeepMind released a paper on effective model routing strategies, discussing how LLM routers are judged on accuracy and cost but can be meaningless if models respond identically.