cost-aware-inference

This paper introduces InflationAgent, a routing system for agentic LLMs that measures token inflation, predicts task difficulty using CoT Branching Entropy, and optimizes model selection to maximize accuracy per cost, achieving higher accuracy with fewer tokens on benchmarks like GSM8K.

This paper introduces value-router, a simulation study for cost-aware routing between cheap heuristics and expensive LLM calls in recommender systems, showing that value-weighted routing improves precision and handles seasonal demand surges with adaptive budgets.