Tag
A paper proposing a latency-aware LLM query router that jointly optimizes latency, accuracy, and cost using a lightweight latency estimator, achieving up to 40% improvement in accuracy–cost utility while maintaining comparable latencies.
EarlyTom is a training-free framework that compresses visual tokens early in the vision encoder to reduce time-to-first-token and computational costs while maintaining accuracy, achieving up to 2.65x TTFT reduction.