Tag
Proposes Cache-Aware Prompt Compression (CAPC), a method that combines query-agnostic compression with caching to reduce LLM API costs, demonstrating significant savings over existing approaches on Anthropic's Sonnet API and production workloads.
This paper formalizes the problem of ordering filters in sequential filtering pipelines under independent cost and selectivity models, proving that ordering by increasing ratio of cost to rejection probability is optimal. Monte Carlo simulations demonstrate that this ordering dominates common heuristics both in expectation and across the full distribution of outcomes.