Tag
The article tests whether DeepSeek Harness maintains high prompt caching rates when using alternative AI models, finding that GLM and Kimi achieve 97-99% cache reuse, while Opus shows no cache activity and GPT test failed.
This paper proposes Global-ImpactCache (GCache), a bilevel optimization framework that learns cache reuse policies for diffusion models by aligning error weighting with final generation quality, instead of relying on local similarity heuristics. It achieves significant speedups and quality improvements on image and video generation tasks, including a 2.17x speedup on Wan2.1 with lower LPIPS.
vLLM integrates Mooncake Store for distributed KV cache reuse, enabling cross-node prefix caching to efficiently serve agentic workloads with high token reuse.
KV Packet proposes a recomputation-free cache reuse framework for LLMs that uses trainable soft-token adapters to bridge context discontinuities, eliminating overhead while maintaining performance comparable to full recomputation baselines on Llama-3.1 and Qwen2.5.