Tag
An ex-vLLM core contributor explains how to reduce LLM inference cost by 10x using LMCache with KV cache offloading to CPU/SSD/remote storage, a technique used by production stacks like Bloomberg.
LMCache v0.5.1 is released with new model support for MiniMax M3, new backends including AMD hipFile and XPU Docker, warm-prefetch across tiers, Device-DAX L1 split, hidden-state caching, and operator automation for vLLM integration.