Tag
An appreciation post for Spiritbuun's llama.cpp fork, highlighting its new Variable Bit Rate (VBR) KV cache feature that dynamically degrades cache precision to fit VRAM, enabling larger auto-fitted context sizes with minimal speed loss.