Tag
An appreciation post for Spiritbuun's llama.cpp fork, highlighting its new Variable Bit Rate (VBR) KV cache feature that dynamically degrades cache precision to fit VRAM, enabling larger auto-fitted context sizes with minimal speed loss.
A practical guide to structuring context for AI agents in production using three separate layers—static (business rules), dynamic (live data), and session (conversation history)—to avoid common failures caused by unmanaged context accumulation.