Tag
Kent C. Dodds announces a package app kit for Kody that enables running full-stack Remix apps with integrated features like design tokens, PWA support, error handling, and realtime sync.
Author released an open-source tool called cache-pressure that benchmarks how well local LLM inference engines actually retain KV cache contexts under pressure, allowing users to verify real cache capacity against advertised claims.
CacheScout is an agent-aware KV-cache runtime layer for multi-agent LLM serving that learns agent execution semantics online to guide cache eviction and prefetching, improving cache hit rate and reducing latency.
This paper proposes a KV cache eviction strategy that scores tokens by counter-causal surprise, removing past tokens that are well-predicted from future context. The method is training-free, in-distribution, and achieves competitive performance with a fast single-layer approximation.
LMCache is an open-source KV cache management layer that separates cache I/O from compute, plugging into vLLM, SGLang, and TensorRT-LLM to achieve up to 14x faster time-to-first-token and 4x faster decoding by parallelizing cache lookups and sharing GPU memory.