Tag
AbridgeKV trains models to compress completed blocks of generated chain-of-thought text into the KV entries of a few learned pause tokens, achieving a 3.5× smaller cache and 3.2× faster decoding on a saturated H100 with no accuracy loss at a quarter of the full KV budget.
Eric Schmidt argues that long reasoning — AI models that can stay on a single problem for hours or days — is the one AI theme he would invest in, and that the market has not yet priced in how different this is from models that simply answer and stop.