Tag
This paper introduces HOLA, a method that gives fast AI models (like linear-attention and state-space models) an additional memory cache to store surprising facts, improving their recall in long-context tasks without sacrificing speed.
This article introduces a new method proposed by Google Research, Cornell, and USC that takes snapshots of RNN memory and caches them, enabling RNNs to efficiently handle long contexts. It combines Transformer-like strong memory with RNN-like low cost, offering a new direction for long-context AI.