Tag
This paper identifies memory retention as the bottleneck in recurrent memory agents for long contexts and proposes Multi-Head Recurrent Memory (MHM), a training-free framework that partitions memory into independent heads with a select-then-update strategy. The lightweight instantiation MHM-LRU significantly improves retention and end-to-end accuracy across 100K–1M token ranges, raising retention from below 30% to 73.96% on RULER-HQA at 896K tokens.
This paper presents preliminary findings from Hierarchos, a 232-million parameter recurrent memory-augmented assistant model, exploring its capabilities in memory retention and assistance tasks.