forgetting

Proposes a cognitively grounded multi-factor value function for agentic memory in LLM agents, learning interpretable weights to decide what to encode, forget, and retrieve under memory constraints. Improves gold-evidence retention significantly over similarity-only or recency-based baselines.

This blog post introduces the concept of the 'Forgetting Wall' in long-horizon video generation and world models, arguing that the primary bottleneck is memory (KV cache growth) rather than compute, and explores compression as a key direction for future models.

Proposes FoLoRA, a forgetting-aware optimization framework for fine-tuning foundation models that balances task utility and forgetting penalty via generalized Rayleigh-quotient optimization, achieving better preservation of non-target capabilities.

This paper introduces the concept of 'initialization memory' to study how much of the random initialization bias survives training in deep networks, showing that low-learning-rate SGD preserves initialization while Adam-family optimizers erase it, and linking this to forgetting dynamics.

Introduces SeqMem-Eval, a diagnostic evaluation framework for sequentially evolving LLM memory that measures multiple dimensions beyond aggregate metrics, revealing trade-offs between adaptability and stability.

Researchers introduce Memora, a benchmark that evaluates LLMs’ ability to retain, update, and forget long-term user memories over weeks-to-months conversations, revealing frequent reuse of obsolete memories.