associative-memory

A Pebble ML paper presents a small DeltaNet variant with fixed token-specific key biases that learns 32 key–value pairings per sequence in a fixed 32 KiB recurrent matrix state, achieving 99.95% recall accuracy even with long filler contexts, while parameter-matched vector and Transformer baselines fail to learn the task.

This paper develops intrinsic dense associative memories on Riemannian manifolds by casting memory as Epanechnikov kernel-density mode seeking, showing that curvature—via Ricci curvature thresholds and geodesic capacity scalings—determines what such a memory retains, creates, and forgets, with demonstrations on WordNet noun hierarchy.

This paper investigates whether gradient-based training can learn the rank necessary to store and compose associations in matrix memories, using experiments with key-value pairs and measuring recovery by cosine similarity.

The paper analyzes a class of associative memories with hidden neurons, deriving phase diagrams and storage capacities using replica methods and linking to softmax attention in transformers.

Introduces Kalman Delta Networks, which improve language modeling by reformulating linear attention as a linear-Gaussian state-space model with Kalman-filter updates to track memory uncertainty, yielding efficient approximations that outperform existing linear-attention models.

This article presents SSAKG 2.0, an open-source software package for constructing and operating Structural Sequential Associative Knowledge Graphs (SSAKGs) for context-based sequence retrieval, with new efficient algorithms implemented in Python and C.

GoldWorm is a native Rust cognitive engine that uses the biologically mapped 302-neuron connectome of C. elegans to process language through a transparent, zero-trust architecture with dual-stream processing and Hebbian associative memory.

This paper proposes H-Res, a method to adapt large transformer models by shaping the energy landscape of associative memories without modifying weights or adding prompts, preserving memory capacity and outperforming LoRA.

Introduces FERNme, an open-source memory layer for AI agents that uses a fuzzy Hebbian graph to simulate associative memory, supporting features like zero-LLM writes, persistence, forgetting, and user ownership.

This paper investigates whether the Engram module, an associative memory mechanism, provides content-addressable retrieval in autoregressive image generation or acts as a gated architectural side-pathway.

Tensor Cache introduces a two-level caching mechanism that compresses evicted key-value pairs from sliding-window attention into a fixed-size associative memory, improving long-context language modeling without unbounded memory growth.

This paper introduces Variational Linear Attention (VLA), a method that stabilizes memory states in linear attention mechanisms for long-context transformers. VLA reframes memory updates as an online regularized least-squares problem, proving bounded state norms and demonstrating significant speedups and improved retrieval accuracy over standard linear attention and DeltaNet.

HeLa-Mem is a bio-inspired memory architecture for LLM agents that models memory as a dynamic graph using Hebbian learning dynamics, featuring episodic and semantic memory stores to improve long-term coherence. Experiments on LoCoMo show superior performance across question categories while using fewer context tokens.