@jiqizhixin: New from NVIDIA! You can edit a model’s compressed memory without scrambling what it already knows! Enter Gated DeltaNe…
Summary
NVIDIA introduces Gated DeltaNet-2, a method for editing compressed model memory without catastrophic forgetting, using independent gates for erase and write operations. It outperforms existing models like Mamba-2 and Mamba-3 on language modeling and long-context tasks.
View Cached Full Text
Cached at: 05/22/26, 09:48 AM
New from NVIDIA!
You can edit a model’s compressed memory without scrambling what it already knows!
Enter Gated DeltaNet-2.
It separates the erase and write operations in linear attention using two independent gates – one for forgetting old info, another for adding new info.
Outperforms Mamba-2, Gated DeltaNet, KDA, and Mamba-3 across language modeling, commonsense reasoning, and retrieval – especially on long-context needle-in-a-haystack benchmarks.
Similar Articles
Sparse Delta Memory: Scaling the State of Linear RNNs through Sparsity
Sparse Delta Memory extends gated linear RNNs with sparse addressing to dramatically increase hidden state capacity for improved long-context learning and retrieval while maintaining computational efficiency.
memt-1.0 architecture breakdown gated deltanet-2, mixture-of-storage, and 66x smaller storage footprint for agent memory [D]
Breakdown of memt-1.0's memory-native architecture, which uses gated DeltaNet-2 with a mixture-of-storage layer to internalize procedural experience into model states, delivering 1.86x inference speedup and 66x smaller storage at 128k context versus full KV caching.
Δ-Mem: Efficient Online Memory for Large Language Models
Proposes delta-Mem, a lightweight online memory mechanism that uses a compact state matrix updated by delta-rule learning to improve long-context performance of frozen LLMs without full fine-tuning or context extension.
nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16 · Hugging Face
NVIDIA 发布 Nemotron Labs Dory1.0-4B-BF16，一款 4B 参数的循环式(latent-recurrent)大语言模型，采用 normalized-GPT 架构与滑窗/全注意力混合层，支持弹性推理计算与最后一次循环 KV 缓存，并原生支持 NVFP4 预训练，采用 OpenMDW-1.1 开放许可。
@che_shr_cat: 1/ We have been treating GPU memory all wrong. What if the GPU didn't need to store your model at all? MegaTrain enable…
MegaTrain enables full-precision training of 100B+ LLMs on a single GPU by treating VRAM as a transient stateless cache, inverting the memory hierarchy.