Memory as Weights: Internalizing Long-Term History for Streaming Videos
Summary
这篇论文提出 MaLoW(Memory as LoRA Weights),将流式视频的长期历史增量编码进固定大小的 LoRA 权重中,冻结主干模型并把当前观测保留在输入里,从而分离长期记忆与实时感知。该方法在 OVO-Bench 和 StreamingBench 上取得领先表现,并在机器人任务上也提升了成功率。
View Cached Full Text
Cached at: 10/16/26, 03:01 AM
Paper page - Memory as Weights: Internalizing Long-Term History for Streaming Videos
Source: https://huggingface.co/papers/2610.22070 💾 What if a streaming AI remembered the past in its weights—not its context window?
Excited to introduce MaLoW (Memory as LoRA Weights) in our new paper, Memory as Weights: Internalizing Long-Term History for Streaming Videos. MaLoW incrementally encodes video history into evolving, fixed-size LoRA weights. The pre-trained backbone stays frozen, while current observations remain in the input—separating long-term memory from real-time perception. 💾 History as weights. The current observations as input. 📥
Highlights from our experiments: 🏆 Strong streaming video performance: Our variants achieve 74.5% on OVO-Bench and 84.8% on StreamingBench, leading the methods compared in our paper. 🌈 Across base VLMs ranging from 0.8B to 11B parameters, MaLoW yields consistent performance gains. 🤖 Beyond video understanding: Incorporating memory improves robot-task success from 59.4% to 68.8% on Simpler WidowX. 📚 A new training resource: MaLoW-Instruct contains 381K question–answer pairs across 56K videos.
Long-term memory is not just about what to remember—but where to store it.
🔗 Explore MaLoW: 📄 arXiv:https://arxiv.org/abs/2610.22070 💻 GitHub:https://github.com/dragonlzm/MaLoW 🌐 Project Page:https://dragonlzm.github.io/MaLoW 🤗 Hugging Face (Checkpoints):https://huggingface.co/collections/zhuomingliu/malow #StreamingVideo #VideoUnderstanding #MultimodalAI #LoRA #Robotics #interactionmodel
Similar Articles
LOCI: Spatial Linear Memory for Streaming World Models
LOCI introduces a hybrid spatial memory architecture for streaming video world models, combining key-value caches with a recurrent linear-attention memory conditioned on camera geometry. It reproduces revisited content more faithfully than full-softmax baselines while reducing peak memory by about 30% and enabling constant-memory streaming of long videos.
Video2LoRA: Parametric Video Internalization for Vision-Language Models
This paper introduces Video2LoRA, a method that predicts Low-Rank Adaptation (LoRA) weights directly from video representations, enabling efficient video processing in frozen vision-language models. It reduces visual token load by up to 1500x and query TTFT by 6-80x while maintaining performance on video summarization and captioning benchmarks.
Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding
LatentStream introduces a progressive latent working memory framework that internalizes streaming visual evidence for continuous reasoning, achieving state-of-the-art results on video benchmarks.
Elephant in the Fridge: Constant-Memory Frame Packing for Long Video Understanding
提出 FramePack-U,一种免训练的常量内存帧打包框架,可将任意长度视频压缩为固定大小的视觉表示,直接插入现成 VLM 推理,在 Long VideoBench、MLVU 和 Video-MME 上显著超越全分辨率基线及现有 token 压缩方法。
VideoLoop: Looped Working Memory Against Semantic Thrashing in Long-Form Video Agents
The paper proposes VideoLoop, a multimodal agent with looped working memory to address semantic thrashing in long-form video understanding, demonstrating performance gains on benchmarks like VideoMME.
