Memory as Weights: Internalizing Long-Term History for Streaming Videos

Hugging Face Daily Papers Papers

Summary

这篇论文提出 MaLoW(Memory as LoRA Weights),将流式视频的长期历史增量编码进固定大小的 LoRA 权重中,冻结主干模型并把当前观测保留在输入里,从而分离长期记忆与实时感知。该方法在 OVO-Bench 和 StreamingBench 上取得领先表现,并在机器人任务上也提升了成功率。

Streaming vision-language models (VLMs) must continuously integrate information from the past while remaining responsive to current observations and user queries. Existing memory mechanisms typically compress, retrieve, or cache historical observations and reintroduce them into the input context, forcing historical memory and current observations to compete for limited context. We propose MaLoW, a streaming memory framework that instead internalizes historical context into evolving lightweight parameter updates. MaLoW uses a recurrent HyperNetwork to incrementally consolidate incoming video observations into fixed-size LoRA weights, allowing historical information to be carried forward without revisiting the full video stream. In doing so, MaLoW represents history as weights, while keeping the current observation as input. To support training, we further construct MaLoW-Instruct, a large-scale streaming video dataset containing 381K question-answer pairs over 56K videos. We conduct extensive experiments to demonstrate that MaLoW achieves strong performance on OVO-Bench, StreamingBench, and StreamArena. On the backward split of OVO-Bench, MaLoW outperforms prior memory approaches by 3.9% to 11.8% across different video backbones while incurring only modest latency overhead. Beyond streaming video understanding, MaLoW also generalizes to vision-language-action models, improving the action success rate on Simpler WidowX by 9.3%.
Original Article
View Cached Full Text

Cached at: 10/16/26, 03:01 AM

Paper page - Memory as Weights: Internalizing Long-Term History for Streaming Videos

Source: https://huggingface.co/papers/2610.22070 💾 What if a streaming AI remembered the past in its weights—not its context window?

Excited to introduce MaLoW (Memory as LoRA Weights) in our new paper, Memory as Weights: Internalizing Long-Term History for Streaming Videos. MaLoW incrementally encodes video history into evolving, fixed-size LoRA weights. The pre-trained backbone stays frozen, while current observations remain in the input—separating long-term memory from real-time perception. 💾 History as weights. The current observations as input. 📥

Highlights from our experiments: 🏆 Strong streaming video performance: Our variants achieve 74.5% on OVO-Bench and 84.8% on StreamingBench, leading the methods compared in our paper. 🌈 Across base VLMs ranging from 0.8B to 11B parameters, MaLoW yields consistent performance gains. 🤖 Beyond video understanding: Incorporating memory improves robot-task success from 59.4% to 68.8% on Simpler WidowX. 📚 A new training resource: MaLoW-Instruct contains 381K question–answer pairs across 56K videos.

Long-term memory is not just about what to remember—but where to store it.

🔗 Explore MaLoW: 📄 arXiv:https://arxiv.org/abs/2610.22070 💻 GitHub:https://github.com/dragonlzm/MaLoW 🌐 Project Page:https://dragonlzm.github.io/MaLoW 🤗 Hugging Face (Checkpoints):https://huggingface.co/collections/zhuomingliu/malow #StreamingVideo #VideoUnderstanding #MultimodalAI #LoRA #Robotics #interactionmodel

teaser_structure

Similar Articles

LOCI: Spatial Linear Memory for Streaming World Models

Hugging Face Daily Papers

LOCI introduces a hybrid spatial memory architecture for streaming video world models, combining key-value caches with a recurrent linear-attention memory conditioned on camera geometry. It reproduces revisited content more faithfully than full-softmax baselines while reducing peak memory by about 30% and enabling constant-memory streaming of long videos.

Video2LoRA: Parametric Video Internalization for Vision-Language Models

Hugging Face Daily Papers

This paper introduces Video2LoRA, a method that predicts Low-Rank Adaptation (LoRA) weights directly from video representations, enabling efficient video processing in frozen vision-language models. It reduces visual token load by up to 1500x and query TTFT by 6-80x while maintaining performance on video summarization and captioning benchmarks.