MemLearner: Learning to Query Context memory for Video World Models
Summary
MemLearner proposes a learning-based adaptive context query method using query tokens to improve scene consistency and memory in video world models, particularly for long sequences with occlusions and dynamic objects.
View Cached Full Text
Cached at: 07/01/26, 03:40 AM
Paper page - MemLearner: Learning to Query Context memory for Video World Models
Source: https://huggingface.co/papers/2606.31734
Abstract
MemLearner improves video world models by using learning-based adaptive context querying with query tokens to enhance scene consistency and memory in long video sequences with occlusions and dynamic objects.
Video World Modelsare interactivevideo generation models that predict future world states based on user actions and history video frames. A critical challenge invideo world modelsis the lack ofmemory, causing inconsistent generated scenes over extended durations. Previous methods explored rule-basedcontext frame retrievalasmemory, but they fail to generalize in scenarios with scene occlusions and dynamic objects. We propose MemLearner, a learning-based adaptive context query method usingquery tokensto bridge context and predicted tokens. By leveraging thevideo generation modelitself for context querying, MemLearner exploits pre-trainedvisual priorswithout training additional modules from scratch, and incorporates efficient strategies for training and inference. We collect a dataset of long videos with scene occlusions and dynamic objects, paired withcamera pose annotations, and propose amulti-dataset training strategyleveraging both annotated rendered and unannotated real-world videos. Extensive experiments demonstrate that MemLearner significantly outperforms priorvideo world modelsin terms ofscene consistencyandmemory, particularly under challenging occlusion and dynamic scenarios.
View arXiv pageView PDFProject pageAdd to collection
Get this paper in your agent:
hf papers read 2606\.31734
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.31734 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.31734 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.31734 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
MemTrain: Self-Supervised Context Memory Training
MemTrain proposes a self-supervised training framework that uses masked reconstruction and intermediate memory recall proxy tasks on Wikipedia corpora to enhance LLM agents' context memory, achieving up to 17.67 point gains on downstream memory-intensive QA benchmarks.
Addressable Memory for Video World Models
This paper introduces WorldTrace, a training-free memory framework for long-horizon video world models that keeps compressed cache addressable, plus LoopBench, a benchmark for episodic recall after long detours. It improves temporal consistency by +15.5% and episodic recall by +19.5% on LoopBench.
MemLens: Benchmarking Multimodal Long-Term Memory in Large Vision-Language Models
MemLens is a new benchmark for evaluating memory capabilities in large vision-language models through multi-session conversations. It compares long-context and memory-augmented approaches, revealing limitations in both and motivating hybrid architectures.
Reasoning with Memory: A Temporal Granularity-Adaptive Framework for Training-Free Long Video Understanding
ReMem introduces a dual-level memory-augmented keyframe selection framework for training-free long video understanding, achieving state-of-the-art zero-shot performance on multiple benchmarks.
EM^2Mem: Event-Centric Multimodal Memory for Large Language Models
EM^2Mem proposes an event-centric multimodal memory framework that binds heterogeneous evidence to event anchors for compact, generation-ready memory in long-video question answering, improving accuracy and reducing latency.