Echo-Memory: A Controlled Study of Memory in Action World Models

Hugging Face Daily Papers Papers
memory-mechanisms world-models video-generation controlled-study action-conditioned state-space-recurrence

Summary

Echo-Memory presents a controlled study of memory mechanisms in action-conditioned world models, revealing that memory structure and capacity significantly impact open-domain return performance beyond replay fidelity. The study introduces a matched evaluation protocol and finds that raw context and state-space recurrence are strong mechanisms.

We present Echo-Memory, a controlled study of memory mechanisms in action-conditioned world models. These models generate multi-segment videos from a first frame, text prompt, and camera-action sequence, but their central failure is often memory rather than local image synthesis: after the camera leaves and returns, the scene or salient object may silently change. Existing memory designs are hard to compare because gains are entangled with backbone, training, retrieval, and evaluation differences. Echo-Memory fixes the action-to-video interface and varies only how history is stored and read by the generator. Under a shared video diffusion backbone, optimizer, camera-action representation, sampler, and evaluation pipeline, we compare raw context, compression-based memory, spatial summaries with different read-out paths, and state-space recurrence. This matched matrix separates four otherwise conflated axes: capacity, compression, read-out, and recurrence. We also evaluate memory through a three-branch protocol: replay quality, in-domain loop revisit, and open-domain return probes. The branches routinely disagree, showing that replay fidelity is not a sufficient proxy for remembering a world. Three findings follow. Raw context is a strong capacity baseline and improves open-domain return far more than it improves replay metrics. Compactness is not a free substitute for capacity: aggressive spatial and hybrid-compression memories lose the salient evidence needed for return. Finally, block-wise state-space recurrence is the strongest open-domain return mechanism in our matrix, showing that the structure of implicit memory matters as much as the decision to use it. These results provide a compact protocol for studying memory in action world models beyond isolated replay metrics.
Original Article
View Cached Full Text

Cached at: 06/09/26, 08:43 AM

Paper page - Echo-Memory: A Controlled Study of Memory in Action World Models

Source: https://huggingface.co/papers/2606.09803 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

Controlled study of memory mechanisms in action-conditioned world models reveals that memory structure and capacity significantly impact open-domain return performance beyond simple replay fidelity measures.

We present Echo-Memory, a controlled study ofmemory mechanismsinaction-conditioned world models. These models generate multi-segment videos from a first frame, text prompt, and camera-action sequence, but their central failure is often memory rather than local image synthesis: after the camera leaves and returns, the scene or salient object may silently change. Existing memory designs are hard to compare because gains are entangled with backbone, training, retrieval, and evaluation differences. Echo-Memory fixes the action-to-video interface and varies only how history is stored and read by the generator. Under a sharedvideo diffusion backbone, optimizer, camera-action representation, sampler, and evaluation pipeline, we compare raw context,compression-based memory,spatial summarieswith different read-out paths, andstate-space recurrence. This matched matrix separates four otherwise conflated axes: capacity, compression, read-out, and recurrence. We also evaluate memory through a three-branch protocol:replay quality, in-domainloop revisit, and open-domainreturn probes. The branches routinely disagree, showing that replay fidelity is not a sufficient proxy for remembering a world. Three findings follow. Raw context is a strong capacity baseline and improves open-domain return far more than it improves replay metrics. Compactness is not a free substitute for capacity: aggressive spatial and hybrid-compression memories lose the salient evidence needed for return. Finally, block-wisestate-space recurrenceis the strongest open-domain return mechanism in our matrix, showing that the structure of implicit memory matters as much as the decision to use it. These results provide a compact protocol for studying memory in action world models beyond isolated replay metrics.

View arXiv pageView PDFProject pageGitHub78Add to collection

Get this paper in your agent:

hf papers read 2606\.09803

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2606.09803 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2606.09803 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.09803 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

WorldMemArena: Evaluating Multimodal Agent Memory Through Action-World Interaction

Hugging Face Daily Papers

WorldMemArena is a new benchmark with 400 multi-session multimodal tasks for evaluating multimodal agent memory, comparing long-context, RAG, and harness-based memory approaches, revealing that better memory writing does not guarantee better performance and that systems struggle with visual evidence.

Composition of Memory Experts for Diffusion World Models

arXiv cs.LG

A new diffusion-based world model framework that uses a composition of specialized memory experts (short-term, long-term episodic, and spatial) to achieve better temporal consistency and long-context modeling without quadratic cost.

Echo-Forcing: A Scene Memory Framework for Interactive Long Video Generation

Hugging Face Daily Papers

Echo-Forcing introduces a scene memory framework for interactive long video generation, using hierarchical temporal memory, scene recall frames, and difference-aware memory decay to handle prompt switching and long-term recall. The method is training-free and achieves strong performance on VBench-Long.

MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memory

Hugging Face Daily Papers

MemEye is a visual-centric evaluation framework that assesses multimodal agent memory by measuring visual evidence granularity and retrieval complexity across 8 life-scenario tasks, revealing that current architectures struggle to preserve fine-grained visual details and reason about state changes over time.