ActWorld: From Explorable to Interactive World Model via Action-Aware Memory
Summary
ActWorld proposes a chunk-autoregressive world model with hierarchical action-aware memory to support object interaction alongside navigation, addressing data and memory bottlenecks in existing interactive world models.
View Cached Full Text
Cached at: 06/17/26, 03:35 AM
Paper page - ActWorld: From Explorable to Interactive World Model via Action-Aware Memory
Source: https://huggingface.co/papers/2606.17730 Authors:
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
ActWorld extends navigation-centric interactive world models to support object interaction through a chunk-autoregressive framework with hierarchical action-aware memory and persistent memory banks.
Interactive world modelsaim to simulate environment dynamics under real-time user actions. However, their action vocabulary is largely confined to navigation: most actions correspond to motion (e.g., walk, turn, look around), while interaction with objects in the scene (e.g., pick up plates, open doors, or trigger physical responses) is either absent, restricted to game domains, or relegated to prompt-to-full-video scenarios. The resulting worlds are visually explorable but not truly actionable. In this work, we present ActWorld, an interactive world model that extends priornavigation-centric generatorsto support mid-rolloutobject interactionwithin achunk-autoregressive framework. We argue that the navigation-interaction gap stems from two bottlenecks. First, a data bottleneck: the lack of human-object interactiondata with accurate, dense labels. Second, a memory bottleneck: recency-biased history compression in existing world models discards theevent-transition framesthat causally determine subsequent object states, leading to anaction-forgetting pathology. On the data side, we construct a 100K interaction video dataset, each annotated withper-chunk captionsviachain-of-thought reasoning. On the model side, we introduce a hierarchicalaction-aware memorydesign that routes history compression by interaction importance, complemented by apersistent memory bankthat maintains event-update and object-identity tokens across long rollouts. Experiments show that ActWorld supports both flexible navigation and richobject interactionwithin a single model, substantially improving interaction fidelity over navigation-only baselines without sacrificing viewpoint control. Project page is available at https://interactwm.github.io/ActWorld.
View arXiv pageView PDFProject pageAdd to collection
Get this paper in your agent:
hf papers read 2606\.17730
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.17730 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.17730 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.17730 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
ReWorld: An Interactive World Model with Long-Horizon Memory
ReWorld presents an interactive world model that decouples short-horizon control from long-horizon memory using mixed attention windows and distribution-matching LoRA distillation, enabling real-time video generation with high action fidelity and recall.
WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes
WorldAct is a framework that converts static 3D generated environments into editable and interactive object-centric scenes using multimodal agents and geometric reconstruction, enabling object-level editing and embodied task execution.
GameWAM: A World Action Model for Video Games
GameWAM introduces the first unified world-action model for native video-game control, jointly predicting future visuals and executable actions using block-causal flow matching and mode-specific distributions.
GameWAM: A World Action Model for Video Games
GameWAM introduces the first world-action model for native closed-loop gameplay and GUI control in video games, jointly generating visual observations and executable actions with competitive task success and revealing a source-sensitivity failure mode.
Micro-World - Action-controlled Interactive world model - AMD
AMD releases Micro-World, an action-controlled interactive world model built on the Wan2.1 family, with open-source weights, code, and a curated dataset for controllable world modeling.