ReWorld: An Interactive World Model with Long-Horizon Memory

Hugging Face Daily Papers Papers
interactive-world-model long-horizon-memory attention-mechanisms lora-distillation video-generation real-time world-modeling

Summary

ReWorld presents an interactive world model that decouples short-horizon control from long-horizon memory using mixed attention windows and distribution-matching LoRA distillation, enabling real-time video generation with high action fidelity and recall.

An interactive world model must follow the user's actions, remember the places it has shown, and stream in real time. The tension is structural: control wants a short horizon, memory wants an unbounded one. ReWorld separates the two during training and bounds them at inference. Mixed per-head attention windows confine most heads to the recent past while a small set of global heads attends over the entire history, and random head routing keeps either capability from binding to particular heads; random chunk dropping makes sparse histories in-distribution. At inference the whole past lives under a fixed budget: a bounded KV cache backed by a pose-indexed landmark bank, from which the model retrieves the landmarks nearest the current pose. A metric-scale-aligned data engine places eight sources -- Unreal-rendered fly-throughs, game roaming, and real-world footage -- on one physical action scale, so the same key press moves the camera the same distance in every source, and palindrome trajectories supply the revisit evidence that memory training needs. Distribution-matching distillation confined to a LoRA adapter then compresses sampling to four steps: one backbone serves both a high-fidelity multi-step mode and a real-time interactive one, streaming 704x1280 video across photorealistic, game-style, and stylized worlds. Under a three-axis protocol covering action following, long-horizon recall, and video quality, against six recent interactive world models it attains the best control fidelity (11.95^circ rotation error and the best camera-motion consistency) and the best generation quality; and on minute-long out-and-back rollouts (64\,s, 384 latents), its fixed 12-chunk cache still regenerates the starting view -- at rollout lengths where a sliding window has long evicted the evidence and full-KV attention runs out of memory.
Original Article
View Cached Full Text

Cached at: 08/25/26, 04:33 AM

Paper page - ReWorld: An Interactive World Model with Long-Horizon Memory

Source: https://huggingface.co/papers/2608.23565 Authors:

,

,

,

,

,

,

,

,

,

Abstract

ReWorld separates short-horizon control and long-horizon memory during training, then bounds both at inference via mixed attention windows, a pose-indexed landmark bank, and distribution-matching LoRA distillation to enable real-time interactive world modeling with strong action fidelity and long-range recall.

An interactive world model must follow the user’s actions, remember the places it has shown, and stream in real time. The tension is structural: control wants a short horizon, memory wants an unbounded one. ReWorld separates the two during training and bounds them at inference.Mixed per-head attention windowsconfine most heads to the recent past while a small set ofglobal headsattends over the entire history, andrandom head routingkeeps either capability from binding to particular heads;random chunk droppingmakes sparse histories in-distribution. At inference the whole past lives under a fixed budget: a boundedKV cachebacked by apose-indexed landmark bank, from which the model retrieves the landmarks nearest the current pose. Ametric-scale-aligned data engineplaces eight sources -- Unreal-rendered fly-throughs, game roaming, and real-world footage -- on one physical action scale, so the same key press moves the camera the same distance in every source, andpalindrome trajectoriessupply the revisit evidence that memory training needs.Distribution-matching distillationconfined to aLoRA adapterthen compresses sampling to four steps: one backbone serves both a high-fidelity multi-step mode and a real-time interactive one, streaming 704x1280 video across photorealistic, game-style, and stylized worlds. Under a three-axis protocol covering action following, long-horizon recall, and video quality, against six recent interactive world models it attains the best control fidelity (11.95^circ rotation error and the best camera-motion consistency) and the best generation quality; and on minute-long out-and-back rollouts (64\,s, 384 latents), its fixed 12-chunk cache still regenerates the starting view -- at rollout lengths where a sliding window has long evicted the evidence and full-KV attention runs out of memory.

View arXiv pageView PDFProject pageGitHub3Add to collection

Get this paper in your agent:

hf papers read 2608\.23565

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.23565 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.23565 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.23565 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Report

Hugging Face Daily Papers

AlayaWorld is a 15B-parameter interactive video world model that generates 24-fps video at 540p and 720p, using autoregressive latent chunk generation with bounded visual context and distillation to reduce inference steps. It achieves state-of-the-art performance on the iWorld-Bench for long-horizon generation.

Addressable Memory for Video World Models

Hugging Face Daily Papers

This paper introduces WorldTrace, a training-free memory framework for long-horizon video world models that keeps compressed cache addressable, plus LoopBench, a benchmark for episodic recall after long detours. It improves temporal consistency by +15.5% and episodic recall by +19.5% on LoopBench.

Memory in Video World Models (6 minute read)

TLDR AI

A best-paper research from NVIDIA and collaborators introduces WorldTrace, a training-free framework that keeps compressed memory addressable in autoregressive video world models by assigning fixed slot-rank positions, enabling coherent long rollouts and long-range recall beyond the training horizon.