Streaming Multi-Agent Autoregressive Diffusion Model with World State Registers

Hugging Face Daily Papers Papers
multi-agent video-diffusion world-model state-registers minecraft generative-ai autoregressive

Summary

WorldWeaver (W²) introduces cross-agent world state registers to multi-agent video diffusion models, enabling shared world state persistence across agents and views, improving logical consistency in two-agent Minecraft video generation.

Multi-agent interactive world models should not only generate consistent observations, but also maintain world states that persist across agents and evolve across views. Existing autoregressive video diffusion pipelines carry forward observation history as conditioning context, which makes shared state difficult to maintain in multi-agent and multi-view settings. We present WorldWeaver (W^2), a streaming multi-agent video diffusion model that augments rollout with cross-agent world state registers: learnable tokens that store shared world information, track individual agent status, and are dynamically updated after each generated chunk. We ground these registers with supervision signals spanning individual agent status, global state views including bird's-eye views, and scene text. We further improve the architecture with a Mixture-of-Transformers design that uses separate weights for world state modeling and visual frame modeling. Extensive experiments in two-agent Minecraft video generation show that explicit world-state modeling improves logical consistency and generation quality.
Original Article
View Cached Full Text

Cached at: 07/24/26, 05:06 AM

Paper page - Streaming Multi-Agent Autoregressive Diffusion Model with World State Registers

Source: https://huggingface.co/papers/2607.21594

Abstract

Multi-agentinteractiveworldmodelsshouldnotonlygenerateconsistentobservations,butalsomaintainworldstatesthatpersistacrossagentsandevolveacrossviews.Existingautoregressivevideodiffusionpipelinescarryforwardobservationhistoryasconditioningcontext,whichmakessharedstatedifficulttomaintaininmulti-agentandmulti-viewsettings.WepresentWorldWeaver(W^2),astreamingmulti-agentvideodiffusionmodelthataugmentsrolloutwithcross-agentworldstateregisters:learnabletokensthatstoresharedworldinformation,trackindividualagentstatus,andaredynamicallyupdatedaftereachgeneratedchunk.Wegroundtheseregisterswithsupervisionsignalsspanningindividualagentstatus,globalstateviewsincludingbird’s-eyeviews,andscenetext.WefurtherimprovethearchitecturewithaMixture-of-Transformersdesignthatusesseparateweightsforworldstatemodelingandvisualframemodeling.Extensiveexperimentsintwo-agentMinecraftvideogenerationshowthatexplicitworld-statemodelingimproveslogicalconsistencyandgenerationquality.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2607\.21594

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.21594 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.21594 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.21594 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

MultiWorld: Scalable Multi-Agent Multi-View Video World Models

Hugging Face Daily Papers

MultiWorld is a unified framework for multi-agent multi-view video world modeling that achieves accurate control of multiple agents while maintaining multi-view consistency through a Multi-Agent Condition Module and Global State Encoder.

MASS: Multiplayer World Models with Authoritative Shared State

Hugging Face Daily Papers

This paper introduces MASS, a method for multiplayer world models that disentangles world dynamics from view rendering using an authoritative shared state, enabling scalable and consistent multi-agent simulation with up to 1,024 concurrent players.

OverSeer: Multi-Agent World Models Beyond What Agents See

Hugging Face Daily Papers

OverSeer introduces a multi-agent world model that augments egocentric agent views with fixed auxiliary camera views acting as a world-centric representation, compressed via delta-VAE to cut token computation, showing improved consistency in Minecraft and Doom.

Multi-Agent World Models (3 minute read)

TLDR AI

γ-World is a generative multi-agent world model that supports independently controllable, permutation-symmetric agents using Simplex Rotary Agent Encoding and Sparse Hub Attention, achieving real-time 24 FPS rollouts and zero-shot generalization from two to four players.