PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives

Hugging Face Daily Papers Papers
benchmark world-models agents evaluation video-generation long-horizon

Summary

PlayWorld is a benchmark for evaluating interactive video world models using multi-modal agent players pursuing long-horizon objectives. It assesses geometry consistency, interaction fidelity, and state evolution, revealing that current models struggle with spatial consistency and persistent state evolution.

Video world models simulate future states conditioned on current observations and user actions. Recent systems have demonstrated impressive video consistency and action controllability over long sequences. However, fairly comparing these interactive models remains challenging. In practice, a human player typically evaluates a world model by pursuing long-horizon objectives through interaction. For example, a user may turn around 360 degrees to see whether the environment remains consistent, or walk into the water and inspect whether realistic water ripples are generated. The action sequence required to achieve the same objective may vary substantially between models, making fixed action-conditioned evaluation unsuitable for cross-model comparison. To address this, we employ multi-modal Agent Players to interact with world models toward specified long-horizon objectives. Building on this paradigm, we introduce PlayWorld, a benchmark providing 171 scenarios, each with a specified objective. To evaluate performance thoroughly, we assess models along four core dimensions: geometry consistency, interaction fidelity, out-of-sight evolution, and insight evolution. In addition, we incorporate basic ability metrics for video quality and controllability. Experiments across nine state-of-the-art world models reveal that current models remain unreliable on long-horizon interactive objectives, particularly in maintaining spatial consistency and persistent state evolution. Code and data are available at https://github.com/kxding/PlayWorld.
Original Article
View Cached Full Text

Cached at: 08/14/26, 03:26 AM

Paper page - PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives

Source: https://huggingface.co/papers/2608.13552 Authors:

,

,

,

,

,

,

,

,

,

,

Abstract

PlayWorld benchmarks interactive video world models by using multi-modal agents to pursue long-horizon objectives, evaluating geometry consistency, interaction fidelity, and state evolution.

Videoworld modelssimulate future states conditioned on current observations and user actions. Recent systems have demonstrated impressive video consistency and action controllability over long sequences. However, fairly comparing these interactive models remains challenging. In practice, a human player typically evaluates a world model by pursuing long-horizon objectives through interaction. For example, a user may turn around 360 degrees to see whether the environment remains consistent, or walk into the water and inspect whether realistic water ripples are generated. The action sequence required to achieve the same objective may vary substantially between models, making fixed action-conditioned evaluation unsuitable for cross-model comparison. To address this, we employmulti-modal Agent Playersto interact withworld modelstoward specified long-horizon objectives. Building on this paradigm, we introducePlayWorld, a benchmark providing 171 scenarios, each with a specified objective. To evaluate performance thoroughly, we assess models along four core dimensions:geometry consistency,interaction fidelity,out-of-sight evolution, andinsight evolution. In addition, we incorporate basic ability metrics for video quality and controllability. Experiments across nine state-of-the-artworld modelsreveal that current models remain unreliable on long-horizon interactive objectives, particularly in maintaining spatial consistency and persistent state evolution. Code and data are available at https://github.com/kxding/PlayWorld.

View arXiv pageView PDFProject pageGitHub8Add to collection

Get this paper in your agent:

hf papers read 2608\.13552

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.13552 in a model README.md to link it from this page.

Datasets citing this paper1

#### jocelynd/playworld-bench Viewer• Updatedabout 2 hours ago • 171 • 1 • 1

Spaces citing this paper1

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles