GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation

Hugging Face Daily Papers Papers
robotics world-models policy-evaluation benchmark embodied-ai robot-simulation

Summary

This paper systematically studies world models for robotic policy evaluation, introduces the WMBench benchmark and GigaWorld-1 model, and shows that long-horizon rollout consistency is more critical than short-term visual realism.

Evaluating embodied robot foundation models remains a critical bottleneck; unlike large language models efficiently assessed via digital benchmarks, robotic policies require slow, costly real-world rollouts limited by hardware and human supervision, which has driven interest in world models as surrogate policy evaluators, yet the key properties that make a world model reliable for policy assessment remain poorly understood. This work presents a systematic study of world models for robotic policy evaluation and introduces WMBench, a benchmark constructed from real-robot teleoperation data and matched policy rollouts covering diverse manipulation tasks to enable controlled comparisons across model families, action encodings, rollout horizons, and evaluation metrics. Using WMBench, we analyze 7 video world models, 4 action representation schemes, and over 324,000 simulated policy rollouts paired with real robot executions, further enriching our analysis with large-scale community submissions from the CVPR 2026 GigaBrain Challenge, curated synthetic trajectories, and a training videos spanning more than 12,000 hours. Our experiments deliver three core insights: evaluator quality is dominated by long-horizon, action-faithful rollout consistency rather than short-term visual realism; pretraining gains stem not only from data scale but from balancing general world knowledge with robot-specific controllability; and architectural choices including action encoding, memory design, and evaluator-focused post-training strongly determine alignment with real-world robot behavior. Drawing on these results, we derive a practical design roadmap and realize it in GigaWorld-1, a world model specially optimized for policy evaluation, and we fully release our code, models, datasets, and toolkits to advance scalable evaluation research for embodied foundation models.
Original Article
View Cached Full Text

Cached at: 07/07/26, 06:42 AM

Paper page - GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation

Source: https://huggingface.co/papers/2607.02642 Published on Jul 2

#3 Paper of the day Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

World models for robotic policy evaluation are systematically studied through a new benchmark, revealing that long-horizon rollout consistency and robot-specific controllability are more important than short-term visual realism for reliable policy assessment.

Evaluating embodied robot foundation models remains a critical bottleneck; unlike large language models efficiently assessed via digital benchmarks,robotic policiesrequire slow, costly real-world rollouts limited by hardware and human supervision, which has driven interest inworld modelsas surrogate policy evaluators, yet the key properties that make a world model reliable for policy assessment remain poorly understood. This work presents a systematic study ofworld modelsfor roboticpolicy evaluationand introduces WMBench, a benchmark constructed fromreal-robot teleoperationdata and matched policy rollouts covering diverse manipulation tasks to enable controlled comparisons across model families, action encodings,rollout horizons, and evaluation metrics. Using WMBench, we analyze 7video world models, 4action representation schemes, and over 324,000 simulated policy rollouts paired with real robot executions, further enriching our analysis with large-scale community submissions from the CVPR 2026 GigaBrain Challenge, curated synthetic trajectories, and a training videos spanning more than 12,000 hours. Our experiments deliver three core insights: evaluator quality is dominated by long-horizon, action-faithful rollout consistency rather than short-term visual realism; pretraining gains stem not only from data scale but from balancing general world knowledge with robot-specific controllability; and architectural choices including action encoding, memory design, and evaluator-focused post-training strongly determine alignment withreal-world robot behavior. Drawing on these results, we derive a practical design roadmap and realize it inGigaWorld-1, a world model specially optimized forpolicy evaluation, and we fully release our code, models, datasets, and toolkits to advance scalable evaluation research for embodied foundation models.

View arXiv pageView PDFProject pageGitHub118Add to collection

Get this paper in your agent:

hf papers read 2607\.02642

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.02642 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.02642 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.02642 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

World Model for Robot Learning: A Comprehensive Survey

Hugging Face Daily Papers

This comprehensive survey reviews the literature on world models for robot learning, covering their roles in policy learning, planning, and simulation. It highlights key paradigms, benchmarks, and future directions for predictive modeling in embodied agents.

A Definition and Roadmap for World Models

arXiv cs.AI

This academic paper provides a scientific definition of world models, discusses key technical aspects, and presents a staged roadmap for developing effective world models across AI subfields such as model-based reinforcement learning, video generation, and embodied robotics.

GigaWorld-Policy-0.5: A Faster and Stronger WAM Empowered by AutoResearch

Hugging Face Daily Papers

GigaWorld-Policy-0.5 is an enhanced World Action Model for robot control that improves training and inference efficiency through a Mixed Action-Conditioned World Modeling strategy and a Mixture-of-Transformers architecture, achieving 85ms latency on a local RTX 4090.

How Should World Models Be Evaluated? A Decision-Making-Centric Position

arXiv cs.LG

This paper surveys evaluation methods for world models and argues for a decision-making-centric framework that prioritizes counterfactual reasoning, planning, and policy optimization over visual quality. It introduces an L0–L7 evaluation ladder and a benchmark protocol to align evaluation with claimed utility.

World Value Models for Robotic Manipulation

Hugging Face Daily Papers

The paper presents World Value Model (WVM), a generalist robotic value model that combines world models with value estimation to accurately assess task progression and improve robotic policy learning from mixed-quality data, achieving state-of-the-art results on standard benchmarks and a new suboptimal data benchmark.