WCM: A World Critic Model for Vision-Language-Action Reinforcement Learning

Hugging Face Daily Papers Papers
vision-language-action reinforcement-learning world-model critic-model robotics manipulation

Summary

Introduces WCM, a World Critic Model that jointly predicts future latent states and estimates values to improve temporal modeling for Vision-Language-Action reinforcement learning, achieving state-of-the-art results across robotic manipulation benchmarks.

Reinforcement learning (RL) post-training of Vision-Language-Action (VLA) models has shown strong promise for robotic manipulation. Among RL methods, critic-based approaches rely on a value estimator that predominantly operates on single-frame observations or single-frame VLM backbone latents, which is a fundamental mismatch with the partially observable nature of robot control. A naive approach to incorporate observation history into the critic incurs exponential complexity with high-dimensional visual space, and still fails because pure scalar-return regression provides insufficient supervision for learning cross-temporal dynamics. We identify the root cause as a state approximation problem: without an explicit world modeling objective, the critic's representation cannot capture the temporal structure needed for accurate value estimation. To address this, we propose the World Critic Model (WCM), built on a lightweight LeJEPA architecture; WCM jointly predicts future latent state and estimates values, such that the critic's representation is explicitly trained to capture temporal dynamics rather than merely regress scalar returns. WCM integrates seamlessly into both on-policy and off-policy training pipelines and is compatible with state-of-the-art VLA backbones including Pi0, Pi0.5, and OpenVLA-OFT. Extensive experiments on 149 tasks across four benchmarks demonstrate that WCM consistently achieves state-of-the-art performance in both in-distribution and out-of-distribution settings, with particularly strong generalization gains. We further validate WCM on seven real-world manipulation tasks using OpenVLA-OFT and Pi0.5 with off-policy RL, confirming stable deployment across diverse settings.
Original Article
View Cached Full Text

Cached at: 08/04/26, 05:37 AM

Paper page - WCM: A World Critic Model for Vision-Language-Action Reinforcement Learning

Source: https://huggingface.co/papers/2607.29613

Abstract

Reinforcementlearning(RL)post-trainingofVision-Language-Action(VLA)modelshasshownstrongpromiseforroboticmanipulation.AmongRLmethods,critic-basedapproachesrelyonavalueestimatorthatpredominantlyoperatesonsingle-frameobservationsorsingle-frameVLMbackbonelatents,whichisafundamentalmismatchwiththepartiallyobservablenatureofrobotcontrol.Anaiveapproachtoincorporateobservationhistoryintothecriticincursexponentialcomplexitywithhigh-dimensionalvisualspace,andstillfailsbecausepurescalar-returnregressionprovidesinsufficientsupervisionforlearningcross-temporaldynamics.Weidentifytherootcauseasastateapproximationproblem:withoutanexplicitworldmodelingobjective,thecritic’srepresentationcannotcapturethetemporalstructureneededforaccuratevalueestimation.Toaddressthis,weproposetheWorldCriticModel(WCM),builtonalightweightLeJEPAarchitecture;WCMjointlypredictsfuturelatentstateandestimatesvalues,suchthatthecritic’srepresentationisexplicitlytrainedtocapturetemporaldynamicsratherthanmerelyregressscalarreturns.WCMintegratesseamlesslyintobothon-policyandoff-policytrainingpipelinesandiscompatiblewithstate-of-the-artVLAbackbonesincludingPi0,Pi0.5,andOpenVLA-OFT.Extensiveexperimentson149tasksacrossfourbenchmarksdemonstratethatWCMconsistentlyachievesstate-of-the-artperformanceinbothin-distributionandout-of-distributionsettings,withparticularlystronggeneralizationgains.WefurthervalidateWCMonsevenreal-worldmanipulationtasksusingOpenVLA-OFTandPi0.5withoff-policyRL,confirmingstabledeploymentacrossdiversesettings.

View arXiv pageView PDFProject pageGitHub11Add to collection

Get this paper in your agent:

hf papers read 2607\.29613

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.29613 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.29613 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.29613 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Learning Visual Feature-Based World Models via Residual Latent Action

Hugging Face Daily Papers

This paper introduces RLA-WM, a visual feature-based world model that leverages residual latent actions and flow matching to efficiently predict future visual states. The method outperforms existing video-diffusion and feature-based approaches while enabling novel robot learning techniques from offline, actionless demonstration videos.

WorldCycle: Self-Verifiable Reinforcement Learning for Long-Horizon Video World Models

Hugging Face Daily Papers

WorldCycle proposes a self-verifiable reinforcement learning method for long-horizon video world models, using reversible action cycles as free supervision to reduce state-returning drift by up to 44% and boost composite-action accuracy nearly 4x. It also introduces CycleBench to evaluate world models as simulators.

In-Context World Modeling for Robotic Control

Hugging Face Daily Papers

This paper introduces In-Context World Modeling (ICWM), a framework that enables robot policies to infer system variables from self-generated interactions, allowing adaptation to novel configurations without parameter updates by treating system identification as an in-context adaptation problem. It outperforms standard VLA baselines on novel camera viewpoints in simulation and real-world experiments.