Hide-and-Seek in Trajectories: Discovering Failure Signals for VLA Runtime Monitoring

Hugging Face Daily Papers Papers
vla-models failure-detection contrastive-learning robotics runtime-monitoring coarse-supervision robot-execution

Summary

Hide-and-Seek is a framework that detects robot execution failures in VLA models by localizing failure-indicative actions through contrastive learning without step-level annotations, achieving state-of-the-art multi-task failure detection.

Vision-Language-Action (VLA) models enable robots to follow natural language instructions and generalize across diverse tasks, but they remain vulnerable to execution failures that compromise reliability in real-world deployment. Detecting such failures during execution is therefore critical for the robust deployment of embodied systems. Existing failure detection methods either rely on expensive action resampling or external models, while alternatives propagate trajectory-level labels uniformly across every timestep, obscuring localized failure signals. In this paper, we propose Hide-and-Seek, a framework that formulates VLA failure detection as a coarsely supervised learning problem. By combining inter-trajectory and intra-trajectory contrastive objectives, Hide-and-Seek localizes failure-indicative actions and induces temporally structured failure signals from trajectory-level supervision alone, without any step-level annotation. We evaluate Hide-and-Seek on LIBERO, VLABench, and a real-world robotic platform across three representative VLA policies: OpenVLA, π_0, and π_{0.5}.Our method achieves state-of-the-art multi-task failure detection performance with a practical accuracy--timeliness trade-off under conformal prediction, and generalizes well to both seen and unseen tasks.
Original Article
View Cached Full Text

Cached at: 06/01/26, 03:18 AM

Paper page - Hide-and-Seek in Trajectories: Discovering Failure Signals for VLA Runtime Monitoring

Source: https://huggingface.co/papers/2605.30834

Abstract

Hide-and-Seek framework detects robot execution failures in vision-language-action models by localizing failure-indicative actions through contrastive learning from trajectory-level supervision without step-level annotations.

Vision-Language-Action (VLA) models enable robots to follow natural language instructions and generalize across diverse tasks, but they remain vulnerable to execution failures that compromise reliability in real-world deployment. Detecting such failures during execution is therefore critical for the robust deployment of embodied systems. Existingfailure detectionmethods either rely on expensive action resampling or external models, while alternatives propagate trajectory-level labels uniformly across every timestep, obscuring localized failure signals. In this paper, we propose Hide-and-Seek, a framework that formulates VLAfailure detectionas acoarsely supervised learningproblem. By combining inter-trajectory andintra-trajectory contrastive objectives, Hide-and-Seek localizes failure-indicative actions and induces temporally structured failure signals from trajectory-level supervision alone, without any step-level annotation. We evaluate Hide-and-Seek on LIBERO, VLABench, and a real-world robotic platform across three representative VLA policies:OpenVLA, π_0, and π_{0.5}.Our method achieves state-of-the-art multi-taskfailure detectionperformance with a practical accuracy--timeliness trade-off underconformal prediction, and generalizes well to both seen and unseen tasks.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2605\.30834

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.30834 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.30834 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.30834 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

MEMENTO: Memory-Guided Memetic Code-as-Policy Evolution

arXiv cs.LG

MEMENTO introduces a memory-guided memetic framework for evolving robot control programs (code-as-policy), outperforming existing methods like Eureka and REvolve in long-horizon embodied tasks, and demonstrating sim-to-real transfer.

Enigma raises $71M to make controlling a robot as easy as adjusting the volume

TechCrunch AI

Enigma emerges from stealth with a $71M seed round led by Index Ventures and Ribbit Capital to develop intuitive human-robot interfaces. The startup launches a large-scale online experiment allowing anyone to interact with over 100 of its proprietary AI robots, aiming to make robot control as effortless as turning a volume knob.