Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models

Hugging Face Daily Papers Papers
vision-language-action tree-search distillation action-evaluation generalization test-time-scaling reinforcement-learning

Summary

Introduces SVA, a framework that decouples action generation from consequence evaluation in frozen VLA models using Monte-Carlo tree search and distillation into a lightweight Q-value model, improving generalization and task success rates while reducing computational costs.

Vision-Language-Action (VLA) models acquire broad embodied capabilities through large-scale pretraining, yet their generalization remains far more fragile than that of LLMs and VLMs. The prevailing remedy, post-training via supervised fine-tuning or reinforcement learning, improves task-specific performance but narrows the generalist capability that makes pretraining valuable. We identify a key bottleneck: VLA failures stem not only from action generation but also from action evaluation. A diagnostic pass@k study confirms that frozen VLAs already contain competent behaviors in their output distribution, with overall success rates rising from 33% at pass@1 to 92% at pass@32. Inspired by this, we propose SVA (Search, Value, and Act), a simple framework that equips frozen VLA policies with long-term consequence awareness. SVA first uses Monte-Carlo tree search in simulation to fully explore the VLA's output distribution and collect diverse trajectories annotated with empirical returns; this knowledge is then distilled into a lightweight Q-value model that predicts the expected consequence of candidate actions; at deployment, the frozen VLA proposes multiple candidates and the evaluator selects the one with the highest uncertainty-regularized Q-value, requiring no simulator access. By decoupling action proposal from consequence evaluation, SVA preserves the generalization capacity of the VLA backbone while substantially improving task success rates. Experiments across embodied benchmarks show that SVA consistently improves generalization on unseen tasks and exhibits strong test-time scaling behavior. Strikingly, SVA enables a 9B VLA to outperform a 27B VLA by 7 points at 27% lower inference latency, suggesting that scaling test-time evaluation is more cost-effective than scaling model size.
Original Article
View Cached Full Text

Cached at: 07/07/26, 06:45 PM

Paper page - Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models

Source: https://huggingface.co/papers/2607.03751

Abstract

A framework called SVA is introduced that enhances Vision-Language-Action models by decoupling action generation from consequence evaluation, thereby improving generalization and task success rates while reducing computational costs.

Vision-Language-Action (VLA) models acquire broad embodied capabilities through large-scale pretraining, yet their generalization remains far more fragile than that of LLMs and VLMs. The prevailing remedy, post-training viasupervised fine-tuningorreinforcement learning, improves task-specific performance but narrows the generalist capability that makes pretraining valuable. We identify a key bottleneck: VLA failures stem not only from action generation but also from action evaluation. A diagnosticpass@k studyconfirms that frozen VLAs already contain competent behaviors in their output distribution, with overall success rates rising from 33% at pass@1 to 92% at pass@32. Inspired by this, we propose SVA (Search, Value, and Act), a simple framework that equips frozen VLA policies with long-term consequence awareness. SVA first usesMonte-Carlo tree searchin simulation to fully explore the VLA’s output distribution and collect diverse trajectories annotated with empirical returns; this knowledge is then distilled into a lightweightQ-value modelthat predicts the expected consequence of candidate actions; at deployment, the frozen VLA proposes multiple candidates and the evaluator selects the one with the highestuncertainty-regularized Q-value, requiring no simulator access. By decoupling action proposal from consequence evaluation, SVA preserves the generalization capacity of the VLA backbone while substantially improving task success rates. Experiments across embodied benchmarks show that SVA consistently improves generalization on unseen tasks and exhibits strongtest-time scalingbehavior. Strikingly, SVA enables a 9B VLA to outperform a 27B VLA by 7 points at 27% lower inference latency, suggesting that scaling test-time evaluation is more cost-effective than scaling model size.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2607\.03751

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.03751 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.03751 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.03751 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

From Foundation to Application: Improving VLA Models in Practice

Papers with Code Trending

This paper presents LingBot-VLA 2.0, which enhances VLA foundation models for robotics by improving generalization across tasks and embodiments, expanding action space to whole-body degrees of freedom, and incorporating predictive dynamics modeling for better temporal reasoning.

Hierarchical Advantage Weighting for Online RL Fine-Tuning of VLAs from Sparse Episode Outcomes

Hugging Face Daily Papers

This paper proposes Hierarchical Advantage-Weighted Behavior Cloning (HABC) for fine-tuning Vision-Language-Action (VLA) policies using online reinforcement learning with sparse binary episode outcomes. HABC separates viability and efficiency objectives via adaptive critic heads and intervention-aware credit assignment, significantly improving success rates on contact-rich bimanual manipulation tasks.

Seeing Before Colliding: Anticipatory Safe RL with Frozen Vision-Language Models

arXiv cs.LG

This paper presents VLM-Safe-RL, a framework that integrates frozen vision-language models into constrained MDP Lagrangian updates to provide anticipatory cost signals for safe reinforcement learning in high-speed visual control tasks. The method outperforms standard constraint-aware baselines on Safety-Gymnasium FormulaOne L2 and generalizes to held-out environments.