Act with Intent: Distilling Behavior Intent for Vision-Language-Action Models

Hugging Face Daily Papers Papers
vision-language-action robotics behavior-cloning distillation action-modeling latent-representation benchmark-improvement

Summary

This paper proposes Intention Distillation (INDI) to distill behavior intent into the action decoder of Vision-Language-Action models, improving performance on benchmarks like SimplerEnv-Bridge and real-world tasks.

Vision-Language-Action (VLA) models can turn multimodal context into robot actions, but their action decoders are still trained largely by behavior cloning. This supervises which motor command was demonstrated while leaving implicit the local objective served by the behavior under the instruction. Future-based supervision enriches action learning with frames, latent observations, trajectories, or motion representations, but these signals capture particular realizations of what may happen rather than the shared semantic objective of the forthcoming behavior. We propose Intention Distillation (INDI), which distills behavior-level intent into the action decoder. During training, a frozen teacher VLM interprets a demonstrated segment from the current observation, instruction, coarse action summary, and corresponding execution video. From its standard inputs, the deployed VLA recovers the resulting multimodal intent representation at an intermediate decoder layer and uses it to organize action prediction together with representations of how the behavior unfolds and what it achieves. On SimplerEnv-Bridge, INDI improves GR00T-N1.7 from 64.3% to 84.7%, and on RoboCasa Kitchen it improves the controlled GR00T-N1.7 baseline from 64.1% to 70.3%, with consistent gains on π_{0.5} across both benchmarks. In real-world tasks, INDI improves average success from 62.0% to 68.7%, with gains of up to 12.0 pp on longer-horizon tasks. Further analyses show that the recovered latent is used by the decoder, captures behavior objective and execution progress, and organizes downstream predictions in an objective-dependent manner. These results show that action decoders benefit from explicitly modeling the semantic objective of the behavior they generate.
Original Article
View Cached Full Text

Cached at: 09/01/26, 12:01 PM

Paper page - Act with Intent: Distilling Behavior Intent for Vision-Language-Action Models

Source: https://huggingface.co/papers/2608.23478

Abstract

Vision-Language-Action(VLA)modelscanturnmultimodalcontextintorobotactions,buttheiractiondecodersarestilltrainedlargelybybehaviorcloning.Thissuperviseswhichmotorcommandwasdemonstratedwhileleavingimplicitthelocalobjectiveservedbythebehaviorundertheinstruction.Future-basedsupervisionenrichesactionlearningwithframes,latentobservations,trajectories,ormotionrepresentations,butthesesignalscaptureparticularrealizationsofwhatmayhappenratherthanthesharedsemanticobjectiveoftheforthcomingbehavior.WeproposeIntentionDistillation(INDI),whichdistillsbehavior-levelintentintotheactiondecoder.Duringtraining,afrozenteacherVLMinterpretsademonstratedsegmentfromthecurrentobservation,instruction,coarseactionsummary,andcorrespondingexecutionvideo.Fromitsstandardinputs,thedeployedVLArecoverstheresultingmultimodalintentrepresentationatanintermediatedecoderlayerandusesittoorganizeactionpredictiontogetherwithrepresentationsofhowthebehaviorunfoldsandwhatitachieves.OnSimplerEnv-Bridge,INDIimprovesGR00T-N1.7from64.3%to84.7%,andonRoboCasaKitchenitimprovesthecontrolledGR00T-N1.7baselinefrom64.1%to70.3%,withconsistentgainsonπ_{0.5}acrossbothbenchmarks.Inreal-worldtasks,INDIimprovesaveragesuccessfrom62.0%to68.7%,withgainsofupto12.0pponlonger-horizontasks.Furtheranalysesshowthattherecoveredlatentisusedbythedecoder,capturesbehaviorobjectiveandexecutionprogress,andorganizesdownstreampredictionsinanobjective-dependentmanner.Theseresultsshowthatactiondecodersbenefitfromexplicitlymodelingthesemanticobjectiveofthebehaviortheygenerate.

View arXiv pageView PDFProject pageGitHub4Add to collection

Get this paper in your agent:

hf papers read 2608\.23478

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.23478 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.23478 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.23478 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

IntentVLA: Short-Horizon Intent Modeling for Aliased Robot Manipulation

Hugging Face Daily Papers

IntentVLA is a history-conditioned visual-language-action framework that improves robot imitation learning stability by encoding short-horizon intents from visual observations, addressing challenges from partial observability and ambiguous observations. It also introduces AliasBench, an ambiguity-aware benchmark for evaluating such methods.

TBD-VLA: Temporal Block Diffusion Vision Language Action Model

Hugging Face Daily Papers

TBD-VLA introduces a discrete vision-language-action framework that combines block diffusion with autoregressive generation to achieve efficient temporal action modeling and faster inference, significantly outperforming prior VLA approaches in simulation and real-world manipulation tasks.