N_0-VTLA: Scaling Vision-Tactile-Language-Action Model with Latent Tactile Tokens

Hugging Face Daily Papers Papers
vision-tactile language-action-model robotics manipulation foundation-model tactile-perception

Summary

Introduces N_0-VTLA, a vision-tactile-language-action foundation model for contact-rich manipulation, featuring large-scale tactile pretraining and advantage-conditioned offline policy improvement, with strong results on real-robot and simulation benchmarks.

We present N_0-VTLA, a vision-tactile-language-action (VTLA) foundation model capable of (1) fine-grained contact-rich manipulation with tactile perception and tactile-feedback control, and (2) offline policy improvement from stored deployment data. Building on current vision-based backbones, we propose a training recipe for tactile integration consisting of visuo-tactile pre-training, staged tactile-pathway integration, and advantage-conditioned offline policy improvement. During pre-training, the policy learns broad contact priors from NeoData, our large-scale visuo-tactile robot dataset; to our knowledge, N_0-VTLA is the first VTLA model pretrained on tactile data at scale. During post-training, we augment the policy with a predictive tactile pathway that distills the contact patterns learned at scale into the fine motion adjustments required by downstream tactile-centric manipulation. For offline policy improvement, we introduce ALTER, an advantage-conditioned offline reinforcement learning method that converts relative progress and trajectory-event comparisons into binary advantage labels for policy training on a fixed deployment corpus, further improving task-specific learning on contact-rich skills such as deformable object manipulation. Across contact-rich benchmarks, N_0-VTLA outperforms strong baselines by wide margins: it wins all nine real-robot NeoReal tasks and reaches 63.8% mean success on a twenty-task simulation suite, against 44.0% for the strongest baseline. N_0-VTLA policies trained with ALTER reach 75-95% success on three long-horizon real-robot tasks. These results lay a foundation for versatile tactile-driven manipulation policies.
Original Article
View Cached Full Text

Cached at: 08/03/26, 05:30 AM

Paper page - N_0-VTLA: Scaling Vision-Tactile-Language-Action Model with Latent Tactile Tokens

Source: https://huggingface.co/papers/2607.23782

Abstract

WepresentN_0-VTLA,avision-tactile-language-action(VTLA)foundationmodelcapableof(1)fine-grainedcontact-richmanipulationwithtactileperceptionandtactile-feedbackcontrol,and(2)offlinepolicyimprovementfromstoreddeploymentdata.Buildingoncurrentvision-basedbackbones,weproposeatrainingrecipefortactileintegrationconsistingofvisuo-tactilepre-training,stagedtactile-pathwayintegration,andadvantage-conditionedofflinepolicyimprovement.Duringpre-training,thepolicylearnsbroadcontactpriorsfromNeoData,ourlarge-scalevisuo-tactilerobotdataset;toourknowledge,N_0-VTLAisthefirstVTLAmodelpretrainedontactiledataatscale.Duringpost-training,weaugmentthepolicywithapredictivetactilepathwaythatdistillsthecontactpatternslearnedatscaleintothefinemotionadjustmentsrequiredbydownstreamtactile-centricmanipulation.Forofflinepolicyimprovement,weintroduceALTER,anadvantage-conditionedofflinereinforcementlearningmethodthatconvertsrelativeprogressandtrajectory-eventcomparisonsintobinaryadvantagelabelsforpolicytrainingonafixeddeploymentcorpus,furtherimprovingtask-specificlearningoncontact-richskillssuchasdeformableobjectmanipulation.Acrosscontact-richbenchmarks,N_0-VTLAoutperformsstrongbaselinesbywidemargins:itwinsallninereal-robotNeoRealtasksandreaches63.8%meansuccessonatwenty-tasksimulationsuite,against44.0%forthestrongestbaseline.N_0-VTLApoliciestrainedwithALTERreach75-95%successonthreelong-horizonreal-robottasks.Theseresultslayafoundationforversatiletactile-drivenmanipulationpolicies.

View arXiv pageView PDFProject pageGitHub31Add to collection

Get this paper in your agent:

hf papers read 2607\.23782

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.23782 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.23782 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.23782 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

LabVLA: Grounding Vision-Language-Action Models in Scientific Laboratories

Hugging Face Daily Papers

LabVLA is a vision-language-action model for scientific laboratory automation, trained with a two-stage approach combining action token pretraining and flow matching. It achieves state-of-the-art success rates on the LabUtopia benchmark by leveraging simulated data to bridge the gap between household demonstrations and lab-specific tasks.