CapVector: Learning Transferable Capability Vectors in Parametric Space for Vision-Language-Action Models
Summary
This paper introduces CapVector, a method that decouples auxiliary training objectives from standard supervised finetuning in Vision-Language-Action models. By extracting transferable capability vectors and applying orthogonal regularization, it enhances model performance and generalization while significantly reducing computational overhead.
View Cached Full Text
Cached at: 05/12/26, 10:52 AM
Paper page - CapVector: Learning Transferable Capability Vectors in Parametric Space for Vision-Language-Action Models
Source: https://huggingface.co/papers/2605.10903
Abstract
A novel approach decouples auxiliary training objectives from standard supervised finetuning to enhance model capabilities while reducing computational overhead through capability vector merging and orthogonal regularization.
This paper proposes a novel approach to address the challenge thatpretrained VLA modelsoften fail to effectively improve performance and reduce adaptation costs during standardsupervised finetuning(SFT). Some advanced finetuning methods withauxiliary training objectivescan improve performance and reduce the number ofconvergence steps. However, they typically incur significant computational overhead due to the additional losses from auxiliary objectives. To simultaneously achieve the enhanced capabilities of auxiliary training with the simplicity of standard SFT, we decouple the two objectives of auxiliary-objective SFT within theparameter space, namely, enhancinggeneral capabilitiesand fittingtask-specific action distributions. To deliver the goal, we only need to train the model to converge on a small-scale task set using two distinct training strategies, resulting in two finetuned models. The parameters’ difference between the two models can then be interpreted ascapability vectorsprovided by auxiliary objectives. These vectors are then merged with pretrained parameters to form a capability-enhancedmeta model. Moreover, when standard SFT is augmented with a lightweightorthogonal regularization loss, the merged model attains performance comparable to auxiliary finetuned baselines with reduced computational overhead. Internal and external experiments demonstrate that ourcapability vectors(1) are effective and versatile across diverse models, (2) can generalize to novel environments and embodiments out of the box.
View arXiv pageView PDFProject pageGitHub2Add to collection
Get this paper in your agent:
hf papers read 2605\.10903
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper1
#### haofuly/capvector_models_collection Robotics• Updatedabout 3 hours ago
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.10903 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.10903 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Large Language Model Teaches Visual Students: Cross-Modality Transfer of Fine-Grained Conceptual Knowledge
This paper introduces LaViD, a framework that transfers semantic knowledge from a language-only LLM to a vision student model by generating multiple-choice questions as conceptual signatures, achieving superior fine-grained classification performance and robustness.
StableVLA: Towards Robust Vision-Language-Action Models without Extra Data
This paper introduces an Information Bottleneck Adapter (IB-Adapter) for Vision-Language-Action (VLA) models to improve robustness against unseen visual disturbances without requiring extra data, achieving up to 30% improvement with minimal parameter overhead.
Beyond English: Uncovering the Multilingual Gap in Vision-Language-Action Models
This paper presents the first systematic study of multilingual instruction following in Vision-Language-Action (VLA) models, revealing significant performance degradation when models trained on English are evaluated on other languages. The authors propose Multilingual Principal Component Alignment (MPCA) to reduce the multilingual performance gap.
Retrieve, Don't Retrain: Extending Vision Language Action Models to New Tasks at Test Time
This paper introduces a retrieval-augmented vision-language-action policy that eliminates per-task fine-tuning by using pre-trained models with indexed demonstrations, enabling efficient cross-embodiment generalization and task adaptation at test time.
APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies
Researchers propose APT, a two-stage training method that pretrains action experts on vision-action pairs before integrating language conditioning, significantly improving out-of-distribution instruction generalization for Vision-Language-Action policies.