PAIR: Bridging Perception and Action in Vision-Language-Action Models

Hugging Face Daily Papers Papers

Summary

PAIR 引入显式的「Bridge Tokens」中间表示,将视觉-语言理解与可执行机器人动作连接起来,通过与动作潜变量对齐并提前注入动作专家,在 LIBERO、CALVIN 及真实操作任务上一致提升性能。

Vision-language-action (VLA) models map visual observations and language instructions to continuous robot actions. This task requires a transition from representations that describe the scene and instruction to representations that support action generation. Many continuous-action VLAs leave this transition implicit and supervise it mainly through the final action-prediction loss. We introduce PAIR, a framework that learns a shared perception-action representation between these two spaces. During training, a Masked Action Autoencoder encodes expert action chunks into horizon-aligned Action Latent Tokens. A Bridge Module extracts task-relevant features from the current visual-language representations. PAIR aligns these features with the Action Latent Tokens to form Bridge Tokens that preserve task information and capture the structure of expert actions. The Bridge Tokens are then projected into the action-token space and injected into the initial Action Tokens, providing an action-ready starting point for Action Expert refinement. At inference, the autoencoder is removed, and the Bridge Tokens are generated only from the current observation and instruction. Experiments on LIBERO, LIBERO-Plus, and CALVIN ABC-D show gains for the evaluated OpenVLA-OFT and VLA-Adapter models. On LIBERO-Plus, PAIR raises VLA-Adapter's success rate from 59.1% to 64.2%. On CALVIN, it increases VLA-Adapter's average completed sequence length from 4.42 to 4.53. Across seven real-world tasks, PAIR raises OpenVLA-OFT's success rate from 51.4% to 65.0%. Representation analyses show that Bridge Tokens retain task information while making continuous-action information accessible before Action Expert refinement. These results support a shared intermediate representation as a useful interface between perception and action in continuous-action VLAs.
Original Article
View Cached Full Text

Cached at: 10/16/26, 03:04 PM

Paper page - PAIR: Bridging Perception and Action in Vision-Language-Action Models

Source: https://huggingface.co/papers/2610.09016 Excited to share PAIR: Bridging Perception and Action in Vision-Language-Action Models ! 🚀

A key challenge in continuous-action VLAs is bridging the gap between visual-language understanding and executable robot actions. Instead of leaving this transition implicit, PAIR learns an explicit intermediate representation that combines task semantics with expert action structure.

Our key idea is to learn Bridge Tokens through alignment with action latents and inject them early into the Action Expert, providing an action-aware starting point for action generation.

PAIR consistently improves performance across LIBERO, LIBERO-Plus, CALVIN, and real-world manipulation tasks.

We hope PAIR inspires further exploration of explicit perception-action representations for VLA models. Feedback and discussions are very welcome!

📄 Paper:https://arxiv.org/abs/2610.09016

Similar Articles

Vision-Language-Action or World-Action? Just Action Models

Hugging Face Daily Papers

论文提出 Just Action Model (JAM),将视觉-语言-动作模型与世界-动作模型解耦为骨干初始化、未来观测目标与推理机制三个可分离选择,并用仅在训练期使用的 foresight loss 在 LIBERO、RoboTwin 等基准上超越 FastWAM/ImageWAM,同时减少 73% 的训练 FLOPs。

PDRVLA: A Vision-Language-Action Model with Physical Dynamics Representations

Hugging Face Daily Papers

PDRVLA 是一篇机器人操作研究论文,提出在视觉-语言-动作(VLA)模型中加入轻量级物理动力学模块,通过世界模型监督预测未来视觉特征,从而在不引入在线视频生成延迟的情况下提升机器人动作生成性能,并在 LIBERO、RoboTwin 2.0 等仿真基准上取得领先成功率。