EyeRobot 2.0: Active Gaze for Precise Manipulation without Wrist Cameras

Hugging Face Daily Papers Papers
robotics active-gaze bimanual-manipulation reinforcement-learning stereo-vision foveation imitation-learning

Summary

EyeRobot 2.0 介绍了一种受人类视觉启发的机器人操作框架，通过双目相机上的主动注视（Active Visual Fixation）实现精细双臂操作，无需腕部相机。研究在真实与模拟任务中进行了上千次试验，证明该方法在物体遮挡腕部相机时可将成功率提升至对手的两倍以上。

Inspired by human vision, we introduce a framework using active gaze to enable fine-grained bimanual manipulation with only a single stereo camera. EyeRobot 2.0 physically attends to a 3D fixation point in the scene by swiveling two eye viewpoints to center their gaze on it. The resulting images are processed foveally by allocating more visual tokens to the image centers, focusing computation on task-relevant features. Such Active Visual Fixation (AVF) requires carefully coordinated gaze during task execution, which we accomplish hierarchically by first training a low-level gaze servoing policy conditioned on a goal object, then training a target selector which emits fixation goals based on task progress. Both modules are trained with RL on real-world data: the first is trained with a dense geometric reward and the second co-trains with the BC gripper policy which allows it to discover fixation sequences that can resemble a human's fixation sequence while performing the task. EyeRobot 2.0 further takes advantage of fixation by canonicalizing gripper information into a fixation-relative SE(3) frame, which compacts the size of the action distribution to learn. We collect teleoperation data for 7 real-world and 6 simulated tasks, and conduct over 1000 physical and 1800 simulated robot trials comparing EyeRobot 2.0 against passive stereo and ego + wrist camera policies trained on the same data. Removing wrist cameras is costly for standard policies: with only passive stereo, real-world success drops from 52% to 27%. EyeRobot 2.0 closes this gap with only stereo, outperforming passive stereo by 40% in real and 20% in sim. It matches ego + wrist policies when their wrist views are clear (69% vs. 64%), and more than doubles their success when grasped objects occlude the wrist cameras (48% vs. 22%)
Original Article
View Cached Full Text

Cached at: 10/05/26, 08:50 PM

Paper page - EyeRobot 2.0: Active Gaze for Precise Manipulation without Wrist Cameras

Source: https://huggingface.co/papers/2610.03710

Abstract

Inspiredbyhumanvision,weintroduceaframeworkusingactivegazetoenablefine-grainedbimanualmanipulationwithonlyasinglestereocamera.EyeRobot2.0physicallyattendstoa3Dfixationpointinthescenebyswivelingtwoeyeviewpointstocentertheirgazeonit.Theresultingimagesareprocessedfoveallybyallocatingmorevisualtokenstotheimagecenters,focusingcomputationontask-relevantfeatures.SuchActiveVisualFixation(AVF)requirescarefullycoordinatedgazeduringtaskexecution,whichweaccomplishhierarchicallybyfirsttrainingalow-levelgazeservoingpolicyconditionedonagoalobject,thentrainingatargetselectorwhichemitsfixationgoalsbasedontaskprogress.BothmodulesaretrainedwithRLonreal-worlddata:thefirstistrainedwithadensegeometricrewardandthesecondco-trainswiththeBCgripperpolicywhichallowsittodiscoverfixationsequencesthatcanresembleahuman’sfixationsequencewhileperformingthetask.EyeRobot2.0furthertakesadvantageoffixationbycanonicalizinggripperinformationintoafixation-relativeSE(3)frame,whichcompactsthesizeoftheactiondistributiontolearn.Wecollectteleoperationdatafor7real-worldand6simulatedtasks,andconductover1000physicaland1800simulatedrobottrialscomparingEyeRobot2.0againstpassivestereoandego+wristcamerapoliciestrainedonthesamedata.Removingwristcamerasiscostlyforstandardpolicies:withonlypassivestereo,real-worldsuccessdropsfrom52%to27%.EyeRobot2.0closesthisgapwithonlystereo,outperformingpassivestereoby40%inrealand20%insim.Itmatchesego+wristpolicieswhentheirwristviewsareclear(69%vs.64%),andmorethandoublestheirsuccesswhengraspedobjectsoccludethewristcameras(48%vs.22%)

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2610\.03710

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.03710 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.03710 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.03710 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Reflex: Visually Grounded Reactive Humanoid Control

Hugging Face Daily Papers

Reflex提出一个三阶段框架，让类人机器人通过自我中心RGB-D视觉实现对抛掷物体的全身心反射式接取控制，在仿真中达到85.7%成功率，并在真实Unitree G1机器人上以2.5米距离实现65%的接取成功率。

ActiveMimic: Egocentric Video Pretraining with Active Perception

Hugging Face Daily Papers

ActiveMimic is a pretraining framework that recovers camera and wrist trajectories from egocentric human video to model active perception as a viewpoint action, enabling robot pretraining that matches the performance of models trained directly on robot data.

TouchScale: 500 Hours of Human Vision and Touch for Visual-Tactile Learning

Hugging Face Daily Papers

TouchScale 引入了一个 500 小时的人类视觉-触觉交互数据集，使用统一的可穿戴设备同步采集 egocentric RGB-D 视频、手腕视频与全手双触觉测量，用于视觉-触觉预训练和机器人操作策略学习，在零样本触觉预测和真实机器人操作成功率上均取得显著提升。

Intelligent whole-body control with Gemini Robotics 2

YouTube AI Channels

Google DeepMind demonstrated the Gemini Robotics 2 model, enabling robot Apollo to understand natural language instructions, autonomously coordinate movements, and complete multi-step tasks such as packing sports equipment in cluttered environments through full-body control and embodied reasoning, validating the potential of general-purpose robots.