ACE-Data-0: Human-Centric Ambient Capture as Embodied Data Engine

Hugging Face Daily Papers Papers
dataset embodied-ai robotics data-engine multimodal human-centric

Summary

Introduces ACE-Data-0, a large-scale embodied AI dataset with 150 hours of synchronized multimodal human demonstrations across 200 task categories, captured by the Ambient Capture Engine (ACE) in real home environments. Includes a hierarchical benchmark exposing gaps in current methods under contact, occlusion, and long horizons.

Embodied intelligence faces a fundamental data bottleneck. Models must capture how first-person perception, whole-body motion, dexterous manipulation, object state, sound, and touch evolve together as humans pursue goals over time. Existing datasets fragment this experience across viewpoints, modalities, or spatial scales, leaving the full perception-action loop only partially observed. We introduce the Ambient Capture Engine (ACE), a human-centric data engine that transforms real home environments into spatially calibrated, temporally synchronized recording studios. ACE operates at two complementary scales: a table-scale configuration resolves hand-object manipulation, while a room-scale configuration captures whole-body motion, locomotion, and interactions across a furnished home. ACE records egocentric and multi-view exocentric video, full-body and articulated hand motion, object geometry and 6-DoF trajectories, audio, and tactile signals as a unified multisensory stream. Using ACE, we build ACE-Data-0, comprising 150 hours and 17M video frames across 200 task categories, performed by 50 participants in 2 environments, for a total of 75,000 interaction episodes. The dataset spans atomic manipulation, long-horizon chains of household activities, and human-scene interaction, while preserving natural behavioral variation through goal-level rather than step-by-step instructions. We further introduce a hierarchical benchmark that progresses from signals to scene components and then to interactions. Evaluations of state-of-the-art methods expose substantial gaps under contact, occlusion, egomotion, and long temporal horizons. ACE-Data-0 provides synchronized human demonstrations with aligned perceptual, kinematic, and contact supervision, offering a scalable foundation for imitation learning, world models, vision-language-action systems, and embodied AI.
Original Article
View Cached Full Text

Cached at: 07/31/26, 05:53 AM

Paper page - ACE-Data-0: Human-Centric Ambient Capture as Embodied Data Engine

Source: https://huggingface.co/papers/2607.28625 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

Embodiedintelligencefacesafundamentaldatabottleneck.Modelsmustcapturehowfirst-personperception,whole-bodymotion,dexterousmanipulation,objectstate,sound,andtouchevolvetogetherashumanspursuegoalsovertime.Existingdatasetsfragmentthisexperienceacrossviewpoints,modalities,orspatialscales,leavingthefullperception-actionlooponlypartiallyobserved.WeintroducetheAmbientCaptureEngine(ACE),ahuman-centricdataenginethattransformsrealhomeenvironmentsintospatiallycalibrated,temporallysynchronizedrecordingstudios.ACEoperatesattwocomplementaryscales:atable-scaleconfigurationresolveshand-objectmanipulation,whilearoom-scaleconfigurationcaptureswhole-bodymotion,locomotion,andinteractionsacrossafurnishedhome.ACErecordsegocentricandmulti-viewexocentricvideo,full-bodyandarticulatedhandmotion,objectgeometryand6-DoFtrajectories,audio,andtactilesignalsasaunifiedmultisensorystream.UsingACE,webuildACE-Data-0,comprising150hoursand17Mvideoframesacross200taskcategories,performedby50participantsin2environments,foratotalof75,000interactionepisodes.Thedatasetspansatomicmanipulation,long-horizonchainsofhouseholdactivities,andhuman-sceneinteraction,whilepreservingnaturalbehavioralvariationthroughgoal-levelratherthanstep-by-stepinstructions.Wefurtherintroduceahierarchicalbenchmarkthatprogressesfromsignalstoscenecomponentsandthentointeractions.Evaluationsofstate-of-the-artmethodsexposesubstantialgapsundercontact,occlusion,egomotion,andlongtemporalhorizons.ACE-Data-0providessynchronizedhumandemonstrationswithalignedperceptual,kinematic,andcontactsupervision,offeringascalablefoundationforimitationlearning,worldmodels,vision-language-actionsystems,andembodiedAI.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2607\.28625

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.28625 in a model README.md to link it from this page.

Datasets citing this paper1

#### ACERobotics/ACE-Data-0 Updatedabout 3 hours ago • 22

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.28625 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

HumanNet: Scaling Human-centric Video Learning to One Million Hours

Hugging Face Daily Papers

HumanNet is a large-scale human-centric video dataset with one million hours of annotated footage, designed to train vision-language-action models. It demonstrates that egocentric human video can effectively replace robot data for embodied intelligence tasks.

Data Pyramid for Embodied Manipulation

Hugging Face Daily Papers

This paper organizes embodied data sources into a five-level pyramid (real-robot, UMI, egocentric/exocentric, simulation, general vision-language), analyzing their trade-offs between scalability and robot alignment, and reviews recent embodied foundation models in terms of data recipes. It also discusses open challenges for building next-generation embodied systems.