HubEgo-100K: In-the-Wild Egocentric Video of Skilled Work for Robot Learning

Hugging Face Daily Papers Papers

Summary

HubEgo-100K presents a large-scale in-the-wild egocentric dataset of skilled work for robot learning: 10,124 hours of 1080p video with synchronized IMU data from 12,227 contributors across 15,548 environments, released as 72,754 annotated episodes in LeRobot and MCAP formats to study diversity-driven policy generalization.

Robot policies trained by imitation generalize poorly outside the environments and objects seen in training. Published data-scaling studies find that generalization grows with the diversity of training environments and objects rather than with more demonstrations per environment. Robot data is costly to collect across many environments, whereas egocentric human video can be recorded in the wild at scale; existing egocentric video, however, mostly shows daily-life activity and contains little skilled work in real workplaces. We present HubEgo-100K, a dataset built for diversity. This version of the paper covers the first release, v1.0: 10,124 hours of 1080p egocentric video with synchronized 100 Hz inertial (IMU) data from one head-mounted phone camera. In v1.0, 12,227 contributors record their own unscripted work in 15,548 environments (each one contributor's own setting of one type, such as their kitchen or workshop) across 11 setting types, 30 work categories, 56 task types and 35 action families, with 83% of hours in workplace settings. The release comprises 72,754 episodes with machine-generated action and language annotations, in LeRobot and MCAP formats under CC BY 4.0. We measure diversity along each axis, show that inside v1.0, with episodes in random order, new contributors and words keep appearing, and relate the measurements to published evidence on six data properties tied to policy performance, including hand visibility and inertial sensing. We also document collection, curation, privacy measures and data format, and provide a datasheet. Later versions of this paper will add benchmarks and cover further releases toward 100,000 hours.
Original Article
View Cached Full Text

Cached at: 11/19/26, 03:40 AM

Paper page - HubEgo-100K: In-the-Wild Egocentric Video of Skilled Work for Robot Learning

Source: https://huggingface.co/papers/2611.21938

Abstract

Robotpoliciestrainedbyimitationgeneralizepoorlyoutsidetheenvironmentsandobjectsseenintraining.Publisheddata-scalingstudiesfindthatgeneralizationgrowswiththediversityoftrainingenvironmentsandobjectsratherthanwithmoredemonstrationsperenvironment.Robotdataiscostlytocollectacrossmanyenvironments,whereasegocentrichumanvideocanberecordedinthewildatscale;existingegocentricvideo,however,mostlyshowsdaily-lifeactivityandcontainslittleskilledworkinrealworkplaces.WepresentHubEgo-100K,adatasetbuiltfordiversity.Thisversionofthepapercoversthefirstrelease,v1.0:10,124hoursof1080pegocentricvideowithsynchronized100Hzinertial(IMU)datafromonehead-mountedphonecamera.Inv1.0,12,227contributorsrecordtheirownunscriptedworkin15,548environments(eachonecontributor’sownsettingofonetype,suchastheirkitchenorworkshop)across11settingtypes,30workcategories,56tasktypesand35actionfamilies,with83%ofhoursinworkplacesettings.Thereleasecomprises72,754episodeswithmachine-generatedactionandlanguageannotations,inLeRobotandMCAPformatsunderCCBY4.0.Wemeasurediversityalongeachaxis,showthatinsidev1.0,withepisodesinrandomorder,newcontributorsandwordskeepappearing,andrelatethemeasurementstopublishedevidenceonsixdatapropertiestiedtopolicyperformance,includinghandvisibilityandinertialsensing.Wealsodocumentcollection,curation,privacymeasuresanddataformat,andprovideadatasheet.Laterversionsofthispaperwilladdbenchmarksandcoverfurtherreleasestoward100,000hours.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2611\.21938

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2611.21938 in a model README.md to link it from this page.

Datasets citing this paper1

#### HubLabs/HubEgo-100K LeRobot• Updatedabout 2 hours ago • 72.8k episodes • 30.8k • 46

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2611.21938 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

HumanNet: Scaling Human-centric Video Learning to One Million Hours

Hugging Face Daily Papers

HumanNet is a large-scale human-centric video dataset with one million hours of annotated footage, designed to train vision-language-action models. It demonstrates that egocentric human video can effectively replace robot data for embodied intelligence tasks.

Ego2Robot: Scalable Robot Data Synthesis from Egocentric Human Data

Hugging Face Daily Papers

Ego2Robot is a scalable pipeline that converts egocentric human manipulation videos into robot training data via action retargeting and visual synthesis, producing 18,561 hours of data across 15 robot morphologies. Experiments show that joint pretraining on this synthesized data improves out-of-distribution generalization for vision-language-action models, including on real-robot deployment.