Grabette: an open system to record robot-manipulation data

Source: https://huggingface.co/blog/grabette Back to Articles

https://huggingface.co/blog/grabette#grabette-an-open-system-to-record-robot-manipulation-data–and-build-a-shared-dataset-togetherGrabette: an open system to record robot-manipulation data. And build a shared dataset, together.

• The bottleneck isn’t the model. It’s the data.
• Standing on the shoulders of UMI
• Meet Grabette
• Built for everyone
• From your hand to a dataset, in two steps- 1. Record - 2. Process, directly in your browser
• What can you do with the data? Here’s one example.
• Now it’s your turn- What’s next

Record your own manipulation tasks in minutes with a handheld gripper, turn them into robot-ready datasets automatically, and help grow an open, collaborative dataset for robot learning.

https://huggingface.co/blog/grabette#the-bottleneck-isnt-the-model-its-the-dataThe bottleneck isn’t the model. It’s the data.

Robot learning has a supply problem. We have capable policy architectures (transformer-based VLAs, diffusion and flow-matching policies, and even world models) and the GPUs to train them. What we lack is large, diverse, real-world manipulation data.

Teleoperating a robot to collect it can be expensive and demanding: first of all,it requiresa robot. And depending on the teleoperation method, data collection can be tedious for the user if it takes hours and involve significant hardware and logistical challenges. That is difficult to scale with the wide variety of tasks and environnements required.

But**you don’t need a robot to collect robot data.**Just a human hand, a gripper, a camera, and a way to recover the 6-DoF trajectory of what the hand did. Capture the demonstration and you have data a robot can learn from.

That’s what we’re releasing today: Grabette, an open, low-cost system for recording manipulation data. Pick it up, record a task with your own hand, and get back a clean, robot-ready dataset. No robot, no lab, no teleop rig.

And that’s the bigger goal: if recording a demonstration is as easy as shooting a video, anyone can contribute. We want Grabette to seed alarge, open, collaborative manipulation dataset. One no single lab could ever build alone.

https://huggingface.co/blog/grabette#standing-on-the-shoulders-of-umiStanding on the shoulders of UMI

Grabette is directly inspired by theUniversal Manipulation Interface(UMI)from Stanford: a handheld gripper with a fisheye camera that records demonstrations “in the wild”, recovers camera trajectories with SLAM, and trains visuomotor policies from them.

UMI proved the recipe works. Other (closed source) devices exists like Agibot’s MEgo gripper, Genrobot’s DAS gripper and Sunday Robotics skill capture glove. Our goal was to make it effortless to use, to get the barrier from “I have a task” to “I have a trained model” as low as possible.

Grabette is built into the modern open ecosystem: LeRobot for datasets, the Hugging Face Hub for sharing, and a processing pipeline you runfrom your browserwith nothing to install. Grabette is something anyone can build on a workbench, use in the field, and contribute data from.

https://huggingface.co/blog/grabette#meet-grabetteMeet Grabette

We have been developing Grabette for months, and we feel it has become usable enough to share. We are excited to share it now with you!

Grabette is ahandheld gripperinstrumented with everything needed to reconstruct a manipulation demonstration.

It carries**two cameras, each with a distinct job.**Splitting the two roles is deliberate: the cheap wide fisheye gives the policy the context-rich, wrist-camera-style view it needs, while the RGBD camera does the heavy lifting of robust 6-DoF tracking.

And while Grabette records data during tasks performed by a user, it relies on its robotic counterpart to execute the movements it has learned after training. So, meetGripette, the robotic arm end-effector twin of Grabette.

The family shares the same hardware DNA:

• **Grabette,**the handheld demonstration device (camera + IMU + gripper, BOM cost ~490€)
• **Gripette,**the motorized gripper (camera + two servomotors, BOM cost ~120€) that closes the loop on a real or simulated robot arm

Grabette hand held recording device (left) and Gripette robot gripper (right)

https://huggingface.co/blog/grabette#built-for-everyoneBuilt for everyone

Everything is open source

Go check the repository!

• Hardware: CAD and production files for Grabette and Gripette
• Capture service: the on-device Raspberry Pi software
• Processing pipeline: run locally, or online via our Hugging Face Space
• Example downstream stack: stock LeRobot training + the OpenArm evaluation, as a reference

**Components.**Standard sensors you can buy, no closed pipeline, no fork lock-in. A Raspberry Pi, a standard Pi camera, an off-the-shelf OAK-D depth camera, magnetic encoders. The whole point is that anyone can build one from parts you can just order.

**Robot-agnostic by design.**Nothing in the capture or the data format assumes a particular arm. Demonstrations are stored as camera-local 6-DoF cartesian pose plus gripper state, the output is a standard LeRobot dataset on the Hugging Face Hub, so the same data can drive different robots and different learning methods. You will still need the matching Gripette gripper on your arm though.

https://huggingface.co/blog/grabette#from-your-hand-to-a-dataset-in-two-stepsFrom your hand to a dataset, in two steps

This release enables anyone to go from “I want to demonstrate a task” to “I have a training-ready dataset” quickly and without prior expertise.

https://huggingface.co/blog/grabette#1-record1. Record

Press the button, and data from the observation camera, the tracking camera (color, depth, and IMU), and the gripper’s encoder joint values are recorded simultaneously, using a single shared clock to ensure proper synchronization. Press the button again to stop the episode, and the data is saved locally on the Raspberry Pi.

https://huggingface.co/blog/grabette#2-process-directly-in-your-browser2. Process, directly in your browser

Open the Grabette dashboard in your browser. Select the episodes you want to add to the dataset, and with one click, post-processing begins.

• The episodes are uploaded to the HF Hub
• The grabette-slam space performs SLAM using RTAB-MAP library and verifies that the trajectory is correct (with no jumps or loss of tracking)
• The episodes are converted to LeRobot format
• A new dataset is uploaded to your space, where you can view the data for each episode using the Le Robot visualizer

Everything is now ready to start training!

https://huggingface.co/blog/grabette#what-can-you-do-with-the-data-heres-one-exampleWhat can you do with the data? Here’s one example.

A dataset is only interesting if it trains something. So, to show the loop end-to-end, we ship a**complete example.**But to be clear, this isanexample of what is enables, not the product: the release is the recording system. Your Grabette data works with any method that consumes LeRobot datasets.

Our example takes 200 recorded demonstrations such as :

and:

• Trains a policywith theLeRobotstack — a Diffusion Policy (ResNet18 + SpatialSoftmax encoder, DDIM scheduler, 6-D rotation actions) that fits on a single consumer GPU.
• Evaluates iton a OpenArm 7-DoF arm with the Gripette gripper, driven over a gRPC API, with this result :

https://huggingface.co/blog/grabette#now-its-your-turnNow it’s your turn

The data bottleneck doesn’t get solved by one lab, but by a community recording demonstrations everywhere. You can now help build the dataset: build a Grabette, record tasks you are interested in, and share them on the Hub. Every episode makes the open dataset bigger and more diverse, and every contributor makes robot learning a little less gated behind expensive hardware.

https://huggingface.co/blog/grabette#whats-nextWhat’s next

This release is only the start of the project! Grabette will keep evolving. We already have more coming, includingCasquette, a head-mounted POV device to complement Grabette for egocentric capture (still a work in progress). But the most important next step isn’t only ours, it’s also yours: start recording, and let’s build the dataset together.

👉Build a Grabette (GitHub)·Process your data (HF Space)· Contribute to the dataset

Built with ❤️ by Pollen Robotics.