@heyshrutimishra: A robot can't learn from a video it can't measure. @Maniformer , the world’s leading physical AI data platform. Its new…

A robot can't learn from a video it can't measure. @Maniformer , the world’s leading physical AI data platform. Its new paper scores a person's hands to 4.29 mm of finger error and their head to between 0.65 and 2.83 mm, checked against a 26-camera motion capture lab. Teaching a robot on a real robot is slow, because every hour of teleoperation uses a machine, an operator and an environment reset. Putting a camera on a person scales with the number of people instead, which is why egocentric datasets have grown from hundreds of hours to thousands. But video only shows what someone did. To turn it into robot actions, you need the hand's 3D position in real units. The wrist path goes through inverse kinematics and the fingers get retargeted to the robot's hand, so supervision is only as good as the hand pose underneath it. The paper's complaint is that datasets carrying those labels were shot in studios or on prepared tabletops, and where labels are released, the accuracy is asserted from hardware specs instead of measured. When I toured Maniformer's headquarters, the data collection centre showed the same problem from the other side. One room runs more than a hundred robots at once, and right beside them people in motion-capture suits perform the same tasks by hand so the robots can learn how a human moves. Maniformer's bet is to take the robot out of that loop. The system is called MEgoVista, and it runs on MEgo View, a head-mounted rig with fisheye cameras at 60 fps and an IMU sampled at 500 Hz. One unprepared recording goes in. Both hands and the head come out in a single gravity-aligned world frame. Three design choices carry it. First, depth is measured instead of guessed. Most existing methods get depth from a single camera with a learned model, so scale is a prior that shifts from frame to frame, and the error sits along the viewing ray where reprojection can't correct it. Here the central camera pairs with each side camera, stereo runs across the wide baseline, and because that baseline is calibrated, the metre comes from the hardware. Scale is fixed once, at initialisation. Second, it knows whose hands are whose. A detector decides at detection time whether each hand belongs to the wearer or a bystander. In the manicure clip the wearer's left hand touches another person's left hand, and both Dyn-HaMR and HaWoR credit the bystander's hand to the wearer. HaPTIC produced no valid output on the multi-person scenes at all. Dyn-HaMR's detection precision came out at 0.73, while MEgoVista scored 1.00 precision and 1.00 recall. Third, it solves the whole sequence instead of frame by frame. A per-frame estimate can look plausible alone and still jitter, drift in depth, or pass through a pose a real hand can't make. MEgoVista optimises one hand trajectory across the full episode with limits on joint range and angular velocity, and lands at 1.11 mm/frame² of jitter against 3.50 for the smoothest baseline. Then comes the scoring, which is the part I found most interesting. Everything was measured against a 26-camera, 120 Hz Chingmu motion-capture system. The protocol charges the method for any hand it declines to predict, and it screens the reference data itself for solver glitches before trusting it. Finger articulation error is 4.2 mm, against 16.30 mm for HaWoR and 49.18 mm for Dyn-HaMR. Absolute wrist position error is 14.30 mm against 46.70 and 184.38. On head tracking, DROID-W suffered severe scale failures on 4 of 22 episodes and MASt3R-SLAM underestimated scale on 21 of the 22, while MEgoVista stayed within a few millimetres and ran about 3x and 3.5x faster on roughly 40 second episodes. One honest note on that last comparison. ORB-SLAM3 was more accurate on one of the two capture sessions, 2.12 mm against 2.83 mm. On the other session MEgoVista won, 0.65 mm against 0.92 mm. The paper reports both, so I will too. The other limit is where the millimetres were measured. They come from inside a motion-capture lab. In the wild there is no ground truth, so the evidence is visual: milk tea preparation with heavy hand occlusion, washing a pet with soiled hands, mold maintenance in industrial gloves, and a working restaurant. The authors also say any error left in the mocap hand reference is charged to their own system, and that slow drift in that reference would slip past their checks. Maniformer says it has collected over 1 million hours of data and delivered the 20,000th MEgo device on August 31. If that scale holds, labels measured against a reference matter more than hours alone. The paper ends by saying it is undecided whether the next millimetre needs a better estimator or a better reference, which is a rare sentence for a company to publish about its own system. Full paper: http://arxiv.org/abs/2609.16684