Learning to Fold: prizewinning solution at LeHome Challenge 2026 (1st place online, 2nd offline)

Hugging Face Daily Papers Papers
robotics reinforcement-learning garment-folding sim-to-real icra-2026 lehome-challenge robot-manipulation

Summary

This paper describes the prizewinning solution for the LeHome Challenge at ICRA 2026, where a two-armed robot learns to fold various garments using a novel RL approach with a self-contained value function, asynchronous training, and heavy sim-to-real augmentation.

I describe my solution to the LeHome Challenge 2026, an ICRA 2026 competition on bimanual garment folding. The system placed 1st of 62 teams in the online (simulation) round and 2nd in the real-world final. It improves a vision-language-action (VLA) policy with a reinforcement-learning loop. The policy is its own value function: the same network that predicts actions also predicts success, progress, and a few task-relevant future quantities, and those predictions drive advantage estimation, live failure detection, and candidate selection. The work mostly recombines existing RL ideas with engineering and optimization contributions that can be used together as one recipe or individually: AWR + RECAP combined for flow-matching VLA; an asynchronous distributed training / rollout pipeline through HuggingFace Hub; inference-time hyperparameters optimization via Thompson sampling; a sim-to-real recipe with camera-alignment tooling, heavy augmentation and DAgger-like HIL data collection.
Original Article
View Cached Full Text

Cached at: 06/29/26, 06:01 AM

Paper page - Learning to Fold: prizewinning solution at LeHome Challenge 2026 (1st place online, 2nd offline)

Source: https://huggingface.co/papers/2606.27163 Earlier this month I placed πŸ₯ˆ 2nd in the LeHome Challenge at ICRA 2026 β€” and πŸ₯‡ 1st of 62 teams in the simulation round before that. This paper explains my solution.

The task: teach a cheap two-armed robot to fold different garments β€” long tops, short tops, long pants, and shorts β€” both in simulation and on a real robot. The robot only sees three cameras and never gets told which garment it’s folding, so it has to figure that out on its own.

Here’s the short version of how it works πŸ‘‡

🧠 The policy is its own value function. From the same forward pass that picks the next action chunk, cheap heads predict success probability, task completion %, garment type, and future keypoint distances + a Q-residual. Those become the advantage signal for RL - no separate critic.

πŸ” A fully asynchronous RL loop coordinated only through the HF Hub: 1 trainer (H200) ships a fresh checkpoint ~every 40 min while N rollout workers (and a human doing teleop / DAgger corrections) collect data in parallel. Nobody waits, it uses the off-policy nature of the loop to the fullest.

πŸ“ˆ Binary success is too sparse, so I densify it by combining multiple approaches simultaneously β€” from objective keypoint distances, the success-probability value baseline, completion %, and relative success rate of different garments.

πŸŽ›οΈ The RL combines AWR (sample good actions more often) + RECAP (feed advantage as a conditioning input, then ask for good actions only at inference, with CFG). I also tune the inference knobs β€” execution length, playback speed, inpainting overlap, CFG scale, best-of-N β€” with a per-parameter Thompson-sampling bandit folded into rollout collection.

πŸ“Š Round 1: 1st of 62, 79.6% success (+6.1 over 2nd, top score on 3 of 4 garments). Round 2: 2nd place, with only ~1 week and no access to the eval robot β€” so the pipeline was sim β†’ my robot β†’ their robot, leaning on heavy augmentation to make the policy more robust.

πŸ’‘ Biggest win left on the table: I ran full RL in sim and BC + human corrections on real separately. They’re very complementary β€” combining them should push success much higher.

Similar Articles