GPT MOMENT of Robotics.

# Gen1-5: A Key Step Toward Robotic Foundation Models, Achieving Instant Learning and Physical Generalization **TL;DR:** The new robotic foundation model Gen1-5 demonstrates unprecedented instant learning, few-shot learning, and physical generalization capabilities. It can rapidly acquire new tasks through brief demonstrations or contextual prompts and even improvise tools on the fly. ## The Vision of Robotic Foundation Models and the Birth of Gen1-5 A core vision in the current robotics field is to create a robotic foundation model. The goal of such a model is to enable us to walk up to a robot and have it perform almost any task immediately, while also generalizing this behavior to adapt to diverse new situations. To realize this vision, researchers have introduced a new model, Gen1-5. Positioned as a generalist with **instant learning capabilities**, Gen1-5's breakthrough lies in being a **one-shot learner** that can learn and generalize new tasks in seconds via contextual prompts. It can combine prompts to learn longer-horizon tasks, even taking cues from simulators and transferring these simulated behaviors to the real world. ## Core Capabilities: Instant Learning and Few-Shot Adaptation Gen1-5’s most striking feature is its extremely efficient learning mode. ### Contextual Prompts and Instant Imitation The model supports variations of **contextual prompts**. The fastest learning method is **zero-shot training on new tasks**, requiring only seconds of demonstration data input into the model’s context. By showing the robot what to do, it can generalize the behavior—this is called "physical prompting." Going further, in some cases, it can directly observe a human and mimic actions on the spot. Simply demonstrating with our own hands how to perform a task allows the robot to immediately imitate using its own hands. This marks the first observation of **highly generalized in-context learning**, though still in a very early stage. ### Few-Shot Learning For tasks requiring a small amount of training, Gen1-5 needs only **1 to 5 minutes of data** and **1 to 10 gradient steps** to learn new tasks. This represents a massive leap compared to traditional deep learning methods, which require large datasets and lengthy training times. ## Demonstrating Physical Generalization and Improvisational Intelligence Gen1-5 not only learns quickly but also exhibits a new level of **physical generalization**, particularly in showing "improvisational intelligence" when handling new tools and objects. ### Improvised Tool Use In experiments, after teaching the robot to sweep blocks into a bowl with a brush: - When given a banana, it figured out how to use the banana as a **temporary brush** to complete the task. - When given a dustpan, it uses its other hand to push the blocks onto the dustpan, then lifts it to pour the blocks into the bowl. - It can also sweep multiple objects and learn to switch hands flexibly to perform complex manipulations. ### Solving Untrained Problems This improvisational intelligence appears in various scenarios: - When one hand has a LEGO block stuck to it, it uses the other hand to remove it. - After being taught to place blocks in a bowl, when the bowl is covered by paper, it **moves the paper away** to complete the task. - Having been taught to open a jar lid with one hand, it can instead do so directly with both hands. - When given different bottles and cups, it also figures out how to open them. These capabilities were not explicitly trained but **emerge** from the new training methods. Although these tasks are currently relatively simple and success rates aren’t exceptionally high, researchers state they have "never seen a model capable of these things," and the level of improvisational intelligence exceeded expectations. ## From Simulation to Reality, from Demonstration to Imitation Gen1-5’s abilities extend beyond simple object manipulation to more complex transfer learning. - **Sim-to-Real Transfer**: The model can take cues from simulators and transfer learned behaviors to the real world. - **Human-to-Robot In-Context Learning**: As mentioned, by observing human hand demonstrations, the robot can quickly imitate and execute new tasks. This opens new possibilities for human-robot interaction and skill transfer. While these capabilities are still in their early stages, together they point to the immense potential of robotic foundation models—an intelligent system capable of rapid adaptation, generalization, and some degree of autonomous problem-solving. ## Summary and Outlook The Gen1-5 model demonstrates a preliminary viable path for robotic foundation models through contextual prompts, few-shot learning, and strong physical generalization. Its emergent capabilities in improvised tool use, obstacle avoidance, and other areas—without explicit training—are incredibly exciting for this new field of intelligence. Although current task complexity is limited, Gen1-5 represents a key step toward robots that "learn new skills quickly like humans," laying the groundwork for more general and flexible robotic systems in the future. Source: GPT MOMENT of Robotics (https://youtu.be/1cllCVK-9lo)