This is a huge moment for robotics and humanity in general, and no one is paying attention.

The fact that this video has only 25k views is astounding to me.

https://t.co/CFfGdwwv0L

Robotic Foundation Model Gen1-5 Enables Instant, One-Shot Task Learning

A new robotic foundation model named Gen1-5 demonstrates the ability to learn and generalize tasks from single demonstrations in seconds, showcasing emergent physical problem-solving capabilities.

Instant Learning for Any Task

The dream of a foundational model for robotics is becoming reality with Gen1-5, a generalist model that allows you to approach a robot and have it perform almost any task immediately. The model is designed for instant learning and generalization.

Its key capabilities include:

• Single-shot learning: Through in-context prompting, it can learn and generalize new tasks within seconds.
• Task composition: It can combine multiple prompts to learn longer-horizon tasks.
• Sim-to-real transfer: It can take prompts from a simulator and transfer those behaviors to the real world.
• Live human observation: In some cases, it can directly observe a human and mimic their actions on-site immediately.

Single-Shot Learning and Generalization

For few-shot learning, the model requires only 1 to 5 minutes of data and can learn new tasks with just 1 to 10 gradient steps. The fastest learning method, however, is zero-shot training for new tasks, needing only a few seconds of demonstration data input into the model’s context. This process is called physical prompting.

Researchers have been studying variants of in-context prompting for about a year, but this is the first time they’ve observed such a highly generalized in-context learning ability. While the current tasks are relatively simple and success rates are not extremely high, this capability has never been seen before in a model. Crucially, this ability was not specifically trained for but emerged spontaneously from the new training scheme.

Physical Generalization: Tool Use and Problem-Solving

Beyond in-context learning, the model exhibits novel physical generalization capabilities, including innovative tool use.

In one example:

• The robot was taught to sweep blocks into a bowl with a brush.
• When given a banana instead, it improvised by using the banana as a brush.
• When given a dustpan, it used its other hand to push blocks onto the dustpan, then lifted it to dump the blocks into the bowl.
• It could also sweep multiple objects and learned to dexterously switch hands.

Overall, it demonstrates more spontaneous intelligence than previous models:

• When a Lego brick stuck to one hand, it used its other hand to remove it.
• After being taught to place blocks in a bowl, when the bowl was covered with paper, it moved the paper aside to complete the task.
• It was taught to open a bottle cap with one hand but then used both hands.
• When presented with different bottles and cups, it figured out how to open them.

Human-to-Robot Imitation Learning

Researchers are also observing a human-to-robot in-context learning phenomenon. A human can show the robot how to perform a task with their own hands, and the robot can immediately imitate it using its own hands. The team is in very early stages but is extremely excited about the potential of this new frontier of intelligence in robotic foundation models.

Source: https://youtu.be/1cllCVK-9lo