On first-order meta-learning algorithms

OpenAI Blog Papers
meta-learning first-order-algorithms maml few-shot-learning optimization

Summary

This paper analyzes first-order meta-learning algorithms for few-shot learning, introducing Reptile and providing theoretical insights into why these computationally efficient methods work well on established benchmarks.

No content available
Original Article Export to Word Export to PDF
View Cached Full Text

Cached at: 04/20/26, 02:56 PM

# On first-order meta-learning algorithms Source: [https://openai.com/index/on-first-order-meta-learning-algorithms/](https://openai.com/index/on-first-order-meta-learning-algorithms/) ## Abstract This paper considers meta\-learning problems, where there is a distribution of tasks, and we would like to obtain an agent that performs well \(i\.e\., learns quickly\) when presented with a previously unseen task sampled from this distribution\. We analyze a family of algorithms for learning a parameter initialization that can be fine\-tuned quickly on a new task, using only first\-order derivatives for the meta\-learning updates\. This family includes and generalizes first\-order MAML, an approximation to MAML obtained by ignoring second\-order derivatives\. It also includes Reptile, a new algorithm that we introduce here, which works by repeatedly sampling a task, training on it, and moving the initialization towards the trained weights on that task\. We expand on the results from Finn et al\. showing that first\-order meta\-learning algorithms perform well on some well\-established benchmarks for few\-shot classification, and we provide theoretical analysis aimed at understanding why these algorithms work\.

Similar Articles

Reptile: A scalable meta-learning algorithm

OpenAI Blog

OpenAI introduces Reptile, a scalable meta-learning algorithm for few-shot classification that achieves comparable performance to MAML while converging faster with lower variance. The paper provides theoretical analysis showing Reptile maximizes inner product between task gradients for improved generalization.

One-shot imitation learning

OpenAI Blog

OpenAI proposes a meta-learning framework for one-shot imitation learning that enables robots to learn new tasks from a single demonstration and generalize to new instances without task-specific engineering. The approach uses soft attention mechanisms to allow neural networks trained on diverse task pairs to perform well on unseen tasks at test time.

Some considerations on learning to explore via meta-reinforcement learning

OpenAI Blog

OpenAI researchers introduce E-MAML and E-RL², two meta-reinforcement learning algorithms designed to improve exploration in tasks where discovering optimal policies requires significant exploration. The work demonstrates these algorithms' effectiveness on novel environments including Krazy World and maze tasks.

FSPO: Few-Shot Optimization of Synthetic Preferences Personalizes to Real Users

arXiv cs.CL

FSPO proposes a few-shot preference optimization algorithm for LLM personalization that reframes reward modeling as meta-learning, enabling models to quickly infer personalized reward functions from limited user preferences. The method achieves 87% personalization performance on synthetic users and 70% on real users through careful synthetic preference dataset construction.