Weak-to-strong generalization

OpenAI Blog Papers
ai-alignment superintelligence rlhf safety superalignment weak-supervision

Summary

OpenAI's Superalignment team introduces weak-to-strong generalization, a new research direction for empirically aligning superhuman AI models by addressing the fundamental challenge of how weak human supervisors can reliably control and steer AI systems vastly smarter than themselves.

We present a new research direction for superalignment, together with promising initial results: can we leverage the generalization properties of deep learning to control strong models with weak supervisors?
Original Article
View Cached Full Text

Cached at: 04/20/26, 02:54 PM

# Weak-to-strong generalization Source: [https://openai.com/index/weak-to-strong-generalization/](https://openai.com/index/weak-to-strong-generalization/) We believe superintelligence—AI vastly smarter than humans—could be developed within the next ten years\. However, we still do not know how to reliably steer and control superhuman AI systems\. Solving this problem is essential for ensuring that even the most advanced AI systems in the future remain safe and beneficial to humanity\. We formed the[Superalignment team⁠](https://openai.com/superalignment/)earlier this year to solve this problem of superintelligence alignment\. Today, we are releasing the team’s first paper, which introduces a new research direction for empirically aligning superhuman models\. Current alignment methods, such as reinforcement learning from human feedback \(RLHF\), rely on human supervision\. However, future AI systems will be capable of extremely complex and creative behaviors that will make it hard for humans to reliably supervise them\. For example, superhuman models may be able to write millions of lines of novel—and potentially dangerous—computer code that would be very hard even for expert humans to understand\. Relative to superhuman AI models, humans will be “weak supervisors\.” This is a core challenge for AGI alignment: how can weak supervisors trust and control substantially stronger models?

Similar Articles

Superalignment Fast Grants

OpenAI Blog

OpenAI announces Superalignment Fast Grants to fund research on aligning superintelligent AI systems, addressing the fundamental challenge of how humans can steer and trust AI systems more capable than themselves. The initiative seeks to rally top researchers to tackle this critical technical problem, which OpenAI believes superintelligence could pose within the next decade.

Toward understanding and preventing misalignment generalization

OpenAI Blog

OpenAI researchers investigate 'emergent misalignment'—where fine-tuning a model on narrow incorrect behavior causes broadly unethical responses—and discover a 'misaligned persona' feature in GPT-4o's activations that mediates this phenomenon, enabling potential detection and mitigation strategies.

Our approach to alignment research

OpenAI Blog

OpenAI outlines their alignment research approach, highlighting reinforcement learning from human feedback (RLHF) as their primary technique for aligning deployed language models like InstructGPT. They discuss achieving significant preference over 100x larger models while using minimal compute, but acknowledge current limitations and propose a long-term strategy of using AI systems to accelerate alignment research beyond what humans can achieve alone.

Weak-to-Strong Learning in Decision Making

arXiv cs.LG

This paper introduces a decision-aware weak-to-strong (W2S) learning framework that uses limited labeled data to train a weak model, which then generates soft supervision on unlabeled data to train a strong model for improved contextual stochastic optimization. Theoretical bounds and empirical experiments show that abundant unlabeled data can reduce downstream decision risk when the correlation between weak and strong feature representations is small.