self-play

Max Rumpf argues that human feedback is becoming obsolete for training advanced AI models, citing examples like chess, math, and search. He advocates for human-free methods like self-play and synthetic data, while a quoted tweet from Will Depue calls for a large-scale data infrastructure parallel to compute scaling.

Trained a superhuman Generals.io agent using self-play reinforcement learning with a JAX-based pipeline and Vision Transformer. Achieved #1 on human 1v1 leaderboard; all code and a fast JAX simulator open-sourced.

EMAgnet introduces parameter-space exponential moving average regularization for policy gradient self-play in large two-player zero-sum games, achieving lower exploitability compared to uniform regularization targets.

A research paper that combines a small amount of human demonstrations as a regularization objective with self-play reinforcement learning, enabling human-compatible driving policies using far less human data (30 minutes vs thousands of hours) and training in 15 hours on a single consumer GPU.

A DeepSeek researcher open-sourced AutoResearch, an autonomous framework that can plan, execute, and debug RL experiments on the DeepSeek 285B model without human intervention, accompanied by a self-play survey paper.

Deli Chen open sources his AutoResearch SKILL tool and releases a survey paper on Self-play, inspired by AlphaZero.

Deli AutoResearch SKILL is open-sourced, an autonomous framework that automates GPU experiments and RL pipelines, with a companion survey paper on Self-play.

This paper presents Delta-Star, a deep reinforcement learning approach using AlphaZero-style self-play to discover superior lattice reduction strategies by interacting with the primitive actions of the LLL algorithm. The learned policy generalizes to higher dimensions and unseen moduli without retraining.

This paper presents a self-play reinforcement learning framework for the four-player imperfect-information card game Big 2, comparing policy-gradient and value-based methods and finding that PPO with entropy regularization outperforms others.

SCOPE is a self-play framework for open-ended tasks that co-evolves a Challenger and Solver policy, achieving up to +10.4 points on benchmarks without external supervision.

A new paper from Meta, CMU, and other labs presents Self-play SWE-RL, a method where coding agents train themselves by manufacturing and fixing bugs in real codebases, achieving significant gains on SWE-bench benchmarks without relying on human-written tasks.

CoSPlay is a training-free framework that jointly improves code generation and unit test quality through cooperative self-play, achieving competitive performance without ground-truth unit tests.

Explores how close a biologically plausible Hebbian agent can get to PPO on Pong, finding only a 2% gap but identifying catastrophic forgetting under self-play as a key bottleneck.

This paper identifies a threshold in decision capacity that determines whether self-play reinforcement learning agents collapse under asymmetric rule perturbations, showing that eliminating all positive-reach contingent decisions leads to rapid convergence to a deterministic exploitation attractor.

This paper studies adversarial action masking in self-play reinforcement learning, where an attacker selectively removes legal actions from a victim's action set. The attack is shown to be significantly more damaging than random masking or perturbation baselines across multiple environments and algorithms, and victims do not recover under extended training.

PopuLoRA introduces a population-based asymmetric self-play framework for RLVR post-training of LLMs, where teacher and student LoRA adapters co-evolve to generate increasingly complex problems, overcoming the self-calibration limitation of single-agent self-play.

Eric Jang announces he has been working on a from-scratch implementation of AlphaGo, the 2016 AI breakthrough that inspired him to enter deep learning.

A thread sharing a video of self-play RL training with lidar and PPO in Unity, followed by a lecture on building AlphaGo from scratch.

A detailed discussion on reinforcement learning and its connection to modern AI, using the reconstruction of AlphaGo with modern tools as a clear example of search and self-play. Key takeaways include neural network amortization of search, credit assignment challenges in LLMs vs AlphaGo, and implications for automated research.

Researchers introduce Self-Guided Self-Play (SGS), a self-play algorithm for LLMs that prevents reward hacking by using a Guide role to score synthetic problems. Applied to theorem proving in Lean4, SGS surpasses RL baselines and allows a 7B model to outperform a 671B model.