Tag
Introduces Dust, the first zeroth-order credit-assignment method capable of pretraining transformers to approach or sometimes exceed backpropagation, reportedly 1,000-10,000x more compute-efficient than the state-of-the-art ES method EGGROLL. Using a parallelized 'virtual population' of activation perturbations, the authors show backprop-like gradients emerge from large populations and that zeroth-order methods actually benefit from overparameterization.
Hill Sampling is a simple test-time scaling method that repeatedly samples edits to the best verified program using frozen LLMs, achieving state-of-the-art results on algorithmic problems like circle packing and Erdős' minimum-overlap problem.
This paper introduces EGGROLL, a method using low-rank Gaussian perturbations to make evolution strategies practical for large language models, and proposes LOO-ROLL, an improved leave-one-out estimator that enhances performance and reduces variance, with significant accuracy gains on the GSM8K benchmark.
This paper investigates Evolution Strategies (ES) as a post-training paradigm for LLM reasoning, showing that ES provides broader reasoning coverage and better Pass@K performance than GRPO through sparse functional updates and population diversity.
This paper proposes Agentic ESOpt, a method using evolution strategies to enable scalable full-parameter fine-tuning of long-horizon LLM agents with minimal GPU memory requirements.
This paper proposes using Evolution Strategies (ES) instead of Reinforcement Learning for post-training LLMs, showing that ES improves solution coverage (pass@k) and achieves better results on math benchmarks.
This paper proposes Hyper-ES, a subspace-based evolution strategy framework for LLM reasoning that obtains descent directions via lightweight gradient-based fine-tuning and then uses CMA-ES to merge layer-wise DARE-TIES coefficients, consistently outperforming GRPO-LoRA while requiring fewer gradient updates.
Proposes a novel self-supervised image clustering framework that uses an evolution-strategy outer loop to maximize a 'surprise score' without needing a per-step loss, paired with a gradient-descent inner loop, achieving state-of-the-art results on standard benchmarks in the strict non-parametric setting.
Introduces Eggroll, a low-rank evolution strategy for gradient-free training of spiking neural networks, reducing memory and time overhead while achieving competitive accuracy on N-MNIST.
NVIDIA and Oxford University introduced EGGROLL, a scalable evolution strategies algorithm that trains billion-parameter models without backpropagation, using only integers and parallel mutations.
OpenAI presents evolution strategies (ES) as a scalable black-box optimization alternative to reinforcement learning for training neural network policies. ES simplifies the optimization problem by treating policy training as a stochastic parameter search that repeatedly samples and selects better parameter configurations based on reward feedback.