black-box-optimization

This research paper evaluates frontier LLMs as batch optimizers in both continuous and discrete settings, finding them competitive in numerical tasks but more effective in semantically rich environments compared to classical methods.

WMLLM is a self-evolving optimization-agent framework that uses predict-then-act world modeling with large language models to improve sample efficiency in black-box optimization, achieving state-of-the-art results in multi-objective molecular optimization.

This paper introduces GRAPE, a two-stage framework for query-efficient high-dimensional Bayesian optimization that refines gradient posteriors and selects update directions based on expected progress, demonstrating superior performance in adversarial attacks and large language model prompt optimization.

The paper tackles multi-fidelity Bayesian optimization where the highest-fidelity function is too expensive to be part of the optimization loop, and proposes incorporating historical high-fidelity data with task descriptors. The method is demonstrated on synthetic functions, chemistry, and hyperparameter optimization tasks.

Introduces BBOWP-Bench, a benchmark suite for evaluating LLMs on black-box optimization word problems, where systems must infer both a search space and an optimization algorithm from natural-language descriptions. Initial results show LLMs can select suitable algorithms but struggle with search space design.

This paper introduces Maximally Robust Satisficing Bayesian Optimization (MRSBO), a method that efficiently finds solutions meeting a quality threshold while being robust to input perturbations after deployment, outperforming previous approaches.

Introduces SPARROW, a black-box optimization algorithm that decouples the generative prior from the reward signal, enabling effective optimization under low budgets with noisy or unreliable feedback.

A new GPU-native parallel optimizer, ChiSao, for multimodal black-box functions that uses convergence-anticonvergence oscillation to find all modes. It achieves 100% mode recovery and up to 34x speedup over baselines on benchmark functions.

Introduces SPO, a stochastic search framework for automatic prompt optimization, with three strategies including SAGE, an agent-guided multi-agent pipeline. Evaluated on benchmarks and deployed on a mental-health chatbot, showing improvements in retention through continuous optimization.

This paper proposes DOMOO, a diversity-driven offline multi-objective optimization method that uses accumulative risk control and nested Pareto set learning to address out-of-distribution issues, achieving superior convergence and diversity on benchmarks.

The paper proposes a permutation-invariant Bayesian optimization method based on Optimal Transport for optimizing offshore wind farm layouts, which reduces computation time by half and yields better layouts compared to vanilla Bayesian optimization.

OpenAI presents evolution strategies (ES) as a scalable black-box optimization alternative to reinforcement learning for training neural network policies. ES simplifies the optimization problem by treating policy training as a stochastic parameter search that repeatedly samples and selects better parameter configurations based on reward feedback.