Tag
This paper proposes a hybrid nested search framework that decouples structural sketching (by an LLM) from numeric parameter optimization (by traditional solvers like CMA-ES) in LLM-driven evolutionary optimization, and validates it across meta-optimization, code-based policies, and Bayesian inference tasks.
This paper proposes Hyper-ES, a subspace-based evolution strategy framework for LLM reasoning that obtains descent directions via lightweight gradient-based fine-tuning and then uses CMA-ES to merge layer-wise DARE-TIES coefficients, consistently outperforming GRPO-LoRA while requiring fewer gradient updates.
TinyRouter is a tiny 10K-parameter LLM router that learns to route each question to the best specialist model from a pool of open-source LLMs, using evolutionary training. It achieves performance matching or exceeding individual models on MMLU and math benchmarks.