hyperparameter-optimization

This paper presents methods to integrate rich auxiliary information into Bayesian Optimization using LLMs, showing superior performance over standard and existing LLM-based approaches in benchmarks and real-world tasks like nuclear fusion.

The paper investigates using LLM-based agents to design expanded hyperparameter search spaces for tabular machine learning models, demonstrating performance improvements without extra tuning costs across multiple datasets.

The paper proposes Finite-Library Input-Warped Bayesian Optimization (FLIWBO), which selects input warps from a finite library to adapt input geometry for improved optimization efficiency while maintaining high-probability convergence guarantees, outperforming standard GP-UCB in various benchmarks.

This paper examines whether larger batch sizes can reduce wall-clock time-to-target in reinforcement learning for large language models by separating algorithmic and systems-level effects, providing a decision rule for optimization.

The paper tackles multi-fidelity Bayesian optimization where the highest-fidelity function is too expensive to be part of the optimization loop, and proposes incorporating historical high-fidelity data with task descriptors. The method is demonstrated on synthetic functions, chemistry, and hyperparameter optimization tasks.

Introduces AgentHPOBench, a sequential benchmark for evaluating LLM agents as hyperparameter optimizers across 30 machine learning tasks, showing current agents have measurable but limited iterative refinement abilities.

Proposes ERAHBO, an efficient heteroscedastic Bayesian optimization method for risk-aware hyperparameter optimization in reinforcement learning, using adaptive re-sampling to improve sample efficiency over fixed-budget approaches.

HOBA proposes a hierarchical reinforcement learning framework for online advertising that uses a large language model for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% improvement in a large-scale A/B test.

This paper presents a method to optimize GPT-5.2 temperature for Naranjo causality assessment in pharmacovigilance, achieving significant agreement improvements via Bayesian hyperparameter optimization with a novel composite metric (EWACS).

This paper introduces SemiScope, an analysis tool designed to disentangle the effects of classifier tuning from joint SSL and classifier optimization in semi-supervised security classification. Results show that most performance gains attributed to joint optimization can be recovered by simply tuning the classifier and its decision threshold with Bayesian optimization.

This paper demonstrates that careful preprocessing—especially context length selection, normalization, and regularization—can make simple linear models like Ridge regression competitive with or superior to large Transformer, MLP, and CNN models on time-series forecasting benchmarks.

LLMZero uses LLM agents to search over training trajectories via tree search, discovering adaptive multi-parameter transitions for RL post-training that outperform fixed schedules and grid search across diverse tasks.

This paper demonstrates that Optuna's constrained Tree-Structured Parzen Estimator (TPE) is a joint density generalization of the c-TPE algorithm, showing its invariance to constraint duplication while independent c-TPE degrades. The authors outline practical tradeoffs and directions for future study.

This paper proposes an automated hyperparameter optimization framework based on Differential Evolution for Latent Factorization of Tensors (LFT) to improve prediction accuracy on large-scale dynamic weighted directed networks, reducing the need for manual tuning.

This paper proposes a staged factorial screening workflow for budget-constrained micro-pretraining, demonstrating that short designed experiments can identify stable hyperparameter penalty directions and support a screen-then-refine strategy.

This paper introduces an auto-research framework using specialist agents to iteratively refine training recipes through an empirical loop of code execution and feedback. The system autonomously improves performance on tasks like Parameter Golf and NanoChat without human intervention by leveraging lineage feedback.