discrete-optimization

This research paper evaluates frontier LLMs as batch optimizers in both continuous and discrete settings, finding them competitive in numerical tasks but more effective in semantically rich environments compared to classical methods.

ReQuant introduces a backpropagation-free, fixed-grid discrete refinement stage for post-training quantization (PTQ) that iteratively improves initial quantized models while preserving the quantized format, showing consistent gains across various LLMs and bit-widths.

This paper presents BayesPO, a Bayesian prompt optimization framework using gradient-guided discrete MCMC with parallel tempering, achieving improved accuracy on instruction-induction tasks.

TROPT is an open-source framework that unifies discrete text-trigger optimization, standardizing development and execution across domains like LLM jailbreaking and model interpretability. It includes over 15 optimizers and 30 recipes, lowering barriers for adoption and advancement.