Tag
The paper proposes a cost-aware algorithm for identifying the best large language model using dueling feedback in a multi-armed bandit framework, demonstrating optimal cost and improvements over existing methods.
This paper formalizes multi-model human evaluation as a best-arm identification problem in a multi-armed bandit setup, adaptively allocating annotation effort to focus on competitive models and improve ranking discrimination.
This paper studies fixed-confidence best-arm identification where costly reward observations are paired with cheap but correlated proxy scores from generative models. It proposes the PROBE algorithm that uses control-variate adjustment and upper certificates on residual variance to achieve oracle-like sample complexity.
This paper introduces a hybrid Track-and-Stop algorithm for best arm identification in generalized linear bandits that unifies absolute and relative feedback. The authors propose a likelihood-ratio-based confidence sequence to adaptively allocate queries, demonstrating improved sample efficiency over baseline methods.