Tag
This paper introduces KappaSharp, a method for condition-number shaping in Preferential Bayesian Optimization that corrects the ill-conditioned Hessian from isolated pairwise comparisons, showing up to +10.9% improvement over the standard PairedGP/EUBO baseline on 11 benchmarks including plasma medicine controller tuning.
This paper proposes a scalable, domain-agnostic framework for automated LLM evaluation that uses pairwise comparisons by multiple LLMs and an Elo rating system to approximate expert judgments, reducing the need for human intervention.
Uses the Bradley-Terry model and Elo rating system to statistically determine a dog's favorite treat through pairwise comparison experiments.
Proposes a prompt perturbation framework that generates perturbed prompt variants, filters out structurally inconsistent comparison patterns using graph-level consistency checks, then applies standard ranking methods to yield more reliable LLM rankings.
MIT researchers present a paper showing that using three-way comparisons instead of pairwise comparisons can significantly improve the accuracy of random utility models for predicting human preferences.