Tag
This paper shows that Luce's choice axiom, assumed by the Bradley–Terry model underlying Chatbot Arena rankings, fails for near-copy models: votes between similar models are up to 1.37× more decisive than Bradley–Terry predicts, based on 1.8 million Arena votes. A small four-to-six parameter correction improves held-out vote prediction, though leaderboard order barely changes.
Uses the Bradley-Terry model and Elo rating system to statistically determine a dog's favorite treat through pairwise comparison experiments.
Reasoning Arena improves reinforcement learning with verifiable rewards by using trace tournaments and Bradley-Terry models to generate meaningful gradients from non-diverse reward groups, resulting in faster training and better reasoning performance.
This paper introduces ECC, an algorithm that calibrates semantic embeddings with limited model comparisons to cluster queries by latent capability requirements, improving LLM capability ranking quality by over 17 percentage points over baselines.
HumorRank introduces a tournament-based leaderboard using pairwise evaluations and Bradley-Terry MLE to rank LLMs on humor generation, showing humor quality depends on comedic mastery rather than scale.