bayesian-accuracy

The paper analyzes length bias in likelihood-based multiple-choice evaluation, showing that standard and length-normalized accuracy introduce biases toward shorter and longer answers respectively. It proposes Bayesian accuracy, a scoring rule that uses an explicit prior over answer length to remove linear length effects, demonstrating consistently lower empirical bias across benchmarks.