scoring-bias

This paper proposes a novel method to mitigate scoring bias in LLM-as-a-Judge by having LLMs randomly generate numbers to measure their latent numerical bias, then rectifying token generation probabilities accordingly. Experiments across four tasks show the method outperforms baselines and reveals that scoring bias varies across models, tasks, and score ranges.