confidence-intervals

This paper analyzes sources of ranking uncertainty in LLM benchmarks like MMLU, proposing modifications to hypothesis tests for constructing rank confidence intervals, and shows that variability across subjects is substantial.

This paper evaluates confidence interval methods for classifier performance metrics under conditions typical of social science text classification, providing guidance on accurate interval estimation for metrics like recall and precision when using large language models.