benchmark-robustness

This paper identifies that standard multiple-choice QA benchmarks are sensitive to phrasing artifacts, conflating knowledge with surface-form familiarity. The authors propose ParaEval, a framework that uses multiple paraphrases per answer option to score models based on most favorable phrasing, reducing false performance gaps and enabling more robust evaluation.