Tag
This paper presents a large-scale evaluation of nine uncertainty estimation methods for LLMs across 22 languages, finding that prompting models to reason in English improves uncertainty estimation for low-resource languages and that the choice of method depends on model scale.
This paper investigates memory manipulation in LLM-based agents for multiple-choice question answering, showing that corrupted memories can cause agents to select incorrect options even when the current query is clean.
This paper identifies that standard multiple-choice QA benchmarks are sensitive to phrasing artifacts, conflating knowledge with surface-form familiarity. The authors propose ParaEval, a framework that uses multiple paraphrases per answer option to score models based on most favorable phrasing, reducing false performance gaps and enabling more robust evaluation.