Tag
ArcticQA/ArcticAbstain introduces a dataset and paired benchmark evaluating whether LLMs appropriately abstain on Arctic science multiple-choice questions, finding large baseline differences and only modest increases in abstention when correct answers are removed.
This paper investigates the two axes of LLM abstention: answer correctness and question answerability. It shows that a single confidence threshold conflates these two failure modes, and proposes a three-class selective acceptance framework with separate budgets. Experiments across five instruction-tuned models reveal that answerability is internally legible but poorly captured by output confidence or self-assessments.