abstention

Introduces Safe-Psych, a sequential benchmark for evaluating how large language models handle diagnostic uncertainty in psychiatry, revealing that even strong models often fail to abstain or seek clarification when clinical evidence is incomplete.

This paper presents AgentAbstain, the first systematic evaluation framework for LLM agents' ability to abstain from acting when appropriate, including a benchmark of 263 paired tasks across 8 abstention scenarios. The best agent achieves only 59.5% paired accuracy, revealing a critical gap independent of general task-solving capability.

This paper defines agentic abstention, the problem of deciding when an LLM agent should stop acting under uncertainty, and evaluates it across web shopping, terminal environments, and question answering. It introduces convolve, a context engineering method that improves timely abstention without updating model parameters.

This paper introduces Know2Guess, a contamination-aware multi-zone benchmark designed to evaluate the transition from answerable knowledge to expected abstention in large language models, addressing data contamination, prompt sensitivity, and refusal behavior. The authors assess FLAN-T5, Qwen2.5-Instruct, and Llama-3-Instruct models, finding that stronger models show selective but incomplete abstention. The benchmark and dataset are publicly released.

OpenBioRQ is a new benchmark of 12,553 unsolved biomedical research questions that tests agentic models' ability to verify sources and avoid false citations. It reveals that current models often link to wrong papers and suffer from agentic collapse on hard questions.

A paper accepted at ICML 2026 introduces predictable hallucination via an information-budget abstention gate, and releases ntkMirror, a training-free open-weight implementation that reduces hallucination by abstaining when information is insufficient, achieving 0.0–0.7% hallucination at ~24% abstention.

This paper argues that current benchmarks for autonomous agents fail to evaluate whether an agent should have proceeded at all, introducing a 'compliance bias'. The authors propose a taxonomy of abstention-warranted scenarios and new evaluation protocols (Safety Rate, Usability Rate, Informed Refusal Rate) with preliminary results showing tunable safety–usability tradeoffs across model families.

The paper introduces SpatialUncertain, a benchmark to evaluate whether vision-language models recognize when they cannot answer spatial questions due to occlusion or perspective ambiguity, revealing overconfidence and poor abstention behavior.

This paper studies fairness in toxicity classification across three axes: ranking, calibration, and abstention. It compares ERM, reweighted ERM, and Group DRO methods with post-hoc interventions, finding that calibration disparity is a hidden fairness violation and that abstention itself can be unfair.

This paper introduces NoisyCoconut, an inference-time method that improves LLM reliability by injecting noise into latent trajectories to generate diverse reasoning paths. The approach enables models to abstain when uncertain, significantly reducing error rates in mathematical reasoning tasks without requiring retraining.