failure-detection

This paper introduces 'legible failures' in language models, where models possess correct information in hidden states but fail to use it, and shows that linear probes can detect and repair such failures through steering interventions.

Agnost AI is a product designed to catch failures in AI agent evaluations that traditional methods might miss.

The paper proposes RUPA, a framework that models LLM agent execution as a dependency graph to propagate uncertainty, improving failure detection and confidence estimation in long trajectories.

A reflection on how AI agents often confuse 'error cleared' with 'task actually resolved', and the need for a structurally different second validation step to catch masked failures.

This paper identifies spectral drift in internal activations of neural networks during misclassifications and introduces Self-Detecting Neural Networks (SDNN) that monitors spectral dynamics to detect failures, achieving 79% AUROC on CIFAR-10, outperforming confidence-based methods by 25-30 percentage points.

This paper proposes a two-stage online learning framework for detecting service-affecting failures in mobile core networks by modeling normal traffic dynamics and analyzing residuals, achieving improved precision-recall trade-off over static thresholds.

Discusses the phenomenon where AI agents appear to succeed at tasks but later reveal failures, highlighting challenges in agent evaluation and monitoring.

This paper proposes a recall-controlled abort cascade that uses lightweight probes on LLM agent internal representations to detect and abort doomed episodes early, saving up to 47% inference compute while maintaining high recall of successful episodes.

Foresight is a failure detection framework for long-horizon robotic manipulation that uses action-conditioned world model latents and functional conformal prediction to monitor trajectories, trained only with final task labels. It demonstrates state-of-the-art performance across simulation and real robot tasks.

A community discussion on agent memory reveals that while various patches exist for what to write down (e.g., plain files, layered memory, post-mortems), the unsolved problem is what to keep—detecting failures is tractable, but deciding which lessons persist still needs human judgment.

This paper characterizes two distinct processes by which language models fail in reasoning—committed failure and persistent uncertainty—using token-level uncertainty signals, and demonstrates implications for self-consistency and failure detection strategies.

AEGIS uses activation-probe early warning to switch to a stronger policy before failures compound in long-horizon robot manipulation, recovering twice as many failures as budget-matched escalation.

Hide-and-Seek is a framework that detects robot execution failures in VLA models by localizing failure-indicative actions through contrastive learning without step-level annotations, achieving state-of-the-art multi-task failure detection.

This paper compares cross-validation ensembles to deep ensembles for uncertainty estimation in medical image segmentation. Deep ensembles outperform cross-validation ensembles in calibration and failure detection, while cross-validation ensembles better approximate inter-rater variability.