diagnostic

OpenMAS-GCom introduces a diagnostic benchmark for evaluating graph-enhanced multi-agent systems by using controlled interventions to attribute performance differences to specific organizational components.

The FDA has cleared the PrecivityAD2 blood test, developed from technology at WashU Medicine, to aid in Alzheimer's evaluation by detecting amyloid plaques with over 90% accuracy using mass spectrometry.

The paper introduces D^CF5, a diagnostic to predict regionwise gains in dynamic ensembling for regression tasks under distribution shift, validated with high correlation across datasets.

This paper introduces EC-Reason-Bench, a training-free diagnostic benchmark to analyze why general LLMs fail on enzyme EC number prediction. It finds that external knowledge is decisive and must precede reasoning, and that reasoning over evidence acts as an arbiter of conflicting nearest neighbors rather than a source of new knowledge.

This paper investigates when machine learning outperforms traditional value sorting for exposure-weighted shipment prioritization using three datasets.

This paper introduces answer-in-context, a diagnostic metric for budget-constrained multi-hop RAG that measures whether the gold answer survives in the packed reader context, and proposes a submodular evidence packing method that improves over heuristics under specific conditions.

This paper proposes a leakage-safe diagnostic to test whether quality-aware multimodal fusion methods actually use reliability scores during inference, by permuting these scores across test examples. Experiments on StressID and CMU-MOSEI show that shuffled reliability scores leave performance unchanged, indicating that quality signals only influence decisions when they reliably predict unimodal correctness.

This paper introduces MoDiCoL, a modular diagnostic continual learning dataset for robust speech recognition, enabling controlled analysis of linguistic content, speaker characteristics, and acoustic environments, and proposes a continual learning curriculum to study how robustness is acquired, transferred, and forgotten.

ToolFailBench, a diagnostic benchmark for tool-using agents, has been accepted at two ICML 2026 workshops, FAGEN and AIWILD.

A tweet observing that much AI cognition will be adequate for tasks, with remaining work involving diagnostic triage such as deciding whether to spend on a lawyer.

This paper investigates a harmful phenomenon in long chain-of-thought (CoT) training traces where post-conclusion continuation reduces training utility, and proposes a diagnostic method called HarmfulContinuationCut (HCC) to detect such harmful continuations.

This paper frames LLM-generated reward shaping for sparse structured RL as a debugging problem, identifying failure modes like reward flooding and semantic misunderstanding. The authors propose diagnostic-driven iterative refinement, achieving dramatic success rate improvements (e.g., DoorKey-8×8 from 2.3% to 97.6%) compared to one-shot generation.

Introduces SeqMem-Eval, a diagnostic evaluation framework for sequentially evolving LLM memory that measures multiple dimensions beyond aggregate metrics, revealing trade-offs between adaptability and stability.