Tag
该论文提出了一个以用户为中心的审计框架，用于区分基准内容的贡献者（源用户）与评估结果所代表的群体（目标用户），并在16个评估资源上发现美国中心化惯例占据主导、文档中普遍存在源-目标用户语境差距，且评分指标会显著影响性能衡量。
这篇论文对IBM Telco客户流失基准进行了可信度审计，揭示了报告高达99%准确率背后的四大问题：SMOTE前划分造成的数据泄漏、TotalCharges字段冗余性导致的SHAP解释失真、概率校准方法选择，以及F1最优阈值与成本最优阈值之间的巨大差距，并提出了一个可复现的报告检查清单。
This paper critically audits benchmark datasets and baselines for spatiotemporal forecasting, showing that classical linear models often compete with graph neural networks, highlighting issues with discriminative reliability and advocating for more rigorous evaluation.
This paper audits knowledge-based VQA benchmarks, revealing systematic violations of assumptions that make accuracy a misleading metric. It introduces a repair protocol and multi-entity augmentation to restore answer derivability and question clarity, showing that corrected settings yield markedly different model rankings.
This paper audits benchmark evaluation in clinical-interview depression detection through four complementary probes across five datasets, finding that standard evaluation protocols may overestimate model performance and that leaderboard rankings lack stability.
This paper audits multimodal physics evaluation pipelines, revealing issues like train-eval contamination, translation drift, and MCQ saturation. It releases new datasets (PhysCorp-A, PhysR1Corp, PhysOlym-A) and a training recipe (Physics-R1) that significantly improves performance on held-out olympiad problems.