Tag
论文研究了基准测试中问题难度和框架方式如何影响模型训练分数的观测提升，指出许多基准隐性地加权了特定难度区域，其排行榜结论需谨慎解读，部分基准甚至覆盖不足难度轴，难以有效监控模型改进过程。
Jev 1.14.0 achieves strong overall calibration on binary questions (ECE of 0.023), but a breakdown by difficulty reveals it is strongly over-confident on hard questions by about 22 percentage points, exceeding the calibration threshold for that slice.