标签
斯坦福大学的研究人员开发了一种基于广义性理论（Generalizability theory）的贝叶斯方差分解框架，用于诊断 agent 基准测试的可靠性。研究发现，增加更多任务并不能消除由脚手架（scaffold）覆盖度不足所引起的不确定性，而整合多样化的基准测试才是获得可靠排名的更低成本途径。
Researchers show agent leaderboards rank task specialization rather than capability, with agent main effects accounting for under 3% of variance across three benchmarks, and propose a Deployment Decision Reliability framework.
This paper applies Generalizability Theory to agent benchmarks, showing leaderboards rank specialization rather than capability, and proposes a framework (DDR) for sizing reliable deployment evaluations.