cascade-evaluation

作者对比了 Jev 与三种 LLM 评审（Sonnet、Haiku、Gemini Flash Lite）在智能体评测中的表现：Jev 成本低 180 倍（每 1k 条裁决 $0.07）但准确率低 2.6 个百分点（90.6% vs 93.2%），建议采用 Jev 先筛 + Sonnet 处理约 16% 不确定样本的级联方案，以 1/6 成本达到 Sonnet 级准确率。

This paper introduces JEV-as-a-Judge, a cost-effective evaluation method for LLMs that uses a decision-only judge with confidence thresholds to accept certain verdicts and escalate uncertain ones, achieving comparable accuracy to state-of-the-art models at significantly lower cost.