inter-judge-agreement

Tag

Cards List
#inter-judge-agreement

Agreement Without Alignment: Label-Free Rubric Clarification Increases LLM-Judge Consensus but Not Agreement with Humans

arXiv cs.CL ↗ · 5d ago Cached

该论文提出一种无标签的评分标准(rubric)修复流程:通过跨评审模型的分歧自动定位有争议的评分条款并由编辑模型重写。实验表明该方法提升了评审间一致性(Krippendorff's α 提升约 0.07),但并未提升与人类专家评分的一致性,甚至点估计略低于随机条款编辑对照组。

0 favorites 0 likes
← Back to home

Submit Feedback