EduPanel: A Three-Agent LLM Judge for Teaching Videos -- Reliability, Complementarity, and Human Trust Calibration

Hugging Face Daily Papers Papers
llm-judge multi-agent teaching-videos educational-evaluation trust-calibration reliability

Summary

EduPanel is a rubric-grounded, learner-conditioned LLM judge that uses three specialized agents to evaluate teaching video quality, achieving reliability comparable to human experts and improving scoring accuracy while maintaining detectability of unreliable outputs.

Teaching videos are becoming a major medium for education, creating a growing need for scalable evaluation of their pedagogical quality. Existing automatic judges do not fully address this setting because teaching quality depends on multimodal evidence and should be evaluated with respect to the intended learner rather than as a universal property. We present EduPanel, a rubric-grounded, learner-conditioned LLM judge that decomposes evaluation across specialized agents to produce interpretable assessments for different aspects of teaching quality. Across expert studies, architecture ablations, and learner-persona analyses, EduPanel achieves reliability comparable to a median human expert. In expert evaluation, its feedback improves scoring accuracy (MAE 0.87 to 0.73), while experts remain able to detect unreliable outputs (AUC = 0.77) instead of accepting them blindly. These results suggest that EduPanel can serve as effective assistants for educational evaluation rather than replacements for human experts.
Original Article
View Cached Full Text

Cached at: 07/22/26, 06:41 AM

Paper page - EduPanel: A Three-Agent LLM Judge for Teaching Videos – Reliability, Complementarity, and Human Trust Calibration

Source: https://huggingface.co/papers/2607.18529 Published on Jul 20

·

Submitted byhttps://huggingface.co/Snooow1029

董家愷on Jul 22

Abstract

Teachingvideosarebecomingamajormediumforeducation,creatingagrowingneedforscalableevaluationoftheirpedagogicalquality.Existingautomaticjudgesdonotfullyaddressthissettingbecauseteachingqualitydependsonmultimodalevidenceandshouldbeevaluatedwithrespecttotheintendedlearnerratherthanasauniversalproperty.WepresentEduPanel,arubric-grounded,learner-conditionedLLMjudgethatdecomposesevaluationacrossspecializedagentstoproduceinterpretableassessmentsfordifferentaspectsofteachingquality.Acrossexpertstudies,architectureablations,andlearner-personaanalyses,EduPanelachievesreliabilitycomparabletoamedianhumanexpert.Inexpertevaluation,itsfeedbackimprovesscoringaccuracy(MAE0.87to0.73),whileexpertsremainabletodetectunreliableoutputs(AUC=0.77)insteadofacceptingthemblindly.TheseresultssuggestthatEduPanelcanserveaseffectiveassistantsforeducationalevaluationratherthanreplacementsforhumanexperts.

View arXiv pageView PDFGitHub0Add to collection

Get this paper in your agent:

hf papers read 2607\.18529

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.18529 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.18529 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.18529 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

LLM-as-Judge in Education: A Curriculum-Grounded Marking Pipeline

arXiv cs.AI

This paper presents a curriculum-grounded LLM-as-Judge pipeline for automated question-level marking in high-stakes exam preparation, using syllabus artefacts and marking guidelines to improve consistency and transparency, with preliminary evaluation showing outcomes comparable to human tutors.

Uncertainty-Aware Trust Estimation for Multi-LLM Systems via Structured Expert Judgement

arXiv cs.LG

This paper introduces an uncertainty-aware trust estimation method for aggregating predictions from multiple LLMs, adapting structured expert judgment with Cooke-style log weighting to penalize overconfident incorrect predictions. Evaluations on MMLU and MMLU-Pro show that this approach achieves superior accuracy-reliability balance under heterogeneous and contaminated expert panels.