EDU-CIRCUIT-HW: Evaluating Multimodal Large Language Models on Real-World University-Level STEM Student Handwritten Solutions
Summary
This paper introduces the EDU-CIRCUIT-HW dataset for evaluating multimodal large language models on real-world university-level STEM handwritten solutions, revealing significant recognition limitations and proposing a hybrid approach that combines automated recognition with minimal human oversight to enhance grading robustness.
View Cached Full Text
Cached at: 05/08/26, 07:00 AM
Paper page - EDU-CIRCUIT-HW: Evaluating Multimodal Large Language Models on Real-World University-Level STEM Student Handwritten Solutions
Source: https://huggingface.co/papers/2602.00095
Abstract
EDU-CIRCUIT-HW dataset reveals significant limitations in MLLMs’ ability to accurately interpret complex STEM handwritten solutions, prompting a hybrid approach combining automated recognition with minimal human oversight for improved educational grading systems.
Multimodal Large Language Models(MLLMs) hold significant promise for revolutionizing traditional education and reducing teachers’ workload. However, accurately interpreting unconstrained STEM studenthandwritten solutionswith intertwined mathematical formulas, diagrams, and textual reasoning poses a significant challenge due to the lack of authentic and domain-specific benchmarks. Additionally, current evaluation paradigms predominantly rely on the outcomes ofdownstream tasks(e.g.,auto-grading), which often probe only a subset of the recognized content, thereby failing to capture the MLLMs’ understanding of complex handwritten logic as a whole. To bridge this gap, we release EDU-CIRCUIT-HW, a dataset consisting of 1,300+ authentic studenthandwritten solutionsfrom a university-level STEM course. Utilizing the expert-verified verbatim transcriptions andgrading reportsof student solutions, we simultaneously evaluate various MLLMs’upstream recognitionfidelity and downstreamauto-gradingperformance. Our evaluation uncovers an astonishing scale oflatent failureswithin MLLM-recognized student handwritten content, highlighting the models’ insufficient reliability forauto-gradingand other understanding-oriented applications in high-stakes educational settings. As a potential solution, we present a case study demonstrating that leveraging identified error patterns to preemptively detect and correct recognition errors, while requiring only minimal human intervention (e.g., routing 3.3% of assignments to human graders and the remainder to the GPT-5.1 grader), can effectively enhance the robustness of the deployedAI-enabled grading system. Code and dataset are available in this GitHub repo: https://gt-learning-innovation.github.io/CIRCUIT_EDU_HW_ACL.
View arXiv pageView PDFProject pageGitHub4Add to collection
Get this paper in your agent:
hf papers read 2602\.00095
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2602.00095 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2602.00095 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2602.00095 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
EduArt: An educational-level benchmark for evaluating art history knowledge in large language models
This paper introduces EduArt, an educational-level benchmark with 871 human-authored questions for evaluating art history knowledge and visual reasoning in multimodal LLMs, revealing that single-format benchmarks overestimate model capabilities.
LiveK12Bench: Have Large Multimodal Models Truly Conquered High School-level Examinations?
Introduces LiveK12Bench, a dynamic multimodal benchmark for evaluating large models on real high school exams across multiple disciplines, revealing substantial performance gaps under realistic constraints.
Large Language Models for Math Education in Low-Resource Languages: A Study in Sinhala and Tamil
This paper evaluates the mathematical reasoning capabilities of large language models in Sinhala and Tamil, two low-resource South Asian languages, using a parallel dataset of independently authored problems. The study demonstrates that while basic arithmetic transfers well across languages, complex reasoning tasks show significant performance degradation in non-English languages, with implications for deploying AI tutoring tools in multilingual educational contexts.
Elmes*: Automated Construction of Fine-Grained Evaluation Rubrics for Large Language Models in Long-Tail Educational Scenarios
This paper introduces Elmes+, an automated framework for constructing fine-grained evaluation rubrics for LLMs in long-tail educational scenarios, and presents the Edu-330 benchmark covering 330 scenarios across 11 subjects. The framework uses a multi-agent engine and self-evolving module to co-optimize evaluation criteria and test data, revealing multidimensional educational capability differences among top LLMs.
Large Language Models as Unified Multimodal Learners for Clinical Prediction
The paper proposes converting multimodal patient data (text, labs, vitals) into a single natural language sequence and fine-tuning LLMs for clinical prediction, achieving comparable or better performance than specialized fusion architectures across three tasks.