LessonBench-V1: A Benchmark Dataset for Evaluating AI Lesson Generation Agents

# LessonBench-V1: A Benchmark Dataset for Evaluating AI Lesson Generation Agents
Source: [https://arxiv.org/html/2607.13041](https://arxiv.org/html/2607.13041)
11institutetext:Department of Computer Science, Nottingham Trent University, Nottingham, UK
Corresponding author:11email:ravidu\.rammunisilva2024@my\.ntu\.ac\.uk22institutetext:Department of Physics and Mathematics, Nottingham Trent University, Nottingham, UKAhmad LotfiIsibor Kennedy IhianleGolnaz ShahtahmassebiJordan J\. Bird

###### Abstract

Large Language Model \(LLM\) based AI educational content generation systems are increasingly being developed, yet no standardised benchmark exists to systematically evaluate them\. This study introducesLessonBench\-V1, a benchmark dataset comprising 647 human\-written lessons paired with LLM\-based reverse\-engineered lesson plans across 240 STEM topics spanning mathematics, physics, chemistry, and computer science\. The lessons are drawn from 97 trusted open sources, including LibreTexts, Brilliant\.org and GeeksForGeeks\. Each lesson plan is human\-reviewed and produced through a pedagogically grounded methodology that synthesises Bloom’s Taxonomy, Gagné’s Events, Merrill’s First Principles, and the 5E Instructional Model\. The lesson plans capture 3,620 learning objectives with pedagogical metadata, enabling systematic, reproducible evaluation of lesson\-generation AI agents and supporting further research\. The study further proposes a three\-dimensional evaluation pipeline for use with the dataset\.

## 1Introduction

Large Language Models \(LLMs\) are increasingly used to generate educational content, ranging from tutoring support to full\-length explanatory lessons\[[4](https://arxiv.org/html/2607.13041#bib.bib10),[9](https://arxiv.org/html/2607.13041#bib.bib6),[11](https://arxiv.org/html/2607.13041#bib.bib9)\]\. A particularly emerging task is*lesson generation*: given a structured, sufficiently detailed lesson plan specifying topics, learning objectives, and preferred pedagogical delivery, an automated AI system produces a complete lesson\. However, evaluating the quality of such AI\-generated lessons against human expert standards remains an open challenge\[[11](https://arxiv.org/html/2607.13041#bib.bib9)\], and there exists no standardised benchmark pairs structured lesson plans with corresponding human\-written lessons for systematic evaluation\.

Existing educational datasets such as MathDial\[[9](https://arxiv.org/html/2607.13041#bib.bib6)\], TutorEval\[[3](https://arxiv.org/html/2607.13041#bib.bib11)\], and EduChat\[[4](https://arxiv.org/html/2607.13041#bib.bib10)\]address similar issues\. TheoremExplainBench\[[6](https://arxiv.org/html/2607.13041#bib.bib12)\]provides 240 STEM topic descriptions with difficulty ratings, but reference lessons are not provided\. Open educational resources such as LibreTexts\[[7](https://arxiv.org/html/2607.13041#bib.bib13)\]offer textbook chapters and other high\-quality articles, but they lack structured paired lesson plans\. These datasets do not capture the full plan\-to\-lesson pipeline that an AI generation agent would follow\.

Addressing these issues, this short paper presents the following two key contributions: 1\)LessonBench\-V1: The first open\-source111LessonBench\-V1 Dataset:[https://github\.com/SuienS/lesson\-bench\-v1](https://github.com/SuienS/lesson-bench-v1)benchmark dataset pairing structured lesson plans with expert\-written lessons across 240 STEM topics and four subjects, comprising 647 paired samples from 97 unique educational sources\. The dataset is constructed using a reproducible, multi\-stage data curation pipeline in collecting, cleaning, structuring, and reviewing educational content from web sources\. 2\) A pedagogical research\-grounded approach to reverse\-engineering lesson plans, incorporating Bloom’s Taxonomy\[[1](https://arxiv.org/html/2607.13041#bib.bib1)\], Gagné’s Events\[[5](https://arxiv.org/html/2607.13041#bib.bib3)\], Merrill’s First Principles\[[10](https://arxiv.org/html/2607.13041#bib.bib4)\], and the 5E Model\[[2](https://arxiv.org/html/2607.13041#bib.bib5)\]to extract detailed, structured instructional blueprints from existing lessons\.

## 2Related Work

#### Educational AI Benchmarks\.

Several benchmarks evaluate AI in educational settings, but none cover the full plan\-to\-lesson pipeline, as shown in Table[1](https://arxiv.org/html/2607.13041#S2.T1)\. MathDial\[[9](https://arxiv.org/html/2607.13041#bib.bib6)\]provides 3,000 teacher\-student dialogues for tutoring evaluation, TutorEval\[[3](https://arxiv.org/html/2607.13041#bib.bib11)\]evaluates LLMs for science tutoring, and EduChat\[[4](https://arxiv.org/html/2607.13041#bib.bib10)\]focuses on educational chatbot design\. TheoremExplainBench\[[6](https://arxiv.org/html/2607.13041#bib.bib12)\]provides 240 STEM topics that form the foundation of this work, but contains no reference lessons\.

#### Instructional Design Frameworks\.

The lesson plans in LessonBench\-V1 were constructed by incorporating four established pedagogical frameworks\. Bloom’s revised Taxonomy\[[1](https://arxiv.org/html/2607.13041#bib.bib1)\]provides six cognitive levels for tagging learning objectives\. Gagné’s Nine Events of Instruction\[[5](https://arxiv.org/html/2607.13041#bib.bib3)\]informs the instructional flow, together with Merrill’s First Principles\[[10](https://arxiv.org/html/2607.13041#bib.bib4)\]and the BSCS 5E Model\[[2](https://arxiv.org/html/2607.13041#bib.bib5)\]\. LessonBench lesson plans unify these into six instructional roles: Motivation, Activation, Demonstration, Application, Integration and Assessment, which holistically cover the instructional flow\.

#### LLMs for Educational Content\.

Recent work has explored LLMs for the generation of educational materials\[[11](https://arxiv.org/html/2607.13041#bib.bib9)\]\. However, most evaluations rely on ad hoc human judgement or LLM\-as\-judge approaches\[[13](https://arxiv.org/html/2607.13041#bib.bib14)\]rather than on reproducible, reference\-based benchmarking\. LessonBench\-V1 aims to provide this missing reference dataset to enable systematic and reproducible evaluation\.

Table 1:Comparison with existing educational AI datasets\.

## 3Dataset Construction

### 3\.1Source Data and Topic Selection

Educational topics chosen for this study were drawn from TheoremExplainBench\[[6](https://arxiv.org/html/2607.13041#bib.bib12)\], which includes 240 topics across Mathematics, Physics, Chemistry, and Computer Science, categorised into three difficulty levels: Easy, Medium, Hard\. For each topic, LessonBench\-V1 contains 1\-5 human\-written lessons from various educational sources, totalling 647 lessons\. Each lesson is accompanied by a corresponding lesson plan\. Source selection prioritised reliable, openly accessible, and pedagogically structured content\. The 97 source websites span open textbook platforms such as LibreTexts \(47%\), which include resources from a variety of high\-quality sources, as well as educational portals such as GeeksForGeeks \(9%\), Byjus \(8%\), and Brilliant\.org \(5%\)\.

### 3\.2Multi\-Stage Curation Pipeline

![Refer to caption](https://arxiv.org/html/2607.13041v1/x1.png)Figure 1:LessonBench construction pipeline1. 1\.Web Data Collection:Relevant sources, including related images, are collected and downloaded\.
2. 2\.HTML\-to\-Markdown Conversion:The downloaded HTML content is converted to Markdown using thehtml\-to\-markdownPython library\.
3. 3\.LLM\-based Markdown Cleanup:The resulting Markdown often contains unwanted web\-scraping artefacts, such as navigational text, broken LaTeX equations, and unrelated web\-page content\. A carefully designed prompt222MD Fixing system prompt:[https://github\.com/SuienS/lesson\-bench\-v1/blob/main/prompts/MD\_Cleanup\_Prompt\.md](https://github.com/SuienS/lesson-bench-v1/blob/main/prompts/MD_Cleanup_Prompt.md)is used with Qwen 3\.6 35B A3B to fix formatting issues across ten identified categories while preserving the relevant educational content verbatim, with no rewriting, summarising, or additional content\.
4. 4\.Lesson Plan Reverse Engineering:A pedagogically research\-backed prompt333Lesson Plan Reverse Engineering system prompt:[https://github\.com/SuienS/lesson\-bench\-v1/blob/main/prompts/LessonPlan\_Creation\_prompt\.md](https://github.com/SuienS/lesson-bench-v1/blob/main/prompts/LessonPlan_Creation_prompt.md)was used with GPT\-4\.1, accessed via the GitHub Copilot API, to reverse engineer a structured lesson from each cleaned lesson\. The prompt is designed to holistically ‘reverse engineer’ the instructional blueprint without summarising content\. The resulting plan captures six sections: Metadata, Learning Objectives, Key Concepts and Essential Questions, Instructional Flow, Practice and Application Strategy, and Pedagogical Strategy\. Each lesson plan444Sample Lesson Plan:[https://github\.com/SuienS/lesson\-bench\-v1/blob/main/LessonBench/data/topic\_000/lesson\_000\_0/lesson\_plan/lesson\_plan\.md](https://github.com/SuienS/lesson-bench-v1/blob/main/LessonBench/data/topic_000/lesson_000_0/lesson_plan/lesson_plan.md)follows a structured template grounded in four pedagogical frameworks\. Learning objectives use Bloom’s revised taxonomy verbs\[[1](https://arxiv.org/html/2607.13041#bib.bib1)\]and are tagged to one of the six cognitive levels\. Instructional flow phases are annotated with roles derived from cross\-mapping Gagné’s Events\[[5](https://arxiv.org/html/2607.13041#bib.bib3)\], Merrill’s Principles\[[10](https://arxiv.org/html/2607.13041#bib.bib4)\], and the 5E model\[[2](https://arxiv.org/html/2607.13041#bib.bib5)\]into six refined roles:*Motivation*,*Activation*,*Demonstration*,*Application*,*Integration*, and*Assessment*\.
5. 5\.Human Review:The cleaned lesson and the created lesson plan are then reviewed to identify any further formatting or structural issues, which are then fixed manually\.

## 4Dataset Analysis

Table[2](https://arxiv.org/html/2607.13041#S4.T2)summarises key statistics for LessonBench\-V1\. The dataset comprises 647 pairs of lessons and lesson plans across 240 topics\. Human\-written lessons average 1,744 words, whereas lesson plans are substantially shorter, averaging 938 words\.

Table 2:LessonBench dataset summary statistics\.#### Bloom’s Taxonomy Distribution\.

Across all 647 lessons, 3,620 learning objectives were extracted, each assigned a Bloom’s taxonomy level, as shown in Figure[2](https://arxiv.org/html/2607.13041#S4.F2)\.

![Refer to caption](https://arxiv.org/html/2607.13041v1/x2.png)Figure 2:The distribution of Bloom’s Taxonomy level \(left\) and Instructional Roles \(right\) across subjects
#### Instructional Role Distribution\.

The distribution of the 5,623 parsed instructional role annotations is shown in Figure[2](https://arxiv.org/html/2607.13041#S4.F2)\. Subject\-specific patterns are evident in the pedagogical metadata\. Mathematics lessons have the highest proportion of Apply\-level objectives, reflecting an emphasis on procedural problem\-solving\. Physics lessons show the highest proportion of ‘Understand’ objectives, consistent with conceptual explanation being central to physics pedagogy\. Computer Science lessons have the most balanced Bloom’s distribution, with notable Analyse\-level content focused on algorithm analysis\.

## 5Benchmarking Framework

LessonBench\-V1 supports the systematic evaluation of AI lesson\-generation systems that take a lesson plan and produce a complete lesson from it\. Given a lesson planPiP\_\{i\}from the dataset, an AI system generates a lessonL^i\\hat\{L\}\_\{i\}, which is then compared with the corresponding human\-written referenceLiL\_\{i\}\. This mirrors the real\-world use case in which a curriculum designer writes a plan and an AI agent produces the lesson\.

LessonBench can be used across three complementary evaluation dimensions: 1\)*Semantic similarity*via BERTScore\[[12](https://arxiv.org/html/2607.13041#bib.bib15)\], measuring contextual overlap; 2\)*Structural similarity*via ROUGE\-L\[[8](https://arxiv.org/html/2607.13041#bib.bib16)\], capturing the longest common subsequence overlap; and 3\)*Pedagogical alignment*by reverse\-engineering a lesson plan from the generated lesson and comparing Bloom’s level coverage and role distributions with the original plan\. Furthermore, the datasets’ metadata, derived from TheoremExplainBench, enables systematic analysis of these dimensions across aspects such as subject and difficulty\.

## 6Conclusion and Future Work

This short paper presents LessonBench\-V1, a benchmark dataset that pairs structured lesson plans with expert\-written lessons across 240 topics spanning four STEM subjects, resulting in 647 paired samples\. The dataset is collected and constructed using a reproducible five\-stage pipeline\. Our initial analysis shows meaningful patterns across subjects and difficulty levels in the collected lessons, affirming its validity\.

#### Limitations and Future Work\.

The lesson plans for the dataset were generated using GPT 4\.1\. Although the generated plans were reviewed prior to the final compilation of the dataset, we plan to conduct systematic experiments to statistically verify their completeness and accuracy against an expert\-written lesson plan baseline\. The dataset is currently drawn only from English\-language sources, and future versions will include lessons from multiple languages\. Furthermore, this dataset is an initial step in a larger research effort to develop a full end\-to\-end lesson\-generation AI agent, which is planned to be evaluated on it\.

## References

- \[1\]L\. W\. Anderson and D\. R\. Krathwohl\(2001\)A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives\.Longman\.Cited by:[§1](https://arxiv.org/html/2607.13041#S1.p3.1),[§2](https://arxiv.org/html/2607.13041#S2.SS0.SSS0.Px2.p1.1),[item 4](https://arxiv.org/html/2607.13041#S3.I1.i4.p1.1)\.
- \[2\]R\. W\. Bybee\(2015\)The BSCS 5E Instructional model: Creating Teachable Moments\.National Science Teachers Association\.Cited by:[§1](https://arxiv.org/html/2607.13041#S1.p3.1),[§2](https://arxiv.org/html/2607.13041#S2.SS0.SSS0.Px2.p1.1),[item 4](https://arxiv.org/html/2607.13041#S3.I1.i4.p1.1)\.
- \[3\]A\. Chevalier, J\. Geng,et al\.\(2024\)Language models as science tutors\.arXiv preprint arXiv:2402\.11111\.Cited by:[§1](https://arxiv.org/html/2607.13041#S1.p2.1),[§2](https://arxiv.org/html/2607.13041#S2.SS0.SSS0.Px1.p1.1),[Table 1](https://arxiv.org/html/2607.13041#S2.T1.1.4.3.1)\.
- \[4\]Y\. Dan, Z\. Lei, Y\. Gu,et al\.\(2025\)EduChat: A Large Language Model\-Based Conversational Agent for Intelligent Education\.InChina Conference on Knowledge Graph and Semantic Computing and International Joint Conference on Knowledge Graphs,Vol\.2229,pp\. 297–308\(en\)\.External Links:ISBN 978\-981\-96\-1808\-8 978\-981\-96\-1809\-5,[Document](https://dx.doi.org/10.1007/978-981-96-1809-5%5F22)Cited by:[§1](https://arxiv.org/html/2607.13041#S1.p1.1),[§1](https://arxiv.org/html/2607.13041#S1.p2.1),[§2](https://arxiv.org/html/2607.13041#S2.SS0.SSS0.Px1.p1.1)\.
- \[5\]R\. M\. Gagné\(1985\)The Conditions of Learning and Theory of Instruction\.4th edition,Holt, Rinehart and Winston\.Cited by:[§1](https://arxiv.org/html/2607.13041#S1.p3.1),[§2](https://arxiv.org/html/2607.13041#S2.SS0.SSS0.Px2.p1.1),[item 4](https://arxiv.org/html/2607.13041#S3.I1.i4.p1.1)\.
- \[6\]M\. Ku, C\. H\. Chong, J\. Leung,et al\.\(2025\)TheoremExplainAgent: Towards Video\-based Multimodal Explanations for LLM Theorem Understanding\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Vienna, Austria,pp\. 6663–6684\(en\)\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.332)Cited by:[§1](https://arxiv.org/html/2607.13041#S1.p2.1),[§2](https://arxiv.org/html/2607.13041#S2.SS0.SSS0.Px1.p1.1),[Table 1](https://arxiv.org/html/2607.13041#S2.T1.1.2.1.1),[§3\.1](https://arxiv.org/html/2607.13041#S3.SS1.p1.1)\.
- \[7\]LibreTexts\(2026\)LibreTexts \- Free the Textbook\.Libretexts\.org\.External Links:[Link](https://libretexts.org/)Cited by:[§1](https://arxiv.org/html/2607.13041#S1.p2.1),[Table 1](https://arxiv.org/html/2607.13041#S2.T1.1.5.4.1)\.
- \[8\]C\. Lin\(2004\)Rouge: a package for automatic evaluation of summaries\.InText summarization branches out,pp\. 74–81\.Cited by:[§5](https://arxiv.org/html/2607.13041#S5.p2.1)\.
- \[9\]J\. Macina, N\. Daheim,et al\.\(2023\)Mathdial: A dialogue tutoring dataset with rich pedagogical properties grounded in math reasoning problems\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 5602–5621\.Cited by:[§1](https://arxiv.org/html/2607.13041#S1.p1.1),[§1](https://arxiv.org/html/2607.13041#S1.p2.1),[§2](https://arxiv.org/html/2607.13041#S2.SS0.SSS0.Px1.p1.1),[Table 1](https://arxiv.org/html/2607.13041#S2.T1.1.3.2.1)\.
- \[10\]M\. D\. Merrill\(2002\-09\)First Principles of Instruction\.Educational Technology Research and Development50,pp\. 43–59\.External Links:[Document](https://dx.doi.org/10.1007/bf02505024)Cited by:[§1](https://arxiv.org/html/2607.13041#S1.p3.1),[§2](https://arxiv.org/html/2607.13041#S2.SS0.SSS0.Px2.p1.1),[item 4](https://arxiv.org/html/2607.13041#S3.I1.i4.p1.1)\.
- \[11\]R\. S\. R\. Silva, A\. Lotfi, I\. K\. Ihianle, G\. Shahtahmassebi, and J\. J\. Bird\(2025\-12\)Generative Artificial Intelligence and the Future of Education: A Systematic Review of Trends, Trustworthiness and Technological Evaluation\.In Review\.External Links:[Document](https://dx.doi.org/10.21203/rs.3.rs-8248369/v1)Cited by:[§1](https://arxiv.org/html/2607.13041#S1.p1.1),[§2](https://arxiv.org/html/2607.13041#S2.SS0.SSS0.Px3.p1.1)\.
- \[12\]T\. Zhang, V\. Kishore,et al\.\(2019\)Bertscore: evaluating text generation with bert\.arXiv preprint arXiv:1904\.09675\.Cited by:[§5](https://arxiv.org/html/2607.13041#S5.p2.1)\.
- \[13\]L\. Zheng, W\. Chiang, Y\. Sheng,et al\.\(2023\)Judging LLM\-as\-a\-judge with MT\-bench and Chatbot Arena\.InProceedings of the 37th International Conference on Neural Information Processing Systems,NIPS ’23,Red Hook, NY, USA\.Cited by:[§2](https://arxiv.org/html/2607.13041#S2.SS0.SSS0.Px3.p1.1)\.