Verified, not generated: expert-verified AI study materials and the distribution of learning gains in a university course

arXiv cs.AI Papers
ai-in-education learning-gains generative-ai study-materials expert-verification economics-education

Summary

一项两组差分研究考察了专家审核的AI生成学习材料（播客、FAQ、测验指南）对大学经济学课程的影响，发现其平均提升2.34分，且约四分之三的收益来自成绩最低的五分之一学生，表明专家验证可缩小而非扩大差距。

arXiv:2610.07097v1 Announce Type: new Abstract: Experimental studies of generative AI in education mostly report average effects, yet field evidence shows that AI can narrow attainment gaps or widen them. We argue that the direction depends on the judgement burden, the expertise a learner must supply to screen AI output before learning from it, and that expert verification before release moves this burden from students to an accountable tutor. We test the argument in a two-cohort difference-in-differences design in which one half of a compulsory firstyear university economics course received AI-generated podcasts, FAQs and quiz-based study guides, produced with a source-grounded model and checked by a named graduate teaching assistant (170 students; 340 examination marks). Access was associated with a 2.34-mark advantage on a 50-mark component. The share of marks below the upper-second classification boundary fell by 24.7 percentage points relative to the counterfactual, effects were significant at every threshold from 23 to 31 marks and at none above, and roughly three-quarters of the average originated in the bottom quintile. The threshold estimate is robust to removing the lowest-scoring students from the pre-intervention cohort; the average effect is not. Interviews and feedback from 36 students indicate that the verification label gave students a reason to engage with AI-generated material without ending their scrutiny of it. Evaluations of AI learning resources that report only mean effects cannot detect whether the students the resources are meant to help are the ones who gain.
Original Article
View Cached Full Text

Cached at: 10/08/26, 12:15 PM 

# Verified, not generated: expert-verified AI study materials and the distribution of learning gains in a university course
Source: [https://arxiv.org/abs/2610.07097](https://arxiv.org/abs/2610.07097)
[View PDF](https://arxiv.org/pdf/2610.07097)

> Abstract:Experimental studies of generative AI in education mostly report average effects, yet field evidence shows that AI can narrow attainment gaps or widen them\. We argue that the direction depends on the judgement burden, the expertise a learner must supply to screen AI output before learning from it, and that expert verification before release moves this burden from students to an accountable tutor\. We test the argument in a two\-cohort difference\-in\-differences design in which one half of a compulsory firstyear university economics course received AI\-generated podcasts, FAQs and quiz\-based study guides, produced with a source\-grounded model and checked by a named graduate teaching assistant \(170 students; 340 examination marks\)\. Access was associated with a 2\.34\-mark advantage on a 50\-mark component\. The share of marks below the upper\-second classification boundary fell by 24\.7 percentage points relative to the counterfactual, effects were significant at every threshold from 23 to 31 marks and at none above, and roughly three\-quarters of the average originated in the bottom quintile\. The threshold estimate is robust to removing the lowest\-scoring students from the pre\-intervention cohort; the average effect is not\. Interviews and feedback from 36 students indicate that the verification label gave students a reason to engage with AI\-generated material without ending their scrutiny of it\. Evaluations of AI learning resources that report only mean effects cannot detect whether the students the resources are meant to help are the ones who gain\.

## Submission history

From: Canh Thien Dang \[[view email](https://arxiv.org/show-email/4f080821/2610.07097)\] **\[v1\]**Mon, 5 Oct 2026 13:53:02 UTC \(1,253 KB\)

Similar Articles

@FinanceYF5: AI enables students to score nearly full marks on their assignments, yet their issues are collectively exposed during t…

X AI KOLs Following

AI帮助学生在作业中拿到接近满分，但在线下期末考试中却集体暴露问题。一位斯坦福教授表示，学生用AI完成作业后不再参加答疑，却无法通过线下期末考试，因此院系重新定义助教职责，取消传统答疑，要求学生一对一向助教讲解每份编程作业以证明真正理解代码。