Beyond Direct Answering: Aligning Educational LLMs as Socratic Guides via Heuristic Reinforcement Learning

# Aligning Educational LLMs as Socratic Guides via Heuristic Reinforcement Learning
Source: [https://arxiv.org/html/2607.22996](https://arxiv.org/html/2607.22996)
11institutetext:East China Normal University, Shanghai, China22institutetext:Shanghai Chuangjie Situo Information Technology Co\., Ltd\., Shanghai, China33institutetext:Shanghai Normal University, Shanghai, China44institutetext:China Association of STEM Education, China
\(✉\)44email:businesssituo@163\.com; stem2603@126\.com###### Abstract

Large language models \(LLMs\) deployed in educational settings often behave as*direct answerers*: they disclose target concepts in the opening turn instead of guiding students through progressive inquiry, as Socratic pedagogy prescribes\. We presentHeuristicEdu, a two\-phase pipeline that aligns Qwen2\.5\-7B toward Socratic tutoring via supervised warm\-up and Group Relative Policy Optimization \(GRPO\)\. Training usesSocraticEdu, 797 multi\-turn Chinese children’s science dialogues reconstructed from a live platform, with a heuristic reward over cognitive depth \(RcogR\_\{\\text\{cog\}\}\), curiosity engagement \(RengR\_\{\\text\{eng\}\}\), and directness \(RdirR\_\{\\text\{dir\}\}\), together with a𝒦query\\mathcal\{K\}\_\{\\text\{query\}\}correction for student\-introduced terms\. We introduce Scaffolding Effectiveness \(SE\) and Conversation Depth \(CD\) to evaluate outcomes beyond surface fluency\. On 30 held\-out questions, the best GRPO variant improvesSEfrom 30\.0% to 63\.3% and lowers keyword leakage from 30\.0% to 13\.3%\. Notably, this best variant omits the directness penalty during optimization, suggesting that explicit anti\-leakage terms can conflict with gradient\-based behavioral alignment\. An unaligned Qwen\-72B baseline reaches 0%SEand 96\.7% leakage, showing that scale alone does not induce Socratic behavior\.

## 1Introduction

Educational psychology has long distinguished between*telling*and*teaching*: a student who is handed the answer to “Why is the sky blue?” acquires a fact, whereas one led to discover Rayleigh scattering through progressive questioning develops durable conceptual understanding\[[18](https://arxiv.org/html/2607.22996#bib.bib18),[3](https://arxiv.org/html/2607.22996#bib.bib3)\]\. Despite this, current LLMs deployed in K\-12 platforms default to comprehensive, immediate answers\. An analysis of 10,512 student queries from a live science platform shows that every tested model—from a 7B instruction\-tuned baseline to Qwen\-72B—leaks core scientific terminology in the opening response at rates above 96%, irrespective of model size\. ThisDirect Answererfailure mode is therefore behavioral rather than capability\-driven, and it cannot be resolved by scaling alone\.

To address this failure mode, we presentHeuristicEdu, a pipeline for aligning Qwen2\.5\-7B\-Instruct toward Socratic tutoring \(Figure[1](https://arxiv.org/html/2607.22996#S1.F1)\)\. We reconstruct platform logs intoSocraticEdu, a 797\-trajectory corpus spanning seven elementary\-science domains, then apply SFT followed by GRPO with a heuristic reward over cognitive lift \(RcogR\_\{\\text\{cog\}\}\), curiosity engagement \(RengR\_\{\\text\{eng\}\}\), and directness \(RdirR\_\{\\text\{dir\}\}\), plus a𝒦query\\mathcal\{K\}\_\{\\text\{query\}\}correction for student\-introduced terms\. We evaluate pedagogical quality with outcome\-oriented metrics in a four\-turn LLM\-simulated tutoring loop, scoring whether the child articulates the target concept without tutor\-side keyword leakage\.

Our main contributions are summarized as follows:

- •Dataset\.We introduceSocraticEdu, a 797\-trajectory Socratic tutoring corpus reconstructed from live student–platform interactions and style\-controlled synthesis across seven elementary\-science domains\.
- •Methodology\.We propose an SFT\-then\-GRPO alignment pipeline with heuristic rewards for cognitive depth, curiosity engagement, and controlled answer disclosure, including a𝒦query\\mathcal\{K\}\_\{\\text\{query\}\}correction for student\-introduced terms\.
- •Evaluation and Findings\.We introduceSEandCDfor closed\-loop Socratic evaluation\. On 30 held\-out questions, PT\-GRPO withoutRdirR\_\{\\text\{dir\}\}reachesSE=0\.633\{\}=0\.633with 13\.3% leakage, outperforming SFT\-only, prompt\-engineering, and unaligned Qwen\-72B baselines;RdirR\_\{\\text\{dir\}\}helps as an inference\-time constraint but hurts when used as a GRPO penalty\.

Raw Data10,512 questions1,172 follow\-upsStep 1Data ReconstructionStep 2GRPO AlignmentStep 3Input PipelineStep 4EvaluationIntent Filter \(F1=0\.98\)High\-value ExtractionTrajectory RebuildRcogR\_\{\\text\{cog\}\}\+RengR\_\{\\text\{eng\}\}\(engagement\-led\)SFT Warm\-up→\\toGRPO𝒦query\\mathcal\{K\}\_\{\\text\{query\}\}correctionSpell CorrectionIntent FilterHistory ConcatSimulator ValidationSE / CD MetricsAblation StudySFT corpus: 797 traj\.\(395 real \+ 402 synth\.\)Qwen2\.5\-7B\(GRPO\-aligned\)GRPO w/oRd​i​rR\_\{dir\}: SE = 0\.633, Leakage = 13\.3%GRPO AlignmentPedagogical Evaluation1234

Figure 1:Overview of theHeuristicEdupipeline: data reconstruction, GRPO alignment, inference\-time preprocessing, and pedagogical evaluation withSEandCD\.
## 2Related Work

##### LLMs in Education\.

Intelligent tutoring systems built on dialogue have a long research history\[[4](https://arxiv.org/html/2607.22996#bib.bib4)\]\. Recent LLM\-based approaches demonstrate strong pedagogical capabilities in mathematics and reading comprehension\[[9](https://arxiv.org/html/2607.22996#bib.bib9),[17](https://arxiv.org/html/2607.22996#bib.bib17),[19](https://arxiv.org/html/2607.22996#bib.bib19),[1](https://arxiv.org/html/2607.22996#bib.bib1)\], yet these systems predominantly operate in an answer\-delivery mode rather than adopting the guided\-inquiry strategy advocated by Socratic pedagogy\. The specific problem of aligning an LLM’s*response behavior*toward Socratic scaffolding—rather than improving answer correctness—has received little formal treatment\.

##### Alignment and Pedagogical Reward Signals\.

RLHF\[[11](https://arxiv.org/html/2607.22996#bib.bib11)\]and process reward models\[[6](https://arxiv.org/html/2607.22996#bib.bib6)\]align LLM behavior with preference or step\-level feedback, but collecting pedagogical preference pairs is costly\. We therefore adopt GRPO\[[16](https://arxiv.org/html/2607.22996#bib.bib16)\], which normalizes rewards within sampled groups and avoids a learned critic\. For the cognitive component of the reward, Bloom’s taxonomy\[[3](https://arxiv.org/html/2607.22996#bib.bib3),[2](https://arxiv.org/html/2607.22996#bib.bib2)\]provides a theory\-grounded scale of educational demand, which we operationalize with an LLM judge\.

##### Beyond Surface\-Level Evaluation\.

BLEU\[[12](https://arxiv.org/html/2607.22996#bib.bib12)\]and ROUGE\[[7](https://arxiv.org/html/2607.22996#bib.bib7)\]correlate poorly with pedagogical quality in open\-domain tutoring, where multiple valid responses exist for every student query\. Reference\-free evaluation methods\[[10](https://arxiv.org/html/2607.22996#bib.bib10)\]are better suited in principle, but they still mainly judge response quality rather than whether a tutoring dialogue changes the learner’s state\. Recent educational\-dialogue work therefore argues for broader benchmarks that assess adaptive guidance, cognitive growth, learner autonomy, and conceptual progression\[[5](https://arxiv.org/html/2607.22996#bib.bib5),[8](https://arxiv.org/html/2607.22996#bib.bib8)\]\. Outside education, progression\-aware dialogue modeling similarly evaluates whether a conversation moves toward a desired outcome\[[14](https://arxiv.org/html/2607.22996#bib.bib14)\]\. OurSEandCDmetrics build on this direction but instantiate it for Socratic tutoring: they ask whether the student reaches the target concept through self\-expression and whether the dialogue continues to advance semantically rather than repeat surface wording\.

## 3TheSocraticEduDataset

### 3\.1Data Source

SocraticEduis constructed from interaction logs ofDr\. Curious, a deployed educational Q&A platform serving Chinese primary\-school students \(ages 8–12\) across 14 schools\. Students submit free\-form science questions in conversational Chinese; the platform returns an initial AI answer and supports follow\-ups, yielding both single\-turn queries and multi\-turn chains\. Each record contains the question text, timestamps, anonymized student/school identifiers, view counts, and follow\-up utterances\. No personally identifiable information is released withSocraticEdu\. Table[1](https://arxiv.org/html/2607.22996#S3.T1)summarizes the corpus statistics after each curation stage\.

Table 1:Corpus statistics forSocraticEdu\.
### 3\.2Scientific Intent Filtering

Because the platform accepts arbitrary free\-form questions, we first train a lightweight*scientific\-intent*filter\. Queries are represented with character\-level TF–IDF features and classified by logistic regression into*scientific*vs\.*non\-scientific*labels\. Five\-fold cross\-validation gives macro\-F1=0\.9805±0\.0022=0\.9805\\pm 0\.0022; applying the classifier removes 1,601 off\-topic items and retains 8,911 science\-oriented queries for clustering and trajectory reconstruction\.

### 3\.3Clustering and Trajectory Construction

We embed filtered questions with multilingual Sentence\-BERT\[[13](https://arxiv.org/html/2607.22996#bib.bib13)\]and apply K\-Means clustering \(K=15K\{=\}15\) to organize heterogeneous topics\. Clusters with above\-median view count and inquiry\-pattern density are treated as*high\-engagement*themes, from which we select 134 seeds with at least 30 views and sustained follow\-up behavior\. For seeds with observed multi\-turn logs, we reconstruct 395*real*trajectories through dialogue replay, using logged student follow\-ups when available\. To cover seeds without long real chains, we further generate 402*synthetic*trajectories under guiding, exploratory, and analogical tutoring styles\. The resulting 797\-trajectory corpus spans seven semantic domains and provides the supervised warm\-up data before GRPO\.

## 4Heuristic Reward Function

We define the composite training reward as:

Rtotal=α​Rcog\+β​Reng−γ​RdirR\_\{\\text\{total\}\}=\\alpha\\,R\_\{\\text\{cog\}\}\+\\beta\\,R\_\{\\text\{eng\}\}\-\\gamma\\,R\_\{\\text\{dir\}\}\(1\)whereα,β,γ≥0\\alpha,\\beta,\\gamma\\geq 0control the relative weight of each component\. Section[7](https://arxiv.org/html/2607.22996#S7)shows thatγ=0\\gamma=0is optimal in practice\.

##### RcogR\_\{\\text\{cog\}\}: Cognitive Depth\.

RcogR\_\{\\text\{cog\}\}encourages the tutor to raise the cognitive demand of the exchange rather than merely confirm facts\. A Qwen\-turbo judge independently assigns Bloom’s Taxonomy levelsb∈\{1,…,6\}b\\in\\\{1,\\ldots,6\\\}\(remember→\\tocreate\) to the preceding student queryqt−1q\_\{t\-1\}and the current model responsertr\_\{t\}\. The reward measures the normalized cognitive*lift*between turns:

Rcog=bresponse−bquery5∈\[−1,1\]R\_\{\\text\{cog\}\}=\\frac\{b\_\{\\text\{response\}\}\-b\_\{\\text\{query\}\}\}\{5\}\\in\[\-1,\\,1\]\(2\)A positive value indicates that the response moves the student toward higher\-order thinking\.

##### RengR\_\{\\text\{eng\}\}: Curiosity Engagement\.

Socratic tutoring must sustain curiosity even when answers are withheld\. We therefore score each tutor turnrtr\_\{t\}with an Engagement Proxy Model \(EPM\) that detects surface cues correlated with child follow\-up in our logs, including rhetorical questions, analogical framing, extreme\-value language, and open\-ended hypothesis starters\. Formally,

Reng=EPM​\(rt\)∈\[0,1\]R\_\{\\text\{eng\}\}=\\textsc\{EPM\}\(r\_\{t\}\)\\in\[0,\\,1\]\(3\)

##### RdirR\_\{\\text\{dir\}\}: Directness Penalty with𝒦query\\mathcal\{K\}\_\{\\text\{query\}\}Correction\.

The directness term discourages premature disclosure of the target concept\. For each seed questionii, we curate a keyword set𝒦i\\mathcal\{K\}\_\{i\}containing core scientific terms that a Socratic tutor should help the student discover rather than state outright \(e\.g\.,*photosynthesis*for a plant\-growth question\)\. Let𝒦query\\mathcal\{K\}\_\{\\text\{query\}\}denote terms in𝒦i\\mathcal\{K\}\_\{i\}already present in the student’s preceding utteranceqt−1q\_\{t\-1\}\. We penalize only*novel*leakage—terms inrtr\_\{t\}that were not introduced by the student:

Rdir=\|\{k∈𝒦i∩rt\}∖𝒦query\|\|𝒦i\|R\_\{\\text\{dir\}\}=\\frac\{\|\\\{k\\in\\mathcal\{K\}\_\{i\}\\cap r\_\{t\}\\\}\\setminus\\mathcal\{K\}\_\{\\text\{query\}\}\|\}\{\|\\mathcal\{K\}\_\{i\}\|\}\(4\)Without this correction, a response that uses “chlorophyll” after a student has asked “Why does photosynthesis need chlorophyll?” would be erroneously penalized\.

Training uses this reward in a two\-stage pipeline\. We first warm up Qwen2\.5\-7B\-Instruct with supervised fine\-tuning on the 797\-trajectorySocraticEducorpus to establish basic Socratic response patterns\. Starting from the SFT checkpoint, the second stage applies GRPO: for each prompt, multiple tutor continuations are sampled, scored byRtotalR\_\{\\text\{total\}\}, normalized within the group to obtain relative advantages, and used for policy updates against a frozen reference model\.

## 5Evaluation Metrics

### 5\.1Scaffolding Effectiveness \(SE\)

Following outcome\-oriented tutoring evaluation\[[5](https://arxiv.org/html/2607.22996#bib.bib5),[8](https://arxiv.org/html/2607.22996#bib.bib8)\],SEmeasures the proportion of test questions where the tutor guides a simulated student to self\-express the core concept*without*uttering any term in𝒦i\\mathcal\{K\}\_\{i\}:

SE=1N∑i=1N𝟙\[c^i=ci∗∧∄t≤T:k∈rt,k∈𝒦i\]\\textsc\{SE\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\mathbb\{1\}\\\!\\left\[\\hat\{c\}\_\{i\}=c\_\{i\}^\{\*\}\\;\\land\\;\\nexists\\,t\\leq T:k\\in r\_\{t\},\\;k\\in\\mathcal\{K\}\_\{i\}\\right\]\(5\)The tutor and simulator interact for up toT=4T\{=\}4rounds; a separate judge then checks final concept match and tutor\-side keyword leakage\.

### 5\.2Conversation Depth \(CD\)

Following progression\-aware dialogue evaluation\[[14](https://arxiv.org/html/2607.22996#bib.bib14)\],CDmeasures the fraction of consecutive guide–student turn pairs\(rt,qt\+1\)\(r\_\{t\},q\_\{t\+1\}\)whose Sentence\-BERT embeddings\[[13](https://arxiv.org/html/2607.22996#bib.bib13)\]have cosine similarity<0\.70<0\.70, indicating semantic progression\. We reportCDwithSEand leakage because novelty alone does not guarantee successful scaffolding\.

### 5\.3Simulator Validity

BecauseSEandCDare simulation\-based, we validate the student proxy against held\-out real follow\-ups\[[15](https://arxiv.org/html/2607.22996#bib.bib15)\]\. Across 30 questions with recorded follow\-ups, each simulated utterance is matched to the nearest real child message from the same seed by SBERT similarity; the mean score is0\.449±0\.2200\.449\\pm 0\.220, indicating on\-topic but non\-duplicate behavior\.

## 6Experiments

We evaluate whether GRPO improves scaffolding beyond SFT and prompting \(RQ1\), which reward components driveSEand leakage \(RQ2\), and whether scale can substitute for task\-specific alignment \(RQ3\)\.

### 6\.1Experimental Setup

##### Compared Systems\.

We compare a*parameter\-training \(PT\) track*on a fixed 7B backbone with a*prompt\-engineering \(PE\) reference track*:

- •PT\-Base: Qwen2\.5\-7B\-Instruct with a Socratic system prompt; no gradient updates\.
- •PT\-SFT: SFT on 797 trajectories only \(Phase 1\)\.
- •PT\-GRPO Full: Phase 1\+\+Phase 2 withα=β=0\.4\\alpha\{=\}\\beta\{=\}0\.4,γ=0\.2\\gamma\{=\}0\.2\.
- •PT\-GRPO w/oRdirR\_\{\\text\{dir\}\}: Phase 1\+\+Phase 2 withα=β=0\.4\\alpha\{=\}\\beta\{=\}0\.4,γ=0\\gamma\{=\}0\.

We also includeQwen\-72B\-Directto test scale andPE\-GRPO\(Qwen\-plus with a Socratic prompt and term constraints\) as a prompting reference\.

##### Implementation Details\.

Table[2](https://arxiv.org/html/2607.22996#S6.T2)summarizes shared settings\. PT variants use the same initialization, chat template, andSocraticEducorpus; only Phase 2 reward weights differ\. The simulator and judge are fixed across systems so score differences reflect tutor behavior rather than evaluation noise\.

Table 2:Training and evaluation hyperparameters\.
##### Test Set\.

We use 30 held\-out seed questions stratified across the seven clustered science domains\. Seeds are excluded from training trajectories and synthetic replay prompts, and each item has a fixed gold conceptci∗c\_\{i\}^\{\*\}plus keyword set𝒦i\\mathcal\{K\}\_\{i\}for leakage detection\.

##### Metrics and Model Selection\.

We reportSE, leakage,CD, andJ=SE−0\.5×LeakRateJ=\\textsc\{SE\}\-0\.5\\times\\text\{LeakRate\}, which penalizes success achieved by keyword leakage\. BecauseN=30N\{=\}30gives per\-runSEvariance of roughly±0\.10\\pm 0\.10, margins below 0\.10 are treated as indicative; a reward\-weight sensitivity check is summarized in Appendix[0\.A](https://arxiv.org/html/2607.22996#Pt0.A1)\.

## 7Results and Analysis

### 7\.1Main Results \(RQ1\)

Table[3](https://arxiv.org/html/2607.22996#S7.T3)compares all systems on the 30\-question test set\. Within the PT track, alignment yields a generally improving pattern: prompt\-only PT\-Base reachesSE=0\.300\{=\}0\.300; SFT alone improves to 0\.500; GRPO withoutRdirR\_\{\\text\{dir\}\}further raisesSEto 0\.633 while cutting leakage from 26\.7% to 13\.3%\. PT\-GRPO w/oRdirR\_\{\\text\{dir\}\}also achieves the highestCD\(0\.722\), suggesting that stronger scaffolding is accompanied by deeper semantic progression rather than simple paraphrase\. Relative to PE\-GRPO, the best 7B PT model is higher inSE\(0\.6330\.633vs\.0\.4000\.400\) andJJ\(0\.5670\.567vs\.0\.3670\.367\), despite using a smaller backbone\. This supports gradient\-based alignment over prompting alone for this behavioral objective\.

Table 3:Results on 30 held\-out questions\.J=SE−0\.5×LeakRateJ=\\textsc\{SE\}\-0\.5\\times\\text\{LeakRate\}\.CDuses SBERT cosine similarity\.Bold= best PT\-track value\.†\\daggerPE\-GRPO is a prompt\-engineering reference, not a direct PT competitor\.CategorySystemSE↑\\uparrowLeakRate↓\\downarrowCD↑\\uparrowJ↑\\uparrowScaleBaselineQwen\-72B\-Direct0\.0000\.967—−\-0\.483PT\-Base \(7B, prompt\)0\.3000\.3000\.6670\.150PT TrackPT\-SFT0\.5000\.2670\.6890\.367PT\-GRPO Full \(α=β=0\.4,γ=0\.2\\alpha\{=\}\\beta\{=\}0\.4,\\,\\gamma\{=\}0\.2\)0\.4330\.2330\.6440\.317PT\-GRPO w/oRdirR\_\{\\text\{dir\}\}\(γ=0\\gamma\{=\}0\)0\.6330\.1330\.7220\.567PE Ref\.†PE\-GRPO0\.4000\.067—0\.367Figure[2](https://arxiv.org/html/2607.22996#S7.F2)visualizes the same PT\-track progression\.SErises from PT\-Base through SFT and peaks at PT\-GRPO w/oRdirR\_\{\\text\{dir\}\}; leakage declines over the same sequence\. The dip at PT\-GRPO Full \(middle bars\) foreshadows the reward\-conflict analysis in Section[7\.2](https://arxiv.org/html/2607.22996#S7.SS2)\.

BaseSFTFullw/oRd​i​rR\_\{dir\}00\.20\.20\.40\.40\.60\.60\.80\.80\.30\.30\.50\.50\.430\.430\.630\.63SE\(a\) Scaffolding Effectiveness \(SE\)

BaseSFTFullw/oRd​i​rR\_\{dir\}00\.10\.10\.20\.20\.30\.30\.40\.40\.50\.50\.30\.30\.270\.270\.230\.230\.130\.13LeakRate\(b\) Keyword Leakage Rate

Figure 2:PT\-trackSEand leakage on the 30\-question test set\. PT\-GRPO w/oRdirR\_\{\\text\{dir\}\}achieves the best performance on both metrics; addingRdirR\_\{\\text\{dir\}\}during GRPO lowersSEdespite reducing leakage\.
### 7\.2Reward Ablation \(RQ2\)

To isolate reward effects \(RQ2\), we compare PT\-track variants with components selectively enabled or disabled\. Table[4](https://arxiv.org/html/2607.22996#S7.T4)summarizes the PT grid\. On the PT track, removingRdirR\_\{\\text\{dir\}\}from optimization strictly improves bothSEand leakage over the full reward\. This contrasts with the PE\-GRPO reference in Table[3](https://arxiv.org/html/2607.22996#S7.T3), where term constraints are applied only at decoding time; the result suggests thatRdirR\_\{\\text\{dir\}\}is risky as a gradient signal even when explicit constraints can reduce inference\-time leakage\.

Table 4:Reward component ablation on the PT track \(n=30n\{=\}30\)\. RemovingRdirR\_\{\\text\{dir\}\}strictly improves both metrics: SE increases by 0\.200 and leakage falls by 10 percentage points relative to the full reward\. “✓” = component active; “✗” = disabled\.
### 7\.3Qualitative Analysis

Table[5](https://arxiv.org/html/2607.22996#S7.T5)illustrates the behavioral shift induced by GRPO alignment\.

Table 5:Qualitative comparison of Base vs\.HeuristicEdu\-GRPO responses\. Underlined terms are core vocabulary from𝒦i\\mathcal\{K\}\_\{i\}\. The GRPO\-aligned model consistently opens with analogies and exploratory questions rather than direct disclosure\.
### 7\.4Discussion of Findings

##### Training trajectory \(RQ1\)\.

The PT track follows an alignment ladder with one important caveat \(Table[3](https://arxiv.org/html/2607.22996#S7.T3), Figure[2](https://arxiv.org/html/2607.22996#S7.F2)\)\. SFT provides the largest single\-step gain inSE\(\+0\.200 over PT\-Base\) by instilling basic questioning patterns fromSocraticEdu, but it still leaks target terms in 26\.7% of dialogues\. GRPO withRcogR\_\{\\text\{cog\}\}andRengR\_\{\\text\{eng\}\}further reshapes turn\-level behavior: the w/o\-RdirR\_\{\\text\{dir\}\}variant improvesSEby another 0\.133 points and halves leakage relative to SFT\. Notably, PT\-GRPO Full*regresses*relative to PT\-SFT onSE, showing that reward design matters as much as the choice to apply RL at all\.

##### Parameter training vs\. prompt engineering \(RQ1\)\.

PT\-GRPO w/oRdirR\_\{\\text\{dir\}\}outperforms PE\-GRPO inSE\(0\.6330\.633vs\.0\.4000\.400\) andJJ\(0\.5670\.567vs\.0\.3670\.367\), while using a 7B backbone rather than an API model\. Gradient\-based alignment directly updates the model’s conditional output distribution, whereas prompt engineering only influences the decoding context at inference time; this structural difference explains the persistent performance gap even with a smaller base model\. PE\-GRPO achieves lower leakage \(6\.7%\) via explicit term constraints, but cannot close theSEgap without parameter updates\.

##### The counterproductive role ofRdirR\_\{\\text\{dir\}\}\(RQ2\)\.

AddingRdirR\_\{\\text\{dir\}\}withγ=0\.2\\gamma\{=\}0\.2reducesSEby 0\.200 and raises leakage by 10 percentage points relative toγ=0\\gamma\{=\}0\. We attribute this to an optimization conflict:RdirR\_\{\\text\{dir\}\}applies a gradient signal that suppresses high\-frequency scientific vocabulary broadly, which in turn reduces the model’s ability to use precise terminology as scaffolding cues\. BecauseRengR\_\{\\text\{eng\}\}rewards engagement through linguistic richness, the two objectives partially cancel whenγ\>0\\gamma\>0, leaving a net degradation\. This interaction is consistent with the 9\-trial Bayesian hyperparameter search \(Appendix[0\.A](https://arxiv.org/html/2607.22996#Pt0.A1)\): all configurations withγ≥0\.20\\gamma\\geq 0\.20andβ≤0\.40\\beta\\leq 0\.40yieldJ<0J<0\. The small grid therefore favors lowγ\\gammaand sufficiently largeβ\\beta, although larger validation sets are needed to draw a sharp boundary\.

##### Scale does not induce Socratic behavior \(RQ3\)\.

Qwen\-72B without task\-specific alignment achieves 0%SEand 96\.7% leakage, matching the direct\-answering failure observed in the unaligned API baseline rather than the prompted PT\-Base system\. Instruction\-following capability, even at large scale, does not produce the behavioral restraint required for Socratic tutoring; explicit optimization against a pedagogical objective is necessary\.

## 8Conclusion

We presentedHeuristicEdu, a training pipeline for redirecting a 7B instruction\-tuned LLM from direct answer delivery toward Socratic guided inquiry\. The approach combines a 797\-trajectory Chinese science dialogue corpus \(SocraticEdu\), a three\-component heuristic reward with a student\-vocabulary correction \(𝒦query\\mathcal\{K\}\_\{\\text\{query\}\}\), and two outcome\-oriented evaluation metrics \(SE,CD\) assessed through an LLM student simulator\.

Three substantive findings emerge from controlled experiments\. First, GRPO parameter training yields stronger Socratic behavior than prompt engineering in our setting: the 7B PT\-GRPO model achieves SE = 0\.633 versus 0\.400 for the PE\-GRPO reference, indicating that gradient\-level alignment can be more effective than inference\-time prompting for this objective\. Second, explicit term constraints can reduce inference\-time leakage, but using the same directness signal as a GRPO reward is harmful in parameter training, where its interaction with the engagement rewardRengR\_\{\\text\{eng\}\}produces a net degradation in bothSEand leakage\. Third, unaligned instruction\-following at 72B scale is no better than a 7B baseline on this task, underscoring that Socratic alignment is a targeted behavioral problem requiring explicit optimization rather than a byproduct of general model capability\.

## References

- \[1\]Achiam, J\., Adler, S\., Agarwal, S\., Ahmad, L\., Akkaya, I\., Aleman, F\.L\., Almeida, D\., Altenschmidt, J\., Altman, S\., Anadkat, S\., et al\.: Gpt\-4 technical report\. arXiv preprint arXiv:2303\.08774 \(2023\)
- \[2\]Anderson, L\.W\., Krathwohl, D\.R\.: A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives: complete edition\. Addison Wesley Longman, Inc\. \(2001\)
- \[3\]Bloom, B\.S\., Engelhart, M\.D\., Furst, E\.J\., Hill, W\.H\., Krathwohl, D\.R\., et al\.: Taxonomy of educational objectives: The classification of educational goals\. Handbook 1: Cognitive domain\. Longman New York \(1956\)
- \[4\]Graesser, A\.C\., Chipman, P\., Haynes, B\.C\., Olney, A\.: Autotutor: An intelligent tutoring system with mixed\-initiative dialogue\. IEEE Transactions on Education48\(4\), 612–618 \(2005\)
- \[5\]Ilkou, E\., Linzbach, S\., Wallat, J\.: Hybrid evaluation of socratic dialogue for teaching\. In: HGAIS@ ISWC \(2024\)
- \[6\]Lightman, H\., Kosaraju, V\., Burda, Y\., Edwards, H\., Baker, B\., Lee, T\., Leike, J\., Schulman, J\., Sutskever, I\., Cobbe, K\.: Let’s verify step by step\. In: International Conference on Learning Representations\. vol\. 2024, pp\. 39578–39601 \(2024\)
- \[7\]Lin, C\.Y\.: Rouge: A package for automatic evaluation of summaries\. In: Text summarization branches out\. pp\. 74–81 \(2004\)
- \[8\]Liu, Y\., Li, C\., Zhang, T\., Wang, M\., Zhu, Q\., Li, J\., Huang, H\.: Discerning minds or generic tutors? evaluating instructional guidance capabilities in socratic llms\. arXiv preprint arXiv:2508\.06583 \(2025\)
- \[9\]Macina, J\., Daheim, N\., Chowdhury, S\., Sinha, T\., Kapur, M\., Gurevych, I\., Sachan, M\.: Mathdial: A dialogue tutoring dataset with rich pedagogical properties grounded in math reasoning problems\. In: Findings of the Association for Computational Linguistics: EMNLP 2023\. pp\. 5602–5621 \(2023\)
- \[10\]Mehri, S\., Eskenazi, M\.: Usr: An unsupervised and reference free evaluation metric for dialog generation\. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics\. pp\. 681–707 \(2020\)
- \[11\]Ouyang, L\., Wu, J\., Jiang, X\., Almeida, D\., Wainwright, C\., Mishkin, P\., Zhang, C\., Agarwal, S\., Slama, K\., Ray, A\., et al\.: Training language models to follow instructions with human feedback\. Advances in neural information processing systems35, 27730–27744 \(2022\)
- \[12\]Papineni, K\., Roukos, S\., Ward, T\., Zhu, W\.J\.: Bleu: a method for automatic evaluation of machine translation\. In: Proceedings of the 40th annual meeting of the Association for Computational Linguistics\. pp\. 311–318 \(2002\)
- \[13\]Reimers, N\., Gurevych, I\.: Sentence\-bert: Sentence embeddings using siamese bert\-networks\. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing \(EMNLP\-IJCNLP\)\. pp\. 3982–3992 \(2019\)
- \[14\]Sanders, A\., Strzalkowski, T\., Si, M\., Chang, A\., Dey, D\., Braasch, J\., Wang, D\.: Towards a progression\-aware autonomous dialogue agent\. In: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies\. pp\. 1194–1212 \(2022\)
- \[15\]Scarlatos, A\., Lee, J\., Woodhead, S\., Lan, A\.: Simulated students in tutoring dialogues: Substance or illusion? arXiv preprint arXiv:2601\.04025 \(2026\)
- \[16\]Shao, Z\., Wang, P\., Zhu, Q\., Xu, R\., Song, J\., Bi, X\., Zhang, H\., Zhang, M\., Li, Y\., Wu, Y\., et al\.: Deepseekmath: Pushing the limits of mathematical reasoning in open language models\. arXiv preprint arXiv:2402\.03300 \(2024\)
- \[17\]Tack, A\., Piech, C\.: The ai teacher test: Measuring the pedagogical ability of blender and gpt\-3 in educational dialogues\. arXiv preprint arXiv:2205\.07540 \(2022\)
- \[18\]Vygotsky, L\.S\.: Mind in society: The development of higher psychological processes, vol\. 86\. Harvard university press \(1978\)
- \[19\]Wei, J\., Tay, Y\., Bommasani, R\., Raffel, C\., Zoph, B\., Borgeaud, S\., Yogatama, D\., Bosma, M\., Zhou, D\., Metzler, D\., et al\.: Emergent abilities of large language models\. arXiv preprint arXiv:2206\.07682 \(2022\)

## Appendix 0\.AReward Weight Sensitivity

We run a nine\-trial Optuna TPE search overα∈\[0\.10,0\.70\]\\alpha\\in\[0\.10,0\.70\],β∈\[0\.20,0\.80\]\\beta\\in\[0\.20,0\.80\], andγ∈\[0\.00,0\.30\]\\gamma\\in\[0\.00,0\.30\], optimizingJ=SE−0\.5×LeakRateJ=\\textsc\{SE\}\-0\.5\\times\\text\{LeakRate\}\. The best searched configuration \(α=0\.45,β=0\.80,γ=0\.10\\alpha\{=\}0\.45,\\beta\{=\}0\.80,\\gamma\{=\}0\.10\) obtainsSE=0\.400\\textsc\{SE\}\{=\}0\.400andJ=0\.217J\{=\}0\.217, below theγ=0\\gamma\{=\}0setting in Table[3](https://arxiv.org/html/2607.22996#S7.T3); all trials withγ≥0\.20\\gamma\\geq 0\.20andβ≤0\.40\\beta\\leq 0\.40yieldJ<0J<0\. Thus the search is used only as sensitivity evidence that high engagement weight is helpful and larger directness penalties are unstable\.

\{credits\}

#### 0\.A\.0\.1Acknowledgements

We thank the partner schools and the Dr\. Curious platform team for data access, and the China Association of STEM Education for research support\.

#### 0\.A\.0\.2\\discintname

The authors have no competing interests to declare\.