通过表征几何正则化缓解大语言模型自我改进中的能力退化

# Mitigating Capability Degradation in LLM Self-Improvement via Representation Geometry Regularization
Source: [https://arxiv.org/html/2610.12855](https://arxiv.org/html/2610.12855)
Jiaqi TianAffiliation:School of Artificial IntelligenceAffiliation:Jilin UniversityAffiliation:ChinaEmail:[tjq26@mails\.jlu\.edu\.cn](mailto:)Ting Long††thanks:Corresponding author\.Affiliation:School of Artificial IntelligenceAffiliation:Jilin UniversityAffiliation:ChinaEmail:[longting@jlu\.edu\.cn](mailto:)

###### Abstract

On\-Policy Self\-distillation is a self\-improvement paradigm that enables large language models \(LLMs\) to improve their capabilities by leveraging the same model as both teacher and student under different conditioning contexts\. However, we find that OPSD methods can progressively degrade previously acquired capabilities: queries that are correctly responded at earlier training stages may become incorrectly responded as optimization proceeds, even the overall performance continues to improve\. Further analysis reveals that this degradation coincides with a deterioration of the geometry of hidden representations, characterized by reduced intra\-cluster compactness and diminished inter\-cluster separation\. Motivated by this observation, we propose Representation Geometry Regularization \(RGR\), which preserves representation geometry during post\-training by encouraging compactness within cluster and separation across cluster\. RGR can be seamlessly integrated into existing OPSD methods without altering their core optimization procedures\. Extensive experiments on multiple OPSD methods and benchmarks show that RGR generally alleviates capability degradation while retaining capability improvements\. Our code is available in[https://github\.com/supermachinelearner/RGR](https://github.com/supermachinelearner/RGR)

## 1Introduction

![Refer to caption](https://arxiv.org/html/2610.12855v1/intro-1.png)Figure 1:\(a\) The capability degradation ratio of existing methods; \(b\) Although the overall performance improves during training, some problems that were previously responded correctly become incorrectly answered as optimization progresses\.On\-Policy Self\-Distillation \(OPSD\)\([Zhao et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib30);[Hübotter et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib7);[Yang et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib26);[Li et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib10)\), is a self\-improvement paradigm for the post\-training of large language models \(LLMs\)\([Brown et al\., 2020](https://arxiv.org/html/2610.12855#bib.bib2);[Vaswani et al\., 2017](https://arxiv.org/html/2610.12855#bib.bib19);[Yang et al\., 2025](https://arxiv.org/html/2610.12855#bib.bib25);[Zeng et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib27);[Xiao et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib23);[Xu et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib24)\), which enables LLMs to improve their capabilities through self\-knowledge distillation\. Unlike conventional knowledge distillation\([Hinton et al\., 2015](https://arxiv.org/html/2610.12855#bib.bib4);[Sanh et al\., 2019](https://arxiv.org/html/2610.12855#bib.bib16)\), which relies on a separately trained teacher, OPSD uses the same LLM as both teacher and student under different conditioning contexts\. Specifically, the student model generates responses to the original query, while teacher model, which share the same parameters with the student model, conditioned on additional privileged information \(e\.g\., verified reasoning traces\) to provide supervision for these generated responses\.

Although recent studies\([Zhao et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib30);[Hübotter et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib7);[Yang et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib26);[Li et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib10);[Wang et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib21)\)have empirically demonstrated that optimizing the LLM with OPSD can outperform conventional post\-training methods such as SFT\([Wei et al\., 2021](https://arxiv.org/html/2610.12855#bib.bib22)\)and GRPO\([Shao et al\., 2024](https://arxiv.org/html/2610.12855#bib.bib17)\), benefiting from the dense and fine\-grained token\-level supervision of OPSD\. We observe a progressive reasoning capability degradation phenomenon during self\-improvement in existing OPSD methods\. As shown in Figure[1](https://arxiv.org/html/2610.12855#S1.F1)\(b\), a query that is responded correctly at earlier training stages gradually becomes incorrectly responded as training proceeds, even as overall performance continues to improve\.

Furthermore, we examine several representative OPSD methods and find that this phenomenon consistently occurs across these methods, albeit to varying degrees, as it is shown in Figure[1](https://arxiv.org/html/2610.12855#S1.F1)\(a\)\. Such degradation may undermine the stability of self\-improvement\.

To better understand this phenomenon, we conduct further analysis, and the results reveal that this capability degradation is accompanied by a progressive deterioration in the geometry of hidden representations\. When a query is answered correctly at earlier stages of training, the hidden representations of its generated responses exhibit a relatively well\-structured clustering pattern, characterized by larger inter\-cluster distances and clearer cluster boundaries\. As training proceeds and the response to the same query gradually becomes incorrect, the relative distances between clusters shrink and the cluster boundaries become increasingly indistinct\. This observation suggests that preserving the geometric structure of the representation space may be important for maintaining previously acquired capabilities during self\-improvement\.

Based on this finding, we theoretically show that preserving intra\-cluster compactness and inter\-cluster separation in the hidden representation space helps maintain the capabilities that the LLM has already acquired\. Based on this theoretical insight, we develop Representation Geometry Regularization \(RGR\), which constrains the evolution of hidden representations during self\-improvement\. RGR incorporates two geometry\-preserving constraints called intra\-cluster compactness and inter\-cluster separation into the OPSD methods, enabling the model to improve targeted capabilities while mitigating the degradation of previously acquired ones\. Experiments across four existing SD models and three widely used benchmarks show that RGR substantially reduces correct\-to\-incorrect transitions during training while retaining its self\-improvement gains\.

In summary, our contributions are summarized as follows:

- •We uncover an overlooked failure mode, where OPSD methods can degrade previously acquired capabilities, causing previously correct predictions to become incorrect\.
- •We identify representation geometry distortion as a key factor underlying this degradation, and theoretically characterize the importance of hidden representation geometry for prediction stability\.
- •We propose RGR, which incorporates geometry\-preserving constraints into OPSD methods to enable self\-improvement while reducing capability degradation\.

## 2Related works

On\-policy self\-distillation \(OPSD\) enables LLMs to improve their capabilities without requiring an external model as a teacher\. Recently, it has attracted significant attention as a promising paradigm for scalable and efficient LLM self\-improvement\. Based on how self\-teacher signals are constructed and how external feedback is incorporated, we organize existing OPSD methods into privileged\-information\-driven methods and external\-feedback\-driven methods\.The privileged\-information\-driven methods construct self\-teacher signals using additional privileged information that is unavailable to the student policy at inference time\. As a representative method, OPSD\([Zhao et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib30)\)uses the same model to construct self\-teacher signals and optimize the student policy under different conditioning contexts: the self\-teacher signals are constructed from both the question and a verified reasoning trace, whereas the student policy is conditioned only on the question\. The external\-feedback\-driven methods retain the self\-teacher structure while incorporating environmental feedback, verification results, or outcome rewards to construct the supervision for policy updating\. Within this category, feedback can be used to construct distillation targets, select training objectives, or modulate token\-level advantages\. SDPO\([Hübotter et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib7)\)conditions the current model on textual or reward\-derived feedback to form a self\-teacher, whose next\-token predictions serve as distillation targets for the student policy\. SRPO\([Li et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib10)\)routes successful trajectories to reward\-driven GRPO updates and failed trajectories to entropy\-weighted, token\-level self\-distillation against a feedback\-conditioned teacher distribution\. RLSD\([Yang et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib26)\)uses verifiable outcomes to determine the direction of policy updates and token\-level teacher–student discrepancies to modulate their magnitude, integrating self\-distillation into reward\-driven learning\.

## 3Preliminaries

Given a queryqq, an LLMπθ\\pi\_\{\\theta\}typically generates a responsey=\(y1,…,yT\)y=\(y\_\{1\},\\ldots,y\_\{T\}\)according toyt∼πθ\(⋅∣q,y<t\)y\_\{t\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid q,y\_\{<t\}\), wherey<t=\(y1,…,yt−1\)y\_\{<t\}=\(y\_\{1\},\\ldots,y\_\{t\-1\}\)denotes the previously generated tokens\. OPSD methods improve the LLM by using the same model under different conditioning contexts as the teacher and student to conduct knowledge distillation\. Specifically, the student is designed to generate an on\-policy response based only on the original query, like the LLM,i\.e\.,y∼πθ\(⋅∣q\)y\\sim\\pi\_\{\\theta\}\(\\cdot\\mid q\)\. While the teacher is designed to generate the response with additional privileged informationzz, such as a verified reasoning trace,πθ\(⋅∣q,z,y<t\)\\pi\_\{\\theta\}\(\\cdot\\mid q,z,y\_\{<t\}\)\. The OPSD methods optimize the model by minimizing the divergence between the teacher and student token distributions along the student’s trajectory:

ℒOPSD\(θ\)=𝔼q∼𝒟,y∼πθ\(⋅∣q\)\[∑t=1TD\(πθ\(⋅∣q,z,y<t\)∥πθ\(⋅∣q,y<t\)\)\]\.\\mathcal\{L\}\_\{\\mathrm\{OPSD\}\}\(\\theta\)=\\mathbb\{E\}\_\{q\\sim\\mathcal\{D\},\\,y\\sim\\pi\_\{\\theta\}\(\\cdot\\mid q\)\}\\left\[\\sum\_\{t=1\}^\{T\}D\\left\(\\pi\_\{\\theta\}\(\\cdot\\mid q,z,y\_\{<t\}\)\\,\\middle\\\|\\,\\pi\_\{\\theta\}\(\\cdot\\mid q,y\_\{<t\}\)\\right\)\\right\]\.\(1\)
whereD\(⋅\|⋅\)D\(\\cdot\|\\cdot\)denotes a divergence measure between the teacher and student token distributions\.DDcan be instantiated by forward KL divergence\([Kullback and Leibler, 1951](https://arxiv.org/html/2610.12855#bib.bib9)\), reverse KL divergence\([Minka et al\., 2005](https://arxiv.org/html/2610.12855#bib.bib15)\), Jensen–Shannon divergence\([Lin, 1991](https://arxiv.org/html/2610.12855#bib.bib12)\), or other suitable distribution\-matching objectives\.

Since the teacher and student share the underlying model parameters, optimizingℒOPSD\\mathcal\{L\}\_\{\\mathrm\{OPSD\}\}updates the LLM itself\. Consequently, the model iteratively improves its responses by distilling the additional information available to the teacher into its ordinary, query\-conditioned behavior\.

## 4The Geometry of Capability Degradation

### 4\.1Progressive Capability Degradation

![Refer to caption](https://arxiv.org/html/2610.12855v1/opsd_figure.png)Figure 2:Hidden\-state representations and response correctness across training epochs\. The query is answered correctly at epochs 25 and 50, but incorrectly at epoch 75\.As discussed previously, although OPSD methods continuously improve the overall performance in the training, some queries that are correctly responded at earlier training stages become incorrectly responded as training proceeds\. To better understand this phenomenon, for a fixed query, we generate a response and visualize the hidden representations corresponding to the generated response every 25 training epochs\. We observe that changes in the geometry of the hidden representations consistently coincide with changes in response correctness\. As illustrated in Figure\.[2](https://arxiv.org/html/2610.12855#S4.F2), when the model generates a correct response at an earlier training stage \(Epoch 25\), the corresponding hidden representations exhibit a relatively structured geometric organization, with clearly distinguishable clusters and relatively large distances between clusters\. As training proceeds and the response becomes incorrect, the relative distances between clusters gradually decrease, while the boundaries between clusters become increasingly indistinct \(Epoch 75\)\. We hypothesize that as a query transitions from being correctly to incorrectly responded, the reasoning process in the generated response gradually becomes less coherent\. When the query is answered correctly, the reasoning tends to exhibit a clear and consistent logical structure, which is reflected in the hidden representation space as distinct and well\-separated clusters corresponding to different reasoning patterns\. As the reasoning becomes increasingly disorganized, these patterns become less distinguishable, causing the corresponding representations to become more entangled and eventually leading to an incorrect response\. Interestingly, this phenomenon is consistently observed in existing OPSD methods, such as OPSD\([Zhao et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib30)\), SDPO\([Hübotter et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib7)\), RLSD\([Yang et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib26)\), SRPO\([Li et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib10)\)\. These observations suggest a strong connection between the geometry of the hidden representation and the stability of the model’s acquired capabilities during self\-improvement\.

### 4\.2Theoretical Analysis for Capability Degradation

To further substantiate the relationship between hidden representation geometry and response correctness, we conduct a theoretical analysis\.

Suppose the LLM undergoing on\-policy self\-distillation is denoted byπθ\(⋅∣q,y<t\)\\pi\_\{\\theta\}\(\\cdot\\mid q,y\_\{<t\}\), whereπθ∈ΠOPSD\\pi\_\{\\theta\}\\in\\Pi\_\{\\mathrm\{OPSD\}\}, andΠOPSD\\Pi\_\{\\text\{OPSD\}\}denote the policy set trained using OPSD\.

Letπθ∗\(⋅∣q,z,y<t\)\\pi\_\{\\theta\*\}\(\\cdot\\mid q,z,y\_\{<t\}\)denote the optimal teacher distribution conditioned on the privileged informationzz, and letπθ∗\(⋅∣q,y<t\)\\pi\_\{\\theta\*\}\(\\cdot\\mid q,y\_\{<t\}\)denote the corresponding optimal student distribution without access tozz\. We define the excess distillation risk as follows\.

###### Definition 4\.1\(The Excess Distillation Risk\)\.

The excess distillation risk of the student modelπθ\(⋅∣q,y<t\)\\pi\_\{\\theta\}\(\\cdot\\mid q,y\_\{<t\}\)is defined as the difference between its distillation risk with respect to the optimal teacher and that achieved by the optimal student, i\.e\.,

ℰKD\(πθ\)=𝔼q∼𝒟,y∼πθ\(⋅∣q\)\[∑t=1TD\(πθ∗\(⋅∣q,z,y<t\)∥πθ\(⋅∣q,y<t\)\)\\displaystyle\\mathcal\{E\}\_\{\\text\{KD\}\}\(\\pi\_\{\\theta\}\)=\\mathbb\{E\}\_\{q\\sim\\mathcal\{D\},y\\sim\\pi\_\{\\theta\}\(\\cdot\\mid q\)\}\\Bigl\[\\sum\_\{t=1\}^\{T\}D\\left\(\\pi\_\{\\theta^\{\*\}\}\(\\cdot\\mid q,z,y\_\{<t\}\)\\,\\middle\\\|\\,\\pi\_\{\\theta\}\(\\cdot\\mid q,y\_\{<t\}\)\\right\)\(2\)−D\(πθ∗\(⋅∣q,z,y<t\)∥πθ∗\(⋅∣q,y<t\)\)\]\.\\displaystyle\-D\\left\(\\pi\_\{\\theta^\{\*\}\}\(\\cdot\\mid q,z,y\_\{<t\}\)\\,\\middle\\\|\\,\\pi\_\{\\theta^\{\*\}\}\(\\cdot\\mid q,y\_\{<t\}\)\\right\)\\Bigr\]\.

This quantity measures the gap between the distillation risk of the current student and that of the optimal student, with a smaller value indicating a closer approximation to the optimal\.

Letℋ^=\{𝐡^i\}i=1T\\hat\{\\mathcal\{H\}\}=\\\{\\hat\{\\mathbf\{h\}\}\_\{i\}\\\}\_\{i=1\}^\{T\}andℋ∗=\{𝐡i∗\}i=1T\\mathcal\{H\}^\{\*\}=\\\{\\mathbf\{h\}\_\{i\}^\{\*\}\\\}\_\{i=1\}^\{T\}denote the hidden representations produced by the current teacherπθ\(⋅∣q,z,y<t\)\\pi\_\{\\theta\}\(\\cdot\\mid q,z,y\_\{<t\}\)and the optimal teacherπθ∗\(⋅∣q,z,y<t\)\\pi\_\{\\theta^\{\*\}\}\(\\cdot\\mid q,z,y\_\{<t\}\), respectively\. Their midpoint representations are denoted byℋ†=\{𝐡i†\}i=1T\\mathcal\{H\}^\{\\dagger\}=\\\{\\mathbf\{h\}\_\{i\}^\{\\dagger\}\\\}\_\{i=1\}^\{T\}, where𝐡i†=\(𝐡^i\+𝐡i∗\)2\\mathbf\{h\}\_\{i\}^\{\\dagger\}=\\frac\{\(\\hat\{\\mathbf\{h\}\}\_\{i\}\+\\mathbf\{h\}\_\{i\}^\{\*\}\)\}\{2\}\. Suppose thatℋ†\\mathcal\{H\}^\{\\dagger\}is partitioned intoKKclusters, denoted by𝒞†=\{C1†,…,CK†\}\\mathcal\{C\}^\{\\dagger\}=\\\{C\_\{1\}^\{\\dagger\},\\ldots,C\_\{K\}^\{\\dagger\}\\\}\. The center of is clusterCk†C\_\{k\}^\{\\dagger\}isμk†=1\|Ck†\|​∑𝐡i†∈Ck†𝐡i†\\mu\_\{k\}^\{\\dagger\}=\\frac\{1\}\{\|C\_\{k\}^\{\\dagger\}\|\}\\sum\_\{\\mathbf\{h\}\_\{i\}^\{\\dagger\}\\in C\_\{k\}^\{\\dagger\}\}\\mathbf\{h\}\_\{i\}^\{\\dagger\}\. Letℋ=\{𝐡i\}i=1T\\mathcal\{H\}=\\\{\\mathbf\{h\}\_\{i\}\\\}\_\{i=1\}^\{T\}denote the student hidden representations corresponding to the current\-teacher representations\{𝐡^i\}i=1T\\\{\\hat\{\\mathbf\{h\}\}\_\{i\}\\\}\_\{i=1\}^\{T\}\. For each clusterCk†C\_\{k\}^\{\\dagger\}, we define𝒳k=\{𝐡i∣𝐡i†∈Ck†\}\\mathcal\{X\}\_\{k\}=\\\{\\mathbf\{h\}\_\{i\}\\mid\\mathbf\{h\}\_\{i\}^\{\\dagger\}\\in C\_\{k\}^\{\\dagger\}\\\}\. That is, if the midpoint𝐡i†\\mathbf\{h\}\_\{i\}^\{\\dagger\}associated with a current\-teacher representation𝐡^i\\hat\{\\mathbf\{h\}\}\_\{i\}belongs toCk†C\_\{k\}^\{\\dagger\}, its corresponding student representation𝐡i\\mathbf\{h\}\_\{i\}is assigned to𝒳k\\mathcal\{X\}\_\{k\}\. The center of is cluster𝒳k\\mathcal\{X\}\_\{k\}isμk𝒳=1\|𝒳k\|​∑hi∈𝒳k𝐡i\\mu\_\{k\}^\{\\mathcal\{X\}\}=\\frac\{1\}\{\|\\mathcal\{X\}\_\{k\}\|\}\\sum\_\{h\_\{i\}\\in\\mathcal\{X\}\_\{k\}\}\\mathbf\{h\}\_\{i\}\. Letμ=1T​∑iT𝐡i\\mu=\\frac\{1\}\{T\}\\sum\_\{i\}^\{T\}\\mathbf\{h\}\_\{i\}denote the mean of hidden representations\. Based on the definition, we have

###### Theorem 4\.1\(Excess Risk Upper Bound\)\.

ℰKD\(πθ\)≤2​‖M‖2N​\|𝒱\|∑Nsupπθ∈ΠOPSD\|Δ\|\[T​d2\+12​∑\|𝒞†\|∑𝐡i∈𝒳k‖𝐡i−μk𝒳‖22−12​∑\|𝒞†\|\|𝒞k†\|​‖μk𝒳−μ‖22\+\\displaystyle\\mathcal\{E\}\_\{\\text\{KD\}\}\(\\pi\_\{\\theta\}\)\\leq\\frac\{2\|\|M\|\|\_\{2\}\}\{N\|\\mathcal\{V\}\|\}\\sum\_\{N\}\\sup\_\{\\pi\_\{\\theta\}\\in\\Pi\_\{\\mathrm\{OPSD\}\}\}\|\\Delta\|\[\\sqrt\{\\frac\{Td\}\{2\}\+\\frac\{1\}\{2\}\\sum\_\{\|\\mathcal\{C\}^\{\\dagger\}\|\}\\sum\_\{\{\\mathbf\{h\}\_\{i\}\}\\in\\mathcal\{X\}\_\{k\}\}\|\|\\mathbf\{h\}\_\{i\}\-\\mu\_\{k\}^\{\\mathcal\{X\}\}\|\|\_\{2\}^\{2\}\-\\frac\{1\}\{2\}\\sum\_\{\|\\mathcal\{C\}^\{\\dagger\}\|\}\|\\mathcal\{C\}^\{\\dagger\}\_\{k\}\|\|\|\\mu\_\{k\}^\{\\mathcal\{X\}\}\-\\mu\|\|^\{2\}\_\{2\}\}\+\(3\)∑\|𝒞†\|\|𝒞k†\|​‖μk𝒳−μk†‖22\+∑\|𝒞†\|∑𝐡i†∈𝒳k‖𝐡i†−μk†‖22\]\\displaystyle\\sqrt\{\\sum\_\{\|\\mathcal\{C\}^\{\\dagger\}\|\}\|\\mathcal\{C\}\_\{k\}^\{\\dagger\}\|\|\|\\mu\_\{k\}^\{\\mathcal\{X\}\}\-\\mu\_\{k\}^\{\\dagger\}\|\|^\{2\}\_\{2\}\+\\sum\_\{\|\\mathcal\{C\}^\{\\dagger\}\|\}\\sum\_\{\{\\mathbf\{h\}\_\{i\}^\{\\dagger\}\}\\in\\mathcal\{X\}\_\{k\}\}\|\|\\mathbf\{h\}\_\{i\}^\{\\dagger\}\-\\mu\_\{k\}^\{\\dagger\}\|\|\_\{2\}^\{2\}\}\]whereNNdenote the number of samples,\|Δ\|=\|\\Delta\|=∑i‖𝐡i^−𝐡i∗‖22\\sqrt\{\\sum\_\{i\}\|\|\\hat\{\\mathbf\{h\}\_\{i\}\}\-\{\\mathbf\{h\}\_\{i\}\}^\{\*\}\|\|\_\{2\}^\{2\}\},M=WT​\(I\|𝒱\|−1\|𝒱\|​𝟏\|𝒱\|​𝟏\|𝒱\|T\)​WM=W^\{T\}\(I\_\{\|\\mathcal\{V\}\|\}\-\\frac\{1\}\{\|\\mathcal\{V\}\|\}\\mathbf\{1\}\_\{\|\\mathcal\{V\}\|\}\\mathbf\{1\}\_\{\|\\mathcal\{V\}\|\}^\{T\}\)W\. The detail proof could be found in Appendix

[D](https://arxiv.org/html/2610.12855#A4)

From Theorem[4\.1](https://arxiv.org/html/2610.12855#S4.Thmtheorem1), we observe that the excess risk upper bound is affected by both intra\-cluster compactness and inter\-cluster separation of the learned representations\. Specifically, reducing the intra\-cluster variance term‖𝐡i−μk𝒳‖22\\\|\\mathbf\{h\}\_\{i\}\-\\mu\_\{k\}^\{\\mathcal\{X\}\}\\\|\_\{2\}^\{2\}and increasing the inter\-cluster dispersion term‖μk𝒳−μ‖22\\\|\\mu\_\{k\}^\{\\mathcal\{X\}\}\-\\mu\\\|\_\{2\}^\{2\}can tighten the excess risk upper bound\. Meanwhile, the term‖μk𝒳−μk†‖22\\\|\\mu\_\{k\}^\{\\mathcal\{X\}\}\-\\mu\_\{k\}^\{\\dagger\}\\\|\_\{2\}^\{2\}reflects the deviation between the learned student representation and the teacher and optimal teacher representation\. Therefore, preventing capability degradation during OPSD requires not only maintaining the alignment with the teacher representation, but also preserving compact intra\-cluster compactness and clear inter\-cluster separation in the representation space\.

## 5Methodology

As discussed previously, we observe that increasingly blurred boundaries between hidden representation clusters coincide with the progressive degradation of model capabilities during OPSD training\. Motivated by this observation, we propose Representation Geometry Regularization \(RGR\) to preserve clear boundaries between hidden representation clusters\. To achieve this goal, we consider two complementary geometric properties: \(1\) intra\-cluster compactness, where representations within each cluster remain sufficiently compact, and \(2\) inter\-cluster separation, where different clusters maintain sufficient separation from each other\. In the following, we discuss how to preserve the two properties in detail and derive the corresponding regularization objectives\.

### 5\.1Intra\-cluster compactness

According to previous works , the intra\-cluster compactness of hidden representations can be evaluated by the Within\-Cluster Sum of Squares \(WCSS\)\([Lloyd, 1982](https://arxiv.org/html/2610.12855#bib.bib13)\), defined as the sum of squared distances between each hidden representation and the center of its corresponding cluster\. However, directly minimizing WCSS may lead to excessive compactness within each cluster\. In the extreme case, hidden representations within the same cluster may become overly similar, reducing the discriminative information encoded in the representation space\. Therefore, we introduce the teacher model as a reference and encourage the WCSS of the student representations within each cluster to remain close to that of the teacher representations\.

Given the queryqq, the hidden representations of the response generated by the student model are denoted as\{𝐡1,…,𝐡T\}\\\{\\mathbf\{h\}\_\{1\},\\ldots,\\mathbf\{h\}\_\{T\}\\\}, while the corresponding hidden representations of the reference model are denoted as\{𝐡1ref,…,𝐡Tref\}\\\{\\mathbf\{h\}\_\{1\}^\{\\text\{ref\}\},\\ldots,\\mathbf\{h\}\_\{T\}^\{\\text\{ref\}\}\\\}\.

Suppose that a clusterCkrefC^\{\\mathrm\{ref\}\}\_\{k\}in the reference model’s representation space corresponds to a clusterCkalignC\_\{k\}^\{\\mathrm\{align\}\}in the student model’s representation space, with cluster centers𝐮kref\\mathbf\{u\}\_\{k\}^\{\\mathrm\{ref\}\}and𝐮kalign\\mathbf\{u\}\_\{k\}^\{\\mathrm\{align\}\}, respectively\.To align the intra\-cluster compactness of the student model with that of the reference model, the learning target for intra\-cluster compactness is defined as:

ℒintra=∑\|𝒞ref\|\|∑𝐡i∈Ckalign,‖𝐡i−𝝁kalign‖22−∑𝐡iref∈Ckref‖𝐡iref−𝝁kref‖22\|,\\mathcal\{L\}\_\{\\mathrm\{intra\}\}=\\sum\_\{\|\\mathcal\{C\}\_\{\\mathrm\{ref\}\}\|\}\\left\|\\sum\_\{\\mathbf\{h\}\_\{i\}\\in C\_\{k\}^\{\\mathrm\{align\}\},\}\\left\\\|\\mathbf\{h\}\_\{i\}\-\\boldsymbol\{\\mu\}\_\{k\}^\{\\mathrm\{align\}\}\\right\\\|\_\{2\}^\{2\}\-\\sum\_\{\\mathbf\{h\}\_\{i\}^\{\\mathrm\{ref\}\}\\in C\_\{k\}^\{\\mathrm\{ref\}\}\}\\left\\\|\\mathbf\{h\}\_\{i\}^\{\\mathrm\{ref\}\}\-\\boldsymbol\{\\mu\}\_\{k\}^\{\\mathrm\{ref\}\}\\right\\\|\_\{2\}^\{2\}\\right\|,\(4\)
where\|𝒞ref\|\|\\mathcal\{C\}\_\{\\mathrm\{ref\}\}\|denotes the number of clusters\.

However, directly optimizing Eq\.[17](https://arxiv.org/html/2610.12855#A5.E17)is difficult\. To address this, we stack the hidden representations of the student model as𝐇=\[𝐡1,…,𝐡T\]∈ℝT×d\\mathbf\{H\}=\[\\mathbf\{h\}\_\{1\},\\ldots,\\mathbf\{h\}\_\{T\}\]\\in\\mathbb\{R\}^\{T\\times d\}, where each hidden representation is the row vector of𝐇\\mathbf\{H\}\. Similarly, we stack the hidden representations of the student model as𝐇ref=\[𝐡1ref,…,𝐡Tref\]\\mathbf\{H\}\_\{\\mathrm\{ref\}\}=\[\\mathbf\{h\}\_\{1\}^\{\\mathrm\{ref\}\},\\ldots,\\mathbf\{h\}\_\{T\}^\{\\mathrm\{ref\}\}\], where each hidden representation is the row vector of𝐇ref\\mathbf\{H\}\_\{\\mathrm\{ref\}\}\. Under the above definitions and assumptions, we have:

###### Theorem 5\.1\(Intra\-cluster Compactness Upper Bound\)\.

The upper bound of the intra\-cluster compactness learning target is

ℒintra≤\(T−\|𝒞ref\|\)12​‖𝐇𝐇⊤−𝐇ref​𝐇refT‖F,\\mathcal\{L\}\_\{\\mathrm\{intra\}\}\\leq\(T\-\|\\mathcal\{C\}\_\{\\mathrm\{ref\}\}\|\)^\{\\frac\{1\}\{2\}\}\|\|\\mathbf\{H\}\\mathbf\{H\}^\{\\top\}\-\\mathbf\{H\}\_\{\\mathrm\{ref\}\}\\mathbf\{H\}\_\{\\mathrm\{ref\}\}^\{T\}\|\|\_\{F\},\(5\)

where\|𝒞ref\|\|\\mathcal\{C\}\_\{\\mathrm\{ref\}\}\|denotes the number of clusters of the reference model\.

The details of the proof can be found in Appendix[E](https://arxiv.org/html/2610.12855#A5)\.

### 5\.2Inter\-cluster separation

We construct a Laplacian matrix\([Von Luxburg, 2007](https://arxiv.org/html/2610.12855#bib.bib20)\)from the pairwise relationships among all hidden representations and use its low\-end spectrum, corresponding to the smallest eigenvalues of the Laplacian matrix, to evaluate the separability between clusters\. However, directly maximizing these low\-end eigenvalues is undesirable, as excessively large inter\-cluster separation can over\-expand the representation space and distort the relative geometric relationships among clusters\. Therefore, we use the teacher model as a reference model to encourage the student model to preserve its inter\-cluster separation, thereby preventing undesirable geometric distortion during optimization\.

Specifically, given the queryqq, the hidden representations of the response generated by the student model are denoted as\{𝐡1,…,𝐡T\}\\\{\\mathbf\{h\}\_\{1\},\\ldots,\\mathbf\{h\}\_\{T\}\\\}, while the corresponding hidden representations of the reference model are denoted as\{𝐡1ref,…,𝐡Tref\}\\\{\\mathbf\{h\}\_\{1\}^\{\\text\{ref\}\},\\ldots,\\mathbf\{h\}\_\{T\}^\{\\text\{ref\}\}\\\}\. We define the Gaussian kernel matrices of the hidden representations of the student model and the reference model as:

Ai​j\\displaystyle A\_\{ij\}=\{exp⁡\(−‖𝐡i−𝐡j‖222​σ2\),i≠j,0,i=j,Ai​jr​e​f\\displaystyle=\\begin\{cases\}\\exp\\left\(\-\\dfrac\{\\left\\\|\\mathbf\{h\}\_\{i\}\-\\mathbf\{h\}\_\{j\}\\right\\\|\_\{2\}^\{2\}\}\{2\\sigma^\{2\}\}\\right\),&i\\neq j,\\\\ 0,&i=j,\\end\{cases\}\\qquad A\_\{ij\}^\{ref\}=\{exp⁡\(−‖𝐡iref−𝐡jref‖222​σ2\),i≠j,0,i=j,\\displaystyle=\\begin\{cases\}\\exp\\left\(\-\\dfrac\{\\left\\\|\\mathbf\{h\}\_\{i\}^\{\\mathrm\{ref\}\}\-\\mathbf\{h\}\_\{j\}^\{\\mathrm\{ref\}\}\\right\\\|\_\{2\}^\{2\}\}\{2\\sigma^\{2\}\}\\right\),&i\\neq j,\\\\ 0,&i=j,\\end\{cases\}\(6\)whereσ\\sigmadenotes the standard deviation of the Gaussian kernel\.

The corresponding degree matrices of the student and reference Gaussian kernel matrices are denoted by𝐃\\mathbf\{D\}and𝐃ref\\mathbf\{D\}^\{\\mathrm\{ref\}\}, respectively, whereDi​i=∑j=1TAi​jD\_\{ii\}=\\sum\_\{j=1\}^\{T\}A\_\{ij\}andDi​iref=∑j=1TAi​jrefD\_\{ii\}^\{\\mathrm\{ref\}\}=\\sum\_\{j=1\}^\{T\}A\_\{ij\}^\{\\mathrm\{ref\}\}\. The unnormalized Laplacian matrices of the student and reference hidden representations are then defined as𝐋=𝐃−𝐀\\mathbf\{L\}=\\mathbf\{D\}\-\\mathbf\{A\}and𝐋ref=𝐃ref−𝐀ref\\mathbf\{L\}^\{\\mathrm\{ref\}\}=\\mathbf\{D\}^\{\\mathrm\{ref\}\}\-\\mathbf\{A\}^\{\\mathrm\{ref\}\}, respectively\. The corresponding eigenvalue vectors are

𝝀\\displaystyle\\boldsymbol\{\\lambda\}=\[λ1​\(𝐋\),…,λT​\(𝐋\)\]⊤,\\displaystyle=\\left\[\\lambda\_\{1\}\(\\mathbf\{L\}\),\\ldots,\\lambda\_\{T\}\(\\mathbf\{L\}\)\\right\]^\{\\top\},\(7\)𝝀ref\\displaystyle\\boldsymbol\{\\lambda\}^\{\\mathrm\{ref\}\}=\[λ1​\(𝐋ref\),…,λT​\(𝐋ref\)\]⊤,\\displaystyle=\\left\[\\lambda\_\{1\}\(\\mathbf\{L\}^\{\\mathrm\{ref\}\}\),\\ldots,\\lambda\_\{T\}\(\\mathbf\{L\}^\{\\mathrm\{ref\}\}\)\\right\]^\{\\top\},
where the eigenvalues are arranged in nondecreasing order\. The learning target for inter\-cluster separation could be represented by the difference between the eigenvalue vectors of the student and the reference, that is

ℒinter=‖𝝀−𝝀ref‖2\.\\mathcal\{L\}\_\{\\mathrm\{inter\}\}=\\left\\\|\\boldsymbol\{\\lambda\}\-\\boldsymbol\{\\lambda\}^\{\\mathrm\{ref\}\}\\right\\\|\_\{2\}\.\(8\)
Then, by the Hoffman–Wielandt inequality\([Hoffman et al\., 1953](https://arxiv.org/html/2610.12855#bib.bib5)\), we obtain

ℒinter≤\(1\+T\)​‖𝐋−𝐋𝐫𝐞𝐟‖F\\mathcal\{L\}\_\{\\mathrm\{inter\}\}\\leq\(1\+\\sqrt\{T\}\)\|\|\\mathbf\{L\}\-\\mathbf\{L^\{ref\}\}\|\|\_\{F\}\(9\)Under the above definitions and assumptions, we have

###### Theorem 5\.2\(Inter\-cluster Separation Upper Bound\)\.

The upper bound of the inter\-cluster separation could be induced by applying Taylor expansion and the Newton–Schwarz inequality:

ℒinter≤\(1\+Tσ\)2​‖𝐇𝐇⊤−𝐇ref​𝐇ref⊤‖F,\\mathcal\{L\}\_\{\\mathrm\{inter\}\}\\leq\\left\(\\frac\{1\+\\sqrt\{T\}\}\{\\sigma\}\\right\)^\{2\}\\left\\\|\\mathbf\{H\}\\mathbf\{H\}^\{\\top\}\-\\mathbf\{H\}\_\{\\mathrm\{ref\}\}\\mathbf\{H\}\_\{\\mathrm\{ref\}\}^\{\\top\}\\right\\\|\_\{F\},\(10\)

The proof is provided in Appendix[F](https://arxiv.org/html/2610.12855#A6)\.

### 5\.3Optimization

Building upon the previous analysis, preserving the intra\-cluster compactness and inter\-cluster separation of the hidden representation space requires minimizingℒintra\\mathcal\{L\}\_\{\\mathrm\{intra\}\}andℒinter\\mathcal\{L\}\_\{\\mathrm\{inter\}\}\. From the above theoretical analysis, we show that both geometric discrepancies can be controlled by the discrepancy between the Gram matrices of the student and reference hidden representations\. Specifically, we haveℒintra≤Cintra​‖𝐇𝐇⊤−𝐇ref​𝐇ref⊤‖F\\mathcal\{L\}\_\{\\mathrm\{intra\}\}\\leq C\_\{\\mathrm\{intra\}\}\\left\\\|\\mathbf\{H\}\\mathbf\{H\}^\{\\top\}\-\\mathbf\{H\}\_\{\\mathrm\{ref\}\}\\mathbf\{H\}\_\{\\mathrm\{ref\}\}^\{\\top\}\\right\\\|\_\{F\}andℒinter≤Cinter​‖𝐇𝐇⊤−𝐇ref​𝐇ref⊤‖F,\\mathcal\{L\}\_\{\\mathrm\{inter\}\}\\leq C\_\{\\mathrm\{inter\}\}\\left\\\|\\mathbf\{H\}\\mathbf\{H\}^\{\\top\}\-\\mathbf\{H\}\_\{\\mathrm\{ref\}\}\\mathbf\{H\}\_\{\\mathrm\{ref\}\}^\{\\top\}\\right\\\|\_\{F\},whereCintraC\_\{\\mathrm\{intra\}\}andCinterC\_\{\\mathrm\{inter\}\}are constants independent of the student representations\. Therefore, minimizing the Gram matrix discrepancy provides a unified objective for preserving both intra\-cluster compactness and inter\-cluster separation\. We thus define the geometry\-preserving objective as

ℒgeo=‖𝐇𝐇⊤−𝐇ref​𝐇ref⊤‖F2\.\\mathcal\{L\}\_\{\\mathrm\{geo\}\}=\\left\\\|\\mathbf\{H\}\\mathbf\{H\}^\{\\top\}\-\\mathbf\{H\}\_\{\\mathrm\{ref\}\}\\mathbf\{H\}\_\{\\mathrm\{ref\}\}^\{\\top\}\\right\\\|\_\{F\}^\{2\}\.\(11\)In addition to representation geometry, recent studies\([Jin et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib8)\)on on\-policy distillation have highlighted the importance of controlling the entropy of the model output distribution during optimization,which motivates us to explicitly incorporate an entropy\-related objective into the optimization process\. Accordingly, we jointly preserve the hidden representation geometry and regulate the output distribution entropy during OPSD optimization\. Combining the original OPSD objective with the geometry\-preserving and entropy\-related objectives, we formulate the overall training objective as

ℒ=ℒOPSD\+α​ℒgeo\+β​ℒent,\\mathcal\{L\}=\\mathcal\{L\}\_\{\\mathrm\{OPSD\}\}\+\\alpha\\mathcal\{L\}\_\{\\mathrm\{geo\}\}\+\\beta\\mathcal\{L\}\_\{\\mathrm\{ent\}\},\(12\)whereℒOPSD\\mathcal\{L\}\_\{\\mathrm\{OPSD\}\}denotes the original on\-policy self\-distillation objective,ℒgeo\\mathcal\{L\}\_\{\\mathrm\{geo\}\}denotes the proposed geometry\-preserving regularization, andℒent=‖𝐇⊤​𝐇−𝐇ref⊤​𝐇ref‖F2\\mathcal\{L\}\_\{\\mathrm\{ent\}\}=\\left\\\|\\mathbf\{H\}^\{\\top\}\\mathbf\{H\}\-\\mathbf\{H\}\_\{\\mathrm\{ref\}\}^\{\\top\}\\mathbf\{H\}\_\{\\mathrm\{ref\}\}\\right\\\|\_\{F\}^\{2\}denotes the entropy\-related objective\. The coefficientsα\\alphaandβ\\betacontrol the contributions of the geometry\-preserving and entropy\-related terms, respectively\.

## 6EXEPERIMENT

In this section, we conduct experiments to evaluate the performance of RGR\. We aim to answer the following research questions: \(1\) Is RGR effective for improving the performance of self distillation \(RQ1\)? \(2\) How do different components of RGR contribute to the performance gains \(RQ2\)? \(3\) Can RGR alleviate the degradation of previously acquired capabilities during OPSD optimization, and how does it affect the evolution of hidden representations \(RQ3\)? \(4\) Can RGR generalize to task\-transfer settings and alleviate the degradation of previously acquired capabilities when learning new tasks \(RQ4\)?

### 6\.1Experiment setup

#### Datasets\.

Following the dataset setup of OPSD\([Zhao et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib30)\), we sample up to 30K problem\-solution pairs containing chain\-of\-thought reasoning from the mathematical reasoning subset of OpenThoughts\([Guha et al\.,](https://arxiv.org/html/2610.12855#bib.bib3)\)for model training\.\. We conduct evaluation on three representative benchmarks: AIME 2024\([Zhang and Math\-AI, 2024](https://arxiv.org/html/2610.12855#bib.bib28)\), AIME 2025\([Zhang and Math\-AI, 2025](https://arxiv.org/html/2610.12855#bib.bib29)\), and HMMT 2025\([Balunovic et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib1)\)\. AIME 2024 consists of problems from the 2024 American Invitational Mathematics Examination, covering diverse areas of high\-school mathematics\. AIME 2025 consists of problems from the 2025 American Invitational Mathematics Examination\. HMMT 2025 consists of problems from the Harvard\-MIT Mathematics Tournament, covering algebra, combinatorics, geometry, and number theory\.

#### Baseline\.

To assess the effectiveness of RGR, we integrate it into four representative self\-distillation methods, namely, OPSD\([Zhao et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib30)\), SDPO\([Hübotter et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib7)\), SRPO\([Li et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib10)\), and RLSD\([Yang et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib26)\), and compare each RGR\-enhanced method with its original counterpart\. These methods differ in how they optimize the self\-distillation objective\. Specifically, OPSD employs forward KL divergence\([Kullback and Leibler, 1951](https://arxiv.org/html/2610.12855#bib.bib9)\), with the policy update constrained by token\-level KL clipping; SDPO uses Jensen–Shannon divergence\([Lin, 1991](https://arxiv.org/html/2610.12855#bib.bib12)\)to align the student with the teacher; SRPO routes successful responses to reward\-based reinforcement and failed responses to self\-distillation; and RLSD uses environment\-derived rewards to determine the direction of policy updates, while reweighting token\-level advantages according to the teacher–student log\-probability gap to modulate the update magnitude\.

#### ImplementationDetails\.

Our experiments are conducted on two foundation LLMs: Qwen3\-1\.7B and Qwen3\-4B\([Yang et al\., 2025](https://arxiv.org/html/2610.12855#bib.bib25)\), with thinking mode enabled throughout training and validation\. All experiments are performed on four NVIDIA A40 GPUs\. All methods use LoRA\([Hu et al\., 2021](https://arxiv.org/html/2610.12855#bib.bib6)\)with a rank of 64 and a scaling factor of 128, with the learning rate set to 5×\\times10−610^\{\-6\}\. The teacher model remains frozen throughout training\. For our method, we fix the parameterα\\alphain Eq\.\([12](https://arxiv.org/html/2610.12855#S5.E12)\) at2\.5×10−22\.5\\times 10^\{\-2\}and the parameterβ\\betain Eq\.\([12](https://arxiv.org/html/2610.12855#S5.E12)\) at5×10−25\\times 10^\{\-2\}across all benchmarks\.

### 6\.2Main results \(RQ1\)

Table 1:Performance gains from RGR\. Best results in bold; RGR improvements in green\.ModelMethodAIME24AIME25HMMT25Avg\.Qwen3\-1\.7BOPSD56\.6743\.0628\.8942\.87OPSD \+ RGR59\.8943\.6030\.0044\.50\+1\.63SDPO38\.6128\.6121\.3929\.54SDPO \+ RGR41\.9431\.3921\.9431\.76\+2\.22RLSD50\.2739\.7025\.8338\.60RLSD \+ RGR51\.3939\.1725\.5638\.70\+0\.10SRPO37\.7830\.0020\.0029\.26SRPO \+ RGR39\.4429\.7221\.6730\.28\+1\.02Qwen3\-4BOPSD74\.1768\.0645\.0062\.41OPSD \+ RGR75\.5670\.5647\.5064\.54\+2\.13SDPO67\.2261\.3938\.6155\.74SDPO \+ RGR72\.5063\.6141\.1159\.07\+3\.33RLSD75\.0065\.0044\.4461\.48RLSD \+ RGR74\.7266\.1144\.7261\.85\+0\.37SRPO72\.2264\.1639\.1758\.52SRPO \+ RGR73\.3366\.6741\.6760\.56\+2\.04Following OPSD\([Zhao et al\., 2026](https://arxiv.org/html/2610.12855#bib.bib30)\), we evaluate each model by generating 12 responses for each problem and report the average accuracy over the 12 responses \(Avg@12\)\.

The results are reported in Table[1](https://arxiv.org/html/2610.12855#S6.T1)\. As it is shown in Table[1](https://arxiv.org/html/2610.12855#S6.T1), we can observe that: \(1\) RGR generally improves different methods across both Qwen3\-1\.7B and Qwen3\-4B\. OPSD, SDPO, and SRPO consistently benefit from RGR, demonstrating that representation geometry preservation provides complementary benefits to different optimization objectives\. Although a few settings show slight decreases, RGR improves the majority of cases, supporting its general effectiveness; \(2\) The gains vary across different methods, with larger improvements generally observed for OPSD and SDPO but limited gains for RLSD\. This can be attributed to their different optimization mechanisms: OPSD and SDPO directly rely on self\-distillation signals for dense token\-level supervision, while RLSD mainly uses self\-distillation to determine the update magnitude and obtains the update direction from RLVR\. Consequently, RGR has more room to benefit methods whose optimization induces stronger changes in the representation space; \(3\) The improvements also vary across AIME24, AIME25, and HMMT25\. This variation may be attributed to the different difficulty levels and reasoning characteristics of these benchmarks, which result in different distributions of correct and incorrect responses during self\-improvement\. Overall, RGR remains beneficial across diverse mathematical reasoning benchmarks, while its improvement varies with the characteristics of the target dataset\.

### 6\.3Ablation Study \(RQ2\)

To investigate the contribution of each component to RGR, we conduct an ablation study with the following variants:

- •w/o Ent: We remove theℒent\\mathcal\{L\}\_\{\\mathrm\{ent\}\}in Eq\.\([12](https://arxiv.org/html/2610.12855#S5.E12)\), removing the constraint on output distribution\.
- •w/o Geo: We remove theℒgeo\\mathcal\{L\}\_\{\\mathrm\{geo\}\}in Eq\.\([12](https://arxiv.org/html/2610.12855#S5.E12)\), eliminating the constraint on intra\-cluster compactness and inter\-cluster separation\.

Table 2:Ablation study of RGR\.MethodAIME24AIME25HMMT25OPSD56\.6743\.0628\.89w/o Ent57\.2241\.3928\.33w/o Geo58\.6142\.2229\.17RGR59\.8943\.6030\.00The results are shown in Table[2](https://arxiv.org/html/2610.12855#S6.T2)that RGR achieves the best performance across all three benchmarks\. From Table[2](https://arxiv.org/html/2610.12855#S6.T2), we can also observe that: \(1\) Removing the geometry constraint \(w/o Geo\) consistently degrades the performance across all benchmarks, compared with the full RGR\. This indicates that explicitly preserving intra\-cluster compactness and inter\-cluster center separation is crucial for maintaining a structured representation space during OPSD optimization\. Without this constraint, the hidden representations may become less organized, leading to limited capability improvement\.

Figure 3:Comparison of the capability degradation ratio between the baseline and the baseline\+RGR model on AIME24 \(%\)\.![Refer to caption](https://arxiv.org/html/2610.12855v1/combined_figure.png)Figure 4:Visualization of hidden state representations across different models and training steps, with response correctness indicated\.\(2\) Removing the entropy constraint \(w/o Ent\) also leads to performance drops on one benchmarks\. This suggests that entropy regularization provides complementary optimization benefits by further stabilizing the output distribution during self\-distillation\. However, the performance gap between RGR andw/o Geois larger than that between RGR andw/o Ent, indicating that representation geometry preservation serves as the primary contributor to the improvements\.

### 6\.4Training Dynamics Visualization \(RQ3\)

Effect of RGR on mitigating progressive capability degradation\.To evaluate whether RGR alleviates progressive capability degradation, we compare the ratio of queries that transition from correct to incorrect responses before and after incorporating RGR into different OPSD variants\. As shown in Figure[3](https://arxiv.org/html/2610.12855#S6.F3), adding RGR consistently reduces the degradation ratio across all evaluated methods\. Specifically, RGR eliminates the degradation in OPSD and reduces the degradation ratios of SDPO, RLSD, and SRPO by 14\.28%, 7\.15%, and 21\.43%, respectively\. These results demonstrate that RGR not only improves self\-improvement performance but also helps preserve previously acquired capabilities by maintaining a more stable hidden representation geometry during optimization\.

Representation geometry analysis\.To further understand how RGR alleviates progressive capability degradation, we visualize the hidden representations of responses corresponding to the degraded queries\. Specifically, we select queries that are initially responded correctly but become incorrectly responded during optimization, and compare the representation geometry before and after incorporating RGR\. As shown in Figure[4](https://arxiv.org/html/2610.12855#S6.F4), without RGR, the hidden representations gradually lose their geometric structure, exhibiting reduced inter\-cluster center separation and less distinguishable cluster boundaries when the response changes from correct to incorrect\. In contrast, RGR maintains more compact clusters with clearer boundaries in most cases\. These results are consistent with our theoretical analysis and demonstrate that preserving representation geometry helps stabilize the evolution of hidden representations during OPSD training\.

Table 3:Forgetting evaluation on AIME24 \(%\)\.Beforedenotes the performance before training on the tool\-use dataset, whileAfterdenotes the performance after tool\-use training\.MethodBeforeAfterForgetting↓\\downarrowOPSD56\.6754\.721\.95OPSD \+ RGR59\.8958\.061\.83SDPO38\.6133\.335\.28SDPO \+ RGR41\.9440\.001\.94RLSD50\.2747\.223\.05RLSD \+ RGR51\.3948\.003\.39SRPO37\.7834\.723\.06SRPO \+ RGR39\.4438\.331\.11Overall, RGR achieves consistent reductions in capability degradation across OPSD, SDPO, RLSD, and SRPO, despite these methods adopting different self\-distillation objectives\. This suggests that RGR serves as a general geometry\-preserving regularization mechanism that can be integrated into different OPSD methods\.

### 6\.5Cross\-Task Forgetting Study \(RQ4\)

To further investigate whether RGR can mitigate capability degradation beyond the original task, we conduct a cross\-task forgetting experiment\. We first post\-train a model on OpenThoughts to acquire mathematical problem\-solving capabilities, and evaluate the model on AIME24 to measure performance\. Then we continue post\-training it on a tool\-use dataset\([Tang et al\., 2023](https://arxiv.org/html/2610.12855#bib.bib18)\), after convergence, we evaluate the model again on AIME24 to measure the performance after tool\-use training\. We apply this setting to OPSD, SDPO, RLSD, and SRPO, and compare their performance before and after training on the tool\-use dataset, as shown in Table[3](https://arxiv.org/html/2610.12855#S6.T3)\. From Table[3](https://arxiv.org/html/2610.12855#S6.T3), we can observe that: \(1\) Sequential training on a new task leads to forgetting of previously acquired capabilities\. After tool\-use training, all four methods exhibit performance degradation on AIME24, consistent with previous studies of catastrophic forgetting in continual fine\-tuning of LLMs\([Luo et al\., 2025](https://arxiv.org/html/2610.12855#bib.bib14);[Li et al\., 2024](https://arxiv.org/html/2610.12855#bib.bib11)\)\. \(2\) RGR generally alleviates cross\-task forgetting\. RGR reduces the forgetting gap for OPSD, SDPO, and SRPO, with particularly notable improvements for SDPO \(5\.28→\\rightarrow1\.94\) and SRPO \(3\.06→\\rightarrow1\.11\)\. This suggests that preserving representation geometry during subsequent optimization can help retain previously acquired capabilities\. \(3\) The effectiveness of RGR depends on the underlying post\-training method\. For RLSD, the forgetting gap slightly increases from 3\.05 to 3\.39 after introducing RGR, suggesting that the effect of representation geometry preservation may vary across different optimization dynamics\. Nevertheless, RGR reduces forgetting for three of the four methods, supporting its broader applicability to sequential post\-training\.

## 7Conclusion

In this work, we identify and investigate the progressive capability degradation phenomenon in on\-policy self\-distillation, where LLM may lose previously acquired capabilities during self\-improvement despite overall performance improvements\. Through empirical analysis and theoretical investigation, we reveal that this degradation is associated with the deterioration of hidden representation geometry, characterized by reduced intra\-cluster compactness and inter\-cluster center separation\. Based on these findings, we propose Representation Geometry Regularization \(RGR\), a general regularization framework that preserves intra\-cluster compactness and inter\-cluster center separation during OPSD optimization\. Extensive experiments across multiple OPSD variants and mathematical reasoning benchmarks demonstrate that RGR effectively reduces correct\-to\-incorrect transitions while maintaining the self\-improvement capability of OPSD\. Future work will explore more adaptive strategies for preserving representation geometry and extending the proposed framework to broader model optimization paradigms beyond self\-distillation\.

## References

- Balunovic et al\. \(2026\)Mislav Balunovic, Jasper Dekoninck, Ivo Petrov, Nikola Jovanović, and Martin Vechev\.Matharena: Evaluating llms on uncontaminated math competitions\.*Advances in Neural Information Processing Systems*, 38, 2026\.
- Brown et al\. \(2020\)Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al\.Language models are few\-shot learners\.*Advances in neural information processing systems*, 33:1877–1901, 2020\.
- \(3\)Etash Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, et al\.Openthoughts: Data recipes for reasoning models, 2025\.*URL https://arxiv\. org/abs/2506\.04178*\.
- Hinton et al\. \(2015\)Geoffrey Hinton, Oriol Vinyals, and Jeff Dean\.Distilling the knowledge in a neural network\.*arXiv preprint arXiv:1503\.02531*, 2015\.
- Hoffman et al\. \(1953\)Alan J Hoffman, Helmut W Wielandt, et al\.The variation of the spectrum of a normal matrix\.*Duke Math\. J*, 20\(1\):37–39, 1953\.
- Hu et al\. \(2021\)Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen\-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen\.Lora: Low\-rank adaptation of large language models\.*arXiv preprint arXiv:2106\.09685*, 2021\.
- Hübotter et al\. \(2026\)Jonas Hübotter, Frederike Lübeck, Lejs Behric, Anton Baumann, Marco Bagatella, Daniel Marta, Ido Hakimi, Idan Shenfeld, Thomas Kleine Buening, Carlos Guestrin, et al\.Reinforcement learning via self\-distillation\.*arXiv preprint arXiv:2601\.20802*, 2026\.
- Jin et al\. \(2026\)Woogyeol Jin, Taywon Min, Yongjin Yang, Dennis Wei, Yi Zhou, Swanand Ravindra Kadhe, Nathalie Baracaldo, and Kimin Lee\.Entropy\-aware on\-policy distillation of language models\.*arXiv preprint arXiv:2603\.07079*, 2026\.
- Kullback and Leibler \(1951\)Solomon Kullback and Richard A Leibler\.On information and sufficiency\.*The annals of mathematical statistics*, 22\(1\):79–86, 1951\.
- Li et al\. \(2026\)Gengsheng Li, Tianyu Yang, Junfeng Fang, Mingyang Song, Mao Zheng, Haiyun Guo, Dan Zhang, Jinqiao Wang, and Tat\-Seng Chua\.Unifying group\-relative and self\-distillation policy optimization via sample routing\.*arXiv preprint arXiv:2604\.02288*, 2026\.
- Li et al\. \(2024\)Hongyu Li, Liang Ding, Meng Fang, and Dacheng Tao\.Revisiting catastrophic forgetting in large language model tuning\.In*Findings of the association for computational linguistics: EMNLP 2024*, pages 4297–4308, 2024\.
- Lin \(1991\)Jianhua Lin\.Divergence measures based on the shannon entropy\.*IEEE Transactions on Information theory*, 37\(1\):145–151, 1991\.
- Lloyd \(1982\)Stuart Lloyd\.Least squares quantization in pcm\.*IEEE transactions on information theory*, 28\(2\):129–137, 1982\.
- Luo et al\. \(2025\)Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang\.An empirical study of catastrophic forgetting in large language models during continual fine\-tuning\.*IEEE Transactions on Audio, Speech and Language Processing*, 2025\.
- Minka et al\. \(2005\)Tom Minka et al\.Divergence measures and message passing\.2005\.
- Sanh et al\. \(2019\)Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf\.Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter\.*arXiv preprint arXiv:1910\.01108*, 2019\.
- Shao et al\. \(2024\)Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al\.Deepseekmath: Pushing the limits of mathematical reasoning in open language models\.*arXiv preprint arXiv:2402\.03300*, 2024\.
- Tang et al\. \(2023\)Qiaoyu Tang, Ziliang Deng, Hongyu Lin, Xianpei Han, Qiao Liang, Boxi Cao, and Le Sun\.Toolalpaca: Generalized tool learning for language models with 3000 simulated cases\.*arXiv preprint arXiv:2306\.05301*, 2023\.
- Vaswani et al\. \(2017\)Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin\.Attention is all you need\.*Advances in neural information processing systems*, 30, 2017\.
- Von Luxburg \(2007\)Ulrike Von Luxburg\.A tutorial on spectral clustering\.*Statistics and computing*, 17\(4\):395–416, 2007\.
- Wang et al\. \(2026\)Jiaxuan Wang, Xuan Ouyang, Zhiyu Chen, Yulan Hu, Zheng Pan, Xin Li, and Lan\-Zhe Guo\.Trace: Distilling where it matters via token\-routed self on\-policy alignment\.*arXiv preprint arXiv:2605\.10194*, 2026\.
- Wei et al\. \(2021\)Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le\.Finetuned language models are zero\-shot learners\.*arXiv preprint arXiv:2109\.01652*, 2021\.
- Xiao et al\. \(2026\)Bangjun Xiao, Bingquan Xia, Bo Yang, Bofei Gao, Bowen Shen, Chen Zhang, Chenhong He, Chiheng Lou, Fuli Luo, Gang Wang, et al\.Mimo\-v2\-flash technical report\.*arXiv preprint arXiv:2601\.02780*, 2026\.
- Xu et al\. \(2026\)Anyi Xu, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, Chenchen Ling, et al\.Deepseek\-v4: Towards highly efficient million\-token context intelligence\.*arXiv preprint arXiv:2606\.19348*, 2026\.
- Yang et al\. \(2025\)An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al\.Qwen3 technical report\.*arXiv preprint arXiv:2505\.09388*, 2025\.
- Yang et al\. \(2026\)Chenxu Yang, Chuanyu Qin, Qingyi Si, Minghui Chen, Naibin Gu, Dingyu Yao, Zheng Lin, Weiping Wang, Jiaqi Wang, and Nan Duan\.Self\-distilled rlvr\.*arXiv preprint arXiv:2604\.03128*, 2026\.
- Zeng et al\. \(2026\)Aohan Zeng, Xin Lv, Zhenyu Hou, Zhengxiao Du, Qinkai Zheng, Bin Chen, Da Yin, Chendi Ge, Chenghua Huang, Chengxing Xie, et al\.Glm\-5: from vibe coding to agentic engineering\.*arXiv preprint arXiv:2602\.15763*, 2026\.
- Zhang and Math\-AI \(2024\)Yifan Zhang and Team Math\-AI\.American invitational mathematics examination \(aime\) 2024, 2024\.
- Zhang and Math\-AI \(2025\)Yifan Zhang and Team Math\-AI\.American invitational mathematics examination \(aime\) 2025, 2025\.
- Zhao et al\. \(2026\)Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover\.Self\-distilled reasoner: On\-policy self\-distillation for large language models\.*arXiv preprint arXiv:2601\.18734*, 2026\.

## Appendix ASensitivity Analysis

#### Hyperparameter sensitivity\.

We investigate the sensitivity of our method to the hyperparametersα\\alphaandβ\\betain Eq\. \([12](https://arxiv.org/html/2610.12855#S5.E12)\)\. We vary one hyperparameter at a time while fixing the other at its selected value\. Specifically, we varyβ\\betaover\{2\.5,4\.0,5\.0,6\.0\}×10−2\\\{2\.5,4\.0,5\.0,6\.0\\\}\\times 10^\{\-2\}while fixingα=2\.5×10−2\\alpha=2\.5\\times 10^\{\-2\}\. Similarly, we varyα\\alphaover\{0\.5,2\.5,3\.5,4\.0,5\.0\}×10−2\\\{0\.5,2\.5,3\.5,4\.0,5\.0\\\}\\times 10^\{\-2\}while fixingβ=5×10−2\\beta=5\\times 10^\{\-2\}\. As shown in Figure[5](https://arxiv.org/html/2610.12855#A1.F5), As shown in the figure, the performance of our model follows a smooth increasing\-then\-decreasing trend with respect to the hyperparameter values\. This observation suggests that the proposed method does not require highly precise hyperparameter tuning and remains stable change across a relatively broad range of parameter settings\.

Figure 5:Sensitivity analysis of the hyperparametersα\\alphaandβ\\betaon AIME24\. Left: performance under different values ofβ\\beta, withα\\alphafixed at2\.5×10−22\.5\\times 10^\{\-2\}\. Right: performance under different values ofα\\alpha, withβ\\betafixed at5×10−25\\times 10^\{\-2\}\. The stars indicate the selected setting:α=2\.5×10−2\\alpha=2\.5\\times 10^\{\-2\}andβ=5×10−2\\beta=5\\times 10^\{\-2\}\.

## Appendix BImplementation Details

Table[4](https://arxiv.org/html/2610.12855#A2.T4)lists the training hyperparameters that are shared across all experiments\. All runs use LoRA for parameter\-efficient fine\-tuning with rankr=64r\{=\}64andα=128\\alpha\{=\}128, and are optimized with a learning rate of5×10−65\\times 10^\{\-6\}under a maximum gradient norm of0\.10\.1\. The maximum sequence length is20,00020\{,\}000tokens, while completions are capped at1,0241\{,\}024tokens\. Rollout sampling uses a temperature of1\.11\.1, top\-ppof0\.950\.95, and top\-kkof2020\. Training is performed in BF16 precision\.

Table 4:Hyperparameters used across all experiments\.HyperparameterValueLearning rate5×10−65\\times 10^\{\-6\}Max gradient norm0\.1LoRA rankrr64LoRAα\\alpha128Max sequence length20,000Max completion length1,024Temperature1\.1Top\-pp0\.95Top\-kk20BF16Enabled
## Appendix CNotation and Additional Preliminaries

Table 5:Summary of the main notation used in the paper\.SymbolDescriptionqqa queryπθ\\pi\_\{\\theta\}LLM policy parameterized byθ\\thetaand trained within the OPSD framework\.θ\\thetaParameters of the LLM policy\.yty\_\{t\}Token generated at decoding stepttbyπθ\\pi\_\{\\theta\}\.𝐲=\(y1,…,yT\)\\mathbf\{y\}=\(y\_\{1\},\\ldots,y\_\{T\}\)Response consisting ofTTtokens generated byπθ\\pi\_\{\\theta\}\.𝐲<t=\(y1,…,yt−1\)\\mathbf\{y\}\_\{<t\}=\(y\_\{1\},\\ldots,y\_\{t\-1\}\)Sequence of tokens generated before steptt\.πθ\(⋅∣q,𝐲<t\)\\pi\_\{\\theta\}\(\\cdot\\mid q,\\mathbf\{y\}\_\{<t\}\)Conditional distribution over the next token given the query and previously generated tokens\.TTLength of the generated response\.ℋ=\{𝐡1,…,𝐡T\}\\mathcal\{H\}=\\\{\\mathbf\{h\}\_\{1\},\\ldots,\\mathbf\{h\}\_\{T\}\\\}the hidden representations corresponding to the generated response\.ℋ^=\{𝐡^1,…,𝐡^T\}\\hat\{\\mathcal\{H\}\}=\\\{\\mathbf\{\\hat\{h\}\}\_\{1\},\\ldots,\\mathbf\{\\hat\{h\}\}\_\{T\}\\\}the hidden representations generated by the teacher model\.ℋ∗=\{𝐡1∗,…,𝐡T∗\}\\mathcal\{H\}^\{\*\}=\\\{\\mathbf\{h\}\_\{1\}^\{\*\},\\ldots,\\mathbf\{h\}\_\{T\}^\{\*\}\\\}the hidden representations generated by the optimal teacher model\.ℋ†=\{𝐡1∗\+𝐡^12\+…,𝐡T∗\+𝐡^T2\}\{\\mathcal\{H\}\}^\{\\dagger\}=\\\{\\frac\{\\mathbf\{h\}\_\{1\}^\{\*\}\+\\mathbf\{\\hat\{h\}\}\_\{1\}\}\{2\}\+\\ldots,\\frac\{\\mathbf\{h\}\_\{T\}^\{\*\}\+\\mathbf\{\\hat\{h\}\}\_\{T\}\}\{2\}\\\}the mixed hidden representations ofℋ\\mathcal\{H\}andℋ^\\hat\{\\mathcal\{H\}\}\.ℋr​e​f=\{𝐡1r​e​f,…,𝐡Tr​e​f\}\\mathcal\{H\}^\{ref\}=\\\{\\mathbf\{h\}\_\{1\}^\{ref\},\\ldots,\\mathbf\{h\}\_\{T\}^\{ref\}\\\}the hidden representations of the reference model\.ℋ=\{𝐡1,…,𝐡T\}\\mathcal\{H\}=\\\{\\mathbf\{h\}\_\{1\},\\ldots,\\mathbf\{h\}\_\{T\}\\\}the hidden representations corresponding to the generated response\.ℋ=\{𝐡1,…,𝐡T\}\\mathcal\{H\}=\\\{\\mathbf\{h\}\_\{1\},\\ldots,\\mathbf\{h\}\_\{T\}\\\}the hidden representations corresponding to the generated response\.𝒞=\{C1,…,CK\}\\mathcal\{C\}=\\\{C\_\{1\},\\ldots,C\_\{K\}\\\}ℋ\\mathcal\{H\}intoKKrepresentation clusters\.𝒞†=\{C1†,…,CK†\}\\mathcal\{C^\{\\dagger\}\}=\\\{C\_\{1\}^\{\\dagger\},\\ldots,C\_\{K\}^\{\\dagger\}\\\},ℋ†\\mathcal\{H^\{\\dagger\}\}intoKKrepresentation clusters\.𝒞r​e​f=\{C1r​e​f,…,CKr​e​f\}\\mathcal\{C\}^\{ref\}=\\\{C\_\{1\}^\{ref\},\\ldots,C\_\{K\}^\{ref\}\\\},ℋr​e​f\\mathcal\{H\}^\{ref\}intoKKrepresentation clusters\.𝒳k\\mathcal\{X\}\_\{k\}:=\{𝐡i∣i∈ℐk†\}\\\{\\mathbf\{h\}\_\{i\}\\mid i\\in\\mathcal\{I\}\_\{k\}^\{\\dagger\}\\\}the associated index set of for each clusterCk†C\_\{k\}^\{\\dagger\}\|𝒞\|\|\\mathcal\{C\}\|the number of clusters of𝒞\\mathcal\{C\}\.\|𝒞†\|\|\\mathcal\{C\}^\{\\dagger\}\|the number of clusters of𝒞†\\mathcal\{C\}^\{\\dagger\}\.\|𝒞ref\|\|\\mathcal\{C\}\_\{\\mathrm\{ref\}\}\|the number of clusters of𝒞ref\\mathcal\{C\}\_\{\\mathrm\{ref\}\}\.𝐇=\[𝐡1,…,𝐡T\]∈ℝT×d\\mathbf\{H\}=\[\\mathbf\{h\}\_\{1\},\\ldots,\\mathbf\{h\}\_\{T\}\]\\in\\mathbb\{R\}^\{T\\times d\}the hidden representation matrix of the student model\.𝐇r​e​f=\[𝐡1r​e​f,…,𝐡Tr​e​f\]∈ℝT×d\\mathbf\{H\}^\{ref\}=\[\\mathbf\{h\}\_\{1\}^\{ref\},\\ldots,\\mathbf\{h\}\_\{T\}^\{ref\}\]\\in\\mathbb\{R\}^\{T\\times d\}the hidden representation matrix of the reference model\.μk\\mu\_\{k\}1\|Ck\|​∑𝐡i∈Ck𝐡i\\frac\{1\}\{\|C\_\{k\}\|\}\\sum\_\{\\mathbf\{h\}\_\{i\}\\in C\_\{k\}\}\\mathbf\{h\}\_\{i\}\.μk†\\mu\_\{k\}^\{\\dagger\}1\|Ck†\|​∑𝐡i†∈Ck†𝐡i†\\frac\{1\}\{\|C\_\{k\}^\{\\dagger\}\|\}\\sum\_\{\\mathbf\{h\}\_\{i\}^\{\\dagger\}\\in C\_\{k\}^\{\\dagger\}\}\\mathbf\{h\}\_\{i\}^\{\\dagger\}\.μkr​e​f\\mu\_\{k\}^\{ref\}1\|Ckr​e​f\|​∑𝐡ir​e​f∈Ckr​e​f𝐡ir​e​f\\frac\{1\}\{\|C\_\{k\}^\{ref\}\|\}\\sum\_\{\\mathbf\{h\}\_\{i\}^\{ref\}\\in C\_\{k\}^\{ref\}\}\\mathbf\{h\}\_\{i\}^\{ref\}\.𝐋\\mathbf\{L\}The Laplacian matrix computed from the hidden representation matrix of the student model with a Gaussian kernel\.𝐋ref\\mathbf\{L\}^\{\\mathrm\{ref\}\}The Laplacian matrix computed from the hidden representation matrix of the reference model with a Gaussian kernel\.𝝀=\[λ1​\(𝐋\),…,λn​\(𝐋\)\]⊤\\boldsymbol\{\\lambda\}=\\left\[\\lambda\_\{1\}\(\\mathbf\{L\}\),\\ldots,\\lambda\_\{n\}\(\\mathbf\{L\}\)\\right\]^\{\\top\}the eigenvalues of the Laplacian matrix𝐋\\mathbf\{L\}\.𝝀ref=\[λ1​\(𝐋ref\),…,λn​\(𝐋ref\)\]⊤\\boldsymbol\{\\lambda\}^\{\\mathrm\{ref\}\}=\\left\[\\lambda\_\{1\}\(\\mathbf\{L\}^\{\\mathrm\{ref\}\}\),\\ldots,\\lambda\_\{n\}\(\\mathbf\{L\}^\{\\mathrm\{ref\}\}\)\\right\]^\{\\top\}the eigenvalues of the Laplacian matrix𝐋ref\\mathbf\{L\}^\{\\mathrm\{ref\}\}\.μkr​e​f\\mu\_\{k\}^\{ref\}1\|Ckr​e​f\|​∑𝐡ir​e​f∈Ckr​e​f𝐡ir​e​f\\frac\{1\}\{\|C\_\{k\}^\{ref\}\|\}\\sum\_\{\\mathbf\{h\}\_\{i\}^\{ref\}\\in C\_\{k\}^\{ref\}\}\\mathbf\{h\}\_\{i\}^\{ref\}\.𝒱\\mathcal\{V\}the vocabulary of the LLM\.\|𝒱\|\|\\mathcal\{V\}\|the size of the vocabulary\.W∈ℝ\|𝒱\|×dW\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\|\\times d\}the weight matrix of the final layer for the trained model\.I\|𝒱\|∈ℝ\|𝒱\|×\|𝒱\|I\_\{\|\\mathcal\{V\}\|\}\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\|\\times\|\\mathcal\{V\}\|\}the identity matrix \.𝟏\|𝒱\|∈ℝ\|𝒱\|\\mathbf\{1\}\_\{\|\\mathcal\{V\}\|\}\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\|\}the all\-ones column vector\.\|Δ\|\|\\Delta\|∑i‖𝐡i^−𝐡i∗‖2\\sqrt\{\\sum\_\{i\}\|\|\\hat\{\\mathbf\{h\}\_\{i\}\}\-\{\\mathbf\{h\}\_\{i\}\}^\{\*\}\|\|\_\{2\}\}\.D\(⋅\|⋅\)D\(\\cdot\|\\cdot\)a divergence measure between the teacher and student token distributions\.ℰKD​\(πθ\)\\mathcal\{E\}\_\{\\text\{KD\}\}\(\\pi\_\{\\theta\}\)a divergence measure between the teacher and student token distributions\.
## Appendix DProof of Theorem[4\.1](https://arxiv.org/html/2610.12855#S4.Thmtheorem1)

By definition[4\.1](https://arxiv.org/html/2610.12855#S4.Thmdefinition1), we obtain

ℰKD\(πθ\)=𝔼q∼𝒟,y∼πθ\(⋅∣q\)\[∑t=1TD\(πθ∗\(⋅∣q,z,y<t\)∥πθ\(⋅∣q,y<t\)\)−D\(πθ∗\(⋅∣q,z,y<t\)∥πθ∗\(⋅∣q,y<t\)\)\]\\mathcal\{E\}\_\{\\text\{KD\}\}\(\\pi\_\{\\theta\}\)=\\mathbb\{E\}\_\{q\\sim\\mathcal\{D\},y\\sim\\pi\_\{\\theta\}\(\\cdot\\mid q\)\}\\left\[\\sum\_\{t=1\}^\{T\}D\\left\(\\pi\_\{\\theta^\{\*\}\}\(\\cdot\\mid q,z,y\_\{<t\}\)\\,\\middle\\\|\\,\\pi\_\{\\theta\}\(\\cdot\\mid q,y\_\{<t\}\)\\right\)\\\\ \-D\\left\(\\pi\_\{\\theta^\{\*\}\}\(\\cdot\\mid q,z,y\_\{<t\}\)\\,\\middle\\\|\\,\\pi\_\{\\theta^\{\*\}\}\(\\cdot\\mid q,y\_\{<t\}\)\\right\)\\right\]\(13\)
To simplify the notation, we define

qt∗:=πθ∗\(⋅∣q,z,y<t\)q^\{\*\}\_\{t\}:=\\pi\_\{\\theta^\{\*\}\}\(\\cdot\\mid q,z,y\_\{<t\}\)pt:=πθ\(⋅∣q,y<t\)p\_\{t\}:=\\pi\_\{\\theta\}\(\\cdot\\mid q,y\_\{<t\}\)qt:=πθ\(⋅∣q,z,y<t\)q\_\{t\}:=\\pi\_\{\\theta\}\(\\cdot\\mid q,z,y\_\{<t\}\)pt∗:=πθ∗\(⋅∣q,y<t\)p^\{\*\}\_\{t\}:=\\pi\_\{\\theta^\{\*\}\}\(\\cdot\\mid q,y\_\{<t\}\)
Letztz\_\{t\}denote the corresponding logits ofptp\_\{t\},zt∗z\_\{t\}^\{\*\}denote the corresponding logits ofqt∗q\_\{t\}^\{\*\}andzt^\\hat\{z\_\{t\}\}denotes the corresponding logit ofqtq\_\{t\}\. For eachf∈ℱO​P​S​Df\\in\\mathcal\{F\}\_\{OPSD\}, letpfp\_\{f\}denote the policy induced byff\.

For one response andq∼𝒟,y∼πθq\\sim\\mathcal\{D\},y\\sim\\pi\_\{\\theta\}, letℰq,θ=∑t\(KL\(qt∗\|\|pt\)\)−E\(KL\(qt∗\|\|pt∗\)\\mathcal\{E\}\_\{q,\\theta\}=\\sum\_\{t\}\(KL\(q^\{\*\}\_\{t\}\|\|p\_\{t\}\)\)\-E\(KL\(q^\{\*\}\_\{t\}\|\|p^\{\*\}\_\{t\}\), we have

ℰKD​\(πθ\)=𝔼q∼𝒟,y∼πθ​ℰq,θ\\mathcal\{E\}\_\{\\text\{KD\}\}\(\\pi\_\{\\theta\}\)=\\mathbb\{E\}\_\{q\\sim\\mathcal\{D\},y\\sim\\pi\_\{\\theta\}\}\\mathcal\{E\}\_\{q,\\theta\}
ℰq,θ=∑t\(KL\(qt∗\|\|pt\)\)−KL\(qt∗\|\|pt∗\)\)\\mathcal\{E\}\_\{q,\\theta\}=\\sum\_\{t\}\(KL\(q^\{\*\}\_\{t\}\|\|p\_\{t\}\)\)\-KL\(q^\{\*\}\_\{t\}\|\|p^\{\*\}\_\{t\}\)\)ℰq,θ=∑t\[KL\(qt∗\|\|pt\)−KL\(qt\|\|pt\)\+KL\(qt\|\|pt\)−KL\(qt∗\|\|qt\)\+KL\(qt∗\|\|qt\)−KL\(qt∗\|\|pt∗\)\]\\mathcal\{E\}\_\{q,\\theta\}=\\sum\_\{t\}\[KL\(q^\{\*\}\_\{t\}\|\|p\_\{t\}\)\-KL\(q\_\{t\}\|\|p\_\{t\}\)\+KL\(q\_\{t\}\|\|p\_\{t\}\)\-KL\(q\_\{t\}^\{\*\}\|\|q\_\{t\}\)\+KL\(q\_\{t\}^\{\*\}\|\|q\_\{t\}\)\-KL\(q^\{\*\}\_\{t\}\|\|p^\{\*\}\_\{t\}\)\]
∑t\[KL\(qt∗\|\|pt\)−KL\(qt\|\|pt\)\]≤supπθ∈ΠOPSD\|∑t\[KL\(qt∗\|\|pf\)−KL\(qt\|\|pf\)\]\|\\sum\_\{t\}\[KL\(q^\{\*\}\_\{t\}\|\|p\_\{t\}\)\-KL\(q\_\{t\}\|\|p\_\{t\}\)\]\\leq sup\_\{\\pi\_\{\\theta\}\\in\{\\Pi\_\{\\text\{OPSD\}\}\}\}\|\\sum\_\{t\}\[KL\(q^\{\*\}\_\{t\}\|\|p\_\{f\}\)\-KL\(q\_\{t\}\|\|p\_\{f\}\)\]\|∑t\(KL\(qt\|\|pt\)−KL\(qt∗\|\|qt\)\)≤0\\sum\_\{t\}\(KL\(q\_\{t\}\|\|p\_\{t\}\)\-KL\(q\_\{t\}^\{\*\}\|\|q\_\{t\}\)\)\\leq 0∑t\(KL\(qt∗\|\|qt\)−KL\(qt∗\|\|pt∗\)\)≤supπθ∈ΠOPSD\|∑t\[KL\(qt∗\|\|pf\)−KL\(qt\|\|pf\)\]\|\\sum\_\{t\}\(KL\(q\_\{t\}^\{\*\}\|\|q\_\{t\}\)\-KL\(q^\{\*\}\_\{t\}\|\|p^\{\*\}\_\{t\}\)\)\\leq\\sup\_\{\\pi\_\{\\theta\}\\in\{\\Pi\_\{\\text\{OPSD\}\}\}\}\|\\sum\_\{t\}\[KL\(q^\{\*\}\_\{t\}\|\|p\_\{f\}\)\-KL\(q\_\{t\}\|\|p\_\{f\}\)\]\|so we have

ℰq,θ≤2∗supπθ∈ΠOPSD\|∑t\[KL\(qt∗\|\|pf\)−KL\(qt\|\|pf\)\]\|\\mathcal\{E\}\_\{q,\\theta\}\\leq 2\*\\sup\_\{\\pi\_\{\\theta\}\\in\{\\Pi\_\{\\text\{OPSD\}\}\}\}\|\\sum\_\{t\}\[KL\(q^\{\*\}\_\{t\}\|\|p\_\{f\}\)\-KL\(q\_\{t\}\|\|p\_\{f\}\)\]\|
ℰq,θ≤2∗supπθ∈ΠOPSD\|∑t\[∑iqt,i∗​l​o​g​qt,i∗−qt,i∗​l​o​g​pt,i−q^t,i​l​o​g​qt,i∗\+q^t,i​l​o​g​pt,i\]\|\\mathcal\{E\}\_\{q,\\theta\}\\leq 2\*\\sup\_\{\\pi\_\{\\theta\}\\in\{\\Pi\_\{\\text\{OPSD\}\}\}\}\|\\sum\_\{t\}\[\\sum\_\{i\}q^\{\*\}\_\{t,i\}logq^\{\*\}\_\{t,i\}\-q^\{\*\}\_\{t,i\}logp\_\{t,i\}\-\\hat\{q\}\_\{t,i\}logq^\{\*\}\_\{t,i\}\+\\hat\{q\}\_\{t,i\}logp\_\{t,i\}\]\|\(14\)
Consider an arbitrary LLM and fix an input context\. Let𝐳~=\(z~1,…,z~V\)⊤∈ℝV\\widetilde\{\\mathbf\{z\}\}=\(\\widetilde\{z\}\_\{1\},\\ldots,\\widetilde\{z\}\_\{V\}\)^\{\\top\}\\in\\mathbb\{R\}^\{V\}denote the logit vector produced by the model, whereVVis the vocabulary size\. The corresponding next\-token probability vector𝐩=\(p1,…,pV\)⊤\\mathbf\{p\}=\(p\_\{1\},\\ldots,p\_\{V\}\)^\{\\top\}is given by

pi:=exp⁡\(z~i\)∑j=1Vexp⁡\(z~j\),i∈\[V\]\.p\_\{i\}:=\\frac\{\\exp\(\\widetilde\{z\}\_\{i\}\)\}\{\\sum\_\{j=1\}^\{V\}\\exp\(\\widetilde\{z\}\_\{j\}\)\},\\qquad i\\in\[V\]\.
Let𝟏∈ℝV\\mathbf\{1\}\\in\\mathbb\{R\}^\{V\}denote the all\-ones vector\. Since softmax is invariant to a constant shift, for anyc∈ℝc\\in\\mathbb\{R\},

softmax⁡\(𝐳~\+c​𝟏\)=softmax⁡\(𝐳~\)\.\\operatorname\{softmax\}\(\\widetilde\{\\mathbf\{z\}\}\+c\\mathbf\{1\}\)=\\operatorname\{softmax\}\(\\widetilde\{\\mathbf\{z\}\}\)\.Define the centering matrix

A:=IV−1V​𝟏𝟏⊤A:=I\_\{V\}\-\\frac\{1\}\{V\}\\mathbf\{1\}\\mathbf\{1\}^\{\\top\}and the centered logit vector

𝐳:=A​𝐳~=𝐳~−\(1V​𝟏⊤​𝐳~\)​𝟏\.\\mathbf\{z\}:=A\\widetilde\{\\mathbf\{z\}\}=\\widetilde\{\\mathbf\{z\}\}\-\\left\(\\frac\{1\}\{V\}\\mathbf\{1\}^\{\\top\}\\widetilde\{\\mathbf\{z\}\}\\right\)\\mathbf\{1\}\.Then

𝟏⊤​𝐳=0,A​𝐳=𝐳,\\mathbf\{1\}^\{\\top\}\\mathbf\{z\}=0,\\qquad A\\mathbf\{z\}=\\mathbf\{z\},and

softmax⁡\(𝐳\)=softmax⁡\(𝐳~\)\.\\operatorname\{softmax\}\(\\mathbf\{z\}\)=\\operatorname\{softmax\}\(\\widetilde\{\\mathbf\{z\}\}\)\.Therefore, without loss of generality, we may work with centered logits and assume throughout that

𝐳=A​𝐳,or equivalently∑j=1Vzj=0\.\\mathbf\{z\}=A\\mathbf\{z\},\\qquad\\text\{or equivalently\}\\qquad\\sum\_\{j=1\}^\{V\}z\_\{j\}=0\.For centered logits satisfying∥𝐳∥2→0\\lVert\\mathbf\{z\}\\rVert\_\{2\}\\to 0, Taylor expansion gives

ezi=1\+zi\+12​zi2\+O⁡\(∥𝐳∥23\)\.e^\{z\_\{i\}\}=1\+z\_\{i\}\+\\frac\{1\}\{2\}z\_\{i\}^\{2\}\+O\\\!\\left\(\\lVert\\mathbf\{z\}\\rVert\_\{2\}^\{3\}\\right\)\.Consequently,

∑j=1Vezj\\displaystyle\\sum\_\{j=1\}^\{V\}e^\{z\_\{j\}\}=V\+∑j=1Vzj\+12​∑j=1Vzj2\+O⁡\(∥𝐳∥23\)\\displaystyle=V\+\\sum\_\{j=1\}^\{V\}z\_\{j\}\+\\frac\{1\}\{2\}\\sum\_\{j=1\}^\{V\}z\_\{j\}^\{2\}\+O\\\!\\left\(\\lVert\\mathbf\{z\}\\rVert\_\{2\}^\{3\}\\right\)=V\+12​∥𝐳∥22\+O⁡\(∥𝐳∥23\),\\displaystyle=V\+\\frac\{1\}\{2\}\\lVert\\mathbf\{z\}\\rVert\_\{2\}^\{2\}\+O\\\!\\left\(\\lVert\\mathbf\{z\}\\rVert\_\{2\}^\{3\}\\right\),where the second equality follows from∑j=1Vzj=0\\sum\_\{j=1\}^\{V\}z\_\{j\}=0\. Therefore,

log⁡\(∑j=1Vezj\)\\displaystyle\\log\\left\(\\sum\_\{j=1\}^\{V\}e^\{z\_\{j\}\}\\right\)=log⁡V\+log⁡\(1\+12​V​∥𝐳∥22\+O⁡\(∥𝐳∥23\)\)\\displaystyle=\\log V\+\\log\\left\(1\+\\frac\{1\}\{2V\}\\lVert\\mathbf\{z\}\\rVert\_\{2\}^\{2\}\+O\\\!\\left\(\\lVert\\mathbf\{z\}\\rVert\_\{2\}^\{3\}\\right\)\\right\)=log⁡V\+12​V​∥𝐳∥22\+O⁡\(∥𝐳∥23\)\.\\displaystyle=\\log V\+\\frac\{1\}\{2V\}\\lVert\\mathbf\{z\}\\rVert\_\{2\}^\{2\}\+O\\\!\\left\(\\lVert\\mathbf\{z\}\\rVert\_\{2\}^\{3\}\\right\)\.In particular,

pi=1V​\(1\+zi\+O⁡\(∥𝐳∥22\)\)\.p\_\{i\}=\\frac\{1\}\{V\}\\left\(1\+z\_\{i\}\+O\(\\lVert\\mathbf\{z\}\\rVert\_\{2\}^\{2\}\)\\right\)\.
For Qwen3, let𝐡∈ℝd\\mathbf\{h\}\\in\\mathbb\{R\}^\{d\}denote the final hidden representation at the current token position and letW∈ℝV×dW\\in\\mathbb\{R\}^\{V\\times d\}denote the weight matrix of its bias\-free language\-modeling head\. The original logits are𝐳~=W​𝐡\\widetilde\{\\mathbf\{z\}\}=W\\mathbf\{h\}\. Defining

A:=IV−1V​𝟏𝟏⊤,A:=I\_\{V\}\-\\frac\{1\}\{V\}\\mathbf\{1\}\\mathbf\{1\}^\{\\top\},the centered logits can be written as

𝐳:=A​𝐳~=A​W​𝐡\.\\mathbf\{z\}:=A\\widetilde\{\\mathbf\{z\}\}=AW\\mathbf\{h\}\.Since𝟏⊤​𝐳=0\\mathbf\{1\}^\{\\top\}\\mathbf\{z\}=0and softmax is invariant to constant shifts,𝐳\\mathbf\{z\}induces the same next\-token distribution as𝐳~\\widetilde\{\\mathbf\{z\}\}\.

Since OPSD employs LoRA without applying it to the output layer, the output\-layer weights remain unchanged\. Therefore, we obtain:

So we can simlify[14](https://arxiv.org/html/2610.12855#A4.E14):

ℰq,θ≤2\|𝒱\|∗supπθ∈ΠOPSD\|∑t\[\(ht−ht∗\+ht^2\)T​WT​\(I\|𝒱\|−1\|𝒱\|​1\|𝒱\|​1\|𝒱\|T\)​W​\(ht∗−h^t\)\]\|\\mathcal\{E\}\_\{q,\\theta\}\\leq\\frac\{2\}\{\|\\mathcal\{V\}\|\}\*\\sup\_\{\\pi\_\{\\theta\}\\in\{\\Pi\_\{\\text\{OPSD\}\}\}\}\|\\sum\_\{t\}\[\(h\_\{t\}\-\\frac\{h\_\{t\}^\{\*\}\+\\hat\{h\_\{t\}\}\}\{2\}\)^\{T\}W^\{T\}\(I\_\{\|\\mathcal\{V\}\|\}\-\\frac\{1\}\{\|\\mathcal\{V\}\|\}1\_\{\\mathcal\{\|V\|\}\}1\_\{\\mathcal\{\|V\|\}\}^\{T\}\)W\(h^\{\*\}\_\{t\}\-\\hat\{h\}\_\{t\}\)\]\|\(15\)
Throughout our analysis, we assume that the hidden representations are normalized using RMSNorm without learnable scaling parameters\.

By the Cauchy–Schwarz inequality, we have

ℰKD\(πθ\)≤2N​\|𝒱\|∑Nsupπθ∈ΠOPSD\|Δ\|\|\|M\|\|2\[T​d2\+12​∑\|𝒞†\|∑𝐡i∈𝒳k‖𝐡i−μk𝒳‖22−12​∑\|𝒞†\|\|𝒞k†\|​‖μk𝒳−μ‖22\+\\displaystyle\\mathcal\{E\}\_\{\\text\{KD\}\}\(\\pi\_\{\\theta\}\)\\leq\\frac\{2\}\{N\|\\mathcal\{V\}\|\}\\sum\_\{N\}\\sup\_\{\\pi\_\{\\theta\}\\in\{\\Pi\_\{\\text\{OPSD\}\}\}\}\|\\Delta\|\|\|M\|\|\_\{2\}\[\\sqrt\{\\frac\{Td\}\{2\}\+\\frac\{1\}\{2\}\\sum\_\{\|\\mathcal\{C\}^\{\\dagger\}\|\}\\sum\_\{\{\\mathbf\{h\}\_\{i\}\}\\in\\mathcal\{X\}\_\{k\}\}\|\|\\mathbf\{h\}\_\{i\}\-\\mu\_\{k\}^\{\\mathcal\{X\}\}\|\|\_\{2\}^\{2\}\-\\frac\{1\}\{2\}\\sum\_\{\|\\mathcal\{C\}^\{\\dagger\}\|\}\|\\mathcal\{C\}^\{\\dagger\}\_\{k\}\|\|\|\\mu\_\{k\}^\{\\mathcal\{X\}\}\-\\mu\|\|^\{2\}\_\{2\}\}\+\(16\)∑\|𝒞†\|\|𝒞k†\|​‖μk𝒳−μk†‖22\+∑\|𝒞†\|∑𝐡i†∈𝒳k‖𝐡i†−μk†‖22\]\\displaystyle\\sqrt\{\\sum\_\{\|\\mathcal\{C\}^\{\\dagger\}\|\}\|\\mathcal\{C\}\_\{k\}^\{\\dagger\}\|\|\|\\mu\_\{k\}^\{\\mathcal\{X\}\}\-\\mu\_\{k\}^\{\\dagger\}\|\|^\{2\}\_\{2\}\+\\sum\_\{\|\\mathcal\{C\}^\{\\dagger\}\|\}\\sum\_\{\{\\mathbf\{h\}\_\{i\}^\{\\dagger\}\}\\in\\mathcal\{X\}\_\{k\}\}\|\|\\mathbf\{h\}\_\{i\}^\{\\dagger\}\-\\mu\_\{k\}^\{\\dagger\}\|\|\_\{2\}^\{2\}\}\]

## Appendix EProof of Theorem[5\.1](https://arxiv.org/html/2610.12855#S5.Thmtheorem1)

###### Proof\.

For each cluster, we stack allhi,r​e​f∈𝒞kr​e​fh\_\{i,ref\}\\in\\mathcal\{C\}\_\{k\}^\{ref\}to formHkr​e​f=\[\(h1r​e​f\)⊤,…,\(h\|Ckr​e​f\|r​e​f\)⊤\]⊤H\_\{k\}^\{ref\}=\[\(h\_\{1\}^\{ref\}\)^\{\\top\},\\ldots,\(h\_\{\|C\_\{k\}^\{ref\}\|\}^\{ref\}\)^\{\\top\}\]^\{\\top\}, then we defineGkr​e​f=Hkr​e​f​Hkr​e​f⊤G\_\{k\}^\{ref\}=H\_\{k\}^\{ref\}H\_\{k\}^\{ref\\top\}\. Accordingly, we stack allhi,a​l​i​g​n∈𝒞ka​l​i​g​nh\_\{i,align\}\\in\\mathcal\{C\}\_\{k\}^\{align\}to formHka​l​i​g​n=\[\(h1a​l​i​g​n\)⊤,…,\(h\|Cka​l​i​g​n\|a​l​i​g​n\)⊤\]⊤H\_\{k\}^\{align\}=\[\(h\_\{1\}^\{align\}\)^\{\\top\},\\ldots,\(h\_\{\|C\_\{k\}^\{align\}\|\}^\{align\}\)^\{\\top\}\]^\{\\top\}, then we defineGka​l​i​g​n=Hka​l​i​g​n​Hka​l​i​g​n⊤G\_\{k\}^\{align\}=H\_\{k\}^\{align\}H\_\{k\}^\{align\\top\}\.

and𝟏\|Ckr​e​f\|∈ℝ\|Ckr​e​f\|\\mathbf\{1\}\_\{\|C\_\{k\}^\{ref\}\|\}\\in\\mathbb\{R\}^\{\|C\_\{k\}^\{ref\}\|\}denotes the all\-ones column vector\.\|𝒞r​e​f\|\|\\mathcal\{C\}\_\{ref\}\|denotes the number of clusters in the reference model\.

First, we can simplify the optimization objective of intra\-cluster compactness for each cluster:

\|∑𝐡i∈Cka​l​i​g​n,‖𝐡i−𝝁ka​l​i​g​n‖22−∑𝐡iref∈Ckref‖𝐡iref−𝝁kref‖22\|,≤‖I\|Cr​e​f\|−𝟏\|Ckr​e​f\|⊤​𝟏\|Ckr​e​f\|‖F​‖Gkr​e​f−Gk‖F\\left\|\\sum\_\{\\mathbf\{h\}\_\{i\}\\in C\_\{k\}^\{align\},\}\\left\\\|\\mathbf\{h\}\_\{i\}\-\\boldsymbol\{\\mu\}\_\{k\}^\{align\}\\right\\\|\_\{2\}^\{2\}\-\\sum\_\{\\mathbf\{h\}\_\{i\}^\{\\mathrm\{ref\}\}\\in C\_\{k\}^\{\\mathrm\{ref\}\}\}\\left\\\|\\mathbf\{h\}\_\{i\}^\{\\mathrm\{ref\}\}\-\\boldsymbol\{\\mu\}\_\{k\}^\{\\mathrm\{ref\}\}\\right\\\|\_\{2\}^\{2\}\\right\|,\\leq\|\|I\_\{\|C\_\{ref\}\|\}\-\\mathbf\{1\}\_\{\|C^\{ref\}\_\{k\}\|\}^\{\\top\}\\mathbf\{1\}\_\{\|C^\{ref\}\_\{k\}\|\}\|\|\_\{F\}\|\|G^\{ref\}\_\{k\}\-G\_\{k\}\|\|\_\{F\}\(17\)
Second,Gka​l​i​g​nG\_\{k\}^\{align\}is a principal submatrix ofH​HTHH^\{T\}andGkr​e​fG\_\{k\}^\{ref\}is a principal submatrix ofH​HTHH^\{T\}\. Because the squared Frobenius norm is additive over pairwise disjoint principal submatrices, Therefore, for all clusters, we can obtain this through the Cauchy\-Schwarz inequality:

∑k‖I\|Ckr​e​f\|−𝟏\|Ckr​e​f\|⊤​𝟏\|Ckr​e​f\|‖F​‖Gkr​e​f−Gka​l​i​g​n‖F≤\(T−\|Cr​e​f\|\)12\|\|∑k\(Gkr​e​f−Gka​l​i​g​n\)\|\|F\\sum\_\{k\}\|\|I\_\{\|C\_\{k\}^\{ref\}\|\}\-\\mathbf\{1\}\_\{\|C\_\{k\}^\{ref\}\|\}^\{\\top\}\\mathbf\{1\}\_\{\|C\_\{k\}^\{ref\}\|\}\|\|\_\{F\}\|\|G\_\{k\}^\{ref\}\-G\_\{k\}^\{align\}\|\|\_\{F\}\\leq\(T\-\|C\_\{ref\}\|\)^\{\\frac\{1\}\{2\}\}\|\|\\sum\_\{k\}\(G\_\{k\}^\{ref\}\-G\_\{k\}^\{align\}\)\|\|\_\{F\}
Third, through the inclusion of the pairwise disjoint principal submatrices, we can get the upper bound ofℒi​n​t​r​a\\mathcal\{L\}\_\{intra\}:

≤\(T−\|𝒞r​e​f\|\)12​‖H​HT−Hr​e​f​Hr​e​f⊤‖F\\displaystyle\\leq\(T\-\|\\mathcal\{C\}\_\{ref\}\|\)^\{\\frac\{1\}\{2\}\}\|\|HH^\{T\}\-H\_\{ref\}H\_\{ref\}^\{\\top\}\|\|\_\{F\}\(18\)∎

## Appendix FProof of Proposition[5\.2](https://arxiv.org/html/2610.12855#S5.Thmtheorem2)

###### Proof\.

Through the definition of the Laplacian matrix and the Cauchy\-Schwarz inequality, we can obtain:

Let𝟏T∈ℝ𝕋\\mathbf\{1\}\_\{T\}\\in\\mathbb\{R^\{T\}\}denote the all\-ones column vector\.

‖L−Lr​e​f‖F=‖d​i​a​g​\(A​𝟏T\)−d​i​a​g​\(Ar​e​f​𝟏T\)\+A−Ar​e​f‖F\\displaystyle\|\|L\-L^\{ref\}\|\|\_\{F\}=\|\|diag\(A\\mathbf\{1\}\_\{T\}\)\-diag\(A^\{ref\}\\mathbf\{1\}\_\{T\}\)\+A\-A^\{ref\}\|\|\_\{F\}≤‖d​i​a​g​\(A−Ar​e​f\)​𝟏T‖F\+\|\|A−Ar​e​f\|\|F,\\displaystyle\\leq\|\|diag\(A\-A^\{ref\}\)\\mathbf\{1\}\_\{T\}\|\|\_\{F\}\+\|\|A\-A^\{ref\}\|\|\_\{F\},
‖d​i​a​g​\(A−Ar​e​f\)​𝟏T‖F=‖A−Ar​e​f‖2≤T\|\|A−Ar​e​f\|\|F\|\|diag\(A\-A^\{ref\}\)\\mathbf\{1\}\_\{T\}\|\|\_\{F\}=\|\|A\-A^\{ref\}\|\|\_\{2\}\\leq\\sqrt\{T\}\|\|A\-A^\{ref\}\|\|\_\{F\}‖d​i​a​g​\(A−Ar​e​f\)​𝟏T‖F\+‖A−Ar​e​f‖F≤\(T\+1\)\|\|A−Ar​e​f\|\|F\|\|diag\(A\-A^\{ref\}\)\\mathbf\{1\}\_\{T\}\|\|\_\{F\}\+\|\|A\-A^\{ref\}\|\|\_\{F\}\\leq\(\\sqrt\{T\}\+1\)\|\|A\-A^\{ref\}\|\|\_\{F\}\(T\+1\)​‖A−Ar​e​f‖F≤1\+T2​σ2​‖D−Dr​e​f‖F\(\\sqrt\{T\}\+1\)\|\|A\-A^\{ref\}\|\|\_\{F\}\\leq\\frac\{1\+\\sqrt\{T\}\}\{2\\sigma^\{2\}\}\|\|D\-D^\{ref\}\|\|\_\{F\}
‖D−Dr​e​f‖F=‖d​i​a​g​\{H​HT\}​𝟏TT\+𝟏T​d​i​a​g​\{H​HT\}−2​H​HT−d​i​a​g​\{Hr​e​f​Hr​e​fT\}​1T−1​d​i​a​g​\{Hr​e​f​Hr​e​fT\}\+2​Hr​e​f​Hr​e​fT‖F\|\|D\-D^\{ref\}\|\|\_\{F\}=\|\|diag\\\{HH^\{T\}\\\}\\mathbf\{1\}\_\{T\}^\{T\}\+\\mathbf\{1\}\_\{T\}diag\\\{HH^\{T\}\\\}\-2HH^\{T\}\-diag\\\{H\_\{ref\}H\_\{ref\}^\{T\}\\\}1^\{T\}\-1diag\\\{H\_\{ref\}H\_\{ref\}^\{T\}\\\}\+2H\_\{ref\}H\_\{ref\}^\{T\}\|\|\_\{F\}Through Cauchy\-Schwarz inequality, we have

∑i\|λ⁡\(L\)−λ⁡\(Lr​e​f\)\|2≤\(1\+Tσ\)2​‖H​HT−Hr​e​f​Hr​e​fT‖F\\sqrt\{\\sum\_\{i\}\|\\lambda\(L\)\-\\lambda\(L^\{ref\}\)\|^\{2\}\}\\leq\(\\frac\{1\+\\sqrt\{T\}\}\{\\sigma\}\)^\{2\}\|\|HH^\{T\}\-H\_\{ref\}H\_\{ref\}^\{T\}\|\|\_\{F\}∎