Hybrid-Policy Self-Editing for Composable Unstructured Knowledge Editing
摘要
This paper introduces HPSE, a hybrid-policy self-editing method for composable unstructured knowledge editing in LLMs, enabling edited models to answer atomic questions and perform multi-hop reasoning about injected knowledge. The approach uses proactive self-distillation with hybrid rollouts and is validated across multiple LLM backbones and editors.
arXiv:2608.11660v1 Announce Type: new
Abstract: Large language models (LLMs) achieve remarkable performance across natural language tasks, yet they are trained on static corpora and their knowledge quickly becomes outdated in a fast-changing world. This motivates knowledge editing (KE), which updates specific knowledge in an LLM without changing unrelated others. Recent works move from structured knowledge triples toward unstructured KE (UKE), where the edit is a free-form passage that may state multiple facts at once. Nonetheless, existing editors inject such a passage yet fail to use it: the edited model can recall the passage, but can neither answer atomic questions about its facts nor compose them into multi-hop reasoning. We attribute this missing property, which we term composability, to editors' passive reliance on the fixed passage as the sole learning source. In response, we cast editing as a proactive self-distillation from a privileged in-context state of the same model, which requires no external supervision. We further reveal that due to the novelty of the injected knowledge, the pre-edited model's own rollouts rarely cover it, which limits the effectiveness of pure on-policy distillation. To close this gap, we propose HPSE, which builds a hybrid rollout that steps in to place missing facts onto the student's own trajectory precisely where its coverage fails, while staying on-policy elsewhere. We theoretically analyze HPSE's advantage over pure on-policy distillation, and empirically establish its plug-and-play improvements across four LLM backbones and two KE editors under various scenarios.
查看缓存全文
缓存时间: 2026/08/13 15:28
# Hybrid-Policy Self-Editing for Composable Unstructured Knowledge Editing Source: [https://arxiv.org/html/2608.11660](https://arxiv.org/html/2608.11660) Tianci Liu1,3Zihan Dong2Tianchun Li3Yi\-Chung Chen3Qiming Cao3Xingchen Wang3Shiyang Wang3Zichen Miao3Linjun Zhang2Haoyu Wang4Jing Gao3 1University of Tennessee2Rutgers University3Purdue University4University at Albany ###### Abstract Large language models \(LLMs\) achieve remarkable performance across natural language tasks, yet they are trained on static corpora and their knowledge quickly becomes outdated in a fast\-changing world\. This motivates knowledge editing \(KE\), which updates specific knowledge in an LLM without changing unrelated others\. Recent works move from structured knowledge triples toward unstructured KE \(UKE\), where the edit is a free\-form passage that may state multiple facts at once\. Nonetheless, existing editors inject such a passage yet fail to*use*it: the edited model can recall the passage, but can neither answer atomic questions about its facts nor compose them into multi\-hop reasoning\. We attribute this missing property, which we term*composability*, to editors’ passive reliance on the fixed passage as the sole learning source\. In response, we cast editing as a proactive self\-distillation from a privileged in\-context state of the same model, which requires no external supervision\. We further reveal that due to the novelty of the injected knowledge, the pre\-edited model’s own rollouts rarely cover it, which limits the effectiveness of pure on\-policy distillation\. To close this gap, we propose HPSE, which builds a hybrid rollout that steps in to place missing facts onto the student’s own trajectory precisely where its coverage fails, while staying on\-policy elsewhere\. We theoretically analyze HPSE’s advantage over pure on\-policy distillation, and empirically establish its plug\-and\-play improvements across four LLM backbones and two KE editors under various scenarios\. Keywords: Knowledge Editing, Large Language Models =Date: August 12, 2026 =Code & Datasets:[https://github\.com/lliutianc/hpse](https://github.com/lliutianc/hpse) =Contact:[tliu43@tennessee\.edu](mailto:[email protected]) ## 1Introduction Large language models \(LLMs\)\([64](https://arxiv.org/html/2608.11660#bib.bib22),[4](https://arxiv.org/html/2608.11660#bib.bib23)\)demonstrate strong generalization across a wide range of language tasks\([54](https://arxiv.org/html/2608.11660#bib.bib79),[27](https://arxiv.org/html/2608.11660#bib.bib80)\), establishing themselves as a new foundation for modern NLP\([3](https://arxiv.org/html/2608.11660#bib.bib81),[92](https://arxiv.org/html/2608.11660#bib.bib82)\)\. As model sizes grow, LLMs further exhibit emergent abilities to follow natural language instructions\([14](https://arxiv.org/html/2608.11660#bib.bib83),[50](https://arxiv.org/html/2608.11660#bib.bib24)\), enabling zero\-shot adaptation to unseen tasks\([32](https://arxiv.org/html/2608.11660#bib.bib84),[5](https://arxiv.org/html/2608.11660#bib.bib25)\)\. However, real\-world LLM deployment remains far from resolved unless LLMs can continually refresh and internalize new knowledge beyond their training data\. In a fast\-changing world, an LLM’s*static*knowledge quickly goes out of date, leading to factual errors or even unsafe generations\([11](https://arxiv.org/html/2608.11660#bib.bib65),[22](https://arxiv.org/html/2608.11660#bib.bib15)\)\. Yet retraining the model to absorb each update is prohibitively expensive\. As a remedy,*Knowledge Editing*\(KE\) has been proposed to address this by updating an LLM with a*specific*piece of knowledge\([68](https://arxiv.org/html/2608.11660#bib.bib30),[86](https://arxiv.org/html/2608.11660#bib.bib14)\)\. Given a direct, well\-structured statement of new knowledge, e\.g\., a knowledge triple naming the new owner of a recently acquired company, KE typically seeks*selective*parameter updates that keep unrelated knowledge and general capabilities intact\. To name a few,[45](https://arxiv.org/html/2608.11660#bib.bib1),[16](https://arxiv.org/html/2608.11660#bib.bib2)apply closed\-form updates to a few causally traced MLP layers that serve as knowledge storage\([10](https://arxiv.org/html/2608.11660#bib.bib85)\),[65](https://arxiv.org/html/2608.11660#bib.bib53)directly learn sparse low\-rank updates\.[40](https://arxiv.org/html/2608.11660#bib.bib38)steer selected representations with non\-linear updates\. Recent works move towards the more realistic setting of*unstructured*KE \(UKE\), where the new knowledge spans multiple aspects in free\-form text\. For example, an actual acquisition announcement is more likely to be a free\-form passage that states multiple entangled facts at once, such as the new parent company, the incoming CEO, and the reshaped reporting structure, with no specification of which facts a user may later query\. UKE pursues this more challenging objective: treating the announcement passage itself as the knowledge to edit\([71](https://arxiv.org/html/2608.11660#bib.bib5)\)\. To this end,[12](https://arxiv.org/html/2608.11660#bib.bib3)extend the edit from a few MLP layers to entire transformer blocks for larger capacity\.[28](https://arxiv.org/html/2608.11660#bib.bib4)decompose the passage into sequential chunks for iterative editing; and[60](https://arxiv.org/html/2608.11660#bib.bib37)preserve the dependencies among chunk\-wise updates\. These fine\-grained designs have collectively demonstrated success in making KE*precise*\. Nonetheless, the edited knowledge turns out largely*useless*compared to knowledge acquired through pretraining in two aspects\. The first is the loss of*decomposition*: after internalizing a long\-form edit, the model can at best recall the whole passage yet cannot answer targeted questions about the individual facts it encodes\([93](https://arxiv.org/html/2608.11660#bib.bib7),[67](https://arxiv.org/html/2608.11660#bib.bib44)\)\. The second is the loss of*composition*: having learned individual facts, the model cannot compose them for multi\-hop reasoning\([91](https://arxiv.org/html/2608.11660#bib.bib6),[90](https://arxiv.org/html/2608.11660#bib.bib39)\)\. Both phenomena have been analyzed in the literature, but mostly in isolation under separate scenarios\([85](https://arxiv.org/html/2608.11660#bib.bib18),[79](https://arxiv.org/html/2608.11660#bib.bib46)\), despite that they co\-occur and are amplified in UKE\. We use*composability*to denote whether edited knowledge, like its pretrained counterpart, can function as atomic facts that are flexibly extracted and recombined, and study decomposition and composition from this unified perspective\. In particular, we introduce an*untargeted*regime for UKE: a free\-form knowledge statement is paired with a generic editing prompt that does not specify which facts are updated, mirroring practical requests where annotators offer only a brief summary of the new passage\. We recast two representative benchmarks into a decomposition probe\([12](https://arxiv.org/html/2608.11660#bib.bib3)\)and a composition probe\([90](https://arxiv.org/html/2608.11660#bib.bib39)\), respectively\. We find that the evaluated methods, despite their different mechanisms, consistently fall short of composability under our regime\. We attribute this failure to their*passive*reliance on the provided editing context as the sole learning source, which induces severe memorization and poor generalization\([85](https://arxiv.org/html/2608.11660#bib.bib18),[39](https://arxiv.org/html/2608.11660#bib.bib19),[8](https://arxiv.org/html/2608.11660#bib.bib86)\); existing augmentation\-based remedies\([93](https://arxiv.org/html/2608.11660#bib.bib7),[79](https://arxiv.org/html/2608.11660#bib.bib46),[67](https://arxiv.org/html/2608.11660#bib.bib44)\)alleviate but do not eliminate this reliance\. In summary, composability remains an open challenge, necessitating a generalizable UKE method; Section[2](https://arxiv.org/html/2608.11660#S2)details these findings\.*Our untargeted regime and formulation of composability offer a unified view of what UKE requires in the wild\. Together with our benchmarking protocol, these constitute the first key contribution of this work\.* Given the pivotal role of the editing context, we pursue a*proactive*strategy through on\-policy self\-distillation \(OPSD\): the edited model generates its own response trajectories, while a*privileged*state of the same model \(one that reads the new passage in context\) provides token\-level distillation targets\([41](https://arxiv.org/html/2608.11660#bib.bib74),[59](https://arxiv.org/html/2608.11660#bib.bib48),[88](https://arxiv.org/html/2608.11660#bib.bib50)\)\. Although OPSD has proved effective for adapting behavioral patterns in post\-training\([8](https://arxiv.org/html/2608.11660#bib.bib86),[41](https://arxiv.org/html/2608.11660#bib.bib74),[59](https://arxiv.org/html/2608.11660#bib.bib48)\), it falls short in UKE: because the injected knowledge is novel, the model’s own rollouts often go off\-topic, yielding limited corrective supervision\([17](https://arxiv.org/html/2608.11660#bib.bib72),[84](https://arxiv.org/html/2608.11660#bib.bib71)\)\. We term this a*coverage failure*\. To mitigate this issue, we propose HPSE, which asks the privileged model not only to grade the edited model’s own responses, but also to*step in*when a rollout deviates too far from a useful trajectory\. This strategy forms hybrid, rather than purely on\-policy, rollouts with better coverage, offering stronger supervision for UKE\. We further support HPSE with theoretical justification\. Notably, HPSE is a data\-centric approach: it replaces only the training signal, making no assumption about which parameters are updated or how the update is parameterized, thereby offering plug\-and\-play flexibility across gradient\-based KE editors\.*The proposed HPSE and our theoretical analysis constitute the second key contribution of this work\.* The remainder of this paper is organized as follows\. Sections[2](https://arxiv.org/html/2608.11660#S2)and[3](https://arxiv.org/html/2608.11660#S3)detail composable UKE and the proposed HPSE, respectively\. Section[4](https://arxiv.org/html/2608.11660#S4)evaluates HPSE across various settings\. Finally, Section[5](https://arxiv.org/html/2608.11660#S5)reviews related work, and Section[6](https://arxiv.org/html/2608.11660#S6)concludes the paper\. ## 2Composability Challenge in Unstructured Knowledge Editing This section presents composability, a critical requirement for practical unstructured knowledge editing \(UKE\) that has been largely overlooked in the literature\. Background on LLMs is also provided\. ### 2\.1Preliminaries Given a text𝐱=\(x1,…,xn\)\\mathbf\{x\}=\(x\_\{1\},\\dots,x\_\{n\}\), where eachxi∈𝒱x\_\{i\}\\in\\mathcal\{V\}is a token from vocabulary𝒱\\mathcal\{V\}, a large language model \(LLM\) parameterized byθ\\thetacomputes probabilityπθ\(𝐱\)\\pi\_\{\\theta\}\(\\mathbf\{x\}\)based on the chain rule\([2](https://arxiv.org/html/2608.11660#bib.bib26)\): πθ\(𝐱\)\\displaystyle\\pi\_\{\\theta\}\(\\mathbf\{x\}\)=∏i=1nπθ\(xi∣x1,…,xi−1\)≜∏i=1nπθ\(xi∣𝐱<i\),\\displaystyle=\\prod\_\{i=1\}^\{n\}\\pi\_\{\\theta\}\(x\_\{i\}\\mid\{x\_\{1\},\\dots,x\_\{i\-1\}\}\)\\triangleq\\prod\_\{i=1\}^\{n\}\\pi\_\{\\theta\}\(x\_\{i\}\\mid\\mathbf\{x\}\_\{<i\}\),whereπθ\(xi∣𝐱<i\)\\pi\_\{\\theta\}\(x\_\{i\}\\mid\\mathbf\{x\}\_\{<i\}\)is the predicted distribution of tokenxix\_\{i\}given the previous𝐱<i\\mathbf\{x\}\_\{<i\}\. The LLM is usually trained with maximum likelihood estimation\([24](https://arxiv.org/html/2608.11660#bib.bib27),[61](https://arxiv.org/html/2608.11660#bib.bib28),[7](https://arxiv.org/html/2608.11660#bib.bib29),[81](https://arxiv.org/html/2608.11660#bib.bib42)\)\. To generate a sentence𝐱\\mathbf\{x\}, the LLM computesπθ\(xi∣𝐱<i\)\\pi\_\{\\theta\}\(x\_\{i\}\\mid\\mathbf\{x\}\_\{<i\}\)and drawsxix\_\{i\}from it; thenxix\_\{i\}is combined with𝐱<i\\mathbf\{x\}\_\{<i\}as new inputs for future steps\. This process terminates when a special token that marks the end of the sentence is generated, or when the maximum length is reached\. Knowledge Editing \(KE\)aims to update a pre\-trained LLM’s specific knowledge precisely while preserving other knowledge unrelated to the update, a requirement known as*locality*\([68](https://arxiv.org/html/2608.11660#bib.bib30),[86](https://arxiv.org/html/2608.11660#bib.bib14)\)\. Structured KE expresses a piece of knowledge as a triple*\(subject, relation, object\)*and requires updating*object*to a new value\. Given a natural\-language pair\(𝐱,𝐲\)\(\\mathbf\{x\},\\mathbf\{y\}\)where the editing prompt𝐱\\mathbf\{x\}describes a*subject*and*relation*, and𝐲\\mathbf\{y\}specifies the corresponding*object*, KE asks the LLM to respond to𝐱\\mathbf\{x\}with the edited object𝐲\\mathbf\{y\}\. Built upon KE,Unstructured KE \(UKE\)studies a more realistic regime, where a free\-form passage𝐜\\mathbf\{c\}encodes*one*or*multiple*pieces of new knowledge, with no labeled knowledge triple, making it more challenging\([71](https://arxiv.org/html/2608.11660#bib.bib5),[77](https://arxiv.org/html/2608.11660#bib.bib43),[28](https://arxiv.org/html/2608.11660#bib.bib4)\)\. ### 2\.2Formulating and Benchmarking UKE Composability Composability Definition\.KE is useful only if the knowledge it injects can be*used*by the LLM, not merely*memorized*in a way that can only be repeated under the exact query used for editing\([91](https://arxiv.org/html/2608.11660#bib.bib6),[12](https://arxiv.org/html/2608.11660#bib.bib3),[85](https://arxiv.org/html/2608.11660#bib.bib18),[90](https://arxiv.org/html/2608.11660#bib.bib39),[93](https://arxiv.org/html/2608.11660#bib.bib7)\)\. Crucially, when it comes to UKE, where multiple pieces of knowledge are injected at once, a*useful*edit entails the following two aspects, which we collectively refer to as*composability*\. - •Decomposition:Multiple pieces of knowledge are often encoded in the passage𝐜\\mathbf\{c\}and are injected jointly by editing the LLM with𝐜\\mathbf\{c\}directly\([12](https://arxiv.org/html/2608.11660#bib.bib3)\)\. In this case, an ideal LLM should be able to decompose these facts and absorb them individually\. After editing, any query about an atomic fact in the passage should be answered with the fact directly, rather than by repeating the whole passage\([93](https://arxiv.org/html/2608.11660#bib.bib7),[67](https://arxiv.org/html/2608.11660#bib.bib44)\)\. - •Composition:Different pieces of new knowledge are usually related\. Consequently, the edited model should be able to proactively combine them for multi\-hop reasoning\. This is closely related to the notion of*portability*\([91](https://arxiv.org/html/2608.11660#bib.bib6),[69](https://arxiv.org/html/2608.11660#bib.bib45),[90](https://arxiv.org/html/2608.11660#bib.bib39)\)\. Untargeted Regime\.We further consider*how*the new knowledge is presented for editing\. In practice, human labelers may want to only summarize the passage briefly, instead of detailing which contents constitute the*updated*knowledge\. To reflect this demand, we formulate UKE in an*untargeted*regime: the passage𝐜\\mathbf\{c\}is paired with a generic editing instruction such as “introduce X”, see Appendix[B](https://arxiv.org/html/2608.11660#A2)for detailed examples\. Benchmarking Composability\.We next conduct a systematic study on UKE composability\. Guided by above principles, we transform existing KE benchmarks into better \(de\)composition probes\. More details on our benchmark construction can be found in Appendix[B](https://arxiv.org/html/2608.11660#A2)\. - •Decomposition probeusesUnKEBench\([12](https://arxiv.org/html/2608.11660#bib.bib3)\), where each sample involves a free\-form passage that contains five atomic facts\. After editing, the LLM is required to answer both the passage\-level question that requires*joint recall*of all facts\([47](https://arxiv.org/html/2608.11660#bib.bib8)\), and the targeted questions about each individual fact, which require*decomposed*recall\. Unlike the original benchmark, whose editing prompt reveals the updated facts, we summarize the*editing*prompt into a more generic description, in line with our untargeted regime\. The testing queries are left unchanged\. To better measure actual knowledge injection rather than lexical repetition of the passage, we use LLM\-as\-Judge\-based scoring, following[47](https://arxiv.org/html/2608.11660#bib.bib8),[12](https://arxiv.org/html/2608.11660#bib.bib3),[77](https://arxiv.org/html/2608.11660#bib.bib43)\. - •Composition proberecastsMQuAKE\-CF\-remastered\([90](https://arxiv.org/html/2608.11660#bib.bib39)\), a cleaned version of the original\([91](https://arxiv.org/html/2608.11660#bib.bib6)\)\. Each editing request in MQuAKE involves 2–4 pieces of new knowledge, each expressed as a separate single\-sentence statement\. After editing, the LLM is required to answer each single\-hop question*individually*, and to*compose*them to solve multi\-hop questions\. In our benchmark, we expand each single statement into an unstructured passage following the protocol of[71](https://arxiv.org/html/2608.11660#bib.bib5)\. Again, we use a generic editing prompt, with testing queries unchanged\. We adopt the original evaluation pipeline from the official benchmark\. Figure 1:Composability results \(Qwen2\.5\)\. Existing methods fail to handle \(de\)composition\.Benchmarking Results\.We benchmark seven representative KE methods on editing Qwen2\.5\-7B\-Instruct\([52](https://arxiv.org/html/2608.11660#bib.bib10)\), with results presented in Figure[1](https://arxiv.org/html/2608.11660#S2.F1): Jnt\. and Dmp\. denote the passage\-level and fact\-level accuracy of the decomposition probe, Ind\. and Cmp\. denote the single\-hop and multi\-hop accuracy of the composition probe, and Locality is measured by MMLU\. The numeric results are deferred to Section[4](https://arxiv.org/html/2608.11660#S4)\. All editors exhibit notable failures in composability\. Some editors, even those originally designed for UKE \(e\.g\., AnyEdit\([28](https://arxiv.org/html/2608.11660#bib.bib4)\)and UnKE\([12](https://arxiv.org/html/2608.11660#bib.bib3)\)\), fall short of recalling the new knowledge in either a joint or a decomposed way, and the structured editors fare no better\. COIN⋆\([93](https://arxiv.org/html/2608.11660#bib.bib7)\), an NTP\-based editor that fine\-tunes on the passage while regularizing against context reliance, achieves the highest decomposed recall \(Dmp\. 53\), though largely by reproducing the whole passage \(as shown later in Section[4](https://arxiv.org/html/2608.11660#S4)\), and it composes little of the injected knowledge either\. Composition also proves hard, with the strongest baseline reaching merely 32\. These failures are exemplified by cases in Figure[2](https://arxiv.org/html/2608.11660#S2.F2)\. On the*decomposition*side, COIN⋆reproduces the injected passage instead of answering the atomic question, carrying the target answer only indirectly; on the*composition*side, AnyEdit cannot chain the two injected edits and falls back on the model’s prior knowledge\. In conclusion, composability remains an open challenge for UKE, leaving a substantial gap between*passively*injecting knowledge and*proactively*using it\. In the next section, we propose our solution\. Failure cases of UKE Composability\.\(a\) Decomposition\.Edit passage \(5 facts\):Elvin Penner is a songwriter who has written hit songs for popular artists such as Taylor Swift and Ed Sheeran…Question:*Which artists has Elvin Penner written hit songs for?*Target:*Taylor Swift and Ed Sheeran\.*COIN⋆✗Elvin Penner is a well\-known songwriter in the music industry\. He has written…\(whole passage repetition\)\(b\) Composition\.Edit passages \(2, one in each\):the religion Louis XV practiced→\\rightarrow*Christian Science*; the founder of Christian Science→\\rightarrow*Ibn Saud*\.Multi\-hop question:*Who founded the religion Louis XV practiced?*Target:*Ibn Saud\.*AnyEdit✗Louis XV was a prominent figure inFrench Catholicism…theJansenist movement…*\(prior knowledge\)*Figure 2:Existing KE methods fall short in handling composability, either failing to answer targeted question directly, or to reason properly with new knowledge\. Errors are highlighted\. ## 3Proposed Method In this section, we propose HPSE, which incorporates*proactive*knowledge use at editing time toward better UKE composability\. Theoretical analysis is also provided\. ### 3\.1Pursuing Composability via Self\-Distillation At a high level, the composability failures above share a common cause: minimizing−logπθ\(𝐜∣𝐱\)\-\\log\\pi\_\{\\theta\}\(\\mathbf\{c\}\\mid\\mathbf\{x\}\)supervises the new facts only under the passage’s own prefixes\. Data augmentation enlarges this context set\([93](https://arxiv.org/html/2608.11660#bib.bib7),[79](https://arxiv.org/html/2608.11660#bib.bib46),[67](https://arxiv.org/html/2608.11660#bib.bib44)\); however, the model remains a*passive*learner from externally fixed targets, limiting its ability to generalize beyond the contexts observed during editing\. Consequently, knowledge injected in this way can remain bound to the contexts in which it was learned\([93](https://arxiv.org/html/2608.11660#bib.bib7)\)\. The recently emerging on\-policy distillation \(OPD\) enables*proactive*learning: rather than fitting externally specified signals, the student generates its own rollout for an external teacher to correct, an approach that has proved effective in enhancing generalization\([59](https://arxiv.org/html/2608.11660#bib.bib48),[37](https://arxiv.org/html/2608.11660#bib.bib49)\)\. Such proactivity lets the edited model invoke the new knowledge in its own generation rather than merely reproduce it under a fixed context, paving a promising path toward better composability\. However, OPD assumes a teacher that already masters the targeted knowledge\. When no such teacher is available111KE is such a case, as the knowledge to edit is by definition novel\., on\-policy self\-distillation\([88](https://arxiv.org/html/2608.11660#bib.bib50), OPSD;\)turns the student itself into the teacher through its in\-context learning capability\([89](https://arxiv.org/html/2608.11660#bib.bib20)\)\. Specifically, given the editing passage𝐜\\mathbf\{c\}and editing prompt𝐱\\mathbf\{x\}, the*privileged*modelπ⋆\\pi^\{\\star\}is defined as the base modelπ0\\pi\_\{0\}reading𝐜\\mathbf\{c\}in context, and OPSD then trains the edited modelπθ\\pi\_\{\\theta\}to matchπ⋆\\pi^\{\\star\}by minimizing the following loss 𝒥\(θ\)=𝔼𝐲∼πθ\(⋅∣𝐱\)\[∑t=1\|𝐲\|DKL\[π⋆\(⋅∣𝐱,𝐲<t\)∥πθ\(⋅∣𝐱,𝐲<t\)\]\],whereπ⋆\(⋅∣𝐱\)≜π0\(⋅∣𝐜,𝐱\)\.\\displaystyle\\mathcal\{J\}\(\\theta\)=\{\\E\}\_\{\\mathbf\{y\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid\\mathbf\{x\}\)\}\\left\[\\sum\_\{t=1\}^\{\|\\mathbf\{y\}\|\}\{D\_\{\\text\{KL\}\}\}\\big\[\\pi^\{\\star\}\(\\cdot\\mid\\mathbf\{x\},\\mathbf\{y\}\_\{<t\}\)\\,\\\|\\,\\pi\_\{\\theta\}\(\\cdot\\mid\\mathbf\{x\},\\mathbf\{y\}\_\{<t\}\)\\big\]\\right\],\\quad\\text\{where\}\\quad\\pi^\{\\star\}\(\\cdot\\mid\\mathbf\{x\}\)\\triangleq\\pi\_\{0\}\(\\cdot\\mid\\mathbf\{c\},\\mathbf\{x\}\)\.\(1\)Here the forward Kullback–Leibler divergenceDKLD\_\{\\text\{KL\}\}matches the privileged distribution in a mass\-covering manner\([37](https://arxiv.org/html/2608.11660#bib.bib49)\)\. As𝐲\\mathbf\{y\}is drawn fromπθ\\pi\_\{\\theta\}itself, the model learns to proactively correct its own output\. Figure 3:OPSD alone brings no consistent gain\.Challenge: The Coverage Failure of OPSD\.Unfortunately, OPSD brings no consistent gain on UKE\. As shown in Figure[3](https://arxiv.org/html/2608.11660#S3.F3), it improves decomposed recall, yet degrades joint and individual recall, exhibiting a failure mode distinct from its post\-training successes\. We provide an intuitive explanation\. Since the new knowledge in𝐜\\mathbf\{c\}is by definition absent from the pre\-edited model, its rollout may follow a path distant from𝐜\\mathbf\{c\}: the response goes*off\-topic*, largely irrelevant to the new facts rather than merely outdated, leaving the privileged model little room to provide corrective signal\. Figure[12](https://arxiv.org/html/2608.11660#A4.F12)shows a coverage failure case\. In summary, OPSD proves insufficient to unlock composability, which motivates HPSE below\. ### 3\.2HPSE: A Plug\-and\-Play Solution Given the coverage failure in UKE, we proposeHybrid\-PolicySelf\-Editing \(HPSE\) to supply self\-distillation with better rollouts\. HPSE is a plug\-and\-play approach that improves the editing signal with hybrid rollout data, and thus applies to a wide range of gradient\-based KE editors\. Algorithm[1](https://arxiv.org/html/2608.11660#alg1)outlines its process\. Specifically, HPSE constructs a*hybrid*rollout that mixes token\-level outputs from the edited model and the privileged state, which is built autoregressively as follows\. At steptt, given the running prefix𝐲<t\\mathbf\{y\}\_\{<t\}, we drawyty\_\{t\}from the hybrid rollout policyπρ\\pi\_\{\\rho\}, a per\-token switch between the two policies, yt∼πρ\(⋅∣𝐱,𝐲<t\)≜\{π⋆\(⋅∣𝐱,𝐲<t\)if*step\-in*is triggered,πθ\(⋅∣𝐱,𝐲<t\)otherwise\.\\displaystyle y\_\{t\}\\;\\sim\\;\\pi\_\{\\rho\}\(\\cdot\\mid\\mathbf\{x\},\\mathbf\{y\}\_\{<t\}\)\\;\\triangleq\\;\\begin\{cases\}\\;\\pi^\{\\star\}\(\\cdot\\mid\\mathbf\{x\},\\mathbf\{y\}\_\{<t\}\)&\\text\{if \\emph\{step\-in\} is triggered\},\\\\\[3\.0pt\] \\;\\pi\_\{\\theta\}\(\\cdot\\mid\\mathbf\{x\},\\mathbf\{y\}\_\{<t\}\)&\\text\{otherwise\}\.\\end\{cases\}\(2\)Here the*step\-in*is triggered when the privileged model and the student significantly disagree, and the privileged is*confident*in its own prediction\. Intuitively, this repairs the off\-topic rollout with a minimal change: it lets the privileged model place the missing facts onto the student’s own trajectory precisely where the student would otherwise stray from𝐜\\mathbf\{c\}\. In practice, we trigger the*step\-in*if logπ⋆\(yt⋆∣𝐱,𝐲<t\)−logπθ\(yt⋆∣𝐱,𝐲<t\)⏟privileged–student gap\>τandπ⋆\(yt⋆∣𝐱,𝐲<t\)⏟privileged confidence\>κ,\\displaystyle\\underbrace\{\\log\\pi^\{\\star\}\(y^\{\\star\}\_\{t\}\\mid\\mathbf\{x\},\\mathbf\{y\}\_\{<t\}\)\-\\log\\pi\_\{\\theta\}\(y^\{\\star\}\_\{t\}\\mid\\mathbf\{x\},\\mathbf\{y\}\_\{<t\}\)\}\_\{\\text\{privileged\-\-student gap\}\}\>\\tau\\;\\;\\text\{and\}\\;\\;\\underbrace\{\\pi^\{\\star\}\(y^\{\\star\}\_\{t\}\\mid\\mathbf\{x\},\\mathbf\{y\}\_\{<t\}\)\}\_\{\\text\{privileged confidence\}\}\>\\kappa,\(3\)whereyt⋆≜argmaxyπ⋆\(y∣𝐱,𝐲<t\)y^\{\\star\}\_\{t\}\\triangleq\\arg\\max\_\{y\}\\pi^\{\\star\}\(y\\mid\\mathbf\{x\},\\mathbf\{y\}\_\{<t\}\)andτ,κ\\tau,\\kappaare hyperparameters\. The first criterion locates the tokens the student is about to miss, and the second ensures the privileged next\-token prediction is itself reliable\. Note that the confidence can also be quantified with other measures such as the privileged distribution entropy\([30](https://arxiv.org/html/2608.11660#bib.bib76)\)\. We adopt max probability due to its simplicity\. As the KE process proceeds, the edited model injects the new knowledge, leading to fewer step\-in triggers\. Consequently, the hybrid distribution gradually converges to the pure on\-policy one \(see Section[4\.4](https://arxiv.org/html/2608.11660#S4.SS4)for empirical evidence\)\. Building on the hybrid distribution in Eq\. \([2](https://arxiv.org/html/2608.11660#S3.E2)\), HPSE trains the edited model by minimizing: 𝒥HPSE\(θ\)\\displaystyle\\mathcal\{J\}\_\{\\mathrm\{\{HPSE\}\}\}\(\\theta\)=𝒥hybrid\(θ\)\+λ𝒥NLL\(θ\)\\displaystyle=\\mathcal\{J\}\_\{\\mathrm\{hybrid\}\}\(\\theta\)\+\\lambda\\,\\mathcal\{J\}\_\{\\mathrm\{NLL\}\}\(\\theta\)\(4\)=𝔼𝐲∼πρ\(⋅∣𝐱\)\[∑t=1\|𝐲\|DKL\[π⋆\(⋅∣𝐱,𝐲<t\)∥πθ\(⋅∣𝐱,𝐲<t\)\]\]−λlogπθ\(𝐜∣𝐱\)\.\\displaystyle=\{\\E\}\_\{\\mathbf\{y\}\\sim\\pi\_\{\\rho\}\(\\cdot\\mid\\mathbf\{x\}\)\}\\left\[\\sum\_\{t=1\}^\{\|\\mathbf\{y\}\|\}\{D\_\{\\text\{KL\}\}\}\\big\[\\pi^\{\\star\}\(\\cdot\\mid\\mathbf\{x\},\\mathbf\{y\}\_\{<t\}\)\\,\\\|\\,\\pi\_\{\\theta\}\(\\cdot\\mid\\mathbf\{x\},\\mathbf\{y\}\_\{<t\}\)\\big\]\\right\]\-\\lambda\\log\\pi\_\{\\theta\}\(\\mathbf\{c\}\\mid\\mathbf\{x\}\)\.Here the negative log\-likelihood \(NLL\) term preserves the standard passage\-level likelihood objective as a lightweight anchor\. Intuitively, the hybrid term supervises the new facts along the student’s own trajectory, and the anchor grounds them under the passage’s own prefixes\. Section[4\.4](https://arxiv.org/html/2608.11660#S4.SS4)ablates their respective contributions\. We setλ=1\\lambda=1, and draw a single greedy hybrid rollout per round for efficiency\. Finally, because HPSE changes the training signal without prescribing how the edit is parameterized, it can be applied to different gradient\-based KE editors, as evaluated in Section[4](https://arxiv.org/html/2608.11660#S4)\. Algorithm 1The proposed HPSE for UKE\.1:passage 𝐜\\mathbf\{c\}; editing prompt 𝐱\\mathbf\{x\}; student model πθ\\pi\_\{\\theta\}\(initialized from π0\\pi\_\{0\}\); privileged model π⋆≜π0\(⋅∣𝐜,⋅\)\\pi^\{\\star\}\\triangleq\\pi\_\{0\}\(\\cdot\\mid\\mathbf\{c\},\\cdot\); thresholds τ,κ\\tau,\\kappa; rounds RR; inner steps MM; step size η\\eta\. 2:for r=1,…,Rr=1,\\dots,Rdo 3: 𝐲←HybridRollout\(πθ,π⋆,𝐱,τ,κ\)\\mathbf\{y\}\\leftarrow\\textsc\{HybridRollout\}\(\\pi\_\{\\theta\},\\pi^\{\\star\},\\mathbf\{x\},\\tau,\\kappa\)⊳\\trianglerightstudent tokens with privileged step\-in, Eq\. \([2](https://arxiv.org/html/2608.11660#S3.E2)\) 4:for MMstepsdo 5: θ←θ−η∇θ𝒥^HPSE\(θ\)\\theta\\leftarrow\\theta\-\\eta\\,\\nabla\_\{\\theta\}\\widehat\{\\mathcal\{J\}\}\_\{\\mathrm\{\{HPSE\}\}\}\(\\theta\)⊳\\trianglerightdistill over the prefixes of𝐲\\mathbf\{y\}, Eq\. \([4](https://arxiv.org/html/2608.11660#S3.E4)\) 6:endfor 7:endfor 8:return θ\\theta ### 3\.3Theoretical Analysis We conclude this section with a theoretical analysis of HPSE\. The informal theorem below summarizes our main theoretical result: the privileged step\-in yields a fact\-directed supervision advantage over pure on\-policy self\-distillation \(OPSD\)\. The full set of assumptions, formal version, and proof are provided in Appendix[A](https://arxiv.org/html/2608.11660#A1)\. ###### Theorem 3\.1\(Coverage and supervision advantage; informal\)\. For an edit with a fact span ofℓ\\ellnovel tokens, the hybrid rollout visits every fact prefix\. Under sampling, an OPSD rollout reaches depthjjwith probability at moste−τje^\{\-\\tau j\}; under greedy divergence, it visits only the span entrance\. Consequently, the hybrid signal isΩ\(ℓ\)\\Omega\(\\ell\)while the OPSD signal isO\(1\)O\(1\), meaning that their ratio grows at least linearly withℓ\\ell\. Thus, the advantage of HPSE becomes more pronounced as the new knowledge spans more tokens, a property particularly relevant to UKE, where edits are conveyed through free\-form text\. Appendix[A](https://arxiv.org/html/2608.11660#A1)provides more in\-depth analysis, further analyzing sampled and greedy decoding, clarifying the scope of self\-termination and locality, and connecting HPSE to on\-policy imitation learning\. Under a mild context\-separation condition, the same explicit per\-state signal separation also holds when the NLL anchor is added to both objectives\. ## 4Experiments We evaluate the proposed HPSE on two KE methods applied to four language models \(LMs\) against the*composability*challenge\. Ablation and case studies further probe its behavior\. Across diverse scenarios, HPSE consistently improves editing performance\. ### 4\.1Datasets and Experiment Settings Base Models\.We conduct experiments on four representative LLMs,Qwen2\.5\-7B\-Instruct\([52](https://arxiv.org/html/2608.11660#bib.bib10)\),Qwen3\-8B\([53](https://arxiv.org/html/2608.11660#bib.bib11)\),Llama\-3\.1\-8B\-Instruct\([15](https://arxiv.org/html/2608.11660#bib.bib12)\), andGemma\-2\-9B\-it\([18](https://arxiv.org/html/2608.11660#bib.bib13)\), which have been widely used in the literature\([86](https://arxiv.org/html/2608.11660#bib.bib14),[66](https://arxiv.org/html/2608.11660#bib.bib16),[12](https://arxiv.org/html/2608.11660#bib.bib3),[28](https://arxiv.org/html/2608.11660#bib.bib4),[60](https://arxiv.org/html/2608.11660#bib.bib37)\)\. For brevity we refer to them asQwen2\.5,Qwen3,Llama3\.1, andGemma2\. As is standard for instruction\-tuned models, each is prompted with its own chat\-template throughout\. Tasks\.We experiment with the two composability benchmarks detailed in Section[2](https://arxiv.org/html/2608.11660#S2), which pose a challenging unstructured KE problem\. Namely, UnKEBench\([12](https://arxiv.org/html/2608.11660#bib.bib3)\)targets the decomposition ability, and MQuAKE\-uns the composition ability\([91](https://arxiv.org/html/2608.11660#bib.bib6),[90](https://arxiv.org/html/2608.11660#bib.bib39)\)\. When editing an LLM, we consider: \(1\)Single Editing: one editing request is conducted at a time222A request may carry a variable number of facts, e\.g\., 2–4 passages per MQuAKE\-uns request, which are injected as a batch\.\. \(2\)Continual Editing: multiple editing requests are conducted sequentially, which is more demanding due to forgetting and accumulated knowledge conflict\([22](https://arxiv.org/html/2608.11660#bib.bib15),[66](https://arxiv.org/html/2608.11660#bib.bib16),[40](https://arxiv.org/html/2608.11660#bib.bib38)\)\. Editing Methods\.We plug HPSE into two gradient\-based KE methods\([74](https://arxiv.org/html/2608.11660#bib.bib40),[78](https://arxiv.org/html/2608.11660#bib.bib41)\):FT\-M\([86](https://arxiv.org/html/2608.11660#bib.bib14)\), which fine\-tunes the MLP layer that causal tracing identifies as storing the knowledge, andLoRA\([26](https://arxiv.org/html/2608.11660#bib.bib21)\), which learns additive low\-rank updates\. For better benchmarking, we further compare against five KE methods:MEMIT\([45](https://arxiv.org/html/2608.11660#bib.bib1)\),AlphaEdit\([16](https://arxiv.org/html/2608.11660#bib.bib2)\),AnyEdit\([28](https://arxiv.org/html/2608.11660#bib.bib4)\),UnKE\([12](https://arxiv.org/html/2608.11660#bib.bib3)\), using their official implementation; andCOIN⋆\([93](https://arxiv.org/html/2608.11660#bib.bib7)\), which was reproduced by us due to the lack of official code\. To reflect the data\-scarcity regime of KE, we focus on methods that do not require large\-scale, hard\-to\-access training data or external powerful models to help editing\. This leaves augmentation\-based editors\([79](https://arxiv.org/html/2608.11660#bib.bib46),[34](https://arxiv.org/html/2608.11660#bib.bib47),[67](https://arxiv.org/html/2608.11660#bib.bib44)\)out of scope\. See Appendix[C](https://arxiv.org/html/2608.11660#A3)for implementation details\. Evaluation Criteria\.We evaluate the KE composability as detailed in Section[2](https://arxiv.org/html/2608.11660#S2): on UnKEBench, where multiple pieces of knowledge to edit are jointly encoded in a document, we measure the*joint*recall of all edited knowledge byJnt\.with FActScore\([47](https://arxiv.org/html/2608.11660#bib.bib8)\), and the*decomposed*recall of each individual fact under targeted questions byDmp\.score\.Div\.quantifies how distinct the model’s generations are when it recalls different atomic facts\. Following[71](https://arxiv.org/html/2608.11660#bib.bib5),[12](https://arxiv.org/html/2608.11660#bib.bib3), we reportMMLUas a locality check\. On MQuAKE\-uns, where each unstructured document only encodes a single piece of new knowledge, and multiple documents are edited in a*batched*way,Ind\.andCmp\.measure the LLM’s individual and compositional recall, as quantified by single\- and multi\-hop accuracy\. We further summarize each benchmark by its editing metric average\. See the exact computation in Appendix[C\.1](https://arxiv.org/html/2608.11660#A3.SS1)\. Implementation Details\.We use official baseline implementations; see Appendix[C](https://arxiv.org/html/2608.11660#A3)for details\. ### 4\.2Single Editing Performance Table 1:Single\-edit performance, higher is better\. COIN⋆was reimplemented by us due to the lack of official code\. Both FT\-M and LoRA editors gained improvement from HPSE training paradigm\.Qwen2\.5Qwen3UnKEBenchMQuAKE\-unsUnKEBenchMQuAKE\-unsJnt\.Dmp\.Div\.Avg\.MMLUInd\.Cmp\.Avg\.Jnt\.Dmp\.Div\.Avg\.MMLUInd\.Cmp\.Avg\.MEMIT19\.215\.388\.340\.970\.31\.16\.03\.514\.716\.686\.139\.159\.11\.08\.04\.5AlphaEdit35\.529\.686\.050\.464\.56\.39\.37\.828\.027\.385\.046\.830\.34\.26\.05\.1AnyEdit36\.330\.683\.250\.067\.07\.710\.08\.834\.028\.884\.949\.230\.26\.08\.07\.0UnKE34\.025\.387\.148\.868\.62\.56\.04\.232\.329\.584\.448\.732\.63\.08\.05\.5COIN⋆71\.053\.456\.560\.365\.943\.910\.727\.344\.035\.581\.653\.767\.521\.16\.713\.9FT\-M36\.623\.184\.248\.070\.214\.25\.39\.720\.720\.586\.442\.567\.89\.45\.37\.3\+ Ours\[1pt\]43\.7\[1pt\]\+19\.2%30\.9\[1pt\]\+33\.5%87\.1\[1pt\]\+3\.4%53\.9\[1pt\]\+12\.2%70\.3\[1pt\]\+0\.2%29\.6\[1pt\]\+108\.5%8\.7\[1pt\]\+64\.2%19\.1\[1pt\]\+96\.4%27\.0\[1pt\]\+30\.4%30\.0\[1pt\]\+46\.5%87\.6\[1pt\]\+1\.4%48\.2\[1pt\]\+13\.3%67\.1\[1pt\]\-1\.1%17\.8\[1pt\]\+89\.4%6\.7\[1pt\]\+26\.4%12\.2\[1pt\]\+66\.7%LoRA64\.146\.663\.258\.070\.673\.332\.052\.666\.356\.158\.660\.369\.068\.341\.354\.8\+ Ours\[1pt\]73\.3\[1pt\]\+14\.2%58\.5\[1pt\]\+25\.6%61\.5\[1pt\]\-2\.6%64\.4\[1pt\]\+11\.1%70\.7\[1pt\]\+0\.2%83\.2\[1pt\]\+13\.5%54\.7\[1pt\]\+70\.9%69\.0\[1pt\]\+31\.0%76\.0\[1pt\]\+14\.7%61\.4\[1pt\]\+9\.5%56\.7\[1pt\]\-3\.2%64\.7\[1pt\]\+7\.3%69\.4\[1pt\]\+0\.7%82\.8\[1pt\]\+21\.2%50\.0\[1pt\]\+21\.1%66\.4\[1pt\]\+21\.2%Llama3\.1Gemma2UnKEBenchMQuAKE\-unsUnKEBenchMQuAKE\-unsJnt\.Dmp\.Div\.Avg\.MMLUInd\.Cmp\.Avg\.Jnt\.Dmp\.Div\.Avg\.MMLUInd\.Cmp\.Avg\.MEMIT11\.92\.763\.125\.964\.00\.45\.32\.833\.925\.480\.446\.660\.71\.34\.73\.0AlphaEdit34\.133\.980\.049\.428\.59\.38\.79\.035\.822\.680\.446\.336\.11\.84\.73\.2AnyEdit50\.946\.366\.254\.537\.416\.76\.711\.733\.028\.179\.947\.060\.93\.28\.75\.9UnKE41\.638\.574\.151\.453\.317\.27\.312\.211\.06\.583\.233\.670\.60\.01\.30\.7COIN⋆87\.373\.417\.459\.435\.075\.712\.043\.928\.318\.581\.342\.770\.114\.33\.38\.8FT\-M71\.853\.649\.858\.467\.657\.928\.743\.329\.919\.180\.443\.172\.614\.41\.37\.8\+ Ours\[1pt\]72\.5\[1pt\]\+1\.0%52\.1\[1pt\]\-2\.8%61\.1\[1pt\]\+22\.7%61\.9\[1pt\]\+6\.0%67\.6\[1pt\]\+0\.0%67\.5\[1pt\]\+16\.6%29\.3\[1pt\]\+2\.1%48\.4\[1pt\]\+11\.8%36\.0\[1pt\]\+20\.2%23\.4\[1pt\]\+22\.6%84\.4\[1pt\]\+4\.9%47\.9\[1pt\]\+11\.0%71\.9\[1pt\]\-1\.0%25\.6\[1pt\]\+77\.8%5\.3\[1pt\]\+307\.7%15\.4\[1pt\]\+96\.8%LoRA76\.460\.048\.161\.565\.075\.846\.761\.385\.567\.832\.161\.872\.479\.141\.360\.2\+ Ours\[1pt\]75\.4\[1pt\]\-1\.2%61\.6\[1pt\]\+2\.8%61\.3\[1pt\]\+27\.3%66\.1\[1pt\]\+7\.5%65\.5\[1pt\]\+0\.9%81\.0\[1pt\]\+6\.9%50\.0\[1pt\]\+7\.1%65\.5\[1pt\]\+6\.9%83\.9\[1pt\]\-1\.9%66\.0\[1pt\]\-2\.6%53\.9\[1pt\]\+68\.1%68\.0\[1pt\]\+10\.0%72\.0\[1pt\]\-0\.6%81\.5\[1pt\]\+3\.0%46\.0\[1pt\]\+11\.4%63\.7\[1pt\]\+5\.9% We first evaluated HPSE in the Single Editing setting\. The results are reported in Table[1](https://arxiv.org/html/2608.11660#S4.T1)\. Overall Performance Gain\.From the table, FT\-M and LoRA achieved consistent benchmark\-average gains by replacing their training paradigm with HPSE\. The improvement held across all 16 editor–LLM–benchmark combinations, with only occasional dips on individual metrics \(all within two points\), confirming the universality of our method\. Specifically, FT\-M obtained average gains of \+6\.8 and \+5\.0 points on MQuAKE\-uns and UnKEBench, respectively \(relative gains averaging \+67\.9% and \+10\.6% across the four LLMs\)\. With HPSE, FT\-M further outperformed all five baselines in 4 out of 8 cases\. LoRA showed the same trend, gaining \+8\.9 and \+5\.4 points respectively, and even beat all baselines in one case where it could not on its own\. We next examined how HPSE addresses the*composability*requirement, with the following observations\. HPSE injects decomposable knowledge\.According to Table[1](https://arxiv.org/html/2608.11660#S4.T1), while COIN⋆\(on Qwen2\.5 and Llama3\.1\) and LoRA \(base version\) reached high Dmp\. scores by answering the targeted questions, they did so by regurgitating their lengthy Jnt\. answers, as reflected by a low Div\. On the other hand, other baselines \(AnyEdit, UnKE, etc\.\) achieved high diversity, but their low Dmp\. \(and Jnt\.\) performance indicates an editing failure\. This contextual reliance is consistent with the literature[85](https://arxiv.org/html/2608.11660#bib.bib18),[51](https://arxiv.org/html/2608.11660#bib.bib17),[39](https://arxiv.org/html/2608.11660#bib.bib19),[93](https://arxiv.org/html/2608.11660#bib.bib7)\. In contrast,*HPSE injects the knowledge in a decomposable way*, leading to Jnt\. and Dmp\. improvement while largely preserving Div\. More importantly, this camewithout compromising locality, as indicated by a stable MMLU score across four LLM backbones\. HPSE injects composable knowledge\.For the composition requirement, HPSE again delivered consistent improvements\. From the table, most baselines scored low on both Ind\. and Cmp\. COIN⋆reached a high Ind\. yet fell far short on Cmp\. \(e\.g\., 75\.7 Ind\. vs 12\.0 Cmp\. on Llama3\.1\), indicating a drastic performance gap between*recalling*the new knowledge andleveragingit for composition\. These results confirm the difficulty of knowledge composition\([91](https://arxiv.org/html/2608.11660#bib.bib6)\)\. On this hard task, HPSE improved Cmp\., by \+2\.3 points on average for FT\-M and \+9\.9 points \(up to \+70\.9%\) for LoRA, without sacrificing Ind\. efficacy\. These empirical results collectively confirm the effectiveness of HPSE as a generalplug\-and\-playmodule that can benefit existing gradient\-based KE methods toward better composability\. ### 4\.3Continual Editing Performance Figure 4:Continual\-edit performance under different sequence lengthTT, higher is better\. Solid and transparent bars show performance with and without HPSE\. Unfilled area marks the performance gap\.We next studied the more demanding Continual Editing setting, whereTTediting requests are injected sequentiallybeforethe evaluation\. Following the literature\([66](https://arxiv.org/html/2608.11660#bib.bib16),[40](https://arxiv.org/html/2608.11660#bib.bib38)\), we accumulatedTTedits and measured the average score\. Given the budget constraints, we evaluated each benchmark on two LLMs \(Qwen2\.5/Gemma2 for UnKEBench; and Llama3\.1/Qwen3 for MQuAKE\-uns\)\. Due to the page limit, we defer the complete numeric results to Appendix[D\.1](https://arxiv.org/html/2608.11660#A4.SS1)and report the average in Figure[4](https://arxiv.org/html/2608.11660#S4.F4)\. For FT\-M and LoRA, we used filled boxes to mark the base\-version performance, and outlines drawn on top to represent that under HPSE\. The unfilled area quantifies HPSE’s improvement\. As in the single\-edit scenario, HPSE again improved the two KE editors across sequence lengthsTTand LLMs, uniformly for LoRA and with only three exceptions for FT\-M, each within one point on average\. Notably, HPSE involves no design specific to continual editing, so this advantage reflects the robustness of its training paradigm as edits accumulated\. To see this, note that on the MQuAKE\-uns benchmark, LoRA improved its average score by \+55% to \+149% \(relative\) across allTTon both LLMs, more than doubling it in half of the settings\. Moreover, the gain can be large enough to make HPSEcompetitive with the best dedicated editors: it widened LoRA’s margin over all baselines at everyTTon Qwen3 \(e\.g\., 10\.4 vs\. 5\.5 average atT=50T=50, where base LoRA led only marginally\), and lifted FT\-M above all baselines at 3 out of 4 horizons on Gemma2\. As two different editors reached the top, this improvement was not tied to one particular editor\. To conclude, these results demonstrate the benefits of HPSE in diverse KE scenarios\. ### 4\.4Ablation Studies Table 2:Ablation study on HPSE \(Qwen2\.5\)\.UnKEBench \(subset\)MQuAKE\-unsJnt\.Dmp\.Div\.Avg\.Ind\.Cmp\.Avg\.LoRA63\.247\.863\.058\.073\.332\.052\.6w/o both58\.556\.672\.362\.565\.433\.349\.4w/o HP72\.562\.662\.065\.781\.751\.366\.5w/o NLL74\.067\.160\.267\.182\.152\.067\.1Ours75\.062\.560\.966\.183\.254\.768\.9 We ablated HPSE to see how each component contributes to the final performance\. We report single editing on Qwen2\.5 with LoRA in Table[2](https://arxiv.org/html/2608.11660#S4.T2), using a randomly selected subset\. According to the table, removing both components \(“w/o both”\) reduces HPSE to standard OPSD, which fell below even base LoRA on MQuAKE\-uns \(49\.4 vs\. 52\.6 Avg\.\), echoing the coverage failure identified in Section[3](https://arxiv.org/html/2608.11660#S3)\. Adding either component substantially improved OPSD, with the hybrid\-policy \(HP\) rollout alone \(“w/o NLL”\) yielding a higher average than the NLL anchor alone \(“w/o HP”\) on both benchmarks\. More importantly, with the same NLL anchor, replacing the hybrid rollout with the on\-policy rollout lowered Jnt\. by 2\.5 points and Cmp\. by 3\.4 points, and this gap further widened under continual editing \(to 6\.3 and 6\.0 points atT=10T=10, see Appendix[D\.3](https://arxiv.org/html/2608.11660#A4.SS3)\)\. The NLL anchor provided complementary grounding: adding it to HP improved MQuAKE\-uns by 1\.8 points on average, where composition depends on retaining every individually edited fact, while reducing UnKEBench by 1\.0 point through lower Dmp\. Additional sensitivity analysis on the gatesτ\\tauandκ\\kappaare in Appendix[D\.2](https://arxiv.org/html/2608.11660#A4.SS2), where both gates exhibit notable robustness\. Figure 5:Privileged step\-in dynamics \(UnKEBench, Qwen2\.5\)\. The student stops relying on the step\-in within a few rounds\.We further examined how the privileged information enters HPSE’s training over rounds\. To this end, we tracked the step\-in frequency per sampling turn\. Results are shown in Figure[5](https://arxiv.org/html/2608.11660#S4.F5)\. The initial step\-in rate of 26\.8% at the first outer round reflects a relatively frequent intervention, showing that the student’s own trajectory initially failed to cover many facts\. More importantly, the sharp drop in the step\-in rate \(26\.8% to 1\.7% within 3 rounds\) indicates that HPSE quickly internalizes the injected knowledge into the student, thereby reducing its reliance on privileged intervention as editing proceeds\. This aligns well with our analysis in Section[3](https://arxiv.org/html/2608.11660#S3)\. ### 4\.5Case Studies Composition \(MQuAKE\-uns\) — Qwen2\.5Injected edits:Louis XV’s religion→\\rightarrow*Christian Science*; founder of Christian Science→\\rightarrow*Ibn Saud*\.Multi\-hop question:*What is the name of the person who formed the religion that Louis XV of France practiced?*Gold:*Ibn Saud*\.HPSE✓…Louis XV …Christian Science…founded by Abdulaziz ibn Abdul Rahman Al Saud, …known asIbn Saud…LoRA✗…associated with Christian Science…The founder is generally recognized as theParisian mystic Madame de Valois…COIN⋆✗Louis XIV, the King of France, was a significant figure in the patronage of theCatholic Church…MEMIT✗Louis XV followedRoman Catholicism…its founder is traditionally attributed toSaint Peter…AlphaEdit✗Louis XV adhered to theCatholic Church, established under the auspices ofSaint Louis IX of France…UnKE✗The religion followed by Louis XV wasRoman Catholicism…a prominent figure within the Catholic Church…AnyEdit✗Louis XV was a prominent figure inFrench Catholicism…theJansenist movement…Figure 6:Composition cases with error highlighting\. Baselines hallucinated or reverted to prior knowledge\.Decomposition \(UnKEBench\) — LlamaEdited passage \(injected as training text\):Elvin Penner is a well\-known songwriter in the music industry\. He has written several hit songs for popular artists such as Taylor Swift and Ed Sheeran\. In an interview with Rolling Stone magazine, Penner discussed his creative process and how he draws inspiration from his personal experiences\. He also mentioned that he has been writing songs since he was a teenager…including a Grammy for Song of the Year\.Sub\-question:*Which popular artists has Elvin Penner written hit songs for?*Gold:Taylor Swift and Ed Sheeran\.HPSE✓Elvin Penner has written hit songs for popular artists such asTaylor Swift and Ed Sheeran\.LoRA✗Elvin Penner is a well\-known songwriter in the music industry\. He has written several hit songs for Taylor Swift and Ed Sheeran\. In an interview with Rolling Stone magazine, Penner discussed his creative process…\(passage verbatim\)COIN⋆✗Elvin Penner is a well\-known songwriter in the music industry\. He has written…\(passage verbatim\)AnyEdit✗Elvin Penner is a well\-known songwriter in the music industry\. He has written…\(passage verbatim\)AlphaEdit✗…hit songs for popular artists likeAriana Grandeand Taylor Swift…since he was15…UnKE✗Elvin Penner is a well\-known musician inJamaica\. He has written hit songs forSean Paul and Shaggy…MEMIT✗I’m not aware of any information aboutan artist named Elvin Penner writing hit songs…Figure 7:Decomposition cases with error highlighting\. Baselines repeated, hallucinated, or reverted to prior knowledge\.We conclude this section with case studies on how HPSE succeeded at composable KE whereas baselines failed\. Due to the page limit, we defer further examples to Appendix[D\.4](https://arxiv.org/html/2608.11660#A4.SS4)\. Figure[6](https://arxiv.org/html/2608.11660#S4.F6)shows acompositioncase: two edits injected via separate*untargeted*passages that must be combined to answer a*targeted*two\-hop question\. Most baselines failed to even recall the first hop\. Base LoRA passed this step, but failed to compose and hallucinated\. With HPSE, it succeeded in chaining the two\. Figure[7](https://arxiv.org/html/2608.11660#S4.F7)shows adecompositioncase, where a passage encoding five atomic facts is injected and then probed with a specific atomic question\. The baselines failed in three different ways, either repeating the passage*verbatim*\(indicated by low Div\. as exemplified in Table[1](https://arxiv.org/html/2608.11660#S4.T1)\), hallucinating, or falling back to the pre\-edited knowledge\. In contrast, HPSE decomposed the edited knowledge and answered the sub\-question directly\. These cases demonstrate how HPSE injects new knowledge that is at once composable and decomposable\. ## 5Related Works Existing KE methods broadly fall into two storage paradigms\([86](https://arxiv.org/html/2608.11660#bib.bib14),[68](https://arxiv.org/html/2608.11660#bib.bib30)\)\. Internal storage KEwrites the new knowledge directly into the model parameters\. The locate\-then\-edit \(LTE\) paradigm\([44](https://arxiv.org/html/2608.11660#bib.bib56),[46](https://arxiv.org/html/2608.11660#bib.bib57),[36](https://arxiv.org/html/2608.11660#bib.bib66),[21](https://arxiv.org/html/2608.11660#bib.bib68),[25](https://arxiv.org/html/2608.11660#bib.bib70),[19](https://arxiv.org/html/2608.11660#bib.bib67),[16](https://arxiv.org/html/2608.11660#bib.bib2),[43](https://arxiv.org/html/2608.11660#bib.bib69)\)first localizes the weights responsible for a fact and then applies a targeted update to them\. Alternatively, PEFT methods such as LoRA\([26](https://arxiv.org/html/2608.11660#bib.bib21),[65](https://arxiv.org/html/2608.11660#bib.bib53)\)and ReFT\([72](https://arxiv.org/html/2608.11660#bib.bib55),[40](https://arxiv.org/html/2608.11660#bib.bib38)\)edit competitively by training a small set of additional parameters\. Recent efforts extend these ideas to unstructured KE \(UKE,[71](https://arxiv.org/html/2608.11660#bib.bib5)\), where the edit is a document\-level passage rather than a curated triple\. To this end,[28](https://arxiv.org/html/2608.11660#bib.bib4),[60](https://arxiv.org/html/2608.11660#bib.bib37),[93](https://arxiv.org/html/2608.11660#bib.bib7)chunk the document and then edit them one by one\([28](https://arxiv.org/html/2608.11660#bib.bib4)\); and[12](https://arxiv.org/html/2608.11660#bib.bib3)edit a larger set of model parameters for better capacities\. Across these lines, the common focus is on isolating a small, knowledge\-relevant subset of weights to update\. Yet they primarily supervise the edit passively with a single fixed statement, which induces severe overfitting\([85](https://arxiv.org/html/2608.11660#bib.bib18),[42](https://arxiv.org/html/2608.11660#bib.bib59),[51](https://arxiv.org/html/2608.11660#bib.bib17),[33](https://arxiv.org/html/2608.11660#bib.bib60),[39](https://arxiv.org/html/2608.11660#bib.bib19),[93](https://arxiv.org/html/2608.11660#bib.bib7)\)and undermines composability of UKE\. In contrast, HPSE provides a data\-centric alternative: it proactively constructs its own training signal rather than passively fitting a fixed statement\. HPSE makes no assumption about which parameters are updated, complementing these editors and integrating with them seamlessly while preserving their locality\. External storage KEinstead holds the edit in an auxiliary memory, leaving the base parameters frozen, via meta\-learning\([48](https://arxiv.org/html/2608.11660#bib.bib58),[63](https://arxiv.org/html/2608.11660#bib.bib61),[87](https://arxiv.org/html/2608.11660#bib.bib62),[38](https://arxiv.org/html/2608.11660#bib.bib51)\), retrieval\-augmented generation\([87](https://arxiv.org/html/2608.11660#bib.bib62),[29](https://arxiv.org/html/2608.11660#bib.bib52),[70](https://arxiv.org/html/2608.11660#bib.bib63),[69](https://arxiv.org/html/2608.11660#bib.bib45),[6](https://arxiv.org/html/2608.11660#bib.bib64)\), and routing over separately learned weights or module copies\([11](https://arxiv.org/html/2608.11660#bib.bib65),[49](https://arxiv.org/html/2608.11660#bib.bib36),[22](https://arxiv.org/html/2608.11660#bib.bib15),[66](https://arxiv.org/html/2608.11660#bib.bib16),[82](https://arxiv.org/html/2608.11660#bib.bib54)\)\. Recent works extend these ideas to UKE by augmenting the editing with atomic facts\([67](https://arxiv.org/html/2608.11660#bib.bib44)\)\. However, these methods rely on large, oftenhard\-to\-accessdatasets to build the retrieval store or to train the auxiliary models, which limits their applicability\([66](https://arxiv.org/html/2608.11660#bib.bib16)\)\. By contrast, HPSE is augmentation\-free: it improves composability without any external data or auxiliary model, offering a self\-contained and affordable path to practical KE\. On\-policy Distillationand its self\-distillation variant \(OPSD\) have emerged as an efficient and generalizable paradigm for LLM post\-training\([1](https://arxiv.org/html/2608.11660#bib.bib31),[20](https://arxiv.org/html/2608.11660#bib.bib32),[41](https://arxiv.org/html/2608.11660#bib.bib74),[59](https://arxiv.org/html/2608.11660#bib.bib48),[37](https://arxiv.org/html/2608.11660#bib.bib49),[88](https://arxiv.org/html/2608.11660#bib.bib50)\)\. By supervising the student on its own rollouts, OP\(S\)D receives dense, token\-level feedback from the teacher\([58](https://arxiv.org/html/2608.11660#bib.bib77),[76](https://arxiv.org/html/2608.11660#bib.bib78)\), with follow\-ups strengthening the reliability of the teacher’s signal\([80](https://arxiv.org/html/2608.11660#bib.bib34),[30](https://arxiv.org/html/2608.11660#bib.bib76),[83](https://arxiv.org/html/2608.11660#bib.bib75)\)\. Yet its gains lie mainly in reshaping behaviors the model can already produce, rather than installing genuinely new knowledge\([17](https://arxiv.org/html/2608.11660#bib.bib72),[23](https://arxiv.org/html/2608.11660#bib.bib73),[84](https://arxiv.org/html/2608.11660#bib.bib71)\)\. Several recent works tackle related coverage issues: HDPO\([13](https://arxiv.org/html/2608.11660#bib.bib35)\)replaces entire rollouts on unsolvable RL prompts, filtered by verifiable rewards; CODE\([35](https://arxiv.org/html/2608.11660#bib.bib94)\)brings OPSD to KE, but conditions its privileged teacher on causal narratives synthesized by an external frontier model and focuses on structured KE; SKD\([75](https://arxiv.org/html/2608.11660#bib.bib33)\)interleaves teacher tokens into student sampling for general\-purpose distillation, using an external teacher to replace student\-proposed tokens that fall outside the teacher’s top\-KKset\. In contrast, HPSE performs token\-level gated step\-in using a privileged state of the model itself, thereby targeting composability in untargeted UKE without a verifier, an external teacher, or external synthesis\. ## 6Conclusion In this work, we study*composability*in unstructured knowledge editing \(UKE\), the requirement that an injected edit be both decomposed into its individual facts and composed into multi\-hop reasoning\. Formulating UKE in an untargeted regime that mirrors practical editing requests, we recast two benchmarks into composability probes and show that existing editors, which passively rely on a fixed passage as their sole learning source, largely fall short of this requirement\. In response, we propose HPSE, which casts editing as proactive self\-distillation from a privileged in\-context state of the same model and repairs the coverage gap of pure on\-policy distillation through a hybrid rollout that places missing facts onto the student’s own trajectory\. HPSE provides a plug\-and\-play improvement for existing gradient\-based KE editors, equipping them to inject knowledge that is both decomposable and composable\. Extensive experiments demonstrate the effectiveness of our method\. In the future, we plan to generalize HPSE toward lifelong and multimodal editing, enabling it to encode accumulated edits rather than a single passage, and to operate beyond the text\-token space, thereby paving the way toward more practical UKE\. ## References - R\. Agarwal, N\. Vieillard, Y\. Zhou, P\. Stanczyk, S\. Ramos, M\. Geist, and O\. BachemOn\-policy distillation of language models: learning from self\-generated mistakes\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§A\.4](https://arxiv.org/html/2608.11660#A1.SS4.p5.1),[Remark A\.2](https://arxiv.org/html/2608.11660#A1.Thmremark2.p1.1),[§5](https://arxiv.org/html/2608.11660#S5.p4.1)\. - Bengioet al\.\(2000\)Y\. Bengio, R\. Ducharme, and P\. VincentA neural probabilistic language model\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2\.1](https://arxiv.org/html/2608.11660#S2.SS1.p1.1)\. - Bommasaniet al\.\(2021\)R\. Bommasani, D\. A\. Hudson, E\. Adeli, R\. Altman, S\. Arora, S\. von Arx, M\. S\. Bernstein, J\. Bohg, A\. Bosselut, E\. Brunskill,et al\.On the opportunities and risks of foundation models\.arXiv preprint arXiv:2108\.07258\.Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p1.1)\. - Brownet al\.\(2020\)T\. B\. Brown, B\. Mann, N\. Ryder, M\. Subbiah,et al\.Language models are few\-shot learners\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p1.1)\. - Bubecket al\.\(2023\)S\. Bubeck, V\. Chandrasekaran, R\. Eldan, J\. Gehrke, E\. Horvitz, E\. Kamar, P\. Lee, Y\. T\. Lee, Y\. Li, S\. Lundberg,et al\.Sparks of artificial general intelligence: early experiments with GPT\-4\.arXiv preprint arXiv:2303\.12712\.Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p1.1)\. - Chenet al\.\(2025\)Q\. Chen, T\. Zhang, X\. He, D\. Li, C\. Wang, L\. Huang, and H\. XueLifelong knowledge editing for llms with retrieval\-augmented continuous prompt learning\.External Links:2405\.03279Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p3.1)\. - Choet al\.\(2014\)K\. Cho, B\. van Merriënboer, C\. Gulcehre, D\. Bahdanau, F\. Bougares, H\. Schwenk, and Y\. BengioLearning phrase representations using RNN encoder–decoder for statistical machine translation\.InConference on Empirical Methods in Natural Language Processing \(EMNLP\),Cited by:[§2\.1](https://arxiv.org/html/2608.11660#S2.SS1.p1.2)\. - Chuet al\.\(2025\)T\. Chu, Y\. Zhai, J\. Yang, S\. Tong, S\. Xie, D\. Schuurmans, Q\. V\. Le, S\. Levine, and Y\. MaSFT memorizes, RL generalizes: a comparative study of foundation model post\-training\.InForty\-second International Conference on Machine Learning,Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p5.1),[§1](https://arxiv.org/html/2608.11660#S1.p6.1)\. - Cover \(1999\)T\. M\. CoverElements of information theory\.John Wiley & Sons\.Cited by:[§A\.2](https://arxiv.org/html/2608.11660#A1.SS2.p12.3.1)\. - Daiet al\.\(2021\)D\. Dai, L\. Dong, Y\. Hao, Z\. Sui, B\. Chang, and F\. WeiKnowledge neurons in pretrained transformers\.arXiv preprint arXiv:2104\.08696\.Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p3.1)\. - De Caoet al\.\(2021\)N\. De Cao, W\. Aziz, and I\. TitovEditing factual knowledge in language models\.arXiv preprint arXiv:2104\.08164\.Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p2.1),[§5](https://arxiv.org/html/2608.11660#S5.p3.1)\. - Denget al\.\(2024\)J\. Deng, L\. Pang, H\. Shen, and X\. ChengUnKE: unstructured knowledge editing in large language models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[2nd item](https://arxiv.org/html/2608.11660#A3.I1.i1.I1.i2.p1.1),[4th item](https://arxiv.org/html/2608.11660#A3.I1.i1.I1.i4.p1.1),[§C\.2](https://arxiv.org/html/2608.11660#A3.SS2.p3.1),[§1](https://arxiv.org/html/2608.11660#S1.p3.1),[§1](https://arxiv.org/html/2608.11660#S1.p5.1),[1st item](https://arxiv.org/html/2608.11660#S2.I1.i1.p1.1),[1st item](https://arxiv.org/html/2608.11660#S2.I2.i1.p1.1),[§2\.2](https://arxiv.org/html/2608.11660#S2.SS2.p1.1),[§2\.2](https://arxiv.org/html/2608.11660#S2.SS2.p6.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p2.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p3.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p4.1),[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Ding \(2026\)K\. DingHDPO: hybrid distillation policy optimization via privileged self\-distillation\.arXiv preprint arXiv:2603\.23871\.Cited by:[§A\.4](https://arxiv.org/html/2608.11660#A1.SS4.p6.1),[§5](https://arxiv.org/html/2608.11660#S5.p4.1)\. - Donget al\.\(2022\)Q\. Dong, L\. Li, D\. Dai, C\. Zheng, Z\. Wu, B\. Chang, X\. Sun, J\. Xu, and Z\. SuiA survey for in\-context learning\.arXiv preprint arXiv:2301\.00234\.Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p1.1)\. - Dubeyet al\.\(2024\)A\. Dubeyet al\.The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[Table 3](https://arxiv.org/html/2608.11660#A3.T3.11.4.1.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p1.1)\. - Fanget al\.\(2025\)J\. Fang, H\. Jiang, K\. Wang, Y\. Ma, J\. Shi, X\. Wang, X\. He, and T\. ChuaAlphaEdit: null\-space constrained model editing for language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§C\.2](https://arxiv.org/html/2608.11660#A3.SS2.p3.1),[§1](https://arxiv.org/html/2608.11660#S1.p3.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p3.1),[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Gandhiet al\.\(2025\)K\. Gandhi, A\. K\. Chakravarthy, A\. Singh, N\. Lile, and N\. GoodmanCognitive behaviors that enable self\-improving reasoners, or, four habits of highly effective STars\.InSecond Conference on Language Modeling,Cited by:[§A\.4](https://arxiv.org/html/2608.11660#A1.SS4.p5.1),[§1](https://arxiv.org/html/2608.11660#S1.p6.1),[§5](https://arxiv.org/html/2608.11660#S5.p4.1)\. - Gemma Team \(2024\)Gemma TeamGemma 2: improving open language models at a practical size\.arXiv preprint arXiv:2408\.00118\.Cited by:[Table 3](https://arxiv.org/html/2608.11660#A3.T3.11.5.1.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p1.1)\. - Guet al\.\(2024a\)J\. Gu, H\. Xu, J\. Ma, P\. Lu, Z\. Ling, K\. Chang, and N\. PengModel editing harms general abilities of large language models: regularization to the rescue\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 16801–16819\.Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Guet al\.\(2024b\)Y\. Gu, L\. Dong, F\. Wei, and M\. HuangMiniLLM: knowledge distillation of large language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p4.1)\. - Guptaet al\.\(2024\)A\. Gupta, D\. Sajnani, and G\. AnumanchipalliA unified framework for model editing\.InFindings of the Association for Computational Linguistics: EMNLP 2024,pp\. 15403–15418\.Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Hartvigsenet al\.\(2023\)T\. Hartvigsen, S\. Sankaranarayanan, H\. Palangi, Y\. Kim, and M\. GhassemiAging with GRACE: lifelong model editing with discrete key\-value adaptors\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p2.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p2.1),[§5](https://arxiv.org/html/2608.11660#S5.p3.1)\. - Heet al\.\(2025\)A\. W\. He, D\. Fried, and S\. WelleckRewarding the unlikely: lifting grpo beyond distribution sharpening\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 25559–25571\.Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p4.1)\. - Hochreiter and Schmidhuber \(1997\)S\. Hochreiter and J\. SchmidhuberLong short\-term memory\.Neural Computation9\(8\),pp\. 1735–1780\.Cited by:[§2\.1](https://arxiv.org/html/2608.11660#S2.SS1.p1.2)\. - Huet al\.\(2024\)C\. Hu, P\. Cao, Y\. Chen, K\. Liu, and J\. ZhaoWilKE: wise\-layer knowledge editor for lifelong knowledge editing\.External Links:2402\.10987Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Huet al\.\(2022\)E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. ChenLoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p3.1),[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Jiet al\.\(2023\)Z\. Ji, N\. Lee, R\. Frieske, T\. Yu, D\. Su, Y\. Xu, E\. Ishii, Y\. J\. Bang, A\. Madotto, and P\. FungSurvey of hallucination in natural language generation\.ACM Computing Surveys55\(12\),pp\. 1–38\.Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p1.1)\. - Jianget al\.\(2025\)H\. Jiang, J\. Fang, N\. Zhang, G\. Ma, M\. Wan, X\. Wang, X\. He, and T\. ChuaEdit any knowledge encoded in language models\.arXiv preprint arXiv:2502\.05628\.Cited by:[§C\.2](https://arxiv.org/html/2608.11660#A3.SS2.p3.1),[§1](https://arxiv.org/html/2608.11660#S1.p3.1),[§2\.1](https://arxiv.org/html/2608.11660#S2.SS1.p2.1),[§2\.2](https://arxiv.org/html/2608.11660#S2.SS2.p6.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p3.1),[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Jianget al\.\(2024\)Y\. Jiang, Y\. Wang, C\. Wu, W\. Zhong, X\. Zeng, J\. Gao, L\. Li, X\. Jiang, L\. Shang, R\. Tang,et al\.Learning to edit: aligning llms with knowledge editing\.arXiv preprint arXiv:2402\.11905\.Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p3.1)\. - Jinet al\.\(2026\)W\. Jin, T\. Min, Y\. Yang, D\. Wei, Y\. Zhou, S\. R\. Kadhe, N\. Baracaldo, and K\. LeeEntropy\-aware on\-policy distillation of language models\.InForty\-third International Conference on Machine Learning,Cited by:[§3\.2](https://arxiv.org/html/2608.11660#S3.SS2.p2.3),[§5](https://arxiv.org/html/2608.11660#S5.p4.1)\. - Kakade and Langford \(2002\)S\. Kakade and J\. LangfordApproximately optimal approximate reinforcement learning\.InProceedings of the Nineteenth International Conference on Machine Learning,ICML ’02,San Francisco, CA, USA,pp\. 267–274\.External Links:ISBN 1558608737Cited by:[§A\.4](https://arxiv.org/html/2608.11660#A1.SS4.p4.1)\. - Kojimaet al\.\(2022\)T\. Kojima, S\. S\. Gu, M\. Reid, Y\. Matsuo, and Y\. IwasawaLarge language models are zero\-shot reasoners\.Advances in neural information processing systems35,pp\. 22199–22213\.Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p1.1)\. - Lampinenet al\.\(2025\)A\. K\. Lampinen, A\. Chaudhry, S\. C\. Chan, C\. Wild, D\. Wan, A\. Ku, J\. Bornschein, R\. Pascanu, M\. Shanahan, and J\. L\. McClellandOn the generalization of language models from in\-context learning and finetuning: a controlled study\.arXiv preprint arXiv:2505\.00661\.Cited by:[Remark A\.1](https://arxiv.org/html/2608.11660#A1.Thmremark1.p1.1),[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Leeet al\.\(2025\)J\. Lee, D\. Jung, and H\. LimStepKE: stepwise knowledge editing for multi\-hop question answering\.InFindings of the Association for Computational Linguistics: EMNLP 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),pp\. 7752–7765\.Cited by:[§C\.2](https://arxiv.org/html/2608.11660#A3.SS2.p3.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p3.1)\. - Liet al\.\(2026a\)S\. Li, K\. Zhang, X\. Wang, J\. Liu, and S\. MoFrom fact overwriting to knowledge evolution: causal editing via on\-policy self\-distillation\.External Links:2605\.28303Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p4.1)\. - Liet al\.\(2024\)X\. Li, S\. Li, S\. Song, J\. Yang, J\. Ma, and J\. YuPmet: precise model editing in a transformer\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 18564–18572\.Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Liet al\.\(2026b\)Y\. Li, Y\. Zuo, B\. He, J\. Zhang, C\. Xiao, C\. Qian, T\. Yu, H\. Gao, W\. Yang, Z\. Liu, and N\. DingRethinking on\-policy distillation of large language models: phenomenology, mechanism, and recipe\.External Links:2604\.13016Cited by:[§3\.1](https://arxiv.org/html/2608.11660#S3.SS1.p2.1),[§3\.1](https://arxiv.org/html/2608.11660#S3.SS1.p3.2),[§5](https://arxiv.org/html/2608.11660#S5.p4.1)\. - Liet al\.\(2025\)Z\. Li, H\. Jiang, H\. Chen, B\. Bi, Z\. Zhou, F\. Sun, J\. Fang, and X\. WangReinforced lifelong editing for language models\.InForty\-second International Conference on Machine Learning,Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p3.1)\. - Liuet al\.\(2025a\)T\. Liu, R\. Li, Z\. Dong, H\. Liu, X\. Tang, Q\. Yin, L\. Zhang, H\. Wang, and J\. GaoMitigating heterogeneous token overfitting in LLM knowledge editing\.InForty\-second International Conference on Machine Learning,Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p5.1),[§4\.2](https://arxiv.org/html/2608.11660#S4.SS2.p4.1),[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Liuet al\.\(2025b\)T\. Liu, R\. Li, Y\. Qi, H\. Liu, X\. Tang, T\. Zheng, Q\. Yin, M\. X\. Cheng, J\. Huan, H\. Wang, and J\. GaoUnlocking efficient, scalable, and continual knowledge editing with basis\-level representation fine\-tuning\.InThe Thirteenth International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p3.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p2.1),[§4\.3](https://arxiv.org/html/2608.11660#S4.SS3.p1.1),[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Lu and Lab \(2025\)K\. Lu and T\. M\. LabOn\-policy distillation\.Thinking Machines Lab: Connectionism\.Note:https://thinkingmachines\.ai/blog/on\-policy\-distillationExternal Links:[Document](https://dx.doi.org/10.64434/tml.20251026)Cited by:[§A\.4](https://arxiv.org/html/2608.11660#A1.SS4.p5.1),[§1](https://arxiv.org/html/2608.11660#S1.p6.1),[§5](https://arxiv.org/html/2608.11660#S5.p4.1)\. - Maet al\.\(2024\)J\. Ma, Z\. Ling, N\. Zhang, and J\. GuNeighboring perturbations of knowledge editing on large language models\.arXiv preprint arXiv:2401\.17623\.Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Maet al\.\(2025\)J\. Ma, H\. Wang, H\. Xu, Z\. Ling, and J\. GuPerturbation\-restrained sequential model editing\.External Links:2405\.16821Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Menget al\.\(2022a\)K\. Meng, D\. Bau, A\. Andonian, and Y\. BelinkovLocating and editing factual associations in gpt\.Advances in Neural Information Processing Systems35,pp\. 17359–17372\.Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Menget al\.\(2023\)K\. Meng, A\. S\. Sharma, A\. J\. Andonian, Y\. Belinkov, and D\. BauMass\-editing memory in a transformer\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§C\.2](https://arxiv.org/html/2608.11660#A3.SS2.p3.1),[§1](https://arxiv.org/html/2608.11660#S1.p3.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p3.1)\. - Menget al\.\(2022b\)K\. Meng, A\. S\. Sharma, A\. Andonian, Y\. Belinkov, and D\. BauMass\-editing memory in a transformer\.arXiv preprint arXiv:2210\.07229\.Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Minet al\.\(2023\)S\. Min, K\. Krishna, X\. Lyu, M\. Lewis, W\. Yih, P\. W\. Koh, M\. Iyyer, L\. Zettlemoyer, and H\. HajishirziFActScore: fine\-grained atomic evaluation of factual precision in long form text generation\.InConference on Empirical Methods in Natural Language Processing \(EMNLP\),Cited by:[2nd item](https://arxiv.org/html/2608.11660#A3.I1.i1.I1.i2.p1.1),[1st item](https://arxiv.org/html/2608.11660#S2.I2.i1.p1.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p4.1)\. - Mitchellet al\.\(2021\)E\. Mitchell, C\. Lin, A\. Bosselut, C\. Finn, and C\. D\. ManningFast model editing at scale\.arXiv preprint arXiv:2110\.11309\.Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p3.1)\. - Mitchellet al\.\(2022\)E\. Mitchell, C\. Lin, A\. Bosselut, C\. D\. Manning, and C\. FinnMemory\-based model editing at scale\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p3.1)\. - Ouyanget al\.\(2022\)L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. L\. Wainwright,et al\.Training language models to follow instructions with human feedback\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p1.1)\. - Qiet al\.\(2025\)S\. Qi, B\. Yang, K\. Jiang, X\. Wang, J\. Li, Y\. Zhong, Y\. Yang, and Z\. ZhengIn\-context editing: learning knowledge from self\-induced distributions\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§4\.2](https://arxiv.org/html/2608.11660#S4.SS2.p4.1),[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Qwen Team \(2024\)Qwen TeamQwen2\.5 technical report\.arXiv preprint arXiv:2412\.15115\.Cited by:[Table 3](https://arxiv.org/html/2608.11660#A3.T3.11.2.1.1),[§2\.2](https://arxiv.org/html/2608.11660#S2.SS2.p6.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p1.1)\. - Qwen Team \(2025\)Qwen TeamQwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[Table 3](https://arxiv.org/html/2608.11660#A3.T3.11.3.1.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p1.1)\. - Raffelet al\.\(2020\)C\. Raffel, N\. Shazeer, A\. Roberts, K\. Lee, S\. Narang, M\. Matena, Y\. Zhou, W\. Li, and P\. J\. LiuExploring the limits of transfer learning with a unified text\-to\-text transformer\.The Journal of Machine Learning Research21\(1\),pp\. 5485–5551\.Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p1.1)\. - Rajaramanet al\.\(2020\)N\. Rajaraman, L\. Yang, J\. Jiao, and K\. RamchandranToward the fundamental limits of imitation learning\.Advances in Neural Information Processing Systems33,pp\. 2914–2924\.Cited by:[§A\.4](https://arxiv.org/html/2608.11660#A1.SS4.p2.1)\. - Ross and Bagnell \(2010\)S\. Ross and J\. A\. BagnellEfficient reductions for imitation learning\.Cited by:[§A\.4](https://arxiv.org/html/2608.11660#A1.SS4.p2.1)\. - Rosset al\.\(2011\)S\. Ross, G\. Gordon, and D\. BagnellA reduction of imitation learning and structured prediction to no\-regret online learning\.InProceedings of the fourteenth international conference on artificial intelligence and statistics,pp\. 627–635\.Cited by:[§A\.4](https://arxiv.org/html/2608.11660#A1.SS4.p3.1)\. - Shenfeldet al\.\(2026\)I\. Shenfeld, M\. Damani, J\. Hübotter, and P\. AgrawalSelf\-distillation enables continual learning\.External Links:2601\.19897Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p4.1)\. - Song and Zheng \(2026\)M\. Song and M\. ZhengA survey of on\-policy distillation for large language models\.External Links:2604\.00626Cited by:[§A\.4](https://arxiv.org/html/2608.11660#A1.SS4.p5.1),[§1](https://arxiv.org/html/2608.11660#S1.p6.1),[§3\.1](https://arxiv.org/html/2608.11660#S3.SS1.p2.1),[§5](https://arxiv.org/html/2608.11660#S5.p4.1)\. - Suet al\.\(2025\)Z\. Su, Z\. Huang, K\. Zhang, and X\. Zhang$\\mu$KE: matryoshka unstructured knowledge editing of large language models\.InSecond Conference on Language Modeling,Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p3.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p1.1),[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Sutskeveret al\.\(2014\)I\. Sutskever, O\. Vinyals, and Q\. V\. LeSequence to sequence learning with neural networks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2\.1](https://arxiv.org/html/2608.11660#S2.SS1.p1.2)\. - Swamyet al\.\(2021\)G\. Swamy, S\. Choudhury, J\. A\. Bagnell, and S\. WuOf moments and matching: a game\-theoretic framework for closing the imitation gap\.InInternational Conference on Machine Learning,pp\. 10022–10032\.Cited by:[§A\.4](https://arxiv.org/html/2608.11660#A1.SS4.p3.1)\. - Tanet al\.\(2024\)C\. Tan, G\. Zhang, and J\. FuMassive editing for large language models via meta learning\.InThe Twelfth International Conference on Learning Representations,Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p3.1)\. - Vaswaniet al\.\(2017\)A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, L\. Kaiser, and I\. PolosukhinAttention is all you need\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p1.1)\. - Wanget al\.\(2024a\)H\. Wang, T\. Liu, R\. Li, M\. Cheng, T\. Zhao, and J\. GaoRoseLoRA: row and column\-wise sparse low\-rank adaptation of pre\-trained language model for knowledge editing and fine\-tuning\.External Links:2406\.10777Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p3.1),[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Wanget al\.\(2024b\)P\. Wang, Z\. Li, N\. Zhang, Z\. Xu, Y\. Yao, Y\. Jiang, P\. Xie, F\. Huang, and H\. ChenWISE: rethinking the knowledge memory for lifelong model editing of large language models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p2.1),[§4\.3](https://arxiv.org/html/2608.11660#S4.SS3.p1.1),[§5](https://arxiv.org/html/2608.11660#S5.p3.1)\. - Wanget al\.\(2026\)P\. Wang, B\. Zhou, X\. Tang, J\. Han, and S\. HuFABLE: fine\-grained fact anchoring for unstructured model editing\.InFindings of the Association for Computational Linguistics: ACL 2026,San Diego, California, United States\.Cited by:[§C\.2](https://arxiv.org/html/2608.11660#A3.SS2.p3.1),[§1](https://arxiv.org/html/2608.11660#S1.p4.1),[§1](https://arxiv.org/html/2608.11660#S1.p5.1),[1st item](https://arxiv.org/html/2608.11660#S2.I1.i1.p1.1),[§3\.1](https://arxiv.org/html/2608.11660#S3.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p3.1),[§5](https://arxiv.org/html/2608.11660#S5.p3.1)\. - Wanget al\.\(2023\)S\. Wang, Y\. Zhu, H\. Liu, Z\. Zheng, C\. Chen, and J\. LiKnowledge editing for large language models: a survey\.arXiv preprint arXiv:2310\.16218\.Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.11660#S2.SS1.p2.1),[§5](https://arxiv.org/html/2608.11660#S5.p1.1)\. - Wanget al\.\(2024c\)Y\. Wang, M\. Chen, N\. Peng, and K\. ChangDeepEdit: knowledge editing as decoding with constraints\.External Links:2401\.10471Cited by:[2nd item](https://arxiv.org/html/2608.11660#S2.I1.i2.p1.1),[§5](https://arxiv.org/html/2608.11660#S5.p3.1)\. - Weiet al\.\(2024\)Z\. Wei, L\. Pang, H\. Ding, J\. Deng, H\. Shen, and X\. ChengStable knowledge editing in large language models\.arXiv preprint arXiv:2402\.13048\.Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p3.1)\. - Wuet al\.\(2024a\)X\. Wu, L\. Pan, W\. Y\. Wang, and A\. T\. LuuAKEW: assessing knowledge editing in the wild\.InConference on Empirical Methods in Natural Language Processing \(EMNLP\),Cited by:[1st item](https://arxiv.org/html/2608.11660#A2.I1.i2.I1.i1.p1.1),[2nd item](https://arxiv.org/html/2608.11660#A3.I1.i1.I1.i2.p1.1),[§1](https://arxiv.org/html/2608.11660#S1.p3.1),[2nd item](https://arxiv.org/html/2608.11660#S2.I2.i2.p1.1),[§2\.1](https://arxiv.org/html/2608.11660#S2.SS1.p2.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p4.1),[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Wuet al\.\(2024b\)Z\. Wu, A\. Arora, Z\. Wang, A\. Geiger, D\. Jurafsky, C\. D\. Manning, and C\. PottsReft: representation finetuning for language models\.arXiv preprint arXiv:2404\.03592\.Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Xieet al\.\(2021\)S\. M\. Xie, A\. Raghunathan, P\. Liang, and T\. MaAn explanation of in\-context learning as implicit bayesian inference\.arXiv preprint arXiv:2111\.02080\.Cited by:[Remark A\.1](https://arxiv.org/html/2608.11660#A1.Thmremark1.p1.1)\. - Xionget al\.\(2025\)H\. Xiong, C\. Tan, and W\. ChenIs fine\-tuning an effective solution? reassessing knowledge editing for unstructured data\.External Links:2506\.09672Cited by:[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p3.1)\. - Xuet al\.\(2025\)W\. Xu, R\. Han, Z\. Wang, L\. T\. Le, D\. Madeka, L\. Li, W\. Y\. Wang, R\. Agarwal, C\. Lee, and T\. PfisterSpeculative knowledge distillation: bridging the teacher\-student gap through interleaved sampling\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p4.1)\. - Xuet al\.\(2026\)Y\. Xu, H\. Sang, Z\. Zhou, R\. He, Z\. Wang, and A\. GeramifardBeyond grpo and on\-policy distillation: an empirical sparse\-to\-dense reward principle for language\-model post\-training\.External Links:2605\.12483Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p4.1)\. - Yanget al\.\(2025\)W\. Yang, F\. Sun, J\. Tan, X\. Ma, Q\. Cao, D\. Yin, H\. Shen, and X\. ChengThe mirage of model editing: revisiting evaluation in the wild\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Vienna, Austria\.Cited by:[2nd item](https://arxiv.org/html/2608.11660#A3.I1.i1.I1.i2.p1.1),[1st item](https://arxiv.org/html/2608.11660#S2.I2.i1.p1.1),[§2\.1](https://arxiv.org/html/2608.11660#S2.SS1.p2.1)\. - Yanget al\.\(2026\)W\. Yang, R\. Tang, H\. Zang, D\. Su, Q\. Cao, J\. Wang, H\. Shen, X\. Cheng, and F\. SunFine\-tuning done right in model editing\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p3.1)\. - Yaoet al\.\(2025\)Y\. Yao, J\. Fang, J\. Gu, N\. Zhang, S\. Deng, H\. Chen, and N\. PengCaKE: circuit\-aware editing enables generalizable knowledge learners\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),pp\. 11366–11382\.Cited by:[§C\.2](https://arxiv.org/html/2608.11660#A3.SS2.p3.1),[§1](https://arxiv.org/html/2608.11660#S1.p4.1),[§1](https://arxiv.org/html/2608.11660#S1.p5.1),[§3\.1](https://arxiv.org/html/2608.11660#S3.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p3.1)\. - Yeet al\.\(2026\)T\. Ye, L\. Dong, X\. Wu, S\. Huang, and F\. WeiOn\-policy context distillation for language models\.arXiv preprint arXiv:2602\.12275\.Cited by:[§A\.4](https://arxiv.org/html/2608.11660#A1.SS4.p6.1),[§5](https://arxiv.org/html/2608.11660#S5.p4.1)\. - Yenduriet al\.\(2023\)G\. Yenduri, R\. M, C\. S\. G, S\. Y, G\. Srivastava, P\. K\. R\. Maddikunta, D\. R\. G, R\. H\. Jhaveri, P\. B, W\. Wang, A\. V\. Vasilakos, and T\. R\. GadekalluGenerative pre\-trained transformer: a comprehensive review on enabling technologies, potential applications, emerging challenges, and future directions\.External Links:2305\.10435Cited by:[§2\.1](https://arxiv.org/html/2608.11660#S2.SS1.p1.2)\. - Yuet al\.\(2024\)L\. Yu, Q\. Chen, J\. Zhou, and L\. HeMelo: enhancing model editing with neuron\-indexed dynamic lora\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 19449–19457\.Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p3.1)\. - Yuet al\.\(2026\)X\. Yu, G\. Li, Q\. Si, G\. Zhang, Y\. Xu, C\. Wang, S\. Dong, K\. Tuo, X\. Zeng, K\. Feng, Q\. Wang, Y\. Shi, X\. Hu, X\. Yue, J\. Wang, and S\. YanDOPD: dual on\-policy distillation\.External Links:2606\.30626Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p4.1)\. - Yueet al\.\(2026\)Y\. Yue, Z\. Chen, R\. Lu, A\. Zhao, Z\. Wang, Y\. Yue, S\. Song, and G\. HuangDoes reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model?\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§A\.4](https://arxiv.org/html/2608.11660#A1.SS4.p5.1),[§1](https://arxiv.org/html/2608.11660#S1.p6.1),[§5](https://arxiv.org/html/2608.11660#S5.p4.1)\. - Zhanget al\.\(2024a\)M\. Zhang, X\. Ye, Q\. Liu, P\. Ren, S\. Wu, and Z\. ChenUncovering overfitting in large language model editing\.arXiv preprint arXiv:2410\.07819\.Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p4.1),[§1](https://arxiv.org/html/2608.11660#S1.p5.1),[§2\.2](https://arxiv.org/html/2608.11660#S2.SS2.p1.1),[§4\.2](https://arxiv.org/html/2608.11660#S4.SS2.p4.1),[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Zhanget al\.\(2024\)N\. Zhanget al\.A comprehensive study of knowledge editing for large language models\.arXiv preprint arXiv:2401\.01286\.Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.11660#S2.SS1.p2.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p3.1),[§5](https://arxiv.org/html/2608.11660#S5.p1.1)\. - Zhanget al\.\(2024b\)N\. Zhang, B\. Tian, S\. Cheng, X\. Liang, Y\. Hu, K\. Xue, Y\. Gou, X\. Chen, and H\. ChenInstructedit: instruction\-based knowledge editing for large language models\.arXiv preprint arXiv:2402\.16123\.Cited by:[§5](https://arxiv.org/html/2608.11660#S5.p3.1)\. - Zhaoet al\.\(2026\)S\. Zhao, Z\. Xie, M\. Liu, J\. Huang, G\. Pang, F\. Chen, and A\. GroverSelf\-distilled reasoner: on\-policy self\-distillation for large language models\.External Links:2601\.18734Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p6.1),[§3\.1](https://arxiv.org/html/2608.11660#S3.SS1.p3.1),[§5](https://arxiv.org/html/2608.11660#S5.p4.1)\. - Zhenget al\.\(2023\)C\. Zheng, L\. Li, Q\. Dong, Y\. Fan, Z\. Wu, J\. Xu, and B\. ChangCan we edit factual knowledge by in\-context learning?\.InConference on Empirical Methods in Natural Language Processing \(EMNLP\),Cited by:[§3\.1](https://arxiv.org/html/2608.11660#S3.SS1.p3.1)\. - Zhonget al\.\(2025\)S\. Zhong, Y\. Lu, L\. Shao, B\. Bhushanam, X\. Du, Y\. Wan, Y\. Shi, D\. Zha, Y\. Wang, N\. Liu, K\. Zhou, S\. Xu, K\. Chang, L\. Feng, V\. Chaudhary, and X\. HuMQuAKE\-remastered: multi\-hop knowledge editing can only be advanced with reliable evaluations\.InThe Thirteenth International Conference on Learning Representations,Cited by:[2nd item](https://arxiv.org/html/2608.11660#A3.I1.i2.I1.i2.p1.1),[§1](https://arxiv.org/html/2608.11660#S1.p4.1),[§1](https://arxiv.org/html/2608.11660#S1.p5.1),[2nd item](https://arxiv.org/html/2608.11660#S2.I1.i2.p1.1),[2nd item](https://arxiv.org/html/2608.11660#S2.I2.i2.p1.1),[§2\.2](https://arxiv.org/html/2608.11660#S2.SS2.p1.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p2.1)\. - Zhonget al\.\(2023\)Z\. Zhong, Z\. Wu, C\. D\. Manning, C\. Potts, and D\. ChenMQuAKE: assessing knowledge editing in language models via multi\-hop questions\.InConference on Empirical Methods in Natural Language Processing \(EMNLP\),Cited by:[2nd item](https://arxiv.org/html/2608.11660#A3.I1.i2.I1.i2.p1.1),[§1](https://arxiv.org/html/2608.11660#S1.p4.1),[2nd item](https://arxiv.org/html/2608.11660#S2.I1.i2.p1.1),[2nd item](https://arxiv.org/html/2608.11660#S2.I2.i2.p1.1),[§2\.2](https://arxiv.org/html/2608.11660#S2.SS2.p1.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p2.1),[§4\.2](https://arxiv.org/html/2608.11660#S4.SS2.p5.1)\. - Zhouet al\.\(2023\)C\. Zhou, Q\. Li, C\. Li, J\. Yu, Y\. Liu, G\. Wang, K\. Zhang, C\. Ji, Q\. Yan, L\. He,et al\.A comprehensive survey on pretrained foundation models: a history from bert to chatgpt\.arXiv preprint arXiv:2302\.09419\.Cited by:[§1](https://arxiv.org/html/2608.11660#S1.p1.1)\. - Zhouet al\.\(2026\)Z\. Zhou, M\. Zhang, S\. Wu, X\. Ye, C\. Zhang, Z\. Chen, and P\. RenUncovering context reliance in unstructured knowledge editing\.arXiv preprint arXiv:2602\.19043\.Cited by:[§C\.2](https://arxiv.org/html/2608.11660#A3.SS2.p3.1),[§1](https://arxiv.org/html/2608.11660#S1.p4.1),[§1](https://arxiv.org/html/2608.11660#S1.p5.1),[1st item](https://arxiv.org/html/2608.11660#S2.I1.i1.p1.1),[§2\.2](https://arxiv.org/html/2608.11660#S2.SS2.p1.1),[§2\.2](https://arxiv.org/html/2608.11660#S2.SS2.p6.1),[§3\.1](https://arxiv.org/html/2608.11660#S3.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.11660#S4.SS1.p3.1),[§4\.2](https://arxiv.org/html/2608.11660#S4.SS2.p4.1),[§5](https://arxiv.org/html/2608.11660#S5.p2.1)\. - Zhuet al\.\(2018\)Y\. Zhu, S\. Lu, L\. Zheng, J\. Guo, W\. Zhang, J\. Wang, and Y\. YuTexygen: a benchmarking platform for text generation models\.InInternational ACM SIGIR Conference on Research and Development in Information Retrieval \(SIGIR\),Cited by:[3rd item](https://arxiv.org/html/2608.11660#A3.I1.i1.I1.i3.p1.1)\. ## Appendix AOmitted Theoretical Analysis In this section we develop the analysis behind Theorem[3\.1](https://arxiv.org/html/2608.11660#S3.Thmtheorem1)that was omitted in the main body due to page limits\. We first introduce the notation used in the analysis, then present the main theoretical results and properties of the method, and finally connect HPSE to on\-policy imitation learning\. ### A\.1Notations For completeness, we first collect the notation that will be used in our analysis\. For discrete distributionsp,qp,qover the vocabulary𝒱\\mathcal\{V\}, the \(forward\) Kullback–Leibler divergence is DKL\[p∥q\]≜∑v∈𝒱p\(v\)logp\(v\)q\(v\)\.D\_\{\\text\{KL\}\}\[p\\\|q\]\\;\\triangleq\\;\\sum\_\{v\\in\\mathcal\{V\}\}p\(v\)\\log\\frac\{p\(v\)\}\{q\(v\)\}\.This is the divergence appearing in the distillation loss of Eq\. \([1](https://arxiv.org/html/2608.11660#S3.E1)\)\. For scalarsa,b∈\(0,1\)a,b\\in\(0,1\), the binary KL between the Bernoulli distributions\(a,1−a\)\(a,1\-a\)and\(b,1−b\)\(b,1\-b\)is dbin\(a∥b\)≜alogab\+\(1−a\)log1−a1−b\.d\_\{\\mathrm\{bin\}\}\(a\\\|b\)\\;\\triangleq\\;a\\log\\frac\{a\}\{b\}\+\(1\-a\)\\log\\frac\{1\-a\}\{1\-b\}\.We writeargmaxvp\(v\)\\arg\\max\_\{v\}p\(v\)for the mode ofpp,\|𝒱\|\|\\mathcal\{V\}\|for the vocabulary size, andDmaxD\_\{\\max\}for a uniform upper bound on the per\-token KL \(Assumption[A\.3](https://arxiv.org/html/2608.11660#A1.Thmassumption3)\)\. Recall from Section[3](https://arxiv.org/html/2608.11660#S3)the studentπθ\\pi\_\{\\theta\}and the privileged teacher π⋆\(⋅∣𝐱,𝐲<t\)≜π0\(⋅∣𝐜,𝐱,𝐲<t\)\.\\pi^\{\\star\}\(\\cdot\\mid\\mathbf\{x\},\\mathbf\{y\}\_\{<t\}\)\\;\\triangleq\\;\\pi\_\{0\}\(\\cdot\\mid\\mathbf\{c\},\\mathbf\{x\},\\mathbf\{y\}\_\{<t\}\)\.At steptt, define the teacher’s greedy token by yt⋆≜argmaxv∈𝒱π⋆\(v∣𝐱,𝐲<t\)\.y^\{\\star\}\_\{t\}\\;\\triangleq\\;\\arg\\max\_\{v\\in\\mathcal\{V\}\}\\pi^\{\\star\}\(v\\mid\\mathbf\{x\},\\mathbf\{y\}\_\{<t\}\)\.The hybrid rollout policyπρ\\pi\_\{\\rho\}of Eq\. \([2](https://arxiv.org/html/2608.11660#S3.E2)\) emitsyt⋆y^\{\\star\}\_\{t\}when both conditions of the step\-in gate in Eq\. \([3](https://arxiv.org/html/2608.11660#S3.E3)\) hold: logπ⋆\(yt⋆∣⋅\)−logπθ\(yt⋆∣⋅\)\>τ,π⋆\(yt⋆∣⋅\)\>κ\.\\log\\pi^\{\\star\}\(y^\{\\star\}\_\{t\}\\mid\\cdot\)\-\\log\\pi\_\{\\theta\}\(y^\{\\star\}\_\{t\}\\mid\\cdot\)\>\\tau,\\qquad\\pi^\{\\star\}\(y^\{\\star\}\_\{t\}\\mid\\cdot\)\>\\kappa\.Otherwise, it emits the student’s token\. We call a run ofℓ\\ellconsecutive positions carrying the injected knowledge a*fact span*, and writeFjF\_\{j\}\(j=0,…,ℓ−1j=0,\\dots,\\ell\-1\) for the prefix whose next token is the\(j\+1\)\(j\{\+\}1\)\-th fact tokenyj\+1⋆y^\{\\star\}\_\{j\+1\}, withF=\{F0,…,Fℓ−1\}F=\\\{F\_\{0\},\\dots,F\_\{\\ell\-1\}\\\}\. The*deep coverage*and*fact\-signal*of a rollout policyμ\\mu\(introduced informally in Theorem[3\.1](https://arxiv.org/html/2608.11660#S3.Thmtheorem1)\) are cj\(μ\)\\displaystyle c\_\{j\}\(\\mu\)≜Pr𝐲∼μ\[𝐲reproducesy1⋆,…,yj⋆\],\\displaystyle\\triangleq\\Pr\\nolimits\_\{\\mathbf\{y\}\\sim\\mu\}\\big\[\\,\\mathbf\{y\}\\text\{ reproduces \}y^\{\\star\}\_\{1\},\\dots,y^\{\\star\}\_\{j\}\\,\\big\],\(5\)Sμ\\displaystyle S\_\{\\mu\}≜𝔼𝐲∼μ\[∑t:𝐲<t∈FDKL\[π⋆\(⋅∣𝐱,𝐲<t\)∥πθ\(⋅∣𝐱,𝐲<t\)\]\]\.\\displaystyle\\triangleq\\E\_\{\\mathbf\{y\}\\sim\\mu\}\\Big\[\\,\\sum\\nolimits\_\{t:\\,\\mathbf\{y\}\_\{<t\}\\in F\}D\_\{\\text\{KL\}\}\\big\[\\pi^\{\\star\}\(\\cdot\\mid\\mathbf\{x\},\\mathbf\{y\}\_\{<t\}\)\\,\\big\\\|\\,\\pi\_\{\\theta\}\(\\cdot\\mid\\mathbf\{x\},\\mathbf\{y\}\_\{<t\}\)\\big\]\\Big\]\.\(6\)SinceFjF\_\{j\}is visited exactly when the firstjjfact tokens are reproduced—an event of probabilitycj\(μ\)c\_\{j\}\(\\mu\)—the fact\-signal decomposes as Sμ=∑j=0ℓ−1cj\(μ\)DKL\[πFj⋆∥πθ,Fj\],πFj⋆≜π⋆\(⋅∣𝐱,Fj\),πθ,Fj≜πθ\(⋅∣𝐱,Fj\)\.\\displaystyle S\_\{\\mu\}\\;=\\;\\sum\_\{j=0\}^\{\\ell\-1\}c\_\{j\}\(\\mu\)\\,D\_\{\\text\{KL\}\}\\big\[\\pi^\{\\star\}\_\{F\_\{j\}\}\\,\\big\\\|\\,\\pi\_\{\\theta,F\_\{j\}\}\\big\],\\qquad\\pi^\{\\star\}\_\{F\_\{j\}\}\\triangleq\\pi^\{\\star\}\(\\cdot\\mid\\mathbf\{x\},F\_\{j\}\),\\quad\\pi\_\{\\theta,F\_\{j\}\}\\triangleq\\pi\_\{\\theta\}\(\\cdot\\mid\\mathbf\{x\},F\_\{j\}\)\.\(7\) ### A\.2Main Result: Signal Separation Given the notation introduced above, we now present our main theoretical result\. Our analysis is built upon the following three assumptions\. ###### Assumption A\.1\(Novel edit\)\. The edit introduces a length\-ℓ\\ellrun of new\-knowledge tokensy1⋆,…,yℓ⋆y^\{\\star\}\_\{1\},\\dots,y^\{\\star\}\_\{\\ell\}that the pre\-edit student lacks but the in\-context teacher supplies\. At every fact prefixFjF\_\{j\}, πθ\(yj\+1⋆∣Fj\)≤ρ,π⋆\(yj\+1⋆∣Fj\)\>κ\.\\pi\_\{\\theta\}\(y^\{\\star\}\_\{j\+1\}\\mid F\_\{j\}\)\\leq\\rho,\\qquad\\pi^\{\\star\}\(y^\{\\star\}\_\{j\+1\}\\mid F\_\{j\}\)\>\\kappa\.Moreover, the edit is*τ\\tau\-detectable*: ρ<κe−τ\.\\rho<\\kappa e^\{\-\\tau\}\. ###### Assumption A\.2\(Stationary student\)\. The analysis holdsπθ\\pi\_\{\\theta\}fixed during each rollout\. This matches the stop\-gradient inner loop of Algorithm[1](https://arxiv.org/html/2608.11660#alg1), which draws one rollout per round and treats it as a fixed dataset for itsMMupdates\. For the sampling\-cost statement \(will be seen in Theorem[A\.3](https://arxiv.org/html/2608.11660#A1.Thmtheorem3)later\), the compared rollouts are also drawn i\.i\.d\. ###### Assumption A\.3\(Regularity\)\. The per\-token KL is uniformly bounded: DKL\[πt⋆∥πθ,t\]≤Dmax<∞\.D\_\{\\text\{KL\}\}\[\\pi^\{\\star\}\_\{t\}\\\|\\pi\_\{\\theta,t\}\]\\leq D\_\{\\max\}<\\infty\. We first prove Theorem[3\.1](https://arxiv.org/html/2608.11660#S3.Thmtheorem1)for student rollouts sampled at temperature11\. Remark[A\.7](https://arxiv.org/html/2608.11660#A1.Thmremark7)later extends the coverage argument to greedy decoding, and Theorem[A\.3](https://arxiv.org/html/2608.11660#A1.Thmtheorem3)examines the per\-round signal and sampling cost under both decoding regimes\. Throughout this section,SμS\_\{\\mu\}refers only to the hybrid\-KL term\. The NLL anchorλ𝒥NLL\\lambda\\mathcal\{J\}\_\{\\mathrm\{NLL\}\}in Eq\. \([4](https://arxiv.org/html/2608.11660#S3.E4)\) is not included in the signal comparison\. Corollary[A\.2](https://arxiv.org/html/2608.11660#A1.Thmtheorem2)below verifies that including the anchor on both sides leaves the comparison unchanged\. We now present the formal version of Theorem[3\.1](https://arxiv.org/html/2608.11660#S3.Thmtheorem1)using the notation of Appendix[A\.1](https://arxiv.org/html/2608.11660#A1.SS1)\. ###### Theorem A\.1\(Signal separation; formal version of Theorem[3\.1](https://arxiv.org/html/2608.11660#S3.Thmtheorem1)\)\. Under Assumptions[A\.1](https://arxiv.org/html/2608.11660#A1.Thmassumption1)–[A\.3](https://arxiv.org/html/2608.11660#A1.Thmassumption3), consider a fact span of lengthℓ\\elland a student rollout sampled at temperature11\. Define d\(κ,τ\)≜κτ\+\(1−κ\)log1−κ1−κe−τ\>0\.d\(\\kappa,\\tau\)\\triangleq\\kappa\\tau\+\(1\-\\kappa\)\\log\\frac\{1\-\\kappa\}\{1\-\\kappa e^\{\-\\tau\}\}\>0\.Then the student and hybrid rollout policies satisfy: 1. 1\.*\(Coverage\.\)*At every depthj≤ℓj\\leq\\ell, cj\(πθ\)≤e−τj,cj\(πρ\)=1,cj\(πρ\)cj\(πθ\)≥eτj\.c\_\{j\}\(\\pi\_\{\\theta\}\)\\leq e^\{\-\\tau j\},\\qquad c\_\{j\}\(\\pi\_\{\\rho\}\)=1,\\qquad\\frac\{c\_\{j\}\(\\pi\_\{\\rho\}\)\}\{c\_\{j\}\(\\pi\_\{\\theta\}\)\}\\geq e^\{\\tau j\}\. 2. 2\.*\(Signal\.\)*The fact\-signals satisfy Sπθ≤Dmax1−e−τ=Θ\(1\),Sπρ≥ℓd\(κ,τ\)=Θ\(ℓ\),S\_\{\\pi\_\{\\theta\}\}\\leq\\frac\{D\_\{\\max\}\}\{1\-e^\{\-\\tau\}\}=\\Theta\(1\),\\qquad S\_\{\\pi\_\{\\rho\}\}\\geq\\ell\\,d\(\\kappa,\\tau\)=\\Theta\(\\ell\),and consequently SπρSπθ=Ω\(ℓ\)\.\\frac\{S\_\{\\pi\_\{\\rho\}\}\}\{S\_\{\\pi\_\{\\theta\}\}\}=\\Omega\(\\ell\)\. We first establish the depth\-wise coverage separation\. ###### Proof of coverage\. Let\(ℱt\)t≥0\(\\mathcal\{F\}\_\{t\}\)\_\{t\\geq 0\}be the filtration generated by the rollout, and define At≜\{𝐲1:t=y1:t⋆\},ct\(μ\)=Prμ\[At\]\.A\_\{t\}\\triangleq\\\{\\mathbf\{y\}\_\{1:t\}=y^\{\\star\}\_\{1:t\}\\\},\\qquad c\_\{t\}\(\\mu\)=\\Pr\_\{\\mu\}\[A\_\{t\}\]\.ThusAtA\_\{t\}is the event that the rollout has reproduced the fact span through depthtt\. We condition throughout on the rollout reaching the span entranceF0F\_\{0\}, soc0=1c\_\{0\}=1for both policies, and analyze coverage*within*the span\. OnAt−1A\_\{t\-1\}, the next token equalsyt⋆y^\{\\star\}\_\{t\}with conditional probability πθ\(yt⋆∣Ft−1\)≤ρ\\pi\_\{\\theta\}\(y^\{\\star\}\_\{t\}\\mid F\_\{t\-1\}\)\\leq\\rhoby novelty \(Assumption[A\.1](https://arxiv.org/html/2608.11660#A1.Thmassumption1)\); offAt−1A\_\{t\-1\}, the eventAtA\_\{t\}is impossible\. Novelty also*derives*the gate: logπ⋆\(yt⋆∣Ft−1\)−logπθ\(yt⋆∣Ft−1\)≥logκρ\>τ,π⋆\(yt⋆∣Ft−1\)≥κ\.\\log\\pi^\{\\star\}\(y^\{\\star\}\_\{t\}\\mid F\_\{t\-1\}\)\-\\log\\pi\_\{\\theta\}\(y^\{\\star\}\_\{t\}\\mid F\_\{t\-1\}\)\\geq\\log\\frac\{\\kappa\}\{\\rho\}\>\\tau,\\qquad\\pi^\{\\star\}\(y^\{\\star\}\_\{t\}\\mid F\_\{t\-1\}\)\\geq\\kappa\.Hence the step\-in fires at each fact token—a consequence, not an assumption\. For the student \(μ=πθ\\mu=\\pi\_\{\\theta\}\), 𝔼\[𝟏At∣ℱt−1\]=𝟏At−1πθ\(yt⋆∣Ft−1\)≤ρ1At−1\.\\E\\\!\\big\[\\mathbf\{1\}\_\{A\_\{t\}\}\\mid\\mathcal\{F\}\_\{t\-1\}\\big\]=\\mathbf\{1\}\_\{A\_\{t\-1\}\}\\,\\pi\_\{\\theta\}\(y^\{\\star\}\_\{t\}\\mid F\_\{t\-1\}\)\\leq\\rho\\,\\mathbf\{1\}\_\{A\_\{t\-1\}\}\.\(8\)Define Mt≜ρ−t𝟏At\.M\_\{t\}\\triangleq\\rho^\{\-t\}\\mathbf\{1\}\_\{A\_\{t\}\}\.ThenMtM\_\{t\}is a nonnegative supermartingale\. Optional stopping \(equivalently, iterating the recursion in Eq\. \([8](https://arxiv.org/html/2608.11660#A1.E8)\)\) gives𝔼\[Mj\]≤M0=1\\E\[M\_\{j\}\]\\leq M\_\{0\}=1, i\.e\. cj\(πθ\)=𝔼\[𝟏Aj\]≤ρj≤e−τjfor allj≤ℓ,c\_\{j\}\(\\pi\_\{\\theta\}\)=\\E\[\\mathbf\{1\}\_\{A\_\{j\}\}\]\\leq\\rho^\{\\,j\}\\leq e^\{\-\\tau j\}\\qquad\\text\{for all \}j\\leq\\ell,\(9\)the last inequality following fromρ<κe−τ≤e−τ\\rho<\\kappa e^\{\-\\tau\}\\leq e^\{\-\\tau\}\. Equivalently, define the first\-deviation time and reproduction depth by T≜inf\{t≥1:𝟏At=0\},D≜T−1\.T\\triangleq\\inf\\\{t\\geq 1:\\mathbf\{1\}\_\{A\_\{t\}\}=0\\\},\\qquad D\\triangleq T\-1\.HereTTis a stopping time because\{T≤t\}=Atc∈ℱt\\\{T\\leq t\\\}=A\_\{t\}^\{c\}\\in\\mathcal\{F\}\_\{t\}\. The reproduction depth satisfies Pr\[D≥j\]≤ρj,𝔼\[D\]≤ρ1−ρ\.\\Pr\[D\\geq j\]\\leq\\rho^\{\\,j\},\\qquad\\E\[D\]\\leq\\frac\{\\rho\}\{1\-\\rho\}\.Thus the student’s own rollout leaves the fact path within a constant number of tokens in expectation\. For the hybrid \(μ=πρ\\mu=\\pi\_\{\\rho\}\), the gate fires at everyFjF\_\{j\}as derived above\. Under greedy step\-in, the teacher tokenyj\+1⋆y^\{\\star\}\_\{j\+1\}is injected deterministically andAℓA\_\{\\ell\}holds surely\. Therefore, cj\(πρ\)=1,cj\(πρ\)cj\(πθ\)≥eτj\.c\_\{j\}\(\\pi\_\{\\rho\}\)=1,\\qquad\\frac\{c\_\{j\}\(\\pi\_\{\\rho\}\)\}\{c\_\{j\}\(\\pi\_\{\\theta\}\)\}\\geq e^\{\\tau j\}\.This completes our proof\. ∎ We next convert the coverage separation into a fact\-signal separation\. ###### Proof of signal\. For μ∈\{πθ,πρ\},\\mu\\in\\\{\\pi\_\{\\theta\},\\pi\_\{\\rho\}\\\},Eq\. \([7](https://arxiv.org/html/2608.11660#A1.E7)\) gives Sμ=∑j=0ℓ−1cj\(μ\)KLj,KLj≜DKL\[πFj⋆∥πθ,Fj\]\.S\_\{\\mu\}=\\sum\_\{j=0\}^\{\\ell\-1\}c\_\{j\}\(\\mu\)\\,\\mathrm\{KL\}\_\{j\},\\qquad\\mathrm\{KL\}\_\{j\}\\triangleq D\_\{\\text\{KL\}\}\[\\pi^\{\\star\}\_\{F\_\{j\}\}\\\|\\pi\_\{\\theta,F\_\{j\}\}\]\.The per\-prefix termKLj\\mathrm\{KL\}\_\{j\}is the same under both rollout policies, so the comparison is determined by the coverage weightscj\(μ\)c\_\{j\}\(\\mu\)\. Assumption[A\.3](https://arxiv.org/html/2608.11660#A1.Thmassumption3)givesKLj≤Dmax\\mathrm\{KL\}\_\{j\}\\leq D\_\{\\max\}\. Combining this bound with the coverage result in Eq\. \([9](https://arxiv.org/html/2608.11660#A1.E9)\), we obtain Sπθ\\displaystyle S\_\{\\pi\_\{\\theta\}\}≤Dmax∑j=0ℓ−1e−τj≤Dmax1−e−τ=Θ\(1\)\.\\displaystyle\\leq D\_\{\\max\}\\sum\_\{j=0\}^\{\\ell\-1\}e^\{\-\\tau j\}\\leq\\frac\{D\_\{\\max\}\}\{1\-e^\{\-\\tau\}\}=\\Theta\(1\)\.This bound is independent ofℓ\\ell: the geometric coverage suppresses every deep term and the student’s signal is front\-loaded on the shallowest tokens\. For the hybrid, cj\(πρ\)=1⟹Sπρ=∑j=0ℓ−1KLj,c\_\{j\}\(\\pi\_\{\\rho\}\)=1\\qquad\\Longrightarrow\\qquad S\_\{\\pi\_\{\\rho\}\}=\\sum\_\{j=0\}^\{\\ell\-1\}\\mathrm\{KL\}\_\{j\},so it remains to lower\-bound eachKLj\\mathrm\{KL\}\_\{j\}\. FixFjF\_\{j\}and set a≜π⋆\(yj\+1⋆∣Fj\),b≜πθ\(yj\+1⋆∣Fj\)\.a\\triangleq\\pi^\{\\star\}\(y^\{\\star\}\_\{j\+1\}\\mid F\_\{j\}\),\\qquad b\\triangleq\\pi\_\{\\theta\}\(y^\{\\star\}\_\{j\+1\}\\mid F\_\{j\}\)\.Novelty gives a≥κ,b≤ρ<κe−τ≤e−τa\.a\\geq\\kappa,\\qquad b\\leq\\rho<\\kappa e^\{\-\\tau\}\\leq e^\{\-\\tau\}a\.Coarsening the vocabulary to the binary event\{y=yj\+1⋆\}\\\{y=y^\{\\star\}\_\{j\+1\}\\\}, the data\-processing inequality\([9](https://arxiv.org/html/2608.11660#bib.bib93)\)gives KLj≥dbin\(a∥b\)\.\\mathrm\{KL\}\_\{j\}\\geq d\_\{\\mathrm\{bin\}\}\(a\\\|b\)\.Forb<ab<a, ∂bdbin\(a∥b\)\\displaystyle\\partial\_\{b\}d\_\{\\mathrm\{bin\}\}\(a\\\|b\)=−ab\+1−a1−b<0,\\displaystyle=\-\\frac\{a\}\{b\}\+\\frac\{1\-a\}\{1\-b\}<0,dbin\(a∥b\)\\displaystyle d\_\{\\mathrm\{bin\}\}\(a\\\|b\)≥dbin\(a∥e−τa\)≕g\(a\),\\displaystyle\\geq d\_\{\\mathrm\{bin\}\}\(a\\\|e^\{\-\\tau\}a\)\\eqqcolon g\(a\),g\(a\)\\displaystyle g\(a\)=aτ\+\(1−a\)log1−a1−e−τa\.\\displaystyle=a\\tau\+\(1\-a\)\\log\\frac\{1\-a\}\{1\-e^\{\-\\tau\}a\}\.Differentiating and usinglogx≥\(x−1\)/x\\log x\\geq\(x\-1\)/x, g′\(a\)\\displaystyle g^\{\\prime\}\(a\)=τ−1\+log1−e−τa1−a\\displaystyle=\\tau\-1\+\\log\\frac\{1\-e^\{\-\\tau\}a\}\{1\-a\}\+\(1−a\)e−τ1−e−τa\\displaystyle\\quad\+\\frac\{\(1\-a\)e^\{\-\\tau\}\}\{1\-e^\{\-\\tau\}a\}≥τ−1\+a\(1−e−τ\)\+\(1−a\)e−τ1−e−τa\\displaystyle\\geq\\tau\-1\+\\frac\{a\(1\-e^\{\-\\tau\}\)\+\(1\-a\)e^\{\-\\tau\}\}\{1\-e^\{\-\\tau\}a\}=τ−1\+a\+e−τ−2ae−τ1−e−τa\.\\displaystyle=\\tau\-1\+\\frac\{a\+e^\{\-\\tau\}\-2ae^\{\-\\tau\}\}\{1\-e^\{\-\\tau\}a\}\.Moreover, a\+e−τ−2ae−τ1−e−τa−e−τ=a\(1−e−τ\)21−e−τa≥0\.\\frac\{a\+e^\{\-\\tau\}\-2ae^\{\-\\tau\}\}\{1\-e^\{\-\\tau\}a\}\-e^\{\-\\tau\}=\\frac\{a\(1\-e^\{\-\\tau\}\)^\{2\}\}\{1\-e^\{\-\\tau\}a\}\\geq 0\.Thusg′\(a\)≥τ−1\+e−τ≥0g^\{\\prime\}\(a\)\\geq\\tau\-1\+e^\{\-\\tau\}\\geq 0forτ\>0\\tau\>0; the last quantity vanishes only atτ=0\\tau=0\. Henceggis nondecreasing on\(0,1\)\(0,1\)and KLj≥g\(κ\)=d\(κ,τ\)\.\\mathrm\{KL\}\_\{j\}\\geq g\(\\kappa\)=d\(\\kappa,\\tau\)\.Using log1−κ1−κe−τ≥−κ\(1−e−τ\)1−κ,\\log\\frac\{1\-\\kappa\}\{1\-\\kappa e^\{\-\\tau\}\}\\geq\-\\frac\{\\kappa\(1\-e^\{\-\\tau\}\)\}\{1\-\\kappa\},we further have d\(κ,τ\)\\displaystyle d\(\\kappa,\\tau\)=κτ\+\(1−κ\)log1−κ1−κe−τ\\displaystyle=\\kappa\\tau\+\(1\-\\kappa\)\\log\\frac\{1\-\\kappa\}\{1\-\\kappa e^\{\-\\tau\}\}≥κ\(τ−1\+e−τ\)\>0\.\\displaystyle\\geq\\kappa\(\\tau\-1\+e^\{\-\\tau\}\)\>0\.Summing over the span yields Sπρ≥ℓd\(κ,τ\)=Θ\(ℓ\),SπρSπθ≥ℓd\(κ,τ\)\(1−e−τ\)Dmax=Ω\(ℓ\)\.S\_\{\\pi\_\{\\rho\}\}\\geq\\ell\\,d\(\\kappa,\\tau\)=\\Theta\(\\ell\),\\qquad\\frac\{S\_\{\\pi\_\{\\rho\}\}\}\{S\_\{\\pi\_\{\\theta\}\}\}\\geq\\frac\{\\ell\\,d\(\\kappa,\\tau\)\(1\-e^\{\-\\tau\}\)\}\{D\_\{\\max\}\}=\\Omega\(\\ell\)\.This completes our proof\. ∎ Theorem[A\.1](https://arxiv.org/html/2608.11660#A1.Thmtheorem1)compares the two rollout policies through the distillation term alone, whereas the full objective in Eq\. \([4](https://arxiv.org/html/2608.11660#S3.E4)\) also carries the NLL anchorλ𝒥NLL\\lambda\\mathcal\{J\}\_\{\\mathrm\{NLL\}\}\. We next verify that including the anchor on both sides does not alter the separation\. The key condition is that the anchor supervises a different set of prefixes\. ###### Assumption A\.4\(Context separation\)\. The fact prefixes and the passage prefixes differ as token sequences: Fj≠𝐜<tfor allj=0,…,ℓ−1andt=1,…,\|𝐜\|\.F\_\{j\}\\neq\\mathbf\{c\}\_\{<t\}\\qquad\\text\{for all \}j=0,\\dots,\\ell\-1\\text\{ and \}t=1,\\dots,\|\\mathbf\{c\}\|\. ###### Corollary A\.2\(Signal separation with the NLL anchor\)\. Under Assumptions[A\.1](https://arxiv.org/html/2608.11660#A1.Thmassumption1)–[A\.3](https://arxiv.org/html/2608.11660#A1.Thmassumption3)and[A\.4](https://arxiv.org/html/2608.11660#A1.Thmassumption4), augment both objectives in Theorem[A\.1](https://arxiv.org/html/2608.11660#A1.Thmtheorem1)with the anchor, i\.e\., compare𝒥\(θ\)\+λ𝒥NLL\(θ\)\\mathcal\{J\}\(\\theta\)\+\\lambda\\mathcal\{J\}\_\{\\mathrm\{NLL\}\}\(\\theta\)of Eq\. \([1](https://arxiv.org/html/2608.11660#S3.E1)\) against𝒥HPSE\(θ\)\\mathcal\{J\}\_\{\\mathrm\{\{HPSE\}\}\}\(\\theta\)of Eq\. \([4](https://arxiv.org/html/2608.11660#S3.E4)\)\. Then: 1. 1\.*\(Zero explicit anchor mass on fact prefixes\.\)*The anchor places no explicit supervision term on anyFjF\_\{j\}\. Hence the fact\-signals of the anchored objectives coincide withSπθS\_\{\\pi\_\{\\theta\}\}andSπρS\_\{\\pi\_\{\\rho\}\}, and the bounds of Theorem[A\.1](https://arxiv.org/html/2608.11660#A1.Thmtheorem1)hold verbatim: Sπθ≤Dmax1−e−τ=Θ\(1\),Sπρ≥ℓd\(κ,τ\)=Θ\(ℓ\)\.S\_\{\\pi\_\{\\theta\}\}\\leq\\frac\{D\_\{\\max\}\}\{1\-e^\{\-\\tau\}\}=\\Theta\(1\),\\qquad S\_\{\\pi\_\{\\rho\}\}\\geq\\ell\\,d\(\\kappa,\\tau\)=\\Theta\(\\ell\)\. 2. 2\.*\(Explicit constraint at fact prefixes\.\)*Viewed as a functional of the per\-state conditionals, the anchored OPSD objective contains no term atFjF\_\{j\}beyond the KL term of weightcj\(πθ\)≤e−τjc\_\{j\}\(\\pi\_\{\\theta\}\)\\leq e^\{\-\\tau j\}, which reduces to the single entrance term \(j=0j=0\) under the greedy hypothesis of Theorem[A\.3](https://arxiv.org/html/2608.11660#A1.Thmtheorem3)\. In contrast, any zero\-loss minimizer of the anchored hybrid objective satisfiesπθ\(⋅∣Fj\)=π⋆\(⋅∣Fj\)\\pi\_\{\\theta\}\(\\cdot\\mid F\_\{j\}\)=\\pi^\{\\star\}\(\\cdot\\mid F\_\{j\}\)at everyFjF\_\{j\}\. ###### Proof\. The anchor decomposes over passage prefixes, 𝒥NLL\(θ\)=−logπθ\(𝐜∣𝐱\)=−∑t=1\|𝐜\|logπθ\(ct∣𝐱,𝐜<t\),\\mathcal\{J\}\_\{\\mathrm\{NLL\}\}\(\\theta\)=\-\\log\\pi\_\{\\theta\}\(\\mathbf\{c\}\\mid\\mathbf\{x\}\)=\-\\sum\_\{t=1\}^\{\|\\mathbf\{c\}\|\}\\log\\pi\_\{\\theta\}\(c\_\{t\}\\mid\\mathbf\{x\},\\mathbf\{c\}\_\{<t\}\),so it depends on the conditionals ofπθ\\pi\_\{\\theta\}only at the states\(𝐱,𝐜<t\)\(\\mathbf\{x\},\\mathbf\{c\}\_\{<t\}\)\. By Assumption[A\.4](https://arxiv.org/html/2608.11660#A1.Thmassumption4)these states differ from everyFjF\_\{j\}, so the anchor contributes no term to the fact\-signal of Eq\. \([6](https://arxiv.org/html/2608.11660#A1.E6)\), which sums only over prefixes inFF\. Claim \(i\) then follows from Theorem[A\.1](https://arxiv.org/html/2608.11660#A1.Thmtheorem1)\. For claim \(ii\), the fact\-prefix contribution to the anchored OPSD objective is the KL term weighted bycj\(πθ\)≤e−τjc\_\{j\}\(\\pi\_\{\\theta\}\)\\leq e^\{\-\\tau j\}from Eq\. \([9](https://arxiv.org/html/2608.11660#A1.E9)\), and is bounded bye−τjDmaxe^\{\-\\tau j\}D\_\{\\max\}under Assumption[A\.3](https://arxiv.org/html/2608.11660#A1.Thmassumption3); under the greedy hypothesis of Theorem[A\.3](https://arxiv.org/html/2608.11660#A1.Thmtheorem3), onlyF0F\_\{0\}is visited and the contribution reduces to the single term there\. The hybrid statement repeats the argument of Proposition[A\.4](https://arxiv.org/html/2608.11660#A1.Thmtheorem4)\(ii\) with the anchor present: the anchor does not constrainπθ\(⋅∣Fj\)\\pi\_\{\\theta\}\(\\cdot\\mid F\_\{j\}\)by Assumption[A\.4](https://arxiv.org/html/2608.11660#A1.Thmassumption4), while the hybrid distillation term assigns weight one to everyFjF\_\{j\}and the forward KL vanishes only at equality\. This completes our proof\. ∎ Theorem[3\.1](https://arxiv.org/html/2608.11660#S3.Thmtheorem1)characterizes the signal available within a single training round\. Under a student rollout, supervision on deeper fact tokens becomes exponentially rare\. We next examine how the decoding regime shapes this per\-round signal and the cost of sampling deep fact prefixes\. ###### Theorem A\.3\(Effect of the decoding regime\)\. Beyond the per\-round signal of Theorem[3\.1](https://arxiv.org/html/2608.11660#S3.Thmtheorem1): 1. 1\.*\(Greedy, per round\.\)*Under greedy decoding, suppose the edit is counterfactual at the span entrance: argmaxvπθ\(v∣F0\)≠y1⋆\.\\arg\\max\_\{v\}\\pi\_\{\\theta\}\(v\\mid F\_\{0\}\)\\neq y^\{\\star\}\_\{1\}\.Then a student\-only \(OPSD\) rollout diverges at the entrance: it visitsF0F\_\{0\}but no deeperFjF\_\{j\}withj≥1j\\geq 1, whereas the hybrid injects the entire span\. Their per\-round fact\-signals satisfy Sπθ=DKL\[πF0⋆∥πθ,F0\]≤Dmax=O\(1\),Sπρ≥ℓd\(κ,τ\)=Ω\(ℓ\)\.S\_\{\\pi\_\{\\theta\}\}=D\_\{\\text\{KL\}\}\\big\[\\pi^\{\\star\}\_\{F\_\{0\}\}\\,\\big\\\|\\,\\pi\_\{\\theta,F\_\{0\}\}\\big\]\\leq D\_\{\\max\}=O\(1\),\\qquad S\_\{\\pi\_\{\\rho\}\}\\geq\\ell\\,d\(\\kappa,\\tau\)=\\Omega\(\\ell\)\. 2. 2\.*\(Sampled, waiting time\.\)*Under temperature\-11sampling, a*fixed*student first reaches depthjjonly afterΩ\(eτj\)\\Omega\(e^\{\\tau j\}\)draws in expectation \(Eq\. \([9](https://arxiv.org/html/2608.11660#A1.E9)\)\)\. #### Greedy decoding\. We first establish the per\-round statement under greedy decoding\. ###### Proof of the greedy statement\. Greedy decoding makes the student rollout the deterministic argmax path\. Conditioned on reaching the span entrance, as assumed throughout, the rollout visitsF0F\_\{0\}and deposits the single KL term there, bounded byDmaxD\_\{\\max\}under Assumption[A\.3](https://arxiv.org/html/2608.11660#A1.Thmassumption3)\. By hypothesis its next token differs fromy1⋆y^\{\\star\}\_\{1\}, so no deeperFjF\_\{j\}is visited and no further term accrues\. The hybrid injects everyyj\+1⋆y^\{\\star\}\_\{j\+1\}by the coverage result in Theorem[A\.1](https://arxiv.org/html/2608.11660#A1.Thmtheorem1)\(i\), and its signal follows from the lower bound in Theorem[A\.1](https://arxiv.org/html/2608.11660#A1.Thmtheorem1)\(ii\)\. This completes our proof\. ∎ #### Sampled decoding\. We next establish the learning\-time statement under temperature\-11sampling\. ###### Proof of the sampled statement\. For a fixed student, the draws are i\.i\.d\. by Assumption[A\.2](https://arxiv.org/html/2608.11660#A1.Thmassumption2), so the first\-success \(geometric\) waiting time to reach depthjjis1/cj\(πθ\)≥eτj1/c\_\{j\}\(\\pi\_\{\\theta\}\)\\geq e^\{\\tau j\}by Eq\. \([9](https://arxiv.org/html/2608.11660#A1.E9)\)\. This completes our proof\. ∎ ### A\.3Self\-Termination, Consistency, and Locality We next study three properties of the proposed intervention: self\-termination, consistency, and locality\. ###### Proposition A\.4\(Self\-termination, consistency, and locality\)\. 1. \(i\)*Self\-termination\.*Once the per\-token gap on the span has fallen to at mostτ\\tau, gate condition \(i\) fails everywhere on the span, and no step\-in fires there: πρ=πθalong the span\.\\pi\_\{\\rho\}=\\pi\_\{\\theta\}\\quad\\text\{along the span\}\.Under the off\-fact condition of part \(iii\), the two policies coincide along the entire rollout, and𝒥HPSE\\mathcal\{J\}\_\{\\mathrm\{\{HPSE\}\}\}reduces to the OPSD KL objective plus the NLL anchorλ𝒥NLL\\lambda\\mathcal\{J\}\_\{\\mathrm\{NLL\}\}\. 2. \(ii\)*Consistency\.*The step\-in changes only the rollout distribution, not the per\-state target\. At each visited statess, both objectives minimize DKL\[π⋆\(⋅∣s\)∥πθ\(⋅∣s\)\],D\_\{\\text\{KL\}\}\[\\pi^\{\\star\}\(\\cdot\\mid s\)\\\|\\pi\_\{\\theta\}\(\\cdot\\mid s\)\],weightingssby its visitation\. On each fact prefixFjF\_\{j\}, the two weights satisfy cj\(πρ\)=1,cj\(πθ\)≤e−τj\.c\_\{j\}\(\\pi\_\{\\rho\}\)=1,\\qquad c\_\{j\}\(\\pi\_\{\\theta\}\)\\leq e^\{\-\\tau j\}\.Hence any zero\-loss minimizer of the hybrid objective satisfies πθ\(⋅∣Fj\)=π⋆\(⋅∣Fj\),\\pi\_\{\\theta\}\(\\cdot\\mid F\_\{j\}\)=\\pi^\{\\star\}\(\\cdot\\mid F\_\{j\}\),while the OPSD constraint there is exponentially weak and, under greedy divergence, absent beyond the span entrance\. 3. \(iii\)*Locality\.*Suppose the edit is local: there isδloc≤τ\\delta\_\{\\mathrm\{loc\}\}\\leq\\tausuch that, at every off\-fact prefixss, logπ⋆\(y⋆∣s\)−logπθ\(y⋆∣s\)≤δloc\.\\log\\pi^\{\\star\}\(y^\{\\star\}\\mid s\)\-\\log\\pi\_\{\\theta\}\(y^\{\\star\}\\mid s\)\\leq\\delta\_\{\\mathrm\{loc\}\}\.Then the gate never fires off\-fact andπρ=πθ\\pi\_\{\\rho\}=\\pi\_\{\\theta\}there\. The step\-in therefore introduces no intervention beyond the OPSD update at any fixed off\-fact state\. #### Self\-termination\. We first prove that the intervention switches off after the fact is learned\. ###### Proof of self\-termination\. The conditiongap≤τ\\text\{gap\}\\leq\\taumakes gate condition \(i\) false at every span position, so the hybrid emits the student token throughout the span\. If in addition the off\-fact condition of part \(iii\) holds, no gate fires anywhere, and the objective is the KL term of Eq\. \([1](https://arxiv.org/html/2608.11660#S3.E1)\) plusλ𝒥NLL\\lambda\\mathcal\{J\}\_\{\\mathrm\{NLL\}\}from Eq\. \([4](https://arxiv.org/html/2608.11660#S3.E4)\)\. This completes our proof\. ∎ #### Consistency\. We next show that the intervention changes state visitation without changing the per\-state target\. ###### Proof of consistency\. For either rollout policy, write the objective as ∑sdμ\(s\)DKL\[π⋆\(⋅∣s\)∥πθ\(⋅∣s\)\],μ∈\{πθ,πρ\}\.\\sum\_\{s\}d\_\{\\mu\}\(s\)\\,D\_\{\\text\{KL\}\}\[\\pi^\{\\star\}\(\\cdot\\mid s\)\\\|\\pi\_\{\\theta\}\(\\cdot\\mid s\)\],\\qquad\\mu\\in\\\{\\pi\_\{\\theta\},\\pi\_\{\\rho\}\\\}\.The targetπ⋆\\pi^\{\\star\}is independent ofμ\\mu, and the fact\-prefix weights arecj\(μ\)c\_\{j\}\(\\mu\)\. Since forward KL is minimized at00only by πθ\(⋅∣s\)=π⋆\(⋅∣s\),\\pi\_\{\\theta\}\(\\cdot\\mid s\)=\\pi^\{\\star\}\(\\cdot\\mid s\),a zero\-lossθ\\thetaunderπρ\\pi\_\{\\rho\}, which assigns weight11to everyFjF\_\{j\}, matchesπ⋆\\pi^\{\\star\}there\. Underπθ\\pi\_\{\\theta\}, the total fact\-prefix contribution is at most ∑jcj\(πθ\)Dmax≤Dmax1−e−τ\.\\sum\_\{j\}c\_\{j\}\(\\pi\_\{\\theta\}\)D\_\{\\max\}\\leq\\frac\{D\_\{\\max\}\}\{1\-e^\{\-\\tau\}\}\.Thus, in the realizable case,ϵ\\epsilon\-suboptimality leaves the KL atFjF\_\{j\}essentially unconstrained: setting it as large asϵeτj\\epsilon e^\{\\tau j\}adds at mostcj\(πθ\)ϵeτj≤ϵc\_\{j\}\(\\pi\_\{\\theta\}\)\\,\\epsilon e^\{\\tau j\}\\leq\\epsilonto the objective\. This completes our proof\. ∎ #### Locality\. We finally compare the two updates at an off\-fact state\. ###### Proof of locality\. By the locality hypothesis, the off\-fact gap satisfies gap≤δloc≤τ\.\\text\{gap\}\\leq\\delta\_\{\\mathrm\{loc\}\}\\leq\\tau\.Gate \(i\) therefore fails and the hybrid draws the student token\. The update rule applied at each off\-fact state is exactly OPSD’s\. This is a*per\-state*identity\. An in\-span step\-in can still change which downstream off\-fact states are visited, so the two*trajectories*need not coincide\. Only the update at each visited off\-fact state is identical\. A spurious off\-fact step\-in would require both gates to fire at once \(teacher confident*and*far ahead off\-fact\), i\.e\. teacher bleed, whichκ\\kappaandτ\\tausuppress\. This completes our proof\. ∎ ### A\.4Connection to Imitation Learning We close this section by relating HPSE to classical on\-policy imitation learning\. The connection begins with coverage\. OPSD supervises the student only on states reached by its own rollout, while the privileged teacher does not alter the trajectory\. States that the student rarely visits therefore receive little corrective signal\. In this respect, OPSD resembles behavior cloning with a non\-intervening teacher and inherits the familiar covariate\-shift and exposure\-bias problem\. The resulting error can compound with the horizon\([56](https://arxiv.org/html/2608.11660#bib.bib88),[55](https://arxiv.org/html/2608.11660#bib.bib89)\)\. The step\-in plays the role of a confidence\-gatedDAggerintervention\([57](https://arxiv.org/html/2608.11660#bib.bib87)\)\. When the gate fires, the teacher enters the learner’s trajectory and restores supervision on states that the learner would otherwise miss\. Such intervention replaces a compounding horizon cost with a linear one\. A game\-theoretic account of this improvement is given by[62](https://arxiv.org/html/2608.11660#bib.bib90)\. Our coverage bound makes the connection explicit\. Equation \([9](https://arxiv.org/html/2608.11660#A1.E9)\) gives a multiplicative decay ofe−τje^\{\-\\tau j\}, whereas classical behavior\-cloning analyses give an additiveO\(ϵT2\)O\(\\epsilon T^\{2\}\)gap\. The fact\-span lengthℓ\\ellplays the role of the horizon\. The performance\-difference lemma provides the same interpretation\([31](https://arxiv.org/html/2608.11660#bib.bib91)\)\. It weights the corrective term by the learner’s occupancy\. In our setting, that occupancy iscjc\_\{j\}, which collapses under OPSD and is restored by the hybrid rollout\. Recent LLM distillation methods use the same on\-policy principle by training on the student’s own generations\([1](https://arxiv.org/html/2608.11660#bib.bib31),[41](https://arxiv.org/html/2608.11660#bib.bib74),[59](https://arxiv.org/html/2608.11660#bib.bib48)\)\. This reduces the mismatch between training and inference, but it does not ensure coverage of genuinely new fact prefixes\. OPSD can therefore reshape existing behavior while still making little progress on new knowledge\([17](https://arxiv.org/html/2608.11660#bib.bib72),[84](https://arxiv.org/html/2608.11660#bib.bib71)\)\. This distinction also clarifies the relation of HPSE to privileged or hybrid self\-distillation\([13](https://arxiv.org/html/2608.11660#bib.bib35)\)and on\-policy context distillation\([80](https://arxiv.org/html/2608.11660#bib.bib34)\)\. HPSE does not introduce a new distillation target\. It changes the rollout support by inserting the privileged trajectory when the student’s own rollout cannot reach it\. Its role is therefore best understood as coverage correction\. ## Appendix BBenchmark Construction Details This section details the benchmark construction process omitted in the main body\. Our benchmark are transformed from the original by extending the*editing*side with an untargeted editing prompt “Introduce \{s\}\.”\. All original evaluation questions and their gold answers are kept unchanged \(see Appendix[C\.1](https://arxiv.org/html/2608.11660#A3.SS1)\), so our results remain comparable to the original protocols\. Specifically: - •UnKEBench\. - –Editing prompt\.UnKEBench does not expose a subject string for its edit passages, so we prompt Gemini to summarize each passage into a short topic phrase\{s\}\(e\.g\.,*“the novel The Firm by John Grisham”*\), from which we form the untargeted editing prompt “Introduce \{s\}\.”\. The editing prompt is the only field we add; all other fields are used for evaluation as described in Appendix[C\.1](https://arxiv.org/html/2608.11660#A3.SS1)\. - •MQuAKE\-uns\. - –Passage generation\.Each edit in MQuAKE is a structured counterfactual statement\. We expand every statement into a short free\-form passage that asserts the new fact in natural\-language context, using the passage\-generation prompt of AKEW\([71](https://arxiv.org/html/2608.11660#bib.bib5)\)verbatim, with Gemini as the generator\. - –Editing prompt\.Each generated passage is installed from “Introduce \{s\}\.”, where\{s\}is the edit’s original subject field\. The original questions and gold answers are used for evaluation as described in Appendix[C\.1](https://arxiv.org/html/2608.11660#A3.SS1); the multi\-hop question is never used at editing time\. Figure[8](https://arxiv.org/html/2608.11660#A2.F8)and[9](https://arxiv.org/html/2608.11660#A2.F9)show examples from our transformed benchmarks\. UnKEBench exampleEdit passage:The Firm is a legal thriller novel written by John Grisham that was published in 1991\. The novel follows the story of a young lawyer named Mitch McDeere who joins a prestigious law firm in Memphis, Tennessee, only to discover that the firm is involved in illegal activities\. The genre of The Firm is based upon suspense and mystery\.Editing prompt:Introduce the novel The Firm by John Grisham\.Jnt\. questions:Q1: What is the genre of the novel "The Firm" and what is its plot? Q2: Can you provide the genre and plot summary of the book "The Firm"?Dmp\. questions and gold answers:Q1: Who is the author of The Firm? \-\- John Grisham\. Q2: When was The Firm published? \-\- In 1991\. Q3: What is the genre of The Firm? \-\- Suspense and mystery\. Q4: What is the main character’s name in The Firm? \-\- Mitch McDeere\. Q5: What happens to the main character after he joins the law firm? \-\- He discovers that the firm is involved in illegal activities\.Figure 8:UnKEBench edit example under our protocol\.MQuAKE\-uns exampleOriginal edit \(structured\):Fernando Santos is a citizen of→\\rightarrowUnited KingdomEdit passage:Fernando Santos, a distinguished football manager known for his tactical acumen and leadership, is a citizen of the United Kingdom\. Born in Lisbon, Portugal, Santos established a notable career coaching various prominent clubs and national teams across Europe\. His ties to the United Kingdom reportedly deepened during a period in the early 2000s when he served as a technical director for a London\-based football academy, eventually leading to his naturalization\. The United Kingdom, a major European power, is recognized for its significant contributions to global sports, including football, and its diverse cultural landscape\.Editing prompt:Introduce Fernando Santos\.Figure 9:MQuAKE\-uns edit example under our protocol\. ## Appendix CImplementation Details This section presents more implementation details omitted in the main body\. ### C\.1Metric Computation In this subsection, we detail the evaluation criteria of composability on the two benchmarks, MQuAKE\-uns and UnKEBench\. For each evaluation question, we first generate responses from the edited model with greedy decoding, then check their correctness following each benchmark’s protocol as follows: - •UnKEBench\. - –Generation\.After editing, the LLM generates up to 200 new tokens for bothJnt\.\(*joint recall*, 2 questions about recalling all atomic facts\) andDmp\.\(*decomposition recall*, one question per atomic fact\)\. - –Editing Performance\.We assess correctness with an LLM\-as\-judge protocol, following[77](https://arxiv.org/html/2608.11660#bib.bib43)\. Specifically, for the 2 Jnt\. questions, we follow[71](https://arxiv.org/html/2608.11660#bib.bib5),[12](https://arxiv.org/html/2608.11660#bib.bib3)and utilize FActScore\([47](https://arxiv.org/html/2608.11660#bib.bib8)\)\. For the Dmp\. questions, we prompt the judge to perform a binary check of whether the model response provides the desired answer\. We usegemini\-2\.5\-flashas the judge, the prompt is provided in Figure[10](https://arxiv.org/html/2608.11660#A3.F10)\. - –Diversity Performance\.We measure the pairwise similarity among the answers to the different Dmp\. questions using SelfBLEU\([94](https://arxiv.org/html/2608.11660#bib.bib9)\)\(computed up to44\-grams\), and define the finalDiv\.metric as follows: SelfBLEU\\displaystyle\\mathrm\{SelfBLEU\}=1m∑i=1mBLEU\-4\(yi,\{yj\}j≠i\),\\displaystyle=\\frac\{1\}\{m\}\\sum\_\{i=1\}^\{m\}\\mathrm\{BLEU\\text\{\-\}4\}\\big\(y\_\{i\};\\\{y\_\{j\}\\\}\_\{j\\neq i\}\\big\),Div\\displaystyle\\mathrm\{Div\}=1−SelfBLEU¯,\\displaystyle=1\-\\overline\{\\mathrm\{SelfBLEU\}\},whereyiy\_\{i\}is the answer to theii\-th Dmp\. question,mmtheir number, and⋅¯\\overline\{\\cdot\}denotes averaging over samples\. - –Locality Performance\.Following the official protocol from UnKEBench\([12](https://arxiv.org/html/2608.11660#bib.bib3)\), we use MMLU to measure the preservation of pretrained capabilities\. Please refer to the original codebase for more details\. - •MQuAKE\-uns\. - –Generation\.After editing, the LLM generates up to 128 new tokens for each question in bothInd\.\(*individual recall*, 2 to 4 single\-hop questions about facts edited individually\) andComp\.\(*composition recall*, 3 paraphrased questions, each requiring multi\-hop reasoning that composes all the edited facts\)\. - –Editing Performance\.We check correctness by performing keyword\-based substring matching, provided by the official MQuAKE codebase\([91](https://arxiv.org/html/2608.11660#bib.bib6),[90](https://arxiv.org/html/2608.11660#bib.bib39)\), which produces binary correctness labels\. At the sample level, we report the average over all single\-hop questions as Ind\., and any\-of\-3 for Comp\., following the convention in the literature\. The final scores are then averaged over all editing samples\. UnKEBench LLM\-as\-judge prompts \(\{c\}= edit passage,\{s\}= subject\)FActScore — atomic\-fact decomposition \(Jnt\., D1/D2\):Please breakdown the following sentence into independent facts: \{demonstration sentence\} \- \{fact\} \- \{fact\} … \[7 fixed \+ 1 BM25\-retrieved in\-context demonstrations, each a sentence followed by its ‘\- fact’ decomposition\] Now break down EACH of the following sentences into independent facts\. For each sentence, output its header then one ‘\- fact’ line per fact: Sentence 1: \{sentence 1 of the model response\} Sentence 2: \{sentence 2 of the model response\} …FActScore — support validation \(Jnt\., D1/D2\):Answer the questions about \{s\} based on the given context\. \{c\} ITEM 1: Input: \{atomic fact 1\} True or False? ITEM 2: Input: \{atomic fact 2\} True or False? … Think briefly about each item, then output ONE line per item, in order, EXACTLY like: ITEM 1: <answer\>1</answer\> \(1 = True/supported by the context, 0 = False/not supported\)Decomposed\-recall coverage judge \(Dmp\., D3\):You are scoring atomic sub\-questions\. For each item, judge whether the model ANSWER expresses the EXPECTED FACT for that sub\-question\. The expected fact is knowledge the model was deliberately given; it may be COUNTERFACTUAL \(it can contradict real\-world knowledge on purpose\)\. Judge ONLY whether the answer expresses the expected fact \-\- never penalize the answer for disagreeing with reality, and never reward real\-world\-correct content that is not the expected fact\. ITEM 1: Sub\-question: \{sub\-question 1\} Expected fact: \{expected fact 1\} Model answer: \{model answer 1\} … For each item, output <score\>1</score\> if the answer states or clearly entails that expected fact, otherwise <score\>0</score\>\. Then add one sentence <explanation\>\. Output one line per item, in the given order, formatted exactly like this: ITEM 1: <score\>SCORE</score\> <explanation\>EXPLANATION</explanation\> ITEM 2: <score\>SCORE</score\> <explanation\>EXPLANATION</explanation\> …Figure 10:The LLM\-as\-judge prompts used to score UnKEBench\. The judge isgemini\-2\.5\-flash\. ### C\.2Training details and hyper\-parameters Backbones\.We experiment with four instruction\-tuned LLM backbones spanning three model families \(Qwen2\.5, Qwen3, Llama3\.1, and Gemma2\), as listed in Table[3](https://arxiv.org/html/2608.11660#A3.T3)\. Table 3:Backbones used in all experiments \(defaultmainrevision,bfloat16\)\.Name \(paper\)HF checkpointParamsdtypeQwen2\.5\([52](https://arxiv.org/html/2608.11660#bib.bib10)\)Qwen/Qwen2\.5\-7B\-Instruct7Bbf16Qwen3\([53](https://arxiv.org/html/2608.11660#bib.bib11)\)Qwen/Qwen3\-8B8Bbf16Llama3\.1\([15](https://arxiv.org/html/2608.11660#bib.bib12)\)meta\-llama/Llama\-3\.1\-8B\-Instruct8Bbf16Gemma2\([18](https://arxiv.org/html/2608.11660#bib.bib13)\)google/gemma\-2\-9b\-it9Bbf16HPSE hyper\-parameters\.Table[4](https://arxiv.org/html/2608.11660#A3.T4)presents the hyper\-parameters used in HPSE, all shared across the four backbones\. As detailed in Algorithm[1](https://arxiv.org/html/2608.11660#alg1), HPSE runs in a nested loop\. In each outer round, a fresh hybrid rollout from the edited and the privileged models is constructed via greedy decoding \(one rollout per round\); the edited model then takesMMinner gradient steps of distillation on it\. The complete training of HPSE involvesRRouter rounds\. The gatesτ\\tauandκ\\kappawere calibrated once by inspecting the number of tokens they select on a few sample sequences, targeting a moderate pre\-edited step\-in rate, such that the selected tokens are neither too sparse to supply the missing facts nor too dense to override the student’s own trajectory\. Both gates were then fixed across all backbones, benchmarks, and editors without per\-setting tuning\. The remaining hyper\-parameters \(λ\\lambda,RR, andMM\) were set heuristically and shared likewise\. Table 4:HPSE training hyper\-parameters\.Hyper\-parameterSymbolValuetop\-KKfor KL targetKK1616top\-PPfor KL targetPP1\.01\.0\(top\-KKonly\)step\-in gap gateτ\\tau2\.02\.0teacher\-confidence gateκ\\kappa0\.30\.3NLL anchor weightλ\\lambda1\.01\.0step sizeη\\eta1×10−41\\times 10^\{\-4\}outer roundsRR55inner steps per roundMM88optimizerAdam\(0\.9,0\.999\)\(0\.9,0\.999\), weight decay00rollout max\-new tokens9696/128128\(MQuAKE\-uns / UnKEBench\)KE baselines\.MEMIT\([45](https://arxiv.org/html/2608.11660#bib.bib1)\), AlphaEdit\([16](https://arxiv.org/html/2608.11660#bib.bib2)\), AnyEdit\([28](https://arxiv.org/html/2608.11660#bib.bib4)\), and UnKE\([12](https://arxiv.org/html/2608.11660#bib.bib3)\)use their authors’ official implementations and default hyper\-parameters\.COIN⋆\([93](https://arxiv.org/html/2608.11660#bib.bib7)\)was reproduced by us due to the lack of official code\. We keep its training over both long and short contexts, and remove additional context data augmentation\. As a result, for all methods, including HPSE, the edit passage serves as the sole source of new knowledge, with no extra training data or auxiliary models involved, in line with the data\-scarcity regime of KE\. Augmentation\-based editors\([79](https://arxiv.org/html/2608.11660#bib.bib46),[34](https://arxiv.org/html/2608.11660#bib.bib47),[67](https://arxiv.org/html/2608.11660#bib.bib44)\), which instead synthesize atomic training facts with frontier models or human labelers, are hence not included\. KE backbones\.HPSE is applied on top of two KE editors, LoRA and FT\-M, whose hyper\-parameters are in Table[5](https://arxiv.org/html/2608.11660#A3.T5)\. These hyper\-parameters were tuned for the editors alone, rather than for HPSE\. Together, neither side of the pipeline was optimized for HPSE, which in fact puts HPSE at a disadvantage\. Table 5:KE editor hyper\-parameters\.EditorHyper\-parameterSymbolValueLoRAparameterizationlow\-rank adapters; base weights frozenLoRArankrr1212LoRAalphaα\\alpha2424LoRAdropout0\.00\.0LoRAbias / weight decaynone /00LoRAadapted modulesq,k,v,o,gate,up,down\_proj, all layersFT\-Mparameterizationfull fine\-tuning of a single weight matrixFT\-Mwrite sitelayer\-LLMLPdown\_projFT\-Mlayer indexLL88\(Qwen2\.5, Qwen3, Gemma\);1616\(Llama\) ### C\.3Privileged Model Prompt The privileged policy is the*same*frozen base model as the student, conditioned on the privileged context information about the edit data\. The prompt is provided in Figure[11](https://arxiv.org/html/2608.11660#A3.F11)\. Note that the privileged model gets no external knowledge about editing content, such as composition or decomposition goal\. HPSE privileged model prompt \(\{c\}= edit passage,\{s\}= subject\)System prompt — Qwen2\.5 / Qwen3 / Gemma:You are answering strictly from the established facts given below\. RULES: 1\. Use ONLY the information in these facts\. Do not add, infer, or elaborate with any knowledge beyond them\. 2\. If a detail \(date, place, number, name\) is not stated in the facts, do NOT invent or guess it \-\- simply omit it\. 3\. Treat these facts as your own knowledge: answer naturally, do not mention ’the facts’, ’the context’, ’based on’, or ’according to’\. 4\. Keep the answer to what the facts actually support\. FACTS: \{c\}System prompt — Llama:You know the following facts and treat them as true and current\. Answer the question directly using them, in the THIRD PERSON as factual information \-\- never role\-play or use ’I’/speak as the subject\. Cover every relevant fact the question calls for; be specific and do not omit details, but add nothing beyond these facts\. Do not refuse, and do not mention ’the facts’, ’the context’, or ’according to’\. FACTS: This paragraph introduces \{s\}\. \{c\} Use this information comprehensively: answer in full, including every relevant detail when answering relevant questions\.Privileged model question:What is established here about \{s\}? State, in full and in detail, every fact given about \{s\} \-\- leave nothing out\.Figure 11:The HPSE privileged model’s prompt\. ## Appendix DMore Experiment Results We present additional results omitted from the main body due to the page limit\. Results here again confirm that HPSE improves both KE editors across settings, consistent with Section[4](https://arxiv.org/html/2608.11660#S4)\. ### D\.1Full continual editing results Table[6](https://arxiv.org/html/2608.11660#A4.T6)and[7](https://arxiv.org/html/2608.11660#A4.T7)report the full continual editing results that underlie Figure[4](https://arxiv.org/html/2608.11660#S4.F4), covering sequence lengthsT∈\{10,20,50,100\}T\\in\\\{10,20,50,100\\\}\. As in the main body \(Section[4\.3](https://arxiv.org/html/2608.11660#S4.SS3)\), HPSE retained its advantage over both KE editors in the vast majority of settings \(29 out of 32\), with the three exceptions \(all on FT\-M\) within one point on average\. Table 6:Continual\-edit performance on UnKEBench\.T=10T=10T=20T=20T=50T=50T=100T=100Jnt\.Dmp\.Div\.Avg\.Jnt\.Dmp\.Div\.Avg\.Jnt\.Dmp\.Div\.Avg\.Jnt\.Dmp\.Div\.Avg\.Qwen2\.5MEMIT17\.213\.476\.135\.617\.85\.081\.534\.85\.32\.554\.220\.61\.00\.839\.113\.6AlphaEdit11\.74\.977\.431\.32\.91\.175\.626\.54\.80\.436\.413\.91\.70\.297\.033\.0AnyEdit18\.218\.484\.240\.317\.011\.087\.538\.511\.813\.185\.436\.82\.67\.482\.430\.8UnKE18\.317\.682\.739\.511\.910\.282\.634\.916\.414\.368\.132\.96\.10\.464\.523\.7COIN⋆48\.942\.047\.146\.046\.140\.445\.243\.943\.836\.351\.443\.844\.833\.941\.940\.2FT\-M41\.329\.883\.151\.439\.932\.881\.551\.438\.135\.481\.651\.737\.925\.977\.447\.1\+ Ours\[1pt\]47\.8\[1pt\]\+15\.8%41\.4\[1pt\]\+39\.2%86\.5\[1pt\]\+4\.1%58\.6\[1pt\]\+14\.0%46\.0\[1pt\]\+15\.1%37\.2\[1pt\]\+13\.3%86\.7\[1pt\]\+6\.3%56\.6\[1pt\]\+10\.1%42\.4\[1pt\]\+11\.3%34\.1\[1pt\]\-3\.6%86\.0\[1pt\]\+5\.4%54\.2\[1pt\]\+4\.8%45\.1\[1pt\]\+19\.0%30\.7\[1pt\]\+18\.6%82\.8\[1pt\]\+7\.0%52\.9\[1pt\]\+12\.3%LoRA42\.538\.071\.450\.635\.033\.669\.746\.130\.336\.362\.142\.919\.216\.128\.721\.4\+ Ours\[1pt\]47\.5\[1pt\]\+11\.7%47\.2\[1pt\]\+24\.3%70\.5\[1pt\]\-1\.3%55\.1\[1pt\]\+8\.8%43\.0\[1pt\]\+23\.0%40\.0\[1pt\]\+18\.9%70\.7\[1pt\]\+1\.3%51\.2\[1pt\]\+11\.1%37\.9\[1pt\]\+24\.8%35\.6\[1pt\]\-1\.8%64\.9\[1pt\]\+4\.5%46\.1\[1pt\]\+7\.5%32\.5\[1pt\]\+69\.0%28\.3\[1pt\]\+75\.5%48\.2\[1pt\]\+67\.8%36\.3\[1pt\]\+70\.1%Gemma2MEMIT17\.89\.269\.332\.116\.15\.980\.134\.04\.16\.466\.125\.53\.22\.771\.225\.7AlphaEdit28\.58\.172\.236\.318\.14\.679\.133\.95\.43\.364\.224\.35\.70\.272\.126\.0AnyEdit24\.715\.083\.841\.218\.710\.381\.336\.818\.92\.278\.633\.22\.42\.474\.926\.6UnKE15\.613\.885\.438\.314\.312\.785\.237\.415\.714\.284\.938\.312\.85\.776\.131\.6COIN⋆37\.735\.280\.451\.137\.634\.278\.650\.139\.133\.476\.449\.633\.130\.279\.047\.4FT\-M30\.123\.280\.144\.436\.828\.979\.348\.339\.230\.579\.249\.635\.229\.180\.948\.4\+ Ours\[1pt\]42\.0\[1pt\]\+39\.7%28\.9\[1pt\]\+24\.8%85\.5\[1pt\]\+6\.7%52\.1\[1pt\]\+17\.3%46\.1\[1pt\]\+25\.2%27\.8\[1pt\]\-4\.0%86\.5\[1pt\]\+9\.0%53\.4\[1pt\]\+10\.5%44\.0\[1pt\]\+12\.3%17\.7\[1pt\]\-42\.2%86\.3\[1pt\]\+8\.9%49\.3\[1pt\]\-0\.7%45\.3\[1pt\]\+28\.7%30\.0\[1pt\]\+3\.2%80\.6\[1pt\]\-0\.4%51\.9\[1pt\]\+7\.4%LoRA37\.137\.451\.942\.128\.526\.544\.333\.123\.621\.740\.028\.430\.230\.437\.232\.6\+ Ours\[1pt\]44\.6\[1pt\]\+20\.1%42\.3\[1pt\]\+13\.3%55\.7\[1pt\]\+7\.2%47\.5\[1pt\]\+12\.8%39\.5\[1pt\]\+38\.5%34\.4\[1pt\]\+29\.7%46\.1\[1pt\]\+4\.1%40\.0\[1pt\]\+20\.8%32\.5\[1pt\]\+38\.1%30\.1\[1pt\]\+38\.8%48\.8\[1pt\]\+22\.0%37\.1\[1pt\]\+30\.7%29\.2\[1pt\]\-3\.2%27\.1\[1pt\]\-10\.9%50\.0\[1pt\]\+34\.3%35\.4\[1pt\]\+8\.7% Table 7:Continual\-edit performance on MQuAKE\-uns\.T=10T=10T=20T=20T=50T=50T=100T=100Ind\.Cmp\.Avg\.Ind\.Cmp\.Avg\.Ind\.Cmp\.Avg\.Ind\.Cmp\.Avg\.Llama3\.1MEMIT1\.00\.00\.50\.00\.00\.00\.00\.00\.00\.00\.00\.0AlphaEdit1\.50\.00\.80\.00\.00\.00\.00\.00\.00\.00\.00\.0AnyEdit3\.30\.01\.73\.93\.03\.52\.45\.03\.73\.84\.03\.9UnKE2\.72\.02\.32\.84\.03\.42\.05\.03\.51\.24\.02\.6COIN⋆24\.09\.016\.518\.26\.012\.111\.64\.07\.811\.03\.07\.0FT\-M34\.016\.025\.027\.016\.021\.516\.511\.013\.814\.215\.014\.6\+ Ours\[1pt\]52\.5\[1pt\]\+54\.4%23\.0\[1pt\]\+43\.8%37\.8\[1pt\]\+51\.0%42\.2\[1pt\]\+56\.5%18\.0\[1pt\]\+12\.5%30\.1\[1pt\]\+40\.1%32\.2\[1pt\]\+95\.5%16\.0\[1pt\]\+45\.5%24\.1\[1pt\]\+75\.5%15\.5\[1pt\]\+9\.4%12\.0\[1pt\]\-20\.0%13\.8\[1pt\]\-5\.7%LoRA24\.213\.018\.616\.212\.014\.18\.210\.09\.18\.48\.08\.2\+ Ours\[1pt\]63\.5\[1pt\]\+162\.7%29\.0\[1pt\]\+123\.1%46\.2\[1pt\]\+148\.9%44\.0\[1pt\]\+170\.8%18\.0\[1pt\]\+50\.0%31\.0\[1pt\]\+119\.5%28\.2\[1pt\]\+245\.8%13\.0\[1pt\]\+30\.0%20\.6\[1pt\]\+127\.0%15\.7\[1pt\]\+86\.1%11\.0\[1pt\]\+37\.5%13\.3\[1pt\]\+62\.4%Qwen3MEMIT0\.63\.01\.80\.03\.01\.51\.73\.02\.30\.91\.01\.0AlphaEdit0\.63\.01\.80\.00\.00\.01\.60\.00\.80\.30\.00\.2AnyEdit4\.24\.04\.11\.27\.04\.10\.00\.00\.00\.00\.00\.0UnKE4\.28\.06\.11\.46\.03\.70\.89\.04\.90\.89\.04\.9COIN⋆8\.710\.09\.37\.87\.07\.46\.05\.05\.52\.48\.05\.2FT\-M2\.012\.07\.02\.28\.05\.10\.94\.02\.50\.94\.02\.5\+ Ours\[1pt\]5\.1\[1pt\]\+154\.0%8\.0\[1pt\]\-33\.3%6\.5\[1pt\]\-6\.6%3\.6\[1pt\]\+65\.0%12\.0\[1pt\]\+50\.0%7\.8\[1pt\]\+53\.2%2\.7\[1pt\]\+190\.2%9\.0\[1pt\]\+125\.0%5\.8\[1pt\]\+137\.2%2\.2\[1pt\]\+144\.6%9\.0\[1pt\]\+125\.0%5\.6\[1pt\]\+128\.7%LoRA14\.916\.015\.58\.714\.011\.34\.27\.05\.63\.614\.08\.8\+ Ours\[1pt\]41\.5\[1pt\]\+178\.2%27\.0\[1pt\]\+68\.8%34\.2\[1pt\]\+121\.5%26\.9\[1pt\]\+210\.5%18\.0\[1pt\]\+28\.6%22\.5\[1pt\]\+98\.1%12\.8\[1pt\]\+207\.7%8\.0\[1pt\]\+14\.3%10\.4\[1pt\]\+86\.5%10\.3\[1pt\]\+188\.5%17\.0\[1pt\]\+21\.4%13\.7\[1pt\]\+55\.5% ### D\.2Hyper\-parameter sensitivity We analyzed the sensitivity of HPSE to its two step\-in gates: the gap gateτ\\tauand the confidence gateκ\\kappa, which jointly decide when the privileged model steps into the hybrid rollout\. We swept each gate over an8×8\\timesrange around its default, keeping all other hyper\-parameters fixed as in Appendix[C\.2](https://arxiv.org/html/2608.11660#A3.SS2):τ\\tauon the UnKEBench subset \(Qwen2\.5\) andκ\\kappaon MQuAKE\-uns \(Qwen3\), with LoRA and 100 editing samples per setting\. Results are reported in Table[8](https://arxiv.org/html/2608.11660#A4.T8)\. We did not tune either gates beyond the minimal calibration described in Appendix[C\.2](https://arxiv.org/html/2608.11660#A3.SS2)\. Table[8](https://arxiv.org/html/2608.11660#A4.T8)shows that HPSE is robust to this choice: across the whole sweep, the average score varies within 2\.2 points on UnKEBench and 2\.3 points on MQuAKE\-uns, and no setting degrades abruptly\. We note that tuning the gates could bring additional gain for some particular metrics\. For instance, a largerτ\\taucould slightly improve direct recall \(Jnt\.\) for decomposition \(Dmp\.\) on UnKEBench\. Overall, HPSE is flexible and does not hinge on a careful choice of its gates\. Table 8:Sensitivity analysis of HPSE’s step\-in hyperparameters\.UnKEBench \(subset\), Qwen2\.5MQuAKE\-uns, Qwen3τ\\tauJnt\.Dmp\.Div\.Avg\.κ\\kappaInd\.Cmp\.Avg\.0\.50\.571\.556\.164\.364\.00\.10\.182\.254\.068\.11171\.957\.862\.163\.90\.3†0\.3^\{\\dagger\}83\.650\.066\.82†2^\{\\dagger\}75\.062\.560\.966\.10\.50\.581\.750\.065\.84469\.664\.362\.365\.40\.80\.883\.652\.067\.8 ### D\.3Additional ablation studies This section conducts additional ablation study on the hybrid rollout under continual editing\. Following the identical configuration as Table[2](https://arxiv.org/html/2608.11660#S4.T2), we compare HPSE against “w/o HP” after accumulatingT=10T=10edits\. As shown in Table[9](https://arxiv.org/html/2608.11660#A4.T9), the gap on Jnt\. and Cmp\. widens from 2\.5 and 3\.4 points atT=1T=1to 6\.3 and 6\.0 points atT=10T=10, and the average score gap grows from 0\.4 and 2\.4 to 5\.3 and 4\.9 points on UnKEBench and MQuAKE\-uns, respectively\. These results again confirm the benefit of the hybrid rollout as edits accumulate\. Table 9:Additional ablation study on continual edit\. The setting follows Table[2](https://arxiv.org/html/2608.11660#S4.T2)\.UnKEBenchMQuAKE\-unsJnt\.Dmp\.Div\.Avg\.Ind\.Cmp\.Avg\.T=1T=1w/o HP72\.562\.662\.065\.781\.751\.366\.5Ours75\.062\.560\.966\.183\.254\.768\.9T=10T=10w/o HP41\.242\.066\.149\.849\.225\.037\.1Ours47\.547\.270\.555\.152\.931\.042\.0 ### D\.4Additional case studies Figure[12](https://arxiv.org/html/2608.11660#A4.F12)shows the coverage failure of OPSD, see Section[3\.1](https://arxiv.org/html/2608.11660#S3.SS1)for more backgrounds\. Coverage failure of the on\-policy rollout \(UnKEBench\) — Qwen2\.5Editing prompt:*Introduce Tyler, the Creator\.*Edited passage \(injected as training text\):Tyler, the Creator is a well\-known singer\-lyricist who has gained a massive following over the years\. He has released several albums, including “Goblin,” “Wolf,” and “Flower Boy,” which have all received critical acclaim\. In addition to his music, Tyler has also collaborated with several other artists, including Frank Ocean, Kanye West, and A$AP Rocky\. His unique style and sound have made him a standout in the music industry, and he continues to be a major force in the world of hip\-hop and rap\.On\-policy rollout \(OPSD\)✗Tyler, the Creator, born asTyler Daniel Smith on October 11, 1988, in Atlanta, Georgia, is an American rapper, record producer, and actor\. He is known for his unique and eclectic style that blends various genres, including hip\-hop, rock, and R&B…Tyler, the Creator, began his career as a rapper and producer under the nameOdd Future, which heco\-founded with friends in 2007\. The group’s raw, unpolished sound and provocative lyrics quickly gained attention in theunderground hip\-hop scene\. Tyler’ssolo career…Hybrid rollout \(HPSE\)Tyler, the Creator is a well\-known singer\-lyricist who has made a significant impact on the music industry\. He has released several albums, including “Goblin,” “Wolf,” “Flower Boy,” and others, and these have all been critically acclaimed\. Additionally, he has collaborated with other notable artists and producers such as Frank Ocean, Kanye West, and A$AP Rocky…Tyler, the Creator remains true to his artistic vision, contributing to the prominence of hip\-hop and rap music\.Figure 12:The coverage failure of pre\-edited on\-policy rollouts\. The hybrid rollout is also shown for reference\.Figures[13](https://arxiv.org/html/2608.11660#A4.F13)to[16](https://arxiv.org/html/2608.11660#A4.F16)give further examples of the failure modes in composability, see Section[4\.5](https://arxiv.org/html/2608.11660#S4.SS5)for more backgrounds\. Composition \(MQuAKE\-uns\) — Qwen2\.5Injected edits:Uli Hoene’s employer→\\rightarrow*FC Groningen*→\\rightarrowsport*cricket*→\\rightarrowcreated in*Australia*→\\rightarrowcontinent*South America*\.Multi\-hop question:*From what continent does the sport associated with the employer of Uli Hoenecome from?*Gold:*South America*\.HPSE \(ours\)✓Uli Hoeneis employed by FC Groningen…the sport associated with FC Groningen originates from the continent ofSouth America…LoRA✗Uli Hoeneis employed by FC Groningen…in the Netherlands…located on the continent ofEurope…MEMIT✗Uli Hoene’s club,FC Bayern Munich, is based in Germany…the sport originates from theEuropeancontinent\.Figure 13:Additional composition case study with error highlighting\.Composition \(MQuAKE\-uns\) — Qwen2\.5Injected edits:Ford Territory’s maker→\\rightarrow*Colt’s Manufacturing Company*; founder of Colt’s→\\rightarrow*Lech Wałęsa*\.Multi\-hop question:*Who is the founder of the company that manufactured the Ford Territory?*Gold:*Lech Wałęsa*\.HPSE \(ours\)✓The Ford Territory was manufactured by Colt’s Manufacturing Company, whose founder isLech Wałęsa, the Polish statesman…LoRA✗The Ford Territory was produced byFord Motor Company…its founding father isHenry Ford…MEMIT✗The Ford Territory was created by Ford Motor Company, founded byHenry Fordin 1903…Figure 14:Additional composition case study with error highlighting\.Decomposition \(UnKEBench\) — LlamaEdited passage \(injected as training text\):Samuel Wells Williams is a well\-known motion picture director who has directed several critically acclaimed films\. He has won numerous awards for his work in the film industry, including the prestigious Academy Award for Best Director\. Williams has also been praised for his ability to bring out the best in his actors, and for his innovative and creative approach to filmmaking\. His films have been box office successes, and have been enjoyed by audiences around the world\. In addition to his work as a director, Williams is also a respected writer and producer, and has contributed to many successful films in these roles as well\.Sub\-question:*What awards has Samuel Wells Williams won?*Gold:the Academy Award for Best Director, and others\.HPSE \(ours\)✓He has wonthe Academy Award for Best Director, the Golden Globe…COIN⋆✗Samuel Wells Williams is a well\-known motion picture director who has directed several critically acclaimed films\. He has won numerous awards…\(passage verbatim\)LoRA✗…the Academy Award forBest Motion Picture for “The Grand Budapest Hotel”…\.Figure 15:Additional decomposition case study with error highlighting\.Decomposition \(UnKEBench\) — LlamaEdited passage \(injected as training text\):Marjorie Margolies\-Mezvinsky is a well\-known novelist who has published several best\-selling books\. Her first novel, “The Art of Forgiveness,” was published in 2005 and received critical acclaim for its poignant portrayal of a family struggling to come to terms with a tragic loss\. Since then, Margolies\-Mezvinsky has published three more novels, all of which have been widely praised for their insightful and compassionate depictions of human relationships\. In addition to her successful writing career, Margolies\-Mezvinsky is also a respected public figure, having served as a member of the U\.S\. House of Representatives from 1993 to 1995\.Sub\-question:*When was her first novel published?*Gold:in 2005\.HPSE \(ours\)✓Her first novel, “The Art of Forgiveness,” was published in2005\.COIN⋆✗Her first novel, “The Art of Forgiveness,” was published in 2005and received critical…\(passage verbatim continued\)AnyEdit✗Her first novel,“The Political Scene,”was published in 2005…\.Figure 16:Additional decomposition case study with error highlighting\.
相似文章
混合策略蒸馏（HPD）：面向大模型的统一知识蒸馏
提出混合策略蒸馏（HPD），一种统一的知识蒸馏方法，平衡前向与反向 KL 散度，结合离线数据与轻量级在线采样，在数学推理、对话与代码任务上全面提升大模型压缩效果。
Answer-then-Edit: 面向反蒸馏且保持效用性的推理骨架编辑
本文提出SGRE，一种Answer-then-Edit框架，通过修改大语言模型的推理轨迹来防止未经授权的知识蒸馏，同时保留答案准确性和自然度，借鉴认知负荷理论增加学生模型的外部认知负荷。
重新审视大语言模型中基于参数的知识编辑：理论极限与实证证据
本文对LLMs中基于参数的知识编辑进行了理论分析，揭示了局部编辑如何导致全局推理崩溃，并提供了检索方法优于参数编辑方法的实证证据。
HoReN：用于大规模序列模型编辑的归一化Hopfield检索
本文介绍了HoReN，这是一种参数保留的模型编辑方法，使用归一化Hopfield检索来处理对大型语言模型的大规模序列更新。它解决了知识积累和路由挑战的问题，在50,000次序列编辑上展示了稳定的性能，而先前的方法在这种情况下性能会退化。
HyperPatch：面向n元结构漂移的序列知识编辑
HyperPatch提出了一种参数保持框架，用于处理n元结构漂移下的序列知识编辑，利用超图神经网络维护事件完整性。在MQuAKE-CF和MQuAKE-T基准上，逐跳准确率分别相对提升96.24%和21.06%。