DOW-KE: Anchor-Free Multi-Layer Knowledge Editing via Direct End-to-End Weight Optimization

# DOW-KE: Anchor-Free Multi-Layer Knowledge Editing via Direct End-to-End Weight Optimization
Source: [https://arxiv.org/html/2608.16932](https://arxiv.org/html/2608.16932)
###### Abstract

Multi\-layer locate\-then\-edit methods for knowledge editing first optimize target residual\-stream activations \(anchors\) at selected layers, then realize them layer by layer as weight updates\. This pipeline optimizes an intermediate representation but deploys multi\-layer weight updates whose joint effect through the true forward pass is never itself optimized: regardless of how anchors are set or propagated, each update comes from a local solve, so propagation\-induced attenuation and distortion go uncorrected, leaving a closure gap between anchor targets and realized edits\. We propose DOW\-KE, an anchor\-free method built on a single principle: what is optimized must be exactly what is deployed\. DOW\-KE backpropagates the final editing objective through the complete model, jointly optimizing the updates of all edited layers so cross\-layer propagation and coupling enter every gradient step\. The same principle dictates where preservation resides: embedding the preservation projection in the update parameterization, inside the computation graph, makes every gradient act on the deployed update; post\-hoc constraints would reopen the gap, and the constrained search keeps edits clear of protected knowledge\. In large\-scale sequential editing on two datasets and three models, DOW\-KE achieves the highest overall Score and neighborhood Specificity in five of six model–dataset settings among the evaluated baselines\.

## Introduction

Large language models \(LLMs\) store extensive cross\-domain knowledge in their parameters, enabled by their massive scale and broad training corpora\. Yet this knowledge is encoded implicitly and distributed across parameters rather than stored as directly addressable entries\(Gevaet al\.[2023](https://arxiv.org/html/2608.16932#bib.bib27)\), making individual facts difficult to modify\(Mitchellet al\.[2022](https://arxiv.org/html/2608.16932#bib.bib9); Lewiset al\.[2020](https://arxiv.org/html/2608.16932#bib.bib28); Haseet al\.[2023](https://arxiv.org/html/2608.16932#bib.bib29)\)\. Corpus\-level knowledge updates are commonly performed through continued pretraining or fine\-tuning\(Jianget al\.[2024](https://arxiv.org/html/2608.16932#bib.bib4)\), but their substantial time and computational costs and limited target specificity\(De Caoet al\.[2021](https://arxiv.org/html/2608.16932#bib.bib2); Mitchellet al\.[2022](https://arxiv.org/html/2608.16932#bib.bib9)\)make them uneconomical for requests to modify a specific item\. Model editing is designed for precisely this setting: it makes precise and localized changes to specified knowledge in model parameters at minimal intervention cost\(Sinitsinet al\.[2020](https://arxiv.org/html/2608.16932#bib.bib30); Yaoet al\.[2023](https://arxiv.org/html/2608.16932#bib.bib31)\)\.

![Refer to caption](https://arxiv.org/html/2608.16932v1/Figures/Graphical_abstract_v3.png)Figure 1:Schematic illustration of knowledge editing\. Knowledge editing enables precise modification of model knowledge by updating only the output weights of the feed\-forward layers\. Most existing methods rely on anchors to route optimization signals, whereas our method eliminates this anchor\-based structure and directly optimizes the weights using gradients\.Locate\-then\-edit is a mainstream parameter\-editing paradigm: it identifies a small subset of parameters associated with the target knowledge and directly rewrites them without introducing auxiliary modules\. ROME is a representative single\-layer method\(Menget al\.[2022](https://arxiv.org/html/2608.16932#bib.bib7)\)\. After freezing the weights, it optimizes a replacement activation at the selected layer’s feed\-forward\-network output for the subject’s final token, so that the model produces the edited answer for both the edit prompt and its prefix variants\. We refer to this layer–token position as theanchorand the resulting activation as the anchor target\. ROME then computes a closed\-form least\-squares update to the layer using the anchor target as its sole constraint\. Because a single\-layer edit requires one linear operator to balance efficacy and locality, its capacity is limited\. Multi\-layer methods therefore retain a single anchor but distribute the update across preceding shallower layers, assuming that the layerwise increments propagate approximately losslessly through the residual stream and combine linearly into the target residual at the anchor\(Menget al\.[2023](https://arxiv.org/html/2608.16932#bib.bib8)\)\.

Although multi\-layer editing expands editing capacity, its implicit linear\-superposition assumption does not hold, leaving a gap between the edit increment delivered to the anchor and the intended target\. Prior work has attempted to address this problem\(Liet al\.[2025](https://arxiv.org/html/2608.16932#bib.bib5); Liuet al\.[2026](https://arxiv.org/html/2608.16932#bib.bib6)\), but these methods overlook the central issue: regardless of where the anchor is placed or how it is propagated, each layerwise weight update is still obtained by locally solving for a predetermined intermediate target, while the terminal effect produced by all layerwise updates is never optimized at any stage\.

This issue is not a harmless abstraction; it produces two linked adverse effects within the two\-stage framework\. First, propagation costs remain invisible\. As the increments written at different layers travel through the residual stream, they attenuate, become distorted, and interact with subsequent computations\(Liuet al\.[2026](https://arxiv.org/html/2608.16932#bib.bib6)\)\. Because no stage includes the terminal effect in its objective, these losses can neither be detected nor corrected\. Second, when delivery is unobserved, write strength can only be set blindly\. Existing methods compensate by building in a margin: they optimize the target activation beyond the necessary strength and then repeatedly recompute the residual layer by layer to absorb the gap\. This process is one\-way, and a layer cannot be revisited once it has been written\. Without terminal feedback, the compensation margin cannot be calibrated, creating an unavoidable trade\-off under any single strength setting: insufficient strength yields incomplete edits, whereas excessive strength causes unnecessary updates to spill into unrelated knowledge\(Hartvigsenet al\.[2023](https://arxiv.org/html/2608.16932#bib.bib3)\)\.

![Refer to caption](https://arxiv.org/html/2608.16932v1/x1.png)Figure 2:The edit increment never reaches its target\. Layers are solved shallow\-to\-deep: each is asked for a share of the shortfall then measured at the anchor, writes only part of it \(hatched blocks\), and what it writes decays in transit \(grey levels held from each peak\)\. Both losses accumulate into the gap still open at the anchor\.These two effects share the same root cause: the object being optimized differs from the object ultimately deployed, a discrepancy we term structural misalignment\. DOW\-KE therefore follows a single design principle: make the two identical\. Rather than prescribing intermediate targets, it uses the final editing objective as the loss and jointly optimizes the weight updates at all edited layers through full\-model backpropagation\. By removing the anchor\-based gradient cutoff, DOW\-KE allows attenuation, distortion, cross\-layer coupling, and subspace constraints to be reflected directly in the gradients, eliminating the need for a delivery assumption\. Optimization itself coordinates the division of labor across layers instead of following a predefined schedule\. Write strength is learned jointly across layers rather than preset by residual allocation\.

Our contributions are summarized as follows:

- •We identify and mechanistically characterize a long\-standing structural misalignment in two\-stage, multi\-layer locate\-then\-edit methods, and quantitatively validate it through controlled experiments\.
- •We propose DOW\-KE, an anchor\-free multi\-layer editing method that uses the final editing objective as its loss and jointly optimizes the weight updates at all edited layers through full\-model backpropagation, thereby explicitly accounting for propagation attenuation, distortion, and cross\-layer coupling\. It also embeds all preservation constraints into the update parameterization\.
- •In sequential editing experiments on two datasets and three models, DOW\-KE achieves the highest overall Score in five of six model\-dataset settings among the evaluated baselines\. Ablations further characterize the behavior of direct weight optimization and in\-graph constraint parameterization in the evaluated setting\.

## Related Work

### Locate\-then\-Edit Methods

ROME\(Menget al\.[2022](https://arxiv.org/html/2608.16932#bib.bib7)\)pioneered this paradigm: causal tracing locates the layers carrying a fact, and a two\-stage procedure optimizes an anchor target by gradient descent before converting it into a closed\-form weight update\. MEMIT\(Menget al\.[2023](https://arxiv.org/html/2608.16932#bib.bib8)\)distributes the residual across several layers to increase capacity; AlphaEdit\(Fanget al\.[2025](https://arxiv.org/html/2608.16932#bib.bib12)\)constrains the update to the null space of protected\-knowledge keys, and EvoEdit\(Lyuet al\.[2026](https://arxiv.org/html/2608.16932#bib.bib14)\)extends that projection to cover prior edits\. The attenuation and distortion introduced by pairing multi\-layer editing with a single anchor, however, went largely unexamined until BLUE\(Liet al\.[2025](https://arxiv.org/html/2608.16932#bib.bib5)\)and FE\(Liuet al\.[2026](https://arxiv.org/html/2608.16932#bib.bib6)\): the former introduces an additional anchor, assigning one anchor target to each boundary layer, at a cost growing with their number; the latter constructs layerwise targets by forward replay\. Progress along this line has thus refined the intermediate targets and the coverage of preservation statistics, while leaving the two\-stage premise itself intact: what is optimized is an activation at the anchor, not the weight update actually deployed\.

### Other Knowledge\-Editing Paradigms

Beyond methods that directly modify the original parameters, knowledge editing encompasses two other paradigms\. The first uses auxiliary parameters or external memory: the original model remains unchanged, while new knowledge is stored in an additional component\. WISE and MELO store edits in routed side memories or dynamic LoRA modules\(Wanget al\.[2024](https://arxiv.org/html/2608.16932#bib.bib18); Yuet al\.[2024](https://arxiv.org/html/2608.16932#bib.bib20)\)\. The second paradigm is meta\-learning, which trains an auxiliary network to learn how to modify parameters\. KE and MEND use hypernetworks to transform naive fine\-tuning gradients into parameter updates that satisfy locality constraints\(De Caoet al\.[2021](https://arxiv.org/html/2608.16932#bib.bib2); Mitchellet al\.[2022](https://arxiv.org/html/2608.16932#bib.bib9)\)\.

External\-memory methods are particularly suited to lifelong editing, where edits arrive one by one and must take effect immediately; however, their storage and routing costs grow continually with the number of edits\. LocFT\-BF revisits direct fine\-tuning by combining localized parameter selection with breadth\-first optimization\(Yanget al\.[2026](https://arxiv.org/html/2608.16932#bib.bib19)\)\. Under the same setting considered by WISE\(Wanget al\.[2024](https://arxiv.org/html/2608.16932#bib.bib18)\), the concurrent work LOKI \(June 2026;Eskandaret al\.[2026](https://arxiv.org/html/2608.16932#bib.bib11)\) instead directly modifies the model’s internal weights\. LOKI removes the intermediate activation anchor\. Its weight\-space null\-space constraint specifies only a general feasible update space and does not use the target edit to predetermine which input features the update should respond to\. Consequently, full\-size weight optimization jointly determines which input features to affect and what content to write\. By contrast, DOW\-KE retains the batched locate\-then\-edit mechanism, in which edit keys and preservation statistics predetermine which input features an update can respond to, and learns only what to write end to end against the final editing objective\. This division keeps each update associated with a specific edit fact, restricts optimization to a low\-rank space that grows with the edit batch, and supports joint coordination among edits within the batch\. DOW\-KE therefore remains a fact\-directed locate\-then\-edit method, replacing the anchor target with the final editing objective; LOKI instead learns the update structure within its general feasible weight space after removing the intermediate activation anchor\.

## Preliminaries

### Residual Stream and FFN Key\-Value Memory

Given an autoregressive language modelGG, let𝐡tl∈ℝd1\\mathbf\{h\}\_\{t\}^\{l\}\\in\\mathbb\{R\}^\{d\_\{1\}\}denote the residual\-stream state at positionttin layerll\. Let𝐚tl\\mathbf\{a\}\_\{t\}^\{l\}and𝐦tl\\mathbf\{m\}\_\{t\}^\{l\}denote the increments written to the residual stream by the attention module and the FFN, respectively\. We useγ\\gammato denote layer normalization andϕl\\phi^\{l\}to denote the FFN feature map before the down projection\. The down\-projection matrix to be edited is denoted byWl∈𝐑d1×d0W^\{l\}\\in\\mathbf\{R\}^\{d\_\{1\}\\times d\_\{0\}\}\. We define the input and output of the down\-projection matrix as the key and value:

𝐤tl≜ϕl​\(γ​\(𝐡tl−1\+𝐚tl\)\)∈𝐑d0,𝐯tl≜Wl​𝐤tl∈ℝd1\.\\mathbf\{k\}\_\{t\}^\{l\}\\triangleq\\phi^\{l\}\\\!\\big\(\\gamma\(\\mathbf\{h\}\_\{t\}^\{l\-1\}\+\\mathbf\{a\}\_\{t\}^\{l\}\)\\big\)\\in\\mathbf\{R\}^\{d\_\{0\}\},\\qquad\\mathbf\{v\}\_\{t\}^\{l\}\\triangleq W^\{l\}\\mathbf\{k\}\_\{t\}^\{l\}\\in\\mathbb\{R\}^\{d\_\{1\}\}\.\(1\)
The feature map can be written in simplified form asϕl​\(𝐱\)=σ​\(Winl​𝐱\)\\phi^\{l\}\(\\mathbf\{x\}\)=\\sigma\(W\_\{\\mathrm\{in\}\}^\{l\}\\mathbf\{x\}\)\. Under the linear associative\-memory view\(Menget al\.[2022](https://arxiv.org/html/2608.16932#bib.bib7)\),WlW^\{l\}stores mappings from input patterns to residual\-stream increments, i\.e\.,Wl​𝐤=𝐯W^\{l\}\\mathbf\{k\}=\\mathbf\{v\}\. Locate\-then\-edit methods accordingly formulate knowledge updates as rewriting specific key\-value associations\.

### Anchor\-Target Optimization

After localization, ROME, MEMIT, and AlphaEdit first optimize a target activation at the deepest edited layerlLl\_\{L\}\. LetGi,𝜹G\_\{i,\\bm\{\\delta\}\}denote the model obtained by adding an increment𝜹\\bm\{\\delta\}to the state𝐡ilL\\mathbf\{h\}\_\{i\}^\{l\_\{L\}\}at the subject’s final token for theii\-th request\. Letℒedit\\mathcal\{L\}\_\{\\mathrm\{edit\}\}denote the negative log\-likelihood of the target object, and letℒKL\\mathcal\{L\}\_\{\\mathrm\{KL\}\}constrain the output distributions before and after the intervention:

𝜹i=arg⁡min𝜹⁡ℒedit​\(Gi,𝜹\)\+λKL​ℒKL​\(Gi,𝜹,G\)\.\\bm\{\\delta\}\_\{i\}=\\arg\\min\_\{\\bm\{\\delta\}\}\\;\\mathcal\{L\}\_\{\\mathrm\{edit\}\}\(G\_\{i,\\bm\{\\delta\}\}\)\+\\lambda\_\{\\mathrm\{KL\}\}\\mathcal\{L\}\_\{\\mathrm\{KL\}\}\(G\_\{i,\\bm\{\\delta\}\},G\)\.\(2\)ℒedit\\mathcal\{L\}\_\{\\mathrm\{edit\}\}is averaged overNNrandom prefixesxjx\_\{j\}to make the target activation robust to contextual variation\.ℒKL\\mathcal\{L\}\_\{\\mathrm\{KL\}\}is evaluated on the essence promptpi′p\_\{i\}^\{\\prime\}to suppress global drift in the subject representation\. The optimized target state is

𝐳i=𝐡ilL\+𝜹i\.\\mathbf\{z\}\_\{i\}=\\mathbf\{h\}\_\{i\}^\{l\_\{L\}\}\+\\bm\{\\delta\}\_\{i\}\.\(3\)Consistent with the terminology in the Introduction, this state is the anchor target\. It specifies the residual\-stream state that the model should reach at the deepest edited layer but does not directly modify the weights\. To avoid confusion with the FFN output value𝐯\\mathbf\{v\}, we follow MEMIT and denote this state by𝐳\\mathbf\{z\}\.

### Closed\-Form Weight Update

The second stage converts the anchor target into weight updates at one or more edited layers\. For any layerllto be updated, the key for each request is first averaged over random prefixes\. The resulting vectors form the edit\-key matrixK1=\[𝐤1l,…,𝐤ul\]∈𝐑d0×uK\_\{1\}=\[\\mathbf\{k\}\_\{1\}^\{l\},\\ldots,\\mathbf\{k\}\_\{u\}^\{l\}\]\\in\\mathbf\{R\}^\{d\_\{0\}\\times u\}for a batch ofuurequests\. Let𝐫i\\mathbf\{r\}\_\{i\}denote the portion of the anchor\-target residual assigned to the current layer for requestii\. Its target value is then𝐯1,i=W​𝐤il\+𝐫i\\mathbf\{v\}\_\{1,i\}=W\\mathbf\{k\}\_\{i\}^\{l\}\+\\mathbf\{r\}\_\{i\}\. When the entire residual is assigned to the deepest edited layer,𝐫i=𝐳i−𝐡ilL\\mathbf\{r\}\_\{i\}=\\mathbf\{z\}\_\{i\}\-\\mathbf\{h\}\_\{i\}^\{l\_\{L\}\}\. Collecting all target values intoV1V\_\{1\}, the weight updateΔ\\Deltais obtained from the following least\-squares problem, in whichK0K\_\{0\}andV0=W​K0V\_\{0\}=WK\_\{0\}are the keys and values of knowledge that must be preserved:

Δ⋆=arg⁡minΔ⁡‖\(W\+Δ\)​K1−V1‖F2\+‖\(W\+Δ\)​K0−V0‖F2\.\\Delta^\{\\star\}=\\arg\\min\_\{\\Delta\}\\big\\\|\(W\+\\Delta\)K\_\{1\}\-V\_\{1\}\\big\\\|\_\{F\}^\{2\}\+\\big\\\|\(W\+\\Delta\)K\_\{0\}\-V\_\{0\}\\big\\\|\_\{F\}^\{2\}\.\(4\)LetR≜V1−W​K1R\\triangleq V\_\{1\}\-WK\_\{1\}denote the value residual\. UsingW​K0=V0WK\_\{0\}=V\_\{0\}, the problem above reduces to regularized least squares in whichRRspecifies the desired correction, while the second\-order statistics of the edit and preserved keys jointly constrain the solution directions\. In practice,K0​K0⊤K\_\{0\}K\_\{0\}^\{\\top\}is approximated byλ​C\\lambda C, withCCthe second moment of keys from a general\-domain corpus, yielding

Δ⋆=R​K1⊤​\(λ​C\+K1​K1⊤\)−1\.\\Delta^\{\\star\}=RK\_\{1\}^\{\\top\}\\big\(\\lambda C\+K\_\{1\}K\_\{1\}^\{\\top\}\\big\)^\{\-1\}\.\(5\)This form handles a batch of requests simultaneously and serves as the basis for the per\-layer closed\-form weight update in multi\-layer editing\.

MEMIT distributes the same batch of requests across a layer windowℒ\\mathcal\{L\}to alleviate the capacity limitation of a single layer\. The anchor target is optimized only once at the deepest layer, after which the layerwise updates are computed sequentially from shallow to deep\. When processing thejj\-th edited layerljl\_\{j\}, the updates to the precedingj−1j\-1layers have already taken effect\. The algorithm performs a new forward pass to recompute the current𝐡ilL\\mathbf\{h\}\_\{i\}^\{l\_\{L\}\}and edit keysK1ljK\_\{1\}^\{l\_\{j\}\}, and then evenly distributes the remaining anchor\-target residual across the remaining layers:

𝐫i\(j\)=𝐳i−𝐡ilLL−j\+1,R\(j\)=\[𝐫1\(j\),…,𝐫u\(j\)\]\.\\mathbf\{r\}\_\{i\}^\{\(j\)\}=\\frac\{\\mathbf\{z\}\_\{i\}\-\\mathbf\{h\}\_\{i\}^\{l\_\{L\}\}\}\{L\-j\+1\},\\qquad R^\{\(j\)\}=\\big\[\\mathbf\{r\}\_\{1\}^\{\(j\)\},\\ldots,\\mathbf\{r\}\_\{u\}^\{\(j\)\}\\big\]\.\(6\)The update to the current layer is

Δlj=R\(j\)​\(K1lj\)⊤​\(λ​Clj\+K1lj​\(K1lj\)⊤\)−1\.\\Delta^\{l\_\{j\}\}=R^\{\(j\)\}\\big\(K\_\{1\}^\{l\_\{j\}\}\\big\)^\{\\top\}\\Big\(\\lambda C^\{l\_\{j\}\}\+K\_\{1\}^\{l\_\{j\}\}\\big\(K\_\{1\}^\{l\_\{j\}\}\\big\)^\{\\top\}\\Big\)^\{\-1\}\.\(7\)Each layerwise closed\-form solution fits only the local target assigned to that layer and does not directly enforce attainment of the final anchor target\.

## Empirical Study

### Single\-Anchor Multi\-Layer Editing

![Refer to caption](https://arxiv.org/html/2608.16932v1/x2.png)Figure 3:Two\-stage mediation leaves the anchor target unmet \(AlphaEdit, Llama\-3\-8B, CounterFact\)\. \(a\) Residual closure as the edit window is written shallow\-to\-deep \(x\-axis: deepest layer written, “—” = pre\-edit; red = still unclosed, blue = closed; the dashed tick is the ideal split under conditions A\+B, which would reach0\); the terminal red segment isrgap≈0\.33r\_\{\\mathrm\{gap\}\}\\approx 0\.33\. \(b\) Per\-layer realization ratio⟨Δ​W​k,r⟩/‖r‖2\\langle\\Delta Wk,r\\rangle/\\\|r\\\|^\{2\}\(condition A\); ideal=1=1\. \(c\) Forward directional fidelityfℓf\_\{\\ell\}\(cosine between the injected target direction and the direction arriving at the anchor\); layer88is the anchor \(no transport\); dashed guide is0\.73z−ℓ0\.73^\{\\,z\-\\ell\}\. \(d\) Terminalrgapr\_\{\\mathrm\{gap\}\}vs\. edit window \(1/2/3/5 layers==\[8\]/\[7,8\]/\[6,7,8\]/\[4–8\]\); the gap shrinks but never closes\.![Refer to caption](https://arxiv.org/html/2608.16932v1/x3.png)Figure 4:The gap resists cheap fixes\. \(a\) Post\-hoc update scaleα\\alpha: efficacy \(EFF\), generalization \(GEN\), specificity \(SPE\), and the composite score; EFF/GEN and SPE move in opposite directions on one trade\-off curve, and the composite peaks at86\.8986\.89\(α=0\.9\\alpha\{=\}0\.9\)—rescaling alone cannot close the gap\. \(b\) Realization ratio, standard AlphaEdit \(uniform apportionment\) vs\. FE \(forward\-replay targets\); both stay near0\.50\.5regardless of how the target is constructed, so better targets do not resolve the realization error\.To support batch editing, single\-anchor multi\-layer methods usually distribute updates across multiple layers while retaining a single anchor\. The anchor target is still optimized only once at the deepest edited layer, while the residual𝐳i−𝐡ilL\\mathbf\{z\}\_\{i\}\-\\mathbf\{h\}\_\{i\}^\{l\_\{L\}\}is allocated across layers with the quota1/\(L−j\+1\)1/\(L\-j\+1\), and each layer independently performs a least\-squares weight update\. This decomposition is no longer lossless\. Its closure depends on two conditions that hold automatically in the single\-layer setting: \(A\) each layer update exactly realizes its allocated target at the edit key, i\.e\.,Δlj​𝐤ilj=𝐫i\(j\)\\Delta^\{l\_\{j\}\}\\mathbf\{k\}\_\{i\}^\{l\_\{j\}\}=\\mathbf\{r\}\_\{i\}^\{\(j\)\}; and \(B\) the increment reaches the deepest edited layer losslessly through the residual stream without interference from increments induced by other requests in the same batch\. If both conditions hold, layerwise recursion yields exact closure at the deepest layer,𝐡ilL=𝐳i\\mathbf\{h\}\_\{i\}^\{l\_\{L\}\}=\\mathbf\{z\}\_\{i\}\.

Empirically, this is not the case\. Figure[3](https://arxiv.org/html/2608.16932#Sx4.F3)\(a\) shows the fraction of the residual that remains unrealized at each edited layer\. Its terminal value,rgap=‖𝐳i−𝐡~ilL‖/‖𝐳i−𝐡ilL‖≈0\.33r\_\{\\mathrm\{gap\}\}=\\big\\\|\\mathbf\{z\}\_\{i\}\-\\tilde\{\\mathbf\{h\}\}\_\{i\}^\{l\_\{L\}\}\\big\\\|\\big/\\big\\\|\\mathbf\{z\}\_\{i\}\-\\mathbf\{h\}\_\{i\}^\{l\_\{L\}\}\\big\\\|\\approx 0\.33, jointly measures the extent to which the two conditions are violated\.

Condition \(A\) fails because the closed\-form solution does not fit only the edit target\. To preserve unrelated knowledge, it incorporates a preservation term in addition to the edit term—MEMIT’s preservation term or AlphaEdit’s null\-space projection—and balances the two within the same least\-squares problem\. Target realization at the edit key is therefore systematically discounted\. Figure[3](https://arxiv.org/html/2608.16932#Sx4.F3)\(b\) measures this discount locally at the updated layer, without transport effects\.

Condition \(B\) fails during transport: subsequent layers dilute the increment’s relative contribution, while unplanned attention and FFN responses distort its direction, as shown in Figure[3](https://arxiv.org/html/2608.16932#Sx4.F3)\(c\)\. The remaining residual is recomputed before every layer update, so any deficit at a shallow layer is folded into the targets of subsequent layers\. Nevertheless, because realization at every layer is itself lossy, the net result along the chain is an unrealized residual at the terminal layer\. The implication is that the property verified at the anchor target is never actually deployed: the edited model ends at an intermediate state that was not optimized at any stage\.

Figure[3](https://arxiv.org/html/2608.16932#Sx4.F3)\(d\) shows that the unrealized residual decreases monotonically as the number of edited layers increases, consistent with the superior performance of multi\-layer editing\. Multi\-layer editing enables multiple lossy realizations of the same target to be performed effectively; however, a gap persists even when the full layer window is used\. A direct remedy would be to scale all increments globally to fill this gap\. Scaling, however, does not alter the update structure; it proportionally amplifies the update’s effect at every position and on every unrelated key\. As shown by the scaling\-coefficient sweep in Figure[4](https://arxiv.org/html/2608.16932#Sx4.F4)\(a\), editing efficacy and locality move in opposite directions along the same trade\-off curve, and no scaling factor improves both simultaneously\. The gap cannot be eliminated by changing amplitude alone; doing so requires changing the optimization structure\.

### Multiple Anchors and Multi\-Layer Editing

Several studies have identified the issue above and proposed two successive remedies at the level of target specification\. BLUE edits only the first and last boundary layers of the editing window and introduces a native anchor target at each layer\. Each layer independently performs full gradient\-based anchor\-target optimization instead of receiving an allocated residual; because each target is optimized through the actual forward computation, downstream transport is already taken into account\. This design incurs two costs\. First, gradient\-based anchor\-target optimization is the most expensive stage of the pipeline and doubling the number of anchors doubles this cost\. Second, the layerwise anchor targets are optimized independently, with no mechanism to guarantee their cross\-layer compatibility\. FE further argues that independently specified layerwise targets are mutually incompatible\. It therefore optimizes an anchor target only at the shallowest edited layer and then uses a forward pass to convert it at low cost into mutually compatible targets for subsequent layers\.

However, both approaches improve only the targets\. Each layer update is still realized by the same closed\-form solution, while realization errors have no feedback channel: once introduced, they propagate and accumulate through the residual stream\. FE also explicitly acknowledges that the layerwise increments are not guaranteed to faithfully realize the targets specified by the anchors\. Our measurements in Figure[4](https://arxiv.org/html/2608.16932#Sx4.F4)\(b\) confirm this point: the layerwise realization errors under FE\-style targets nearly overlap those under standard residual\-allocation targets\. Realization loss is largely independent of how the targets are specified; it is a property of the closed\-form weight update itself\. As long as the two\-stage structure of “specify a target, then realize it” remains unchanged, better targets still cannot be guaranteed to be attained\.

### Removing the Intermediate Anchor

Both remedies stay inside the same information boundary: the target stage cannot observe realization loss, the realization stage receives no end\-to\-end feedback, and transport enters the objective of neither\. DOW\-KE therefore removes the anchor and optimizes the layerwise weight updates directly against the final editing objective\. No intermediate target is left that can fail to be realized, and transport, cross\-layer coupling, and within\-batch interference all move inside the forward pass through which that objective is evaluated\. Only the anchor’s role as an intermediate target is removed: the mechanism by which edit keys and preservation statistics predetermine an update’s response scope is retained in full and folded into the update parameterization, so preservation and editing no longer compete inside a per\-layer least\-squares problem\.

## Method

DOW\-KE builds on a structural observation about the closed\-form update: for both MEMIT and AlphaEdit, Eq\.[5](https://arxiv.org/html/2608.16932#Sx3.E5)factors into

Δ=R​B,R∈𝐑d1×u,B∈𝐑u×d0\.\\Delta\\;=\\;R\\,B,\\qquad R\\in\\mathbf\{R\}^\{d\_\{1\}\\times u\},\\quad B\\in\\mathbf\{R\}^\{u\\times d\_\{0\}\}\.\(8\)The columns ofR=\[𝐫1,…,𝐫u\]R=\[\\mathbf\{r\}\_\{1\},\\dots,\\mathbf\{r\}\_\{u\}\]are the residuals allocated from the anchor target, so the anchor influences the update only throughRR;BBis a solution operator built solely from the edit keys and the preservation statistics, namelyK1⊤​\(λ​C\+K1​K1⊤\)−1K\_\{1\}^\{\\top\}\(\\lambda C\+K\_\{1\}K\_\{1\}^\{\\top\}\)^\{\-1\}for MEMIT andK1⊤​P​\[\(K1​K1⊤\+Kp​Kp⊤\)​P\+I\]−1K\_\{1\}^\{\\top\}P\[\(K\_\{1\}K\_\{1\}^\{\\top\}\+K\_\{p\}K\_\{p\}^\{\\top\}\)P\+I\]^\{\-1\}for AlphaEdit, withPPthe null\-space projection derived fromCCandKpK\_\{p\}the edit keys of previous batches\.

DOW\-KE keepsBBfixed during optimization and freesRR: at each edited layer, the allocated residual is replaced by a directly optimized variableAl∈𝐑d1×uA^\{l\}\\in\\mathbf\{R\}^\{d\_\{1\}\\times u\}, givingΔl=Al​Bl\\Delta^\{l\}=A^\{l\}B^\{l\}\. HereBlB^\{l\}is obtained by projecting a standard ridge basis onto the input directions allowed by AlphaEdit\. LetUτlU\_\{\\tau\}^\{l\}contain the leading principal directions ofClC^\{l\}whose cumulative spectral energy reaches at leastτ\\tau, and setPl=I−Uτl​\(Uτl\)⊤P^\{l\}=I\-U\_\{\\tau\}^\{l\}\(U\_\{\\tau\}^\{l\}\)^\{\\top\}\. The variables\{Al\}\\\{A^\{l\}\\\}are jointly optimized against the final editing objective of the deployed model\. For any𝐤∈span⁡\(Uτl\)\\mathbf\{k\}\\in\\operatorname\{span\}\(U\_\{\\tau\}^\{l\}\), the construction above givesBl​𝐤=𝟎B^\{l\}\\mathbf\{k\}=\\mathbf\{0\}; henceΔl​𝐤=𝟎\\Delta^\{l\}\\mathbf\{k\}=\\mathbf\{0\}for everyAlA^\{l\}\. Thus, suppression on the selected key subspace is built into the parameterization rather than learned during optimization\. The thresholdτ\\taucontrols the spectral\-energy coverage of this protected subspace, and Table[3](https://arxiv.org/html/2608.16932#Sx6.T3)shows that varying it shifts the empirical trade\-off between Specificity and Generalization\.

Preservation is applied on the output side as well\. WithD0D\_\{0\}the second moment of unrelated\-knowledge outputs at layerll, we buildPoutlP\_\{\\mathrm\{out\}\}^\{l\}fromD0D\_\{0\}by the same energy rule, giving the deployed update

Δl=Poutl​Al​Bl\.\\Delta^\{l\}\\;=\\;P\_\{\\mathrm\{out\}\}^\{l\}\\,A^\{l\}\\,B^\{l\}\.\(9\)BlB^\{l\}constrains which inputs the update responds to;PoutlP\_\{\\mathrm\{out\}\}^\{l\}keeps its write directions out of the subspace occupied by unrelated\-knowledge outputs\.

A third restriction is positional and acts only in the backward pass\. The objective scores the target object at the end of the sequence, so it can also be reduced by writing at positions before the subject, which under causal attention have not read the complete subject and cannot be fact\-directed\. Writingsis\_\{i\}for the subject’s final position in rewriting sequenceiiandsg​\[⋅\]\\mathrm\{sg\}\[\\cdot\]for the stop\-gradient operator, the increment contributed at positionttisPoutl​Al​\(Bl​𝐤tl\)P\_\{\\mathrm\{out\}\}^\{l\}A^\{l\}\(B^\{l\}\\mathbf\{k\}\_\{t\}^\{l\}\)fort≥sit\\geq s\_\{i\}andsg​\[Poutl​Al\]​\(Bl​𝐤tl\)\\mathrm\{sg\}\[P\_\{\\mathrm\{out\}\}^\{l\}A^\{l\}\]\(B^\{l\}\\mathbf\{k\}\_\{t\}^\{l\}\)fort<sit<s\_\{i\}\. The two expressions are numerically identical, so the forward logits and loss are unchanged relative to the deployed update\. Only the backward gradient routing changes: for positions before the subject, the direct gradient path throughPoutl​AlP\_\{\\mathrm\{out\}\}^\{l\}A^\{l\}is stopped, while the path through𝐤tl\\mathbf\{k\}\_\{t\}^\{l\}remains active, allowing the loss to backpropagate through the network\.

Removing residual allocation also leaves the division of labor across layers to the end\-to\-end objective, so unfavorable transport at one layer can be absorbed by others; this is a by\-product of joint optimization that we do not validate separately\.

Collecting the pieces, DOW\-KE solves

\{Al\}⋆=arg⁡min\{Al\}⁡ℒedit​\(GΔ\)\+λKL​ℒKL​\(GΔ,G\),\\\{A^\{l\}\\\}^\{\\star\}=\\arg\\min\_\{\\\{A^\{l\}\\\}\}\\;\\mathcal\{L\}\_\{\\mathrm\{edit\}\}\\big\(G\_\{\\Delta\}\\big\)\+\\lambda\_\{\\mathrm\{KL\}\}\\,\\mathcal\{L\}\_\{\\mathrm\{KL\}\}\\big\(G\_\{\\Delta\},G\\big\),\(10\)whereGΔG\_\{\\Delta\}is the model with the updates in Eq\.[9](https://arxiv.org/html/2608.16932#Sx5.E9)applied at all edited layers\. We use the same losses as in Eq\.[2](https://arxiv.org/html/2608.16932#Sx3.E2):ℒedit\\mathcal\{L\}\_\{\\mathrm\{edit\}\}is the negative log\-likelihood of the target object, averaged over theuurequests and theNNprefixesxjx\_\{j\}, whileℒKL\\mathcal\{L\}\_\{\\mathrm\{KL\}\}is computed on the essence promptspi′p\_\{i\}^\{\\prime\}\. Unlike anchor optimization, both losses are evaluated on the model containing all layer updates and are used to optimize\{Al\}\\\{A^\{l\}\\\}jointly\. The positional rule is applied only when computingℒedit\\mathcal\{L\}\_\{\\mathrm\{edit\}\}: it stops direct gradients toAlA^\{l\}from positions before the subject without changing the forward outputs or the deployed updates\. The KL loss uses the standard gradient path\.

## Experiments

### Models and Datasets

To enable fair comparisons with prior work, we evaluate our method on three autoregressive language models commonly used in model\-editing research: Llama\-3\-8B\-Instruct\(Meta AI[2024](https://arxiv.org/html/2608.16932#bib.bib24)\), GPT\-J\-6B\(Wang and Komatsuzaki[2021](https://arxiv.org/html/2608.16932#bib.bib26)\), and GPT2\-XL\-1\.5B\(Radfordet al\.[2019](https://arxiv.org/html/2608.16932#bib.bib25)\)\. Following the standard configuration of locate\-then\-edit methods, all edits are applied to the FFN down\-projection matrix at each layer within the layer window identified by causal tracing: 4–8 for Llama\-3\-8B\-Instruct, 3–8 for GPT\-J\-6B, and 13–17 for GPT2\-XL\-1\.5B\(Fanget al\.[2025](https://arxiv.org/html/2608.16932#bib.bib12)\)\. All experiments are run on NVIDIA L40 GPUs\. We conduct experiments on CounterFact\(Menget al\.[2022](https://arxiv.org/html/2608.16932#bib.bib7)\)and zsRE\(Levyet al\.[2017](https://arxiv.org/html/2608.16932#bib.bib23)\), two of the most influential benchmarks for model editing\.

### Evaluation Metrics and Experiment Setup

Efficacy, Generalization, and Specificity are the three core metrics for knowledge editing\. They are measured on edit prompts, rephrased prompts, and neighborhood prompts, respectively, although their precise computation differ slightly between the two datasets\. The main tables additionally report their harmonic mean as the overall Score\. CounterFact further reports Fluency and Consistency\. Fluency uses the n\-gram entropy of generated text to detect repetitive degeneration, whereas Consistency measures the TF\-IDF similarity between generated text and a reference passage describing the new fact, thereby assessing whether the edit is reflected in free\-form generation\. We follow the original AlphaEdit implementation for all metric calculations\(Fanget al\.[2025](https://arxiv.org/html/2608.16932#bib.bib12)\)\. Under the sequential editing protocol, we sample 2,000 requests from each dataset and apply them successively to the same model in 20 batches of 100 requests\. Further details of the experimental setup are provided in Appendix A of the supplementary material\.

Table 1:Comparison of DOW\-KE with existing methods on the sequential model editing task\. The best results are highlighted in bold, while the second\-best results are underlined\. Baseline results are taken fromFanget al\.\([2025](https://arxiv.org/html/2608.16932#bib.bib12)\)andLiet al\.\([2025](https://arxiv.org/html/2608.16932#bib.bib5)\)\. For FE, no official results have been reported for the setting of 2,000 requests applied sequentially in 20 batches; the values in the table are from our own evaluation\. Confidence intervals are reported in appendix of the supplementary material\.Table 2:Operating\-point\-matched ablation results\. The scaling coefficientα\\alphacontrols the trade\-off between editing performance and Specificity\. Rows 1–2 compare Efficacy and Generalization at approximately matched Specificity, whereas Rows 2–3 compare Specificity at approximately matched editing performance\.
### Comparative Experiments

Table[1](https://arxiv.org/html/2608.16932#Sx6.T1)reports DOW\-KE’s evaluation results on three models and two datasets\. All values are averaged over five runs; under the same experimental settings, we compare DOW\-KE with other locate\-then\-edit methods and fine\-tuning\-based baselines\(Fanget al\.[2025](https://arxiv.org/html/2608.16932#bib.bib12)\)\. DOW\-KE achieves the best result in most settings and the second\-best result in the remaining settings\. Compared with BLUE, it has a clear advantage in five of the six model–dataset combinations; in particular, it outperforms BLUE on every reported metric on zsRE\. We also include FE as a comparison method, whose results are relatively weak under the multi\-batch sequential\-editing protocol\. Thus, DOW\-KE not only consistently improves on AlphaEdit but also outperforms, overall, methods that retain the anchor structure, including BLUE and FE\.

### Ablation Studies

Table[2](https://arxiv.org/html/2608.16932#Sx6.T2)examines two key design choices of DOW\-KE: removing the intermediate activation anchor and directly optimizing the weights, and incorporating the preservation\-subspace constraint into the differentiable computation graph\. We use AlphaEdit with the energy\-threshold rule as the comparison baseline\. Although this setting differs slightly from the original AlphaEdit, it uses the same thresholding rule as DOW\-KE\. Letα\\alphadenote the weight\-scaling coefficient,Δ​W=α​Δ​Wgrad\\Delta W=\\alpha\\Delta W\_\{\\mathrm\{grad\}\}, with a default value of 1\. We treatα\\alphaas an operating\-point control along the trade\-off between editing performance and Specificity\. At matched Specificity \(77\.59 versus 77\.85\), NLL\-to\-W\. withα=1\.2\\alpha=1\.2improves Efficacy and Generalization over the baseline, demonstrating the benefit of direct weight optimization\. At comparable Efficacy and Generalization, adding the in\-graph constraint with the defaultα=1\\alpha=1improves Specificity from 77\.85 to 80\.24\. These matched comparisons support the complementary contributions of direct optimization and in\-graph constraint parameterization\. Row 4 adds subject\-conditioned gradient routing, which is designed to prevent the optimizer from directly shapingAlA^\{l\}through positions that precede and therefore do not fully encode the subject\. Routing improves Generalization by 1\.02 points while reducing Specificity by 1\.38 points, resulting in a modest 0\.31\-point decrease in Score\. We include it in the full configuration for its structural role in promoting fact\-conditioned optimization, rather than as a score\-maximizing component\.

Table 3:Ablation results for the subspace threshold and output\-side projection on LLaMA3\-8B and CounterFact\. Experiments are conducted without subject routing to isolate the effects of the subspace threshold and output\-side projection\.Table[3](https://arxiv.org/html/2608.16932#Sx6.T3)shows a threshold\-controlled trade\-off: increasingτ\\tauimproves Specificity but reduces Generalization\. One possible explanation is that the enlarged preservation subspace leaves fewer editable directions, encouraging narrower fitting to the optimization prompts and weaker transfer to paraphrases\. We leave direct verification of this mechanism to future work\. Table[3](https://arxiv.org/html/2608.16932#Sx6.T3)also compares performance with and without the output\-side preservation constraint\. Across all thresholds shown, adding output\-side preservation improves Specificity and, in some settings, also improves Generalization\. This suggests that output\-side preservation provides a constraint complementary to key\-side preservation, thereby strengthening the protection of unrelated knowledge\.

## Conclusion

We introduce DOW\-KE, an anchor\-free knowledge\-editing method that addresses the substantial gap between pre\-specified intermediate targets and their actual realization in multi\-layer editing\. DOW\-KE removes the intermediate activation anchor and layerwise residual allocation, and brings preservation constraints, transport attenuation, directional distortion, and cross\-layer coupling into a single differentiable computation graph\. The update that is optimized is therefore exactly the update that is deployed, eliminating the mechanism by which a prescribed intermediate target can remain unrealized\. At the same time, the method retains the mechanism by which edit keys and preservation statistics predetermine the response scope of an update, so that each edit remains tied to a specific fact\. Experiments on three models and two datasets show that DOW\-KE achieves the highest composite Score in five of six evaluated settings\.

## References

- N\. De Cao, W\. Aziz, and I\. Titov \(2021\)Editing factual knowledge in language models\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,M\. Moens, X\. Huang, L\. Specia, and S\. W\. Yih \(Eds\.\),Online and Punta Cana, Dominican Republic,pp\. 6491–6506\.External Links:[Document](https://dx.doi.org/10.18653/v1/2021.emnlp-main.522),[Link](https://aclanthology.org/2021.emnlp-main.522/)Cited by:[Introduction](https://arxiv.org/html/2608.16932#Sx1.p1.1),[Other Knowledge\-Editing Paradigms](https://arxiv.org/html/2608.16932#Sx2.SSx2.p1.1)\.
- M\. Eskandar, M\. Sirera Perelló, S\. Ioannidis, and J\. Dy \(2026\)LOKI: memory\-free null\-space constrained lifelong knowledge editing\.Note:arXiv preprintExternal Links:2606\.19679,[Document](https://dx.doi.org/10.48550/arXiv.2606.19679),[Link](https://arxiv.org/abs/2606.19679)Cited by:[Other Knowledge\-Editing Paradigms](https://arxiv.org/html/2608.16932#Sx2.SSx2.p2.1)\.
- J\. Fang, H\. Jiang, K\. Wang, Y\. Ma, J\. Shi, X\. Wang, X\. He, and T\. Chua \(2025\)AlphaEdit: null\-space constrained knowledge editing for language models\.InThe Thirteenth International Conference on Learning Representations,Singapore\.External Links:[Link](https://openreview.net/forum?id=HvSytvg3Jh)Cited by:[Locate\-then\-Edit Methods](https://arxiv.org/html/2608.16932#Sx2.SSx1.p1.1),[Models and Datasets](https://arxiv.org/html/2608.16932#Sx6.SSx1.p1.1),[Evaluation Metrics and Experiment Setup](https://arxiv.org/html/2608.16932#Sx6.SSx2.p1.1),[Comparative Experiments](https://arxiv.org/html/2608.16932#Sx6.SSx3.p1.1),[Table 1](https://arxiv.org/html/2608.16932#Sx6.T1)\.
- M\. Geva, J\. Bastings, K\. Filippova, and A\. Globerson \(2023\)Dissecting recall of factual associations in auto\-regressive language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 12216–12235\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.751),[Link](https://aclanthology.org/2023.emnlp-main.751/)Cited by:[Introduction](https://arxiv.org/html/2608.16932#Sx1.p1.1)\.
- T\. Hartvigsen, S\. Sankaranarayanan, H\. Palangi, Y\. Kim, and M\. Ghassemi \(2023\)Aging with GRACE: lifelong model editing with discrete key\-value adaptors\.InAdvances in Neural Information Processing Systems,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Vol\.36,pp\. 47934–47959\.External Links:[Document](https://dx.doi.org/10.52202/075280-2079),[Link](https://proceedings.neurips.cc/paper_files/paper/2023/file/95b6e2ff961580e03c0a662a63a71812-Paper-Conference.pdf)Cited by:[Introduction](https://arxiv.org/html/2608.16932#Sx1.p4.1)\.
- P\. Hase, M\. Bansal, B\. Kim, and A\. Ghandeharioun \(2023\)Does localization inform editing? surprising differences in causality\-based localization vs\. knowledge editing in language models\.InAdvances in Neural Information Processing Systems,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Vol\.36,pp\. 17643–17668\.External Links:[Document](https://dx.doi.org/10.52202/075280-0774),[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/3927bbdcf0e8d1fa8aa23c26f358a281-Abstract-Conference.html)Cited by:[Introduction](https://arxiv.org/html/2608.16932#Sx1.p1.1)\.
- Z\. Jiang, Z\. Sun, W\. Shi, P\. Rodriguez, C\. Zhou, G\. Neubig, X\. V\. Lin, W\. Yih, and S\. Iyer \(2024\)Instruction\-tuned language models are better knowledge learners\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 5421–5434\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.296),[Link](https://aclanthology.org/2024.acl-long.296/)Cited by:[Introduction](https://arxiv.org/html/2608.16932#Sx1.p1.1)\.
- O\. Levy, M\. Seo, E\. Choi, and L\. Zettlemoyer \(2017\)Zero\-shot relation extraction via reading comprehension\.InProceedings of the 21st Conference on Computational Natural Language Learning \(CoNLL 2017\),R\. Levy and L\. Specia \(Eds\.\),Vancouver, Canada,pp\. 333–342\.External Links:[Document](https://dx.doi.org/10.18653/v1/K17-1034),[Link](https://aclanthology.org/K17-1034/)Cited by:[Models and Datasets](https://arxiv.org/html/2608.16932#Sx6.SSx1.p1.1)\.
- P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. Kiela \(2020\)Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.InAdvances in Neural Information Processing Systems,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\. Balcan, and H\. Lin \(Eds\.\),Vol\.33,pp\. 9459–9474\.External Links:[Link](https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html)Cited by:[Introduction](https://arxiv.org/html/2608.16932#Sx1.p1.1)\.
- X\. Li, S\. Wang, S\. Li, S\. Song, B\. Ji, J\. Ma, and J\. Yu \(2025\)Rethinking residual distribution in locate\-then\-edit model editing\.InAdvances in Neural Information Processing Systems,D\. Belgrave, C\. Zhang, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, and N\. Chen \(Eds\.\),Vol\.38,pp\. 78348–78374\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2025/file/70d4ef44dc973586cfa3ea92b4868b72-Paper-Conference.pdf)Cited by:[Introduction](https://arxiv.org/html/2608.16932#Sx1.p3.1),[Locate\-then\-Edit Methods](https://arxiv.org/html/2608.16932#Sx2.SSx1.p1.1),[Table 1](https://arxiv.org/html/2608.16932#Sx6.T1)\.
- W\. Liu, H\. Liu, Z\. Deng, Y\. W\. Teh, and W\. S\. Lee \(2026\)From backward spreading to forward replay: revisiting target construction in LLM parameter editing\.InProceedings of the 43rd International Conference on Machine Learning,Proceedings of Machine Learning Research,Seoul, South Korea\.External Links:[Link](https://icml.cc/virtual/2026/poster/61038)Cited by:[Introduction](https://arxiv.org/html/2608.16932#Sx1.p3.1),[Introduction](https://arxiv.org/html/2608.16932#Sx1.p4.1),[Locate\-then\-Edit Methods](https://arxiv.org/html/2608.16932#Sx2.SSx1.p1.1)\.
- S\. Lyu, Y\. Gu, X\. Wang, J\. Huang, S\. Luan, Y\. Cui, X\. Chang, and P\. Lu \(2026\)EvoEdit: evolving null\-space alignment for robust and efficient knowledge editing\.InFindings of the Association for Computational Linguistics: ACL 2026,M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 1520–1540\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.75),[Link](https://aclanthology.org/2026.findings-acl.75/)Cited by:[Locate\-then\-Edit Methods](https://arxiv.org/html/2608.16932#Sx2.SSx1.p1.1)\.
- K\. Meng, D\. Bau, A\. Andonian, and Y\. Belinkov \(2022\)Locating and editing factual associations in GPT\.InAdvances in Neural Information Processing Systems,S\. Koyejo, S\. Mohamed, A\. Agarwal, D\. Belgrave, K\. Cho, and A\. Oh \(Eds\.\),Vol\.35,pp\. 17359–17372\.External Links:[Document](https://dx.doi.org/10.52202/068431-1262),[Link](https://proceedings.neurips.cc/paper_files/paper/2022/file/6f1d43d5a82a37e89b0665b33bf3a182-Paper-Conference.pdf)Cited by:[Introduction](https://arxiv.org/html/2608.16932#Sx1.p2.1),[Locate\-then\-Edit Methods](https://arxiv.org/html/2608.16932#Sx2.SSx1.p1.1),[Residual Stream and FFN Key\-Value Memory](https://arxiv.org/html/2608.16932#Sx3.SSx1.p3.3),[Models and Datasets](https://arxiv.org/html/2608.16932#Sx6.SSx1.p1.1)\.
- K\. Meng, A\. S\. Sharma, A\. J\. Andonian, Y\. Belinkov, and D\. Bau \(2023\)Mass\-editing memory in a transformer\.InThe Eleventh International Conference on Learning Representations,Kigali, Rwanda\.External Links:[Link](https://openreview.net/forum?id=MkbcAHIYgyS)Cited by:[Introduction](https://arxiv.org/html/2608.16932#Sx1.p2.1),[Locate\-then\-Edit Methods](https://arxiv.org/html/2608.16932#Sx2.SSx1.p1.1)\.
- Meta AI \(2024\)Introducing Meta Llama 3: the most capable openly available LLM to date\.Note:https://ai\.meta\.com/blog/meta\-llama\-3/Accessed: 2026\-07\-25Cited by:[Models and Datasets](https://arxiv.org/html/2608.16932#Sx6.SSx1.p1.1)\.
- E\. Mitchell, C\. Lin, A\. Bosselut, C\. Finn, and C\. D\. Manning \(2022\)Fast model editing at scale\.InThe Tenth International Conference on Learning Representations,Virtual Event\.External Links:[Link](https://openreview.net/forum?id=0DcZxeWfOPt)Cited by:[Introduction](https://arxiv.org/html/2608.16932#Sx1.p1.1),[Other Knowledge\-Editing Paradigms](https://arxiv.org/html/2608.16932#Sx2.SSx2.p1.1)\.
- A\. Radford, J\. Wu, R\. Child, D\. Luan, D\. Amodei, and I\. Sutskever \(2019\)Language models are unsupervised multitask learners\.Technical reportOpenAI\.Note:Available athttps://cdn\.openai\.com/better\-language\-models/language\-models\.pdfCited by:[Models and Datasets](https://arxiv.org/html/2608.16932#Sx6.SSx1.p1.1)\.
- A\. Sinitsin, V\. Plokhotnyuk, D\. Pyrkin, S\. Popov, and A\. Babenko \(2020\)Editable neural networks\.InThe Eighth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=HJedXaEtvS)Cited by:[Introduction](https://arxiv.org/html/2608.16932#Sx1.p1.1)\.
- B\. Wang and A\. Komatsuzaki \(2021\)GPT\-J\-6B: a 6 billion parameter autoregressive language model\.Note:https://github\.com/kingoflolz/mesh\-transformer\-jaxEleutherAI model release; accessed: 2026\-07\-25Cited by:[Models and Datasets](https://arxiv.org/html/2608.16932#Sx6.SSx1.p1.1)\.
- P\. Wang, Z\. Li, N\. Zhang, Z\. Xu, Y\. Yao, Y\. Jiang, P\. Xie, F\. Huang, and H\. Chen \(2024\)WISE: rethinking the knowledge memory for lifelong model editing of large language models\.InAdvances in Neural Information Processing Systems,A\. Globerson, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. Tomczak, and C\. Zhang \(Eds\.\),Vol\.37,Rio de Janeiro, Brazil,pp\. 53764–53797\.External Links:[Document](https://dx.doi.org/10.52202/079017-1703),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/60960ad78868fce5c165295fbd895060-Abstract-Conference.html)Cited by:[Other Knowledge\-Editing Paradigms](https://arxiv.org/html/2608.16932#Sx2.SSx2.p1.1),[Other Knowledge\-Editing Paradigms](https://arxiv.org/html/2608.16932#Sx2.SSx2.p2.1)\.
- W\. Yang, R\. Tang, H\. Zang, D\. Su, Q\. Cao, J\. Wang, H\. Shen, X\. Cheng, and F\. Sun \(2026\)Fine\-tuning done right in model editing\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=cfHuA5jsPt)Cited by:[Other Knowledge\-Editing Paradigms](https://arxiv.org/html/2608.16932#Sx2.SSx2.p2.1)\.
- Y\. Yao, P\. Wang, B\. Tian, S\. Cheng, Z\. Li, S\. Deng, H\. Chen, and N\. Zhang \(2023\)Editing large language models: problems, methods, and opportunities\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 10222–10240\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.632),[Link](https://aclanthology.org/2023.emnlp-main.632/)Cited by:[Introduction](https://arxiv.org/html/2608.16932#Sx1.p1.1)\.
- L\. Yu, Q\. Chen, J\. Zhou, and L\. He \(2024\)MELO: enhancing model editing with neuron\-indexed dynamic LoRA\.Proceedings of the AAAI Conference on Artificial Intelligence38\(17\),pp\. 19449–19457\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v38i17.29916),[Link](https://ojs.aaai.org/index.php/AAAI/article/view/29916)Cited by:[Other Knowledge\-Editing Paradigms](https://arxiv.org/html/2608.16932#Sx2.SSx2.p1.1)\.