LadderEdit：面向大模型高效终身编辑的编辑级残差压缩

# Edit-Level Residual Compression for Memory-Efficient Lifelong Editing of LLMs
Source: [https://arxiv.org/html/2610.11160](https://arxiv.org/html/2610.11160)
Xiaobing YuPeijie QiuJin YangXuanzhao DongWeiwei MaZhaoqi An††thanks:This work was done prior to joining Amazon\.Affiliation:Yale University;Affiliation:Washington University in St\. Louis;Affiliation:Icahn School of Medicine at Mount Sinai;Affiliation:Arizona State UniversityEmail:[xiaobing\.yu@yale\.edu](mailto:)Xiaoqi ZhaoAffiliation:Yale University;Email:[xiaoqi\.zhao@yale\.edu](mailto:)Affiliation:co\-corresponding authorsCode:[https://github\.com/VisualReasoner/LadderEdit](https://github.com/VisualReasoner/LadderEdit)Xiaofeng LiuAffiliation:Yale University;Email:[xiaofeng\.liu@yale\.edu](mailto:)Affiliation:co\-corresponding authorsCode:[https://github\.com/VisualReasoner/LadderEdit](https://github.com/VisualReasoner/LadderEdit)

###### Abstract

Lifelong editing of LLMs requires storing thousands of edits after acquisition\. A widely used family of approaches attaches one LoRA adapter per edit, which preserves behavior but grows linearly in storage\. To address this challenge, we propose LadderEdit, a method that compresses each LoRA adapter after it is acquired\. Each edit is first stored at low rank as a cheap sketch\. We then check whether this sketch still satisfies the rewrite, generalization, and locality contract on probe prompts\. Edits that pass keep the sketch; those that fail are promoted to a higher rank along a ladder until the contract is met\. Because every edit retains some representation, coverage is maintained, and only hard edits consume more rank\. Across ZsRE, CounterFact, and WikiBigEdit benchmarks on LLaMA\-3\-8B, Mistral\-7B, and Qwen2\.5\-7B, LadderEdit tracks exact LoRA storage at5\.2×5\.2\\timesless memory and remains effective at50,00050\{,\}000sequential edits\.

## 1Introduction

Lifelong model editing asks a deployed large language model to absorb a stream of factual corrections and knowledge updates without full retraining\. This setting is increasingly important: world knowledge changes, model errors are discovered after deployment, and domain\-specific assistants require controlled updates over time\. A useful lifelong editor must therefore satisfy three requirements simultaneously\. It must remain plastic enough to incorporate new edits, stable enough to preserve previous edits and unrelated knowledge, and efficient enough to serve after thousands of updates\.

Existing approaches expose a fundamental storage\-plasticity tension\. Shared\-parameter and constant\-memory editors bound the deployed memory footprint, but every new edit competes with previous edits for limited capacity[Hartvigsen et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib4);[Wang et al\. \(2025\)](https://arxiv.org/html/2610.11160#bib.bib7)\. As the stream grows, such methods can preserve locality while losing edit plasticity: the model avoids damaging unrelated knowledge, but new edits may fail to generalize beyond the rewrite prompt\. Retrieval\- and routing\-based editors reduce direct parameter interference but introduce a separate edit\-selection problem and can fail when a query is semantically close to, but not exactly matched with, the stored edit[Li et al\. \(2024a\)](https://arxiv.org/html/2610.11160#bib.bib8);[Wang and Li \(2024b\)](https://arxiv.org/html/2610.11160#bib.bib25);[Zhang et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib27);[Wang and Li \(2024a\)](https://arxiv.org/html/2610.11160#bib.bib24)\. Direct parameter\-editing methods avoid explicit per\-edit storage, yet repeated updates can interfere destructively over long edit streams[Meng et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib3);[Meng et al\. \(2022\)](https://arxiv.org/html/2610.11160#bib.bib2);[De Cao et al\. \(2021\)](https://arxiv.org/html/2610.11160#bib.bib16);[Dai et al\. \(2022\)](https://arxiv.org/html/2610.11160#bib.bib18);[Fang et al\. \(2025\)](https://arxiv.org/html/2610.11160#bib.bib6);[Wang et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib20);[Mitchell et al\. \(2022a\)](https://arxiv.org/html/2610.11160#bib.bib14)\.

Figure 1:Performance comparison of LadderEdit, exact LoRA, and baseline methods across sequential editing tasks\. LadderEdit usually tracks exact LoRA closely while reducing persistent memory, with small losses in some compressed settings\.An alternative is to store edit\-local update memories\. Each edit is written into an isolated update representation, so later edits do not overwrite earlier ones[Hu et al\. \(2022\)](https://arxiv.org/html/2610.11160#bib.bib1);[Yu et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib9)\. This design preserves edit\-specific plasticity and provides a clean full\-storage reference: if the correct edit memory is selected, the system can recover the behavior of the original update\. The drawback is persistent storage\. If every edit is retained at full resolution, memory grows linearly with the number of edits\. Thus, the bottleneck in long\-horizon editing is not only how to acquire an edit but also how much of each acquired edit must remain stored\.

To address this dilemma, we reframe lifelong edit storage as an edit\-level resolution allocation problem\. The key is not simply which edits should be kept or dropped but how much representational resolution each edit needs to preserve its behavioral contract\. This distinction matters because keep/drop caching couples memory reduction to edit absence: once an edit is dropped, it has no representation and cannot support rewrite, generalization, or locality\. In contrast, a resolution\-allocation policy can preserve coverage for every edit through a cheap representation, while spending additional memory only on edits that require higher fidelity\.

We proposeLadderEdit, a storage\-layer method based on this coverage\-before\-fidelity principle\. Each incoming edit is first acquired as a full\-resolution update memory; in our implementation, this update memory is an exact LoRA update\. LadderEdit is a two\-stage controller\. A*spectral predictor*scores each candidate rank using the singular\-value tail of the update together with the edit’s behavioral slack, and proposes the cheapest rankr^i\\hat\{r\}\_\{i\}whose score falls below a calibrated threshold; this proposal requires only one SVD per edit and no behavioral evaluation\. A*behavioral audit*then verifies the proposed sketch against rewrite, generalization, and locality probes\. If the audit passes, the edit is stored atr^i\\hat\{r\}\_\{i\}; if it fails, the audit promotes the edit along the spectrally\-ordered ladder until a passing rung is found\. This design avoids exhaustive auditing across all rungs and gives every edit coverage through its predicted rung while spending additional resolution only when audit demands it\.

We validate on ZsRE[Levy et al\. \(2017\)](https://arxiv.org/html/2610.11160#bib.bib10), CounterFact \(CF\)[Meng et al\. \(2022\)](https://arxiv.org/html/2610.11160#bib.bib2);[Meng et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib3), and WikiBigEdit[Thede et al\. \(2025\)](https://arxiv.org/html/2610.11160#bib.bib36)using LLaMA\-3\-8B[Grattafiori et al\. \(2024\)](https://arxiv.org/html/2610.11160#bib.bib11), Mistral\-7B[Jiang et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib12), and Qwen2\.5\-7B[Yang et al\. \(2024\)](https://arxiv.org/html/2610.11160#bib.bib28);[Bai et al\. \(2025\)](https://arxiv.org/html/2610.11160#bib.bib40);[Yang et al\. \(2025\)](https://arxiv.org/html/2610.11160#bib.bib13)\. Figure[1](https://arxiv.org/html/2610.11160#S1.F1)shows LadderEdit tracks the full\-resolution reference at a fraction of the memory, and budget\-matched frontiers confirm that residual\-ladder allocation substantially outperforms exact\-cache: preserving coverage before fidelity is the better storage primitive for scalable lifelong editing\.

Our contributions are as follows:*i\)*We formulate lifelong editing storage as a resolution\-allocation problem, separating edit acquisition, selection, and persistent representation\.*ii\)*We introduce LadderEdit, a residual\-ladder controller that covers every edit with a low\-rank sketch and restores detail only when behavioral audits demand it\.*iii\)*We identify*coverage\-before\-fidelity*as the principle for scalable edit memory, nearly doubling utility over exact\-cache at matched budget\.*iv*We validate LadderEdit on ZsRE, CF, and WikiBigEdit, tracking full\-resolution memory at5\.2×5\.2\\timesless storage and remaining effective at 50,000 edits\.

![Refer to caption](https://arxiv.org/html/2610.11160v1/ladderedit-main.png)Figure 2:LadderEdit insertion pipeline\. We first fit an exact LoRA update for the incoming edit, then a candidate rankr^i\\hat\{r\}\_\{i\}from the update spectrum and exact\-edit margin \(one SVD, no behavioral cost\)\. We further decompose the edit into ladder rungs: low\-rank sketches at increasing rank\. And finally we audit the predicted sketch against rewrite, generalization, and locality\. If the audit passes, store atr^i\\hat\{r\}\_\{i\}; otherwise promote along the spectral ladder until a passing rung is found\.
## 2Related Work

#### Model editing and lifelong editing\.

Model editing methods update a model’s behavior without full retraining, either by directly modifying parameters[Sinitsin et al\. \(2020\)](https://arxiv.org/html/2610.11160#bib.bib17);[De Cao et al\. \(2021\)](https://arxiv.org/html/2610.11160#bib.bib16);[Meng et al\. \(2022\)](https://arxiv.org/html/2610.11160#bib.bib2);[Meng et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib3), learning editor networks[Mitchell et al\. \(2022a\)](https://arxiv.org/html/2610.11160#bib.bib14), or routing edited cases through external/contextual memories[Mitchell et al\. \(2022b\)](https://arxiv.org/html/2610.11160#bib.bib15);[Zheng et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib19)\. These methods establish the basic reliability, generalization, and locality criteria, but most are not designed for long edit streams where thousands of updates must coexist\. Lifelong editing methods address this sequential setting through key\-value memories, side memories, null\-space constraints, or adapter/MoE\-based mechanisms[Hartvigsen et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib4);[Wang et al\. \(2024a\)](https://arxiv.org/html/2610.11160#bib.bib5);[Fang et al\. \(2025\)](https://arxiv.org/html/2610.11160#bib.bib6);[Wang et al\. \(2025\)](https://arxiv.org/html/2610.11160#bib.bib7);[Yu et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib9);[Li et al\. \(2024a\)](https://arxiv.org/html/2610.11160#bib.bib8);[Wang and Li \(2024b\)](https://arxiv.org/html/2610.11160#bib.bib25);[Wang and Li \(2024a\)](https://arxiv.org/html/2610.11160#bib.bib24)\. However, they still face a storage\-plasticity tension: shared or constant\-memory systems can lose plasticity as capacity saturates, while edit\-local memories preserve behavior but grow with the number of edits\. LadderEdit targets this storage layer directly\.

#### Low\-rank adaptation and edit\-memory storage\.

LoRA and its variants show that low\-rank updates provide an efficient representation for adapting large models[Hu et al\. \(2022\)](https://arxiv.org/html/2610.11160#bib.bib1);[Dettmers et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib26);[Liu et al\. \(2024\)](https://arxiv.org/html/2610.11160#bib.bib29);[Lialin et al\. \(2024\)](https://arxiv.org/html/2610.11160#bib.bib31);[Yu et al\. \(2025\)](https://arxiv.org/html/2610.11160#bib.bib41)\. Adaptive\-rank methods such as AdaLoRA[Zhang et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib27)allocate rank across layers or modules during training, while serving systems such as S\-LoRA[Sheng et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib30)study how to deploy many adapters efficiently\. LadderEdit addresses a different problem: after an edit has already been acquired, how much of its update memory must be persistently stored? The unit of allocation is an edit rather than a layer, and the criterion is behavioral contract satisfaction rather than gradient\-derived importance\. Thus LadderEdit is not ordinary LoRA compression or training\-time rank allocation; it is an edit\-level resolution\-allocation method that keeps every edit covered by a sketch and promotes to higher rank only when needed\.

#### Evaluation under compression\.

Recent surveys and evaluations emphasize that edited models should be judged not only by rewrite success but also by generalization, locality, long\-form behavior, and degradation over many edits[Yao et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib21);[Zhang et al\. \(2024\)](https://arxiv.org/html/2610.11160#bib.bib22);[Wang et al\. \(2024b\)](https://arxiv.org/html/2610.11160#bib.bib23);[Rosati et al\. \(2024\)](https://arxiv.org/html/2610.11160#bib.bib32);[Li et al\. \(2024b\)](https://arxiv.org/html/2610.11160#bib.bib33);[Zheng et al\. \(2024\)](https://arxiv.org/html/2610.11160#bib.bib34);[Wu et al\. \(2024\)](https://arxiv.org/html/2610.11160#bib.bib35)\. LadderEdit follows this view at the storage level: spectral compressibility proposes candidate rungs, but the final decision is made by behavioral auditing over rewrite, generalization, and locality\. This is the central difference from exact\-cache allocation, which reduces memory by dropping edits rather than by allocating resolution\.

LadderEdit’s formulation differs from ordinary LoRA compression and adaptive rank training\. LadderEdit does not learn a better adapter at acquisition time and does not solve retrieval or task identification; it operates after the edit has been acquired, inheriting the selection mechanism of the full\-resolution memory bank, and asks a storage\-layer question: once an edit is selected, can the system return a cheaper representation without violating the measured edit contract? The advantage comes from changing the memory primitive\. Exact\-cache allocation saves memory by storing a subset of edits exactly and dropping the rest, reducing coverage\. LadderEdit first gives every edit coverage through a sketch, then allocates fidelity through additional rank only where the audit shows it is needed\. The two policies fail differently under compression: exact\-cache fails by absence, whereas LadderEdit degrades by resolution\. A focused comparison that distinguishes LadderEdit from training\-time rank allocation[Zhang et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib27), multi\-adapter serving[Sheng et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib30), and adapter merging is provided in Appendix[D](https://arxiv.org/html/2610.11160#A4)\.

## 3Method

### 3\.1Problem Setting

We consider lifelong editing for a pretrained LLMfθf\_\{\\theta\}, where edits arrive as a sequence\{ei\}i=1N\\\{e\_\{i\}\\\}\_\{i=1\}^\{N\}\. After each insertion, the system must answer the edited prompt correctly, generalize the edit to related prompts, and preserve unrelated knowledge\. We refer to these requirements as reliability, generalization, and locality\. The scaling challenge is not only to satisfy this behavioral contract for one edit, but to keep satisfying it asNNgrows\.

We instantiate each edit with a LoRA\-style update\. LetΔiE\\Delta\_\{i\}^\{E\}denote the exact update produced by the edit writer for editeie\_\{i\}\. Two extremes bracket the space of persistent representations: a full\-resolution edit\-local memory bank stores everyΔiE\\Delta\_\{i\}^\{E\}exactly, scaling linearly asN​PENP\_\{E\}wherePEP\_\{E\}is the cost of one exact edit; a shared or constant\-memory editor folds all edits into a fixed footprint, but plasticity degrades as later edits compete with earlier ones for finite capacity\. Neither extreme matches the empirical structure of an edit stream: acrossN=2,000N=2\{,\}000edits on both ZsRE and CounterFact, roughly80%80\\%of edits are behaviorally sketch\-sufficient at rank 1, a∼19%\\sim\\\!19\\%hard tail requires rank 2 or higher, and a smaller∼5%\\sim\\\!5\\%population is*compression\-regularized*, the rank\-1 sketch satisfies the behavioral contract that the exact update fails \(Appendix[A\.1](https://arxiv.org/html/2610.11160#A1.SS1)\)\. LadderEdit occupies the intermediate regime that this distribution implies: every edit retains its own representation, but that representation may be a compressed rung rather than the full exact adapter\. Table[4](https://arxiv.org/html/2610.11160#S4.T4)reports the population structure of the edit stream that motivates this design\.

The system rests on three commitments\. It is*exact first*: each edit is acquired at full resolution by the standard writer, so the acquired update is preserved without loss\. It is*retrieval agnostic*: it inherits whatever selection mechanism the deployment uses, with the storage gain preserved under a learned sentence encoder retriever \(Appendix[A](https://arxiv.org/html/2610.11160#A1)\)\. It is*contract grounded*: at insertion, a behavioral audit certifies that the returned representation satisfies the same rewrite, generalization, and locality test asΔiE\\Delta\_\{i\}^\{E\}\. At inference, retrieval selects the edit and LadderEdit returns the cheapest audited rung; the acquisition cost ofΔiE\\Delta\_\{i\}^\{E\}is paid once, and the persistent cost is whatever the audit permits\. LadderEdit decides which directions ofΔiE\\Delta\_\{i\}^\{E\}to retain and composes with quantization, which acts on bit width \(Appendix[A\.4](https://arxiv.org/html/2610.11160#A1.SS4)\)\.

### 3\.2Residual\-Ladder Allocation

LadderEdit is a residual\-ladder allocation mechanism for scaling edit\-local update memories in lifelong editing\. It keeps the isolation and plasticity of per\-edit adapters, but stores most edits at lower resolution\. The method has three steps: acquire an exact update, compile that update into candidate rungs, and audit the cheapest safe rung\. Figure[2](https://arxiv.org/html/2610.11160#S1.F2)summarizes the full insertion\-time pipeline, and Algorithm[1](https://arxiv.org/html/2610.11160#alg1)gives the controller\.

For an incoming editeie\_\{i\}, letΔiE\\Delta\_\{i\}^\{E\}denote the exact LoRA\-style update acquired by the edit writer\. LadderEdit constructs a rank\-rrsketch:

Δi\(r\)=Πr​\(ΔiE\),\\Delta\_\{i\}^\{\(r\)\}=\\Pi\_\{r\}\(\\Delta\_\{i\}^\{E\}\),\(1\)We use the rank menuℛ=\{1,2,…,Rmax\}\\mathcal\{R\}=\\\{1,2,\\ldots,R\_\{\\max\}\\\}, matching the same acquisition rank asRmaxR\_\{\\max\}from the exact LoRA update\.

whereΠr\\Pi\_\{r\}truncates the singular directions of the effective LoRA update\. The omitted detail is the residual:

ρi\(r\)=ΔiE−Δi\(r\)\.\\rho\_\{i\}^\{\(r\)\}=\\Delta\_\{i\}^\{E\}\-\\Delta\_\{i\}^\{\(r\)\}\.\(2\)The deployed edit is one rung of the ladder:

Δi=Δi\(ri\)\.\\Delta\_\{i\}=\\Delta\_\{i\}^\{\(r\_\{i\}\)\}\.\(3\)Whenri=Rmaxr\_\{i\}=R\_\{\\max\}, the stored update equals the exact LoRA edit\. The key difference from exact\-cache allocation is that every edit keeps a representation, and memory is spent on resolution rather than on a binary keep/drop decision\.

Rank selection starts from a cheap spectral proposal\. LetTi​\(r\)T\_\{i\}\(r\)be the normalized spectral tail after rankrr\. We define the exact\-edit margin as the minimum behavioral slack of the full\-resolution edit:

μiE=min⁡\{RiE−τR,GiE−τG,LiE−τL\}\.\\mu\_\{i\}^\{E\}=\\min\\\{R\_\{i\}^\{E\}\-\\tau\_\{R\},\\;G\_\{i\}^\{E\}\-\\tau\_\{G\},\\;L\_\{i\}^\{E\}\-\\tau\_\{L\}\\\}\.\(4\)Positive margin means the acquired edit satisfies all three measured constraints, and larger margin indicates more tolerance for compression\.

LadderEdit scores:

Si​\(r\)=Ti​\(r\)\[μiE\]\+\+ϵ,S\_\{i\}\(r\)=\\frac\{T\_\{i\}\(r\)\}\{\[\\mu\_\{i\}^\{E\}\]\_\{\+\}\+\\epsilon\},\(5\)whereϵ\>0\\epsilon\>0prevents unstable division\. Lower scores indicate that little spectral mass remains outside the sketch relative to the exact edit’s behavioral slack\. The cheapest rank whose score falls below a calibrated thresholdτ\\tauis the predicted rankr^i\\hat\{r\}\_\{i\}:

r^i=min⁡\{r∈ℛ:Si​\(r\)≤τ\}\.\\hat\{r\}\_\{i\}=\\min\\\{r\\in\\mathcal\{R\}:S\_\{i\}\(r\)\\leq\\tau\\\}\.\(6\)The spectral score is a proposal, not a safety proof; it selects*which single rung*the audit will inspect first, so the audit sees one candidate per edit instead of\|ℛ\|\|\\mathcal\{R\}\|candidates\.

Behavioral auditing remains the final gate\. For a candidate stored representationmm, letRimR\_\{i\}^\{m\},GimG\_\{i\}^\{m\}, andLimL\_\{i\}^\{m\}denote rewrite, generalization, and locality scores\. We define the soft contract gap:

Him=\\displaystyle H\_\{i\}^\{m\}=\{\}wR​\[τR−Rim\]\+\+wG​\[τG−Gim\]\+\\displaystyle w\_\{R\}\[\\tau\_\{R\}\-R\_\{i\}^\{m\}\]\_\{\+\}\+w\_\{G\}\[\\tau\_\{G\}\-G\_\{i\}^\{m\}\]\_\{\+\}\(7\)\+wL​\[τL−Lim\]\+\.\\displaystyle\+w\_\{L\}\[\\tau\_\{L\}\-L\_\{i\}^\{m\}\]\_\{\+\}\.The audit score in Eq\.[7](https://arxiv.org/html/2610.11160#S3.E7)uses weightswR=1\.0w\_\{R\}=1\.0,wG=0\.8w\_\{G\}=0\.8, andwL=1\.2w\_\{L\}=1\.2\. Locality receives the highest weight because locality failures are catastrophic and irreversible, while reliability and generalization failures trigger rank promotion and recover within the same edit\. We sweptwLw\_\{L\}over\{0\.8,1\.0,1\.2,1\.5,2\.0\}\\\{0\.8,1\.0,1\.2,1\.5,2\.0\\\}on the validation split and Avg\. stayed within 0\.01 of the default forwL∈\[1\.0,1\.5\]w\_\{L\}\\in\[1\.0,1\.5\]and dropped by 0\.02 and 0\.03 at the endpoints 0\.8 and 2\.0 respectively\. The defaultwL=1\.2w\_\{L\}=1\.2sits at the midpoint of the stable region\. The weightswRw\_\{R\}andwGw\_\{G\}follow the standard LoRA loss\-weighting ratio\. We demonstrate more details in Appendix[A\.3](https://arxiv.org/html/2610.11160#A1.SS3)\.

#### Downgrade policy\.

Algorithm[1](https://arxiv.org/html/2610.11160#alg1)includes budget\-based downgrading\. When the audit succeeds at rankrr, the controller checks whether the rank\-\(r−1\)\(r\{\-\}1\)approximationU:,1:r−1Σ1:r−1,1:r−1V1:r−1,:TU\_\{:,1:r\-1\}\\Sigma\_\{1:r\-1,1:r\-1\}V\_\{1:r\-1,:\}^\{T\}captures more than 92% of the Frobenius energy of the full LoRA update\. If so, the controller commits the rank\-\(r−1\)\(r\{\-\}1\)sketch and re\-runs the audit on the downgraded sketch, and if the downgrade also passes the audit, it is kept\. Downgrade attempts are bounded at one per edit to keep insertion cost bounded\. Downgrades occur for 3\.1% of edits atT=1,000T=1\{,\}000on LLaMA\-3\-8B / ZsRE and save approximately 0\.6 GB of persistent memory atT=10,000T=10\{,\}000\. The 0\.92 Frobenius threshold was selected on the validation split, and varying it from 0\.88 to 0\.95 changes Avg\. by less than 0\.005\.

IfHim=0H\_\{i\}^\{m\}=0, the candidate satisfies the edit contract\. If the audit atr^i\\hat\{r\}\_\{i\}fails \(Hi\(r^i\)\>0H\_\{i\}^\{\(\\hat\{r\}\_\{i\}\)\}\>0\), LadderEdit re\-audits at the next rung along the spectrally\-ordered ladderr^i\+1,r^i\+2,…\\hat\{r\}\_\{i\}\+1,\\hat\{r\}\_\{i\}\+2,\\ldotsuntil the contract is met orRmaxR\_\{\\max\}is reached\. In practice the predicted rung passes on the first audit for the majority of edits \(Appendix[A\.5](https://arxiv.org/html/2610.11160#A1.SS5)\), so the expected number of audits per edit is close to one\.

Under a memory budget, higher\-rank promotion is allocated where it removes the most contract violation per parameter\. LetPEP\_\{E\}be the exact edit cost andPrP\_\{r\}the rank\-rrsketch cost\. The promotion value density is:

RVDi​\(r\)=Hi\(r\)−HiEPE−Pr\.\\mathrm\{RVD\}\_\{i\}\(r\)=\\frac\{H\_\{i\}^\{\(r\)\}\-H\_\{i\}^\{E\}\}\{P\_\{E\}\-P\_\{r\}\}\.\(8\)If allNNedits use rankrrandKKedits are promoted to the exact rung, persistent memory is:

M⁡\(r,K\)=N​Pr\+K⁡\(PE−Pr\)\.M\(r,K\)=NP\_\{r\}\+K\(P\_\{E\}\-P\_\{r\}\)\.\(9\)For analytical clarity, Eq\.[8](https://arxiv.org/html/2610.11160#S3.E8)–[9](https://arxiv.org/html/2610.11160#S3.E9)use a two\-state limit in which promoted edits jump directly toRmaxR\_\{\\max\}\. The deployed controller supports intermediate promotion along the full ladderℛ\\mathcal\{R\}\. The first term gives every edit a cheap plastic representation\. The second term spends exact memory only on the difficult subset\. This yields a practical route for one\-LoRA\-per\-edit scaling wheneverK≪NK\\ll N\.

Algorithm 1LadderEdit: spectral\-prediction with audited promotion1:Edit stream

\{ei\}i=1N\\\{e\_\{i\}\\\}\_\{i=1\}^\{N\}, frozen model

fθf\_\{\\theta\}, rank menu

ℛ=\{1,…,Rmax\}\\mathcal\{R\}=\\\{1,\\ldots,R\_\{\\max\}\\\}, threshold

τ\\tau, memory budget

BB
2:Initialize stored memory

ℳ←∅\\mathcal\{M\}\\leftarrow\\emptyset
3:foreach incoming edit

eie\_\{i\}do

4:Acquire exact LoRA update

ΔiE\\Delta\_\{i\}^\{E\}
5:Compute SVD of

ΔiE\\Delta\_\{i\}^\{E\}and form spectral tails

Ti​\(r\)T\_\{i\}\(r\)for all

r∈ℛr\\in\\mathcal\{R\}
6:Estimate exact\-edit margin

μiE\\mu\_\{i\}^\{E\}on rewrite, generalization, and locality probes

7:Predict:

r^i←min⁡\{r∈ℛ:Si​\(r\)≤τ\}\\hat\{r\}\_\{i\}\\leftarrow\\min\\\{r\\in\\mathcal\{R\}:S\_\{i\}\(r\)\\leq\\tau\\\}⊳\\trianglerightsingle proposal, no behavioral cost

8:

ri←r^ir\_\{i\}\\leftarrow\\hat\{r\}\_\{i\}
9:Audit:build sketch

Δi\(ri\)\\Delta\_\{i\}^\{\(r\_\{i\}\)\}, compute contract gap

Hi\(ri\)H\_\{i\}^\{\(r\_\{i\}\)\}
10:while

Hi\(ri\)\>0H\_\{i\}^\{\(r\_\{i\}\)\}\>0and

ri<Rmaxr\_\{i\}<R\_\{\\max\}do⊳\\trianglerightaudited promotion

11:

ri←ri\+1r\_\{i\}\\leftarrow r\_\{i\}\+1
12:Rebuild sketch

Δi\(ri\)\\Delta\_\{i\}^\{\(r\_\{i\}\)\}, recompute

Hi\(ri\)H\_\{i\}^\{\(r\_\{i\}\)\}
13:endwhile

14:Store

Δi\(ri\)\\Delta\_\{i\}^\{\(r\_\{i\}\)\}in

ℳ\\mathcal\{M\}
15:ifstored memory exceeds

BBthen

16:Downgrade edits with lowest value density

17:endif

18:endfor

19:returnstored edit memory

ℳ\\mathcal\{M\}

The rank\-analysis package also clarifies how to interpret storage rungs\. An edit is*sketch\-sufficient*if both sketch and exact pass,*detail\-needed*if the sketch fails but exact passes,*compression\-regularized*if the sketch passes but exact fails, and*exact\-limited*if both fail\. More details in Appendix[D\.6](https://arxiv.org/html/2610.11160#A4.SS6)\.

## 4Experiments

### 4\.1Experimental Setup

We evaluate LadderEdit as a storage\-layer method for edit\-local memories\. The central question is whether lifelong edit storage should be allocated by*coverage*and*fidelity*, rather than by exact keep/drop caching\. We therefore compare exact LoRA as the full\-resolution reference, fixed\-rank compression as a nonadaptive baseline, exact\-cache as a keep/drop policy, and LadderEdit as an edit\-level resolution\-allocation policy across standard sequential editing benchmarks\.

Table 1:Q&A task results on the ZsRE dataset\.TTdenotes the number of edits\. Mean±\\pmstandard deviation over three random seeds\. Avg\. is the arithmetic mean of Rel\., Gen\., and Loc\. The best average performance per column is marked inbold\.Table 2:Sequential editing results on CF\.TTdenotes the number of edits\. Mean±\\pmstandard deviation over three random seeds\. The best performance per column is marked inbold\.MethodLLaMA\-3\-8BMistral\-7BT=100T=100T=500T=500T=1,000T=1\{,\}000T=2,000T=2\{,\}000T=100T=100T=500T=500T=1,000T=1\{,\}000T=2,000T=2\{,\}000Rel\.Loc\.Rel\.Loc\.Rel\.Loc\.Rel\.Loc\.Rel\.Loc\.Rel\.Loc\.Rel\.Loc\.Rel\.Loc\.FT0\.42±\\pm\.0040\.55±\\pm\.0010\.25±\\pm\.0050\.30±\\pm\.0010\.12±\\pm\.0050\.12±\\pm\.0010\.05±\\pm\.0020\.03±\\pm\.0020\.39±\\pm\.0050\.52±\\pm\.0010\.22±\\pm\.0060\.27±\\pm\.0010\.10±\\pm\.0060\.10±\\pm\.0020\.03±\\pm\.0020\.02±\\pm\.001ROME[Meng et al\. \(2022\)](https://arxiv.org/html/2610.11160#bib.bib2)0\.28±\\pm\.0030\.45±\\pm\.0020\.08±\\pm\.0040\.15±\\pm\.0020\.02±\\pm\.0010\.05±\\pm\.0020\.00±\\pm\.0000\.02±\\pm\.0010\.25±\\pm\.0030\.42±\\pm\.0020\.06±\\pm\.0040\.12±\\pm\.0030\.01±\\pm\.0010\.04±\\pm\.0020\.00±\\pm\.0000\.01±\\pm\.001MEMIT[Meng et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib3)0\.35±\\pm\.0020\.52±\\pm\.0010\.12±\\pm\.0020\.20±\\pm\.0010\.03±\\pm\.0020\.08±\\pm\.0010\.01±\\pm\.0010\.03±\\pm\.0020\.31±\\pm\.0020\.48±\\pm\.0010\.09±\\pm\.0030\.16±\\pm\.0010\.02±\\pm\.0010\.06±\\pm\.0020\.00±\\pm\.0000\.02±\\pm\.001GRACE[Hartvigsen et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib4)0\.75±\\pm\.0010\.96±\\pm\.0010\.71±\\pm\.0010\.96±\\pm\.0010\.68±\\pm\.0010\.96±\\pm\.0010\.62±\\pm\.0010\.95±\\pm\.0010\.72±\\pm\.0010\.95±\\pm\.0010\.68±\\pm\.0010\.95±\\pm\.0010\.65±\\pm\.0010\.95±\\pm\.0010\.59±\\pm\.0010\.94±\\pm\.001WISE[Wang et al\. \(2024a\)](https://arxiv.org/html/2610.11160#bib.bib5)0\.68±\\pm\.0140\.88±\\pm\.0020\.62±\\pm\.0170\.86±\\pm\.0020\.55±\\pm\.0190\.83±\\pm\.0030\.45±\\pm\.0210\.77±\\pm\.0030\.65±\\pm\.0160\.86±\\pm\.0020\.58±\\pm\.0190\.83±\\pm\.0030\.50±\\pm\.0220\.80±\\pm\.0030\.40±\\pm\.0240\.73±\\pm\.003AlphaEdit[Fang et al\. \(2025\)](https://arxiv.org/html/2610.11160#bib.bib6)0\.62±\\pm\.0120\.85±\\pm\.0080\.58±\\pm\.0140\.70±\\pm\.0100\.52±\\pm\.0160\.48±\\pm\.0110\.42±\\pm\.0180\.22±\\pm\.0120\.58±\\pm\.0140\.82±\\pm\.0090\.54±\\pm\.0170\.66±\\pm\.0110\.47±\\pm\.0190\.43±\\pm\.0120\.37±\\pm\.0210\.18±\\pm\.014MEMOIR[Wang et al\. \(2025\)](https://arxiv.org/html/2610.11160#bib.bib7)0\.78±\\pm\.0080\.88±\\pm\.0060\.75±\\pm\.0100\.94±\\pm\.0070\.72±\\pm\.0110\.93±\\pm\.0080\.66±\\pm\.0120\.90±\\pm\.0090\.76±\\pm\.0090\.86±\\pm\.0070\.72±\\pm\.0110\.93±\\pm\.0080\.69±\\pm\.0120\.91±\\pm\.0090\.63±\\pm\.0140\.88±\\pm\.010ELDER[Li et al\. \(2024a\)](https://arxiv.org/html/2610.11160#bib.bib8)0\.52±\\pm\.0140\.82±\\pm\.0090\.46±\\pm\.0170\.78±\\pm\.0110\.39±\\pm\.0190\.73±\\pm\.0120\.30±\\pm\.0210\.66±\\pm\.0130\.49±\\pm\.0160\.80±\\pm\.0100\.43±\\pm\.0190\.76±\\pm\.0120\.36±\\pm\.0220\.71±\\pm\.0140\.27±\\pm\.0240\.64±\\pm\.016MELO[Yu et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib9)0\.56±\\pm\.0120\.86±\\pm\.0080\.50±\\pm\.0140\.82±\\pm\.0100\.43±\\pm\.0160\.78±\\pm\.0110\.34±\\pm\.0180\.71±\\pm\.0120\.53±\\pm\.0140\.84±\\pm\.0090\.47±\\pm\.0170\.80±\\pm\.0110\.40±\\pm\.0190\.76±\\pm\.0120\.31±\\pm\.0210\.69±\\pm\.014AdaLoRA[Zhang et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib27)0\.18±\\pm\.0200\.32±\\pm\.0100\.02±\\pm\.0010\.02±\\pm\.0010\.00±\\pm\.0000\.01±\\pm\.0010\.00±\\pm\.0000\.00±\\pm\.0000\.15±\\pm\.0230\.30±\\pm\.0110\.01±\\pm\.0010\.02±\\pm\.0010\.00±\\pm\.0000\.01±\\pm\.0010\.00±\\pm\.0000\.00±\\pm\.000LoRA \(Exact\)[Hu et al\. \(2022\)](https://arxiv.org/html/2610.11160#bib.bib1)0\.82±\\pm\.0050\.85±\\pm\.0020\.78±\\pm\.0060\.96±\\pm\.0020\.75±\\pm\.0070\.95±\\pm\.0030\.71±\\pm\.0070\.93±\\pm\.0030\.80±\\pm\.0060\.83±\\pm\.0020\.76±\\pm\.0070\.95±\\pm\.0030\.72±\\pm\.0080\.93±\\pm\.0030\.68±\\pm\.0090\.91±\\pm\.003LadderEdit \(Ours\)0\.83±\\pm\.0040\.90±\\pm\.0020\.78±\\pm\.0050\.96±\\pm\.0020\.77±\\pm\.0050\.96±\\pm\.0030\.74±\\pm\.0060\.94±\\pm\.0030\.81±\\pm\.0050\.88±\\pm\.0020\.77±\\pm\.0060\.95±\\pm\.0030\.74±\\pm\.0060\.94±\\pm\.0030\.71±\\pm\.0070\.92±\\pm\.003

#### Datasets\.

We conduct experiments on three benchmarks: ZsRE[Levy et al\. \(2017\)](https://arxiv.org/html/2610.11160#bib.bib10), CounterFact \(CF\)[Meng et al\. \(2022\)](https://arxiv.org/html/2610.11160#bib.bib2);[Meng et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib3), and WikiBigEdit[Thede et al\. \(2025\)](https://arxiv.org/html/2610.11160#bib.bib36)\. ZsRE is a question\-answering dataset for evaluating factual knowledge and generalization in language models\. CF tests the ability to edit and preserve specific factual associations in a controlled setting\. WikiBigEdit is a large\-scale benchmark for long\-horizon, up to 50,000 edits, sequential editing\.

We evaluate LadderEdit on LLaMA\-3\-8B[Grattafiori et al\. \(2024\)](https://arxiv.org/html/2610.11160#bib.bib11), Mistral\-7B[Jiang et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib12), and Qwen2\.5\-7B[Yang et al\. \(2025\)](https://arxiv.org/html/2610.11160#bib.bib13)\. We compare against representative direct\-editing methods, memory\- and routing\-based lifelong editors, constrained\-update methods, LoRA/MoE\-style edit\-memory baselines, adaptive\-rank LoRA, and the full\-resolution exact LoRA reference\. Detailed baseline descriptions are deferred to Appendix[F\.3](https://arxiv.org/html/2610.11160#A6.SS3)\.

#### Edit lookup protocol\.

LadderEdit evaluates storage for edit\-local memories, not edit selection\. For edit\-local methods, each rewrite, generalization, and locality probe is associated with the edit that generated it\. Exact LoRA and LadderEdit use the same association, so they differ only in the stored representation returned after lookup\. Methods whose routing is part of the original algorithm, such as memory\- or MoE\-based lifelong editors, are evaluated with their native routing mechanisms\.

We report reliability, generalization, locality, and their average when applicable\. Locality is the fraction of unrelated probes whose prediction remains unchanged relative to the frozen base model\. 1\.00 means no failures on the finite locality probe set\.

#### Probe split protocol\.

For each edit, the benchmark\-provided prompts are partitioned into three disjoint sets at edit\-construction time\. The*audit set*contains the rewrite prompt and a held\-out subset of rephrase/generalization variants, used only at insertion time to scoreSi​\(r\)S\_\{i\}\(r\)and gate the contract gapHi\(r\)H\_\{i\}^\{\(r\)\}\. The*validation set*is a separate stream used only to select audit thresholds, which are then frozen before the test stream is touched\. The*test set*contains the remaining rephrase/generalization variants and unrelated locality probes; it is used only to report Reliability, Generalization, and Locality in the main tables and is never seen by the audit\. Audit and test sets share neither prompts nor paraphrases of the same prompt\. Full split statistics, audit\-test correlation, and a leaky\-versus\-clean comparison are reported in Appendix[C](https://arxiv.org/html/2610.11160#A3)\. More details in Appendix[B\.1](https://arxiv.org/html/2610.11160#A2.SS1)\.

### 4\.2Results

#### Main Results\.

Tables[1](https://arxiv.org/html/2610.11160#S4.T1)and[2](https://arxiv.org/html/2610.11160#S4.T2)show that LadderEdit preserves the plasticity of edit\-local adapters while reducing their persistent storage\. On ZsRE, FT, ROME, and MEMIT degrade quickly, and GRACE preserves locality but fails to generalize, especially on Mistral\-7B and Qwen2\.5\-7B\. Exact LoRA remains the strongest full\-storage reference because each edit has isolated capacity, but LadderEdit tracks it after compression: on LLaMA\-3\-8B it improves the average over exact LoRA atT=500T=500\(0\.97 vs\. 0\.93\) and matches it atT=1,000T=1\{,\}000andT=2,000T=2\{,\}000; on Mistral\-7B it slightly improves reliability or generalization at longer horizons; on Qwen2\.5\-7B it stays within 0\.01 to 0\.02 average of exact LoRA at the largest horizons\. CounterFact gives the same message in a setting that emphasizes locality: atT=2,000T=2\{,\}000, LadderEdit moves exact LoRA from 0\.71/0\.93 to 0\.74/0\.94 in reliability/locality on LLaMA\-3\-8B and from 0\.68/0\.91 to 0\.71/0\.92 on Mistral\-7B\. These gains suggest that compression is not merely lossy: low\-rank sketches can remove overly specific directions, while the audit promotes edits to higher rank when more fidelity is needed\.

#### WikiBigEdit Long\-Horizon Scaling\.

AtT=50,000T=50\{,\}000on WikiBigEdit \(Table[3](https://arxiv.org/html/2610.11160#S4.T3)\), LadderEdit reaches 0\.75 reliability, 0\.84 generalization, 0\.98 locality, and 0\.86 average\. GRACE holds reliability and locality but drops generalization to 0\.28; WISE loses both reliability and generalization; MEMOIR reaches only 0\.69 average\. Together with the memory trade\-off in Figure[3](https://arxiv.org/html/2610.11160#S4.F3), LadderEdit preserves edit\-local behavior at this scale while substantially reducing persistent storage\.

Table 3:Editing performance on Qwen2\.5\-7B at long\-horizon scale on WikiBigEdit\. Mean±\\pmstandard deviation over three random seeds\.Figure 3:Scaling reference on WikiBigEdit: average performance against persistent edit memory\.
#### Memory Efficiency and Ablation\.

The memory results support LadderEdit’s central claim: lifelong edit storage should be treated as per\-edit resolution allocation\. Table[4](https://arxiv.org/html/2610.11160#S4.T4)reports the empirical population structure of the edit stream and confirms the heterogeneity assumed in Section[3\.1](https://arxiv.org/html/2610.11160#S3.SS1)\. On both ZsRE and CounterFact, roughly 80% of edits are sketch\-sufficient at rank 1, a 13 to 17% tail requires rank 2 or higher to satisfy the behavioral contract, and a smaller 3 to 5% subset requires the full acquisition rank\. A 4 to 7% compression\-regularized population, where the rank\-1 sketch passes the contract that exact LoRA fails, shows that the full update is not a behavioral ceiling\. This distribution justifies the ladder design: a single fixed rank cannot serve all edits, but a small higher\-rank budget plus a cheap default rung suffices for the bulk of the stream\. Per\-rank compression statistics \(memory, retained energy, false retirements\) are reported in Appendix[F](https://arxiv.org/html/2610.11160#A6)\. A predictor ablation isolating the spectral score, the behavioral margin, and their product against random and oracle baselines is reported in Appendix[E](https://arxiv.org/html/2610.11160#A5); the combined predictor approaches oracle utility at far fewer audits per edit than any component alone\.

Table[5](https://arxiv.org/html/2610.11160#S4.T5)tests the memory primitive directly\. The exact\-cache baseline is validation\-selected: under a budget ofKKfull edits, it keeps those with the largest sketch\-only contract gap and drops the rest\. At5\.57×5\.57\\times\(CF\) and5\.63×5\.63\\times\(ZsRE\) compression, it retains about 91 of 512 edits, reaching utility 0\.434 on CF and 0\.420 on ZsRE; dropped edits have no representation\. LadderEdit covers every edit with a rank\-1 sketch: at8×8\\timescompression, the sketch alone reaches 0\.753 \(CF\) and 0\.765 \(ZsRE\), exceeding exact\-cache by roughly 0\.33 at lower memory, so coverage drives most of the gap\. Behavioral rung selection adds the rest\. At the matched budget, a static mixed\-rank policy reaches 0\.815 \(CF\) and 0\.796 \(ZsRE\); the dual\-budget ladder, spending that budget selectively on audit\-flagged edits, reaches 0\.862 \(CF\) and 0\.866 \(ZsRE\)\. The 0\.05 gap between static and dual\-budget at identical memory is the signature of audit\-driven allocation\. Exact\-cache fails by absence; LadderEdit degrades by resolution\.

Finally, Table[6](https://arxiv.org/html/2610.11160#S4.T6)shows that this frontier translates into deployed storage savings\. LadderEdit is exact\-first, so it does not reduce the initial LoRA acquisition cost; its benefit is retained edit memory after acquisition\. AtT=10,000T=10\{,\}000on LLaMA\-3\-8B, the audit increases per\-edit insertion time relative to exact LoRA, but persistent memory drops from 100\.93 GB to 19\.44 GB, a5\.19×5\.19\\timesreduction\. Because exact LoRA and LadderEdit use the same edit\-selection assumption, this is a storage\-layer gain: the lookup mechanism is unchanged, and LadderEdit changes only the representation returned after lookup\. Additional qualitative examples in Appendix[D\.6](https://arxiv.org/html/2610.11160#A4.SS6)and stress tests in Appendix[D\.7](https://arxiv.org/html/2610.11160#A4.SS7)show the same mechanism at the prompt and robustness levels: the edit stream is heterogeneous, many edits are sketch\-sufficient, and a smaller subset needs higher rank to satisfy the measured contract\.

Table 4:Population structure of edit streams atN=2,000N=2\{,\}000\. Categories are determined by the behavioral audit on audit\-split probes\. Most edits are over\-resolved by the full LoRA update\.Table 5:Budget\-matched strict\-memory frontier atN=512N=512\. LadderEdit preserves coverage and allocates residual fidelity through behavioral rung selection\.\(a\)ZsRE\(b\)CF
Figure 4:Spectral\-filter frontier analysis on ZsRE and CF atN=512N=512edits\. The plots show the tradeoff between memory compression and behavioral contract utility for different rank tiers and allocation strategies\.Table 6:Cost decomposition atT=10,000T=10\{,\}000on LLaMA\-3\-8B\. LadderEdit reduces this storage by5\.19×5\.19\\timesrelative to exact LoRA\. Serving path summarizes the additional edit mechanism used at inference\.

## 5Conclusion

We introduced LadderEdit, a residual\-ladder storage controller for lifelong editing\. LadderEdit first uses a spectral predictor to select a candidate rank for each acquired edit, and a behavioral audit verifies the proposal and promotes the edit along the ladder only when the performance of that edit demands it\. Across ZsRE, CF, and WikiBigEdit on LLaMA\-3\-8B, Mistral\-7B, and Qwen2\.5\-7B, LadderEdit tracks exact LoRA while reducing persistent memory by5\.2×5\.2\\timesand remaining effective at 50,000 edits\. These results show that most exact adapters are over\-resolved: low\-rank sketches preserve the edit contract for the bulk of the stream, while higher resolution is needed only for a smaller hard edits subset\. The key question in lifelong edit storage is not only*which*edits to keep, but also*how much*of each edit should be retained\.

## 6Limitations

LadderEdit is not constant\-memory: every edit retains at least a rank\-1 sketch, so persistent storage grows linearly with the stream, and future work could explore sketch downgrading or merging to bound the total\. The behavioral audit adds insertion\-time latency \(124 ms versus 86 ms atT=10,000T\{=\}10\{,\}000on LLaMA\-3\-8B\), which could be reduced by batching audits or running them asynchronously\. The audit also depends on probe coverage and does not guarantee correctness outside the probed set, motivating richer probe construction or learned probe selection\. LadderEdit is a storage\-layer method and inherits the retrieval mechanism used by the full\-resolution memory bank; integration with broader production retrieval stacks remains open\. The spectral score that proposes candidate ranks is a calibrated heuristic, and replacing it with a predictor trained on audit outcomes is a natural extension\. Finally, our experiments span three benchmarks and three 7B and 8B backbones, with long\-horizon scaling shown to50,00050\{,\}000edits; validating LadderEdit at larger scales and with non\-LoRA edit writers remains future work\.

#### Potential Risks\.

LadderEdit reduces the persistent cost of injecting and retaining edits in a large language model, which could lower the operational barrier to large\-scale knowledge modification\. We see three concrete risks\. First, the same machinery that enables efficient correction of outdated or harmful facts can be used to insert false or adversarial edits at scale, since the audit verifies behavioral contract satisfaction rather than factual correctness of the target\. Mitigations include provenance tracking on the edit stream, signed edit acquisition, and downstream factuality checking, none of which are part of the method itself\. Second, the behavioral audit relies on a probe bank whose coverage is finite; edits whose effects fall outside the probed semantic neighborhood may pass the audit while exhibiting unintended behavior on unprobed prompts\. This is a soft\-coverage rather than a hard\-correctness guarantee, and we emphasize in the appendix that the audit should not be treated as a safety certificate\. Third, the residual\-ladder representation makes individual edits less inspectable than uncompressed adapters: a low\-rank sketch can be behaviorally equivalent to an exact LoRA update on the probed contract while retaining different parameter directions\. In safety\-critical deployments, this argues for keeping edit\-level audit logs and periodic reaudits against the exact update rather than relying on stored representations alone\. We do not release any new model weights or edit corpora and rely on public benchmarks; the method’s broader societal effect depends on the editing workflow it is deployed in rather than on the storage primitive itself\.

## 7Ethics Considerations

Model editing can be used to correct errors and update knowledge, but it can also be misused to insert false or harmful facts\. Edits require provenance, authorization, validation, and auditing to ensure responsible deployment\. LadderEdit is a memory allocation mechanism for model editing; it does not serve as a truth or safety filter\.

## Acknowledgment

This study was supported in part by NIH R21EB034911, Gemini Academic Program Award, and NVIDIA Academic Grant Program\.

## References

- S\. Bai, Y\. Cai, R\. Chen, K\. Chen, X\. Chen, Z\. Cheng, L\. Deng, W\. Ding, C\. Gao, C\. Ge,et al\.Qwen3\-vl technical report\.arXiv preprint arXiv:2511\.21631\.Cited by:[§G\.1](https://arxiv.org/html/2610.11160#A7.SS1.p1.1),[§1](https://arxiv.org/html/2610.11160#S1.p6.1)\.
- Daiet al\.\(2022\)D\. Dai, L\. Dong, Y\. Hao, Z\. Sui, B\. Chang, and F\. WeiKnowledge neurons in pretrained transformers\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics,Cited by:[§1](https://arxiv.org/html/2610.11160#S1.p2.1)\.
- De Caoet al\.\(2021\)N\. De Cao, W\. Aziz, and I\. TitovEditing factual knowledge in language models\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,Cited by:[§1](https://arxiv.org/html/2610.11160#S1.p2.1),[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px1.p1.1)\.
- Dettmerset al\.\(2023\)T\. Dettmers, A\. Pagnoni, A\. Holtzman, and L\. ZettlemoyerQLoRA: efficient finetuning of quantized LLMs\.InAdvances in Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px2.p1.1)\.
- Fanget al\.\(2025\)J\. Fang, H\. Jiang, K\. Wang, Y\. Ma, J\. Shi, X\. Wang, X\. He, and T\. ChuaAlphaEdit: null\-space constrained knowledge editing for language models\.InInternational Conference on Learning Representations,Cited by:[§F\.3](https://arxiv.org/html/2610.11160#A6.SS3.SSS0.Px3.p1.1),[§1](https://arxiv.org/html/2610.11160#S1.p2.1),[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px1.p1.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.22.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.35.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.9.1),[Table 2](https://arxiv.org/html/2610.11160#S4.T2.4.1.9.1),[Table 3](https://arxiv.org/html/2610.11160#S4.T3.2.1.6.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§1](https://arxiv.org/html/2610.11160#S1.p6.1),[§4\.1](https://arxiv.org/html/2610.11160#S4.SS1.SSS0.Px1.p2.1)\.
- Hartvigsenet al\.\(2023\)T\. Hartvigsen, S\. Sankaranarayanan, H\. Palangi, Y\. Kim, and M\. GhassemiAging with GRACE: lifelong model editing with discrete key\-value adaptors\.InAdvances in Neural Information Processing Systems,Cited by:[§F\.3](https://arxiv.org/html/2610.11160#A6.SS3.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.11160#S1.p2.1),[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px1.p1.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.20.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.33.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.7.1),[Table 2](https://arxiv.org/html/2610.11160#S4.T2.4.1.7.1),[Table 3](https://arxiv.org/html/2610.11160#S4.T3.2.1.4.1)\.
- Huet al\.\(2022\)E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, W\. Chen,et al\.LoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations,Cited by:[§F\.3](https://arxiv.org/html/2610.11160#A6.SS3.SSS0.Px6.p1.1),[§1](https://arxiv.org/html/2610.11160#S1.p3.1),[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px2.p1.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.14.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.27.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.40.1),[Table 2](https://arxiv.org/html/2610.11160#S4.T2.4.1.14.1),[Table 6](https://arxiv.org/html/2610.11160#S4.T6.2.1.3.1)\.
- Izacardet al\.\(2021\)G\. Izacard, M\. Caron, L\. Hosseini, S\. Riedel, P\. Bojanowski, A\. Joulin, and E\. GraveUnsupervised dense information retrieval with contrastive learning\.arXiv preprint arXiv:2112\.09118\.Cited by:[Table 26](https://arxiv.org/html/2610.11160#A7.T26.2.3.1.1.1)\.
- Jianget al\.\(2023\)A\. Q\. Jiang, A\. Sablayrolles, A\. Mensch, C\. Bamford, D\. S\. Chaplot, D\. de Las Casas, F\. Bressand, G\. Lengyel, G\. Lample, L\. Saulnier,et al\.Mistral 7b\.arXiv preprint arXiv:2310\.06825\.Cited by:[§1](https://arxiv.org/html/2610.11160#S1.p6.1),[§4\.1](https://arxiv.org/html/2610.11160#S4.SS1.SSS0.Px1.p2.1)\.
- Levyet al\.\(2017\)O\. Levy, M\. Seo, E\. Choi, and L\. ZettlemoyerZero\-shot relation extraction via reading comprehension\.InProceedings of the 21st Conference on Computational Natural Language Learning \(CoNLL 2017\),pp\. 333–342\.Cited by:[§1](https://arxiv.org/html/2610.11160#S1.p6.1),[§4\.1](https://arxiv.org/html/2610.11160#S4.SS1.SSS0.Px1.p1.1)\.
- Liet al\.\(2024a\)J\. Li, Q\. Wang, Z\. Wang, Y\. Zhang, and Z\. MaoELDER: enhancing lifelong model editing with mixture\-of\-LoRA\.arXiv preprint arXiv:2408\.11869\.Cited by:[§F\.3](https://arxiv.org/html/2610.11160#A6.SS3.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.11160#S1.p2.1),[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px1.p1.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.11.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.24.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.37.1),[Table 2](https://arxiv.org/html/2610.11160#S4.T2.4.1.11.1),[Table 3](https://arxiv.org/html/2610.11160#S4.T3.2.1.8.1),[Table 6](https://arxiv.org/html/2610.11160#S4.T6.2.1.5.1)\.
- Liet al\.\(2024b\)Q\. Li, X\. Liu, Z\. Tang, P\. Dong, Z\. Li, X\. Pan, and X\. ChuShould we really edit language models? on the evaluation of edited language models\.Advances in Neural Information Processing Systems\.Cited by:[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px3.p1.1)\.
- Lialinet al\.\(2024\)V\. Lialin, S\. Muckatira, N\. Shivagunde, and A\. RumshiskyReLoRA: high\-rank training through low\-rank updates\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px2.p1.1)\.
- Liuet al\.\(2024\)S\. Liu, C\. Wang, H\. Yin, P\. Molchanov, Y\. F\. Wang, K\. Cheng, and M\. ChenDoRA: weight\-decomposed low\-rank adaptation\.arXiv preprint arXiv:2402\.09353\.Cited by:[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px2.p1.1)\.
- Menget al\.\(2022\)K\. Meng, D\. Bau, A\. Andonian, and Y\. BelinkovLocating and editing factual associations in GPT\.InAdvances in Neural Information Processing Systems,Cited by:[§F\.3](https://arxiv.org/html/2610.11160#A6.SS3.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.11160#S1.p2.1),[§1](https://arxiv.org/html/2610.11160#S1.p6.1),[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2610.11160#S4.SS1.SSS0.Px1.p1.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.18.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.31.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.5.1),[Table 2](https://arxiv.org/html/2610.11160#S4.T2.4.1.5.1)\.
- Menget al\.\(2023\)K\. Meng, A\. S\. Sharma, A\. Andonian, Y\. Belinkov, and D\. BauMass\-editing memory in a transformer\.InInternational Conference on Learning Representations,Cited by:[§F\.3](https://arxiv.org/html/2610.11160#A6.SS3.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.11160#S1.p2.1),[§1](https://arxiv.org/html/2610.11160#S1.p6.1),[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2610.11160#S4.SS1.SSS0.Px1.p1.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.19.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.32.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.6.1),[Table 2](https://arxiv.org/html/2610.11160#S4.T2.4.1.6.1),[Table 3](https://arxiv.org/html/2610.11160#S4.T3.2.1.3.1)\.
- Mitchellet al\.\(2022a\)E\. Mitchell, C\. Lin, A\. Bosselut, C\. Finn, and C\. D\. ManningFast model editing at scale\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2610.11160#S1.p2.1),[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px1.p1.1)\.
- Mitchellet al\.\(2022b\)E\. Mitchell, C\. Lin, A\. Bosselut, C\. D\. Manning, and C\. FinnMemory\-based model editing at scale\.InInternational Conference on Machine Learning,Cited by:[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px1.p1.1)\.
- Rosatiet al\.\(2024\)D\. Rosati, R\. Gonzales, J\. Chen, X\. Yu, Y\. Kayani, F\. Rudzicz, and H\. SajjadLong\-form evaluation of model editing\.Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics\.Cited by:[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px3.p1.1)\.
- Shenget al\.\(2023\)Y\. Sheng, S\. Cao, D\. Li, C\. Hooper, N\. Lee, S\. Yang, C\. Chou, B\. Zhu, L\. Zheng, K\. Keutzer,et al\.S\-lora: serving thousands of concurrent lora adapters\.arXiv preprint arXiv:2311\.03285\.Cited by:[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px3.p2.1)\.
- Sinitsinet al\.\(2020\)A\. Sinitsin, V\. Plokhotnyuk, D\. Pyrkin, S\. Popov, and A\. BabenkoEditable neural networks\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px1.p1.1)\.
- Songet al\.\(2020\)K\. Song, X\. Tan, T\. Qin, J\. Lu, and T\. LiuMpnet: masked and permuted pre\-training for language understanding\.Advances in neural information processing systems33,pp\. 16857–16867\.Cited by:[Table 26](https://arxiv.org/html/2610.11160#A7.T26.2.5.1.1.1)\.
- Thedeet al\.\(2025\)L\. Thede, K\. Roth, M\. Bethge, Z\. Akata, and T\. HartvigsenWikibigedit: understanding the limits of lifelong knowledge editing in llms\.arXiv preprint arXiv:2503\.05683\.Cited by:[§1](https://arxiv.org/html/2610.11160#S1.p6.1),[§4\.1](https://arxiv.org/html/2610.11160#S4.SS1.SSS0.Px1.p1.1)\.
- Wanget al\.\(2025\)K\. Wang, Y\. Qin, N\. Dimitriadis, A\. Favero, and P\. FrossardMEMOIR: lifelong model editing with minimal overwrite and informed retention for LLMs\.InAdvances in Neural Information Processing Systems,Cited by:[§F\.3](https://arxiv.org/html/2610.11160#A6.SS3.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.11160#S1.p2.1),[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px1.p1.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.10.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.23.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.36.1),[Table 2](https://arxiv.org/html/2610.11160#S4.T2.4.1.10.1),[Table 3](https://arxiv.org/html/2610.11160#S4.T3.2.1.7.1),[Table 6](https://arxiv.org/html/2610.11160#S4.T6.2.1.6.1)\.
- Wanget al\.\(2022\)L\. Wang, N\. Yang, X\. Huang, B\. Jiao, L\. Yang, D\. Jiang, R\. Majumder, and F\. WeiText embeddings by weakly\-supervised contrastive pre\-training\.arXiv preprint arXiv:2212\.03533\.Cited by:[Table 26](https://arxiv.org/html/2610.11160#A7.T26.2.4.1.1.1)\.
- Wanget al\.\(2024a\)P\. Wang, Z\. Li, N\. Zhang, Z\. Xu, Y\. Yao, Y\. Jiang, P\. Xie, F\. Huang, and H\. ChenWISE: rethinking the knowledge memory for lifelong model editing of large language models\.arXiv preprint arXiv:2405\.14768\.Cited by:[§F\.3](https://arxiv.org/html/2610.11160#A6.SS3.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px1.p1.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.21.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.34.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.8.1),[Table 2](https://arxiv.org/html/2610.11160#S4.T2.4.1.8.1),[Table 3](https://arxiv.org/html/2610.11160#S4.T3.2.1.5.1)\.
- Wanget al\.\(2023\)P\. Wang, N\. Zhang, B\. Tian, Z\. Xi, Y\. Yao, Z\. Xu, M\. Wang, S\. Mao, X\. Wang, S\. Cheng,et al\.EasyEdit: an easy\-to\-use knowledge editing framework for large language models\.arXiv preprint arXiv:2308\.07269\.Cited by:[§1](https://arxiv.org/html/2610.11160#S1.p2.1)\.
- Wang and Li \(2024a\)R\. Wang and P\. LiLEMoE: advanced mixture of experts adaptor for lifelong model editing of large language models\.arXiv preprint arXiv:2406\.20030\.Cited by:[§F\.3](https://arxiv.org/html/2610.11160#A6.SS3.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.11160#S1.p2.1),[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px1.p1.1)\.
- Wang and Li \(2024b\)R\. Wang and P\. LiMEMoE: enhancing model editing with mixture of experts adaptors\.arXiv preprint arXiv:2405\.19086\.Cited by:[§F\.3](https://arxiv.org/html/2610.11160#A6.SS3.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.11160#S1.p2.1),[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px1.p1.1)\.
- Wanget al\.\(2024b\)S\. Wang, Y\. Zhu, H\. Liu, Z\. Zheng, C\. Chen, and J\. LiKnowledge editing for large language models: a survey\.arXiv preprint arXiv:2310\.16218\.Cited by:[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px3.p1.1)\.
- Wuet al\.\(2024\)T\. Wu, L\. Luo, Y\. Li, S\. Pan, T\. Vu, and G\. HaffariContinual learning for large language models: a survey\.arXiv preprint arXiv:2402\.01364\.Cited by:[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px3.p1.1)\.
- Yanget al\.\(2025\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§G\.1](https://arxiv.org/html/2610.11160#A7.SS1.p1.1),[§1](https://arxiv.org/html/2610.11160#S1.p6.1),[§4\.1](https://arxiv.org/html/2610.11160#S4.SS1.SSS0.Px1.p2.1)\.
- Yanget al\.\(2024\)A\. Yang, B\. Yang, B\. Hui, B\. Zheng, B\. Yu, C\. Zhou, C\. Li, C\. Li, D\. Liu, F\. Huang,et al\.Qwen2\.5 technical report\.arXiv preprint arXiv:2412\.15115\.Cited by:[§G\.1](https://arxiv.org/html/2610.11160#A7.SS1.p1.1),[§1](https://arxiv.org/html/2610.11160#S1.p6.1)\.
- Yaoet al\.\(2023\)Y\. Yao, P\. Wang, B\. Tian, S\. Cheng, Z\. Li, S\. Deng, H\. Chen, and N\. ZhangEditing large language models: problems, methods, and opportunities\.arXiv preprint arXiv:2305\.13172\.Cited by:[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px3.p1.1)\.
- Yuet al\.\(2023\)L\. Yu, Q\. Chen, J\. Zhou, and L\. HeMELO: enhancing model editing with neuron\-indexed dynamic LoRA\.arXiv preprint arXiv:2312\.11795\.Cited by:[§F\.3](https://arxiv.org/html/2610.11160#A6.SS3.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.11160#S1.p3.1),[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px1.p1.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.12.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.25.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.38.1),[Table 2](https://arxiv.org/html/2610.11160#S4.T2.4.1.12.1),[Table 3](https://arxiv.org/html/2610.11160#S4.T3.2.1.9.1),[Table 6](https://arxiv.org/html/2610.11160#S4.T6.2.1.4.1)\.
- Yuet al\.\(2025\)X\. Yu, J\. Yang, X\. Wu, P\. Qiu, and X\. LiuFm\-lora: factorized low\-rank meta\-prompting for continual learning\.In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops \(CVPRW\),pp\. 6399–6408\.Cited by:[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px2.p1.1)\.
- Zhanget al\.\(2024\)N\. Zhang, Y\. Yao, B\. Tian, P\. Wang, S\. Deng, M\. Wang, Z\. Xi, S\. Mao, J\. Zhang, Y\. Ni,et al\.A comprehensive study of knowledge editing for large language models\.arXiv preprint arXiv:2401\.01286\.Cited by:[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px3.p1.1)\.
- Zhanget al\.\(2023\)Q\. Zhang, M\. Chen, A\. Bukharin, N\. Karampatziakis, P\. He, Y\. Cheng, W\. Chen, and T\. ZhaoAdalora: adaptive budget allocation for parameter\-efficient fine\-tuning\.arXiv preprint arXiv:2303\.10512\.Cited by:[§F\.3](https://arxiv.org/html/2610.11160#A6.SS3.SSS0.Px5.p1.1),[§1](https://arxiv.org/html/2610.11160#S1.p2.1),[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px3.p2.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.13.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.26.1),[Table 1](https://arxiv.org/html/2610.11160#S4.T1.4.1.39.1),[Table 2](https://arxiv.org/html/2610.11160#S4.T2.4.1.13.1),[Table 3](https://arxiv.org/html/2610.11160#S4.T3.2.1.10.1)\.
- Zhenget al\.\(2023\)C\. Zheng, L\. Li, Q\. Dong, Y\. Fan, Z\. Wu, J\. Xu, and B\. ChangCan we edit factual knowledge by in\-context learning?\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Cited by:[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px1.p1.1)\.
- Zhenget al\.\(2024\)J\. Zheng, S\. Qiu, C\. Shi, and Q\. MaTowards lifelong learning of large language models: a survey\.arXiv preprint arXiv:2406\.06391\.Cited by:[§2](https://arxiv.org/html/2610.11160#S2.SS0.SSS0.Px3.p1.1)\.

## Appendix

## Appendix AComposing LadderEdit with Learned Retrieval

The main experiments use matched edit association to isolate the storage layer from the retrieval layer, so that any performance gap among storage policies is attributable to the persistent representation rather than to retrieval quality\. This subsection examines the orthogonal question of whether the storage savings reported in the main paper survive when the matched association is replaced with a learned retriever\.

We evaluate three retrieval strategies on LLaMA\-3\-8B / ZsRE atT=1,000T=1\{,\}000: a learned Contriever\-style dual encoder trained on edit prompts, a learned E5\-large retriever, and a frozen off\-the\-shelf MPNet embedding \(no retrieval training\)\. In all cases, the retriever selects a single edit\-local memory, and the same retrieved edit identity is passed to both exact LoRA and LadderEdit, so the comparison still measures storage rather than retrieval quality\.

Table[7](https://arxiv.org/html/2610.11160#A1.T7)reports sensitivity to retrieval on ZsRE / LLaMA\-3\-8B atT=1,000T=1\{,\}000\.

Table 7:Retrieval sensitivity on ZsRE / LLaMA\-3\-8B atT=1,000T=1\{,\}000\. The persistent\-memory ratio is preserved under all retrieval settings in the table because the retriever is shared between exact LoRA and LadderEdit\. Locality is barely affected \(0\.99→\\rightarrow0\.98 under MPNet\) because retrieved adapters still land on real edit directions rather than random directions\.Both methods degrade by an equal amount under imperfect retrieval, because the association layer is shared between exact LoRA and LadderEdit\. The relative ordering of storage methods is therefore preserved under all retrieval strategies in Table[7](https://arxiv.org/html/2610.11160#A1.T7)\.

### A\.1Edit\-Stream Heterogeneity

Table[4](https://arxiv.org/html/2610.11160#S4.T4)of the main text reports the overall population structure of the edit stream\. Within the compression\-regularized population, we find a positive correlation \(Pearsonr=0\.38r=0\.38on ZsRE,r=0\.41r=0\.41on CF\) between the lexical overlap of the rewrite prompt and locality probes and the magnitude of the locality improvement obtained by sketch storage\. This is consistent with low\-rank truncation removing direction components that overfit to surface features of the rewrite prompt, which the exact update carries but the locality probes penalize\. Section[A\.5](https://arxiv.org/html/2610.11160#A1.SS5)reports the corresponding ladder allocation decisions\.

### A\.2Comparison with Static Mixed\-Rank Allocation

The exact\-cache baseline in Section 4 isolates the keep\-versus\-drop extreme of fixed\-allocation policies\. A second non\-adaptive baseline, closer to LadderEdit in absolute memory and not dropping edits, assigns every edit a fixed rank with no behavioral audit\. The audit’s contribution is most visible on the worst\-edit Avg, the per\-edit Avg averaged over the bottom 10% of edits—which the pooled Avg averages away\. Against the strongest fixed\-rank baseline \(rank\-8\), audited LadderEdit gains\+0\.05\+0\.05on pooled Avg but\+0\.12\+0\.12on the worst\-edit metric\. An oracle controller that selects the per\-edit optimal rank reaches 0\.96 pooled and 0\.78 worst\-edit, indicating LadderEdit is within 0\.01 pooled and 0\.04 worst\-edit of the oracle\. The audit’s main contribution is therefore on the hard tail of edits where fixed\-rank baselines fail systematically\.

Table 8:Audit contribution against fixed\-rank storage atT=2,000T=2\{,\}000on ZsRE / LLaMA\-3\-8B across three seeds\. Memory is normalized to exact LoRA\. Pooled Avg\. is the pooled evaluation Avg\. across edits; worst\-edit Avg\. is the per\-edit Avg\. averaged over only the bottom 10% of edits\. Standard deviations over three seeds are reported in parentheses when available\.#### Trade\-offs with the strongest baselines\.

GRACE achieves 1\.00 reliability and locality on ZsRE by codebook retrieval, but its generalization \(0\.34 on LLaMA\-3 atT=2,000T=2\{,\}000\) means it cannot answer paraphrased queries\. MEMOIR achieves 0\.69 Avg\. atT=10,000T=10\{,\}000in only 0\.14 GB of persistent memory, a138×138\\timesmemory advantage over LadderEdit \(19\.4 GB\)\. LadderEdit’s 0\.11 Avg\. improvement over MEMOIR atT=10,000T=10\{,\}000is therefore an explicit memory\-for\-fidelity trade, not a strict dominance\. Practitioners with extreme memory constraints should prefer MEMOIR; practitioners with extreme paraphrase\-coverage requirements should consider GRACE; LadderEdit is recommended for the regime where retrieval\-quality edits at scale are the bottleneck\.

### A\.3Audit Score Weights

The audit score in Eq\.[7](https://arxiv.org/html/2610.11160#S3.E7)combines reliability, generalization, and locality residuals as a weighted sum,s⁡\(z\)=wR⋅ΔR\+wG⋅ΔG\+wL⋅ΔLs\(z\)=w\_\{R\}\\cdot\\Delta\_\{R\}\+w\_\{G\}\\cdot\\Delta\_\{G\}\+w\_\{L\}\\cdot\\Delta\_\{L\}, with default weightswR=1\.0w\_\{R\}=1\.0,wG=0\.8w\_\{G\}=0\.8, andwL=1\.2w\_\{L\}=1\.2\. The asymmetry betweenwLw\_\{L\}and the other two weights is deliberate\. A reliability or generalization failure on a candidate rung triggers rank promotion: the audit rejects the sketch, the controller moves up the ladder, and the contract is reassessed at the next rung\. The failure is recoverable within the same insertion\. A locality failure, by contrast, alters behavior on prompts the edit was never supposed to touch\. If the audit lets such a sketch pass, the corruption is committed to persistent memory and propagates to every later query that lands on that edit; there is no within\-insertion repair\. We therefore weight locality higher than the recoverable metrics, treatingwL\>wR,wGw\_\{L\}\>w\_\{R\},w\_\{G\}as a soft asymmetry between catastrophic and recoverable failure modes\.

The defaultwL=1\.2w\_\{L\}=1\.2is selected by a sweep on the validation split\. We evaluatewL∈\{0\.8,1\.0,1\.2,1\.5,2\.0\}w\_\{L\}\\in\\\{0\.8,1\.0,1\.2,1\.5,2\.0\\\}holdingwRw\_\{R\}andwGw\_\{G\}fixed; Figure[5](https://arxiv.org/html/2610.11160#A1.F5)reports the result on ZsRE / LLaMA\-3\-8B atT=1,000T=1\{,\}000\. The choice is not knife\-edge: average performance stays within0\.010\.01of the peak forwL∈\[1\.0,1\.5\]w\_\{L\}\\in\[1\.0,1\.5\], and drops by0\.020\.02atwL=0\.8w\_\{L\}=0\.8and0\.030\.03atwL=2\.0w\_\{L\}=2\.0\. The default sits at the midpoint of this stable region\.

The per\-metric decomposition \(Figure[5](https://arxiv.org/html/2610.11160#A1.F5), middle\) clarifies why the stable region is bounded on both sides\. AswLw\_\{L\}falls below1\.01\.0, locality begins to leak: the audit no longer penalizes locality violations strongly enough to reject sketches that overwrite neighboring knowledge, and the Loc\. metric drops sharply \(from1\.001\.00at the default to0\.880\.88atwL=0\.8w\_\{L\}=0\.8\)\. AswLw\_\{L\}rises above1\.51\.5, the audit becomes over\-restrictive: many sketches that would have preserved locality adequately are rejected for marginal locality residuals, the controller promotes them to higher ranks or beyond the ladder, and reliability and generalization both degrade as a result\. The defaultwL=1\.2w\_\{L\}=1\.2is the unique value in the swept range at which all three metrics are simultaneously near their per\-metric optima\.

The failure\-mode decomposition \(Figure[5](https://arxiv.org/html/2610.11160#A1.F5), right\) reports the same trade\-off in absolute terms\. AtwL=0\.8w\_\{L\}=0\.8, locality failures account for14\.2%14\.2\\%of edits: a non\-trivial fraction of the stream commits a sketch that should have been rejected, and the resulting corruption cannot be undone\. AtwL=2\.0w\_\{L\}=2\.0, locality failures drop to0\.4%0\.4\\%but the audit rejects9\.5%9\.5\\%of edits that the more permissive default would have accepted; these rejections are recoverable in principle \(the controller can re\-audit at a different operating point\), but in our deployed pipeline they translate to higher rank consumption and reduced per\-edit efficiency\. The default minimizes the total failure rate at1\.6%1\.6\\%, balancing the two error modes against the asymmetry the prose above describes\.

We also verified thatwRw\_\{R\}andwGw\_\{G\}are not separately tuned\. SettingwR=wGw\_\{R\}=w\_\{G\}over the same five\-point sweep and varying their common value in\{0\.6,0\.8,1\.0\}\\\{0\.6,0\.8,1\.0\\\}changes Avg\. by less than0\.0050\.005, and we therefore fix\(wR,wG\)=\(1\.0,0\.8\)\(w\_\{R\},w\_\{G\}\)=\(1\.0,0\.8\)to follow the standard LoRA loss\-weighting ratio\. The locality weight is the only audit hyperparameter with a non\-trivial effect on the headline metrics; all main\-paper results use the defaultwL=1\.2w\_\{L\}=1\.2without per\-benchmark retuning\.

Figure 5:Sensitivity of LadderEdit to the locality weightwLw\_\{L\}on ZsRE / LLaMA\-3\-8B atT=1,000T=1\{,\}000, withwRw\_\{R\}andwGw\_\{G\}held fixed\. Left: average performance, peaking at the defaultwL=1\.2w\_\{L\}=1\.2with a stable region across\[1\.0,1\.5\]\[1\.0,1\.5\]\. Middle: per\-metric decomposition, showing that locality cliffs upward betweenwL=0\.8w\_\{L\}=0\.8andwL=1\.0w\_\{L\}=1\.0while reliability and generalization degrade forwL≥1\.5w\_\{L\}\\geq 1\.5\. Right: failure\-mode decomposition, with locality failures dominating at lowwLw\_\{L\}and reliability or generalization rejections dominating at highwLw\_\{L\}; the default minimizes the total failure rate\.
### A\.4Composition with Post\-Training Quantization

LadderEdit reduces the number of stored parameters per edit, whereas quantization reduces the bit\-width of each stored parameter\. The two reductions act on orthogonal axes of the persistent\-storage cost and compose multiplicatively in principle\. We verify this empirically by applying symmetric per\-tensor INT8 quantization to the stored LoRA factors, both for the exact\-LoRA reference and for the LadderEdit rungs\.

Table 9:Composition of LadderEdit with INT8 quantization atT=2,000T=2\{,\}000on ZsRE / LLaMA\-3\-8B\. The two compression axes are orthogonal: INT8 reduces per\-edit bit\-width while LadderEdit reduces per\-edit rank\. Composing them yields a20×20\\timesreduction over FP16 exact LoRA\. The composed configuration \(LadderEdit \+ INT8\) reaches 0\.90 Avg\. at 0\.05 normalized memory, dominating exact LoRA \+ INT8 alone \(0\.93 Avg\. at 0\.25 memory\) on the cost axis\.Quantization is a complementary compression primitive that operates within the cost of one stored representation, while LadderEdit operates across representations by deciding how much rank each edit retains\. We therefore treat quantization as an orthogonal post\-processing step rather than as a competing baseline; the right deployment configuration uses both\.

### A\.5Per\-Condition Rung Allocation

The default audit thresholds in the main paper produce the following distribution of stored ranks across the rank menuℛ=\{1,2,…,8\}\\mathcal\{R\}=\\\{1,2,\\ldots,8\\\}, whereRmax=8R\_\{\\max\}=8matches the exact LoRA acquisition rank\. Table[10](https://arxiv.org/html/2610.11160#A1.T10)reports the distribution atT=1,000T=1\{,\}000on ZsRE / LLaMA\-3\-8B; comparable distributions are observed for the other backbones and the other horizons\.

The stored rank distribution reflects two stages of decision\-making\. Each edit first receives a*predicted rank*r^i\\hat\{r\}\_\{i\}from the spectral proposal \(Eq\.[5](https://arxiv.org/html/2610.11160#S3.E5)\): LadderEdit computes the normalized spectral tailTi​\(r\)T\_\{i\}\(r\)at everyr∈ℛr\\in\\mathcal\{R\}from a single SVD, evaluates the scoreSi​\(r\)=Ti​\(r\)/\(\[μiE\]\+\+ϵ\)S\_\{i\}\(r\)=T\_\{i\}\(r\)/\(\[\\mu\_\{i\}^\{E\}\]\_\{\+\}\+\\epsilon\), and selects the cheapestr^i\\hat\{r\}\_\{i\}whose score falls below the calibrated thresholdτ\\tau\. This proposal uses only spectral information and the exact\-edit margin, so it costs one rank\-search per edit and no behavioral evaluation\. The behavioral audit then verifies the proposal: if the rank\-r^i\\hat\{r\}\_\{i\}sketch satisfies the contract \(Hi\(r^i\)=0H\_\{i\}^\{\(\\hat\{r\}\_\{i\}\)\}=0\), the edit is stored atr^i\\hat\{r\}\_\{i\}; otherwise the audit promotes the edit along the spectrally\-ordered ladderr^i\+1,r^i\+2,…\\hat\{r\}\_\{i\}\+1,\\hat\{r\}\_\{i\}\+2,\\ldotsuntil a passing rung is reached\. Table[10](https://arxiv.org/html/2610.11160#A1.T10)reports the distribution of*stored*ranksrir\_\{i\}, which equalsr^i\\hat\{r\}\_\{i\}for the majority of edits whose proposal passes on first audit and exceedsr^i\\hat\{r\}\_\{i\}only for the minority that require promotion\.

Table 10:Distribution of stored representations \(N=1,000 edits atT=1,000T=1\{,\}000on LLaMA\-3\-8B / ZsRE; sums to 100\.0%\) across the fullr=1r\{=\}1tor=8r\{=\}8ladder\. The majority of edits are served by the cheapest rung, and the distribution decays sharply: the top two rungs together cover95\.5%95\.5\\%of edits, while only the remaining∼4\.5%\\sim 4\.5\\%requires ranks 3 or above\. The audit promotes edits to higher rungs only when the low\-rank sketch fails to satisfy the behavioral contract, which corresponds approximately to the detail\-needed population of Table[4](https://arxiv.org/html/2610.11160#S4.T4)\. The parity between LadderEdit and exact LoRA at this horizon \(Table 1 of the main text\) therefore reflects targeted rank promotion on a small minority of edits rather than uniformly high\-rank storage across the stream\.The per\-edit allocation decisions are released alongside the code to support reproducibility\. The audit cost reported in Appendix[D\.7](https://arxiv.org/html/2610.11160#A4.SS7)is dominated by the rank\-1 evaluation path because most edits terminate there; only the∼20%\\sim 20\\%of edits that are promoted incur the higher\-rank audit cost\.

### A\.6Memory Ratio Stability

The5\.2×5\.2\\timesmemory ratio in the abstract and main text refers to the \(LLaMA\-3\-8B,T=10,000T=10\{,\}000\) configuration on WikiBigEdit\. We report ratios at every backbone\-horizon pair to document the stability of this number\. AtT=10,000T=10\{,\}000the ratio is5\.2×5\.2\\timeson LLaMA\-3\-8B,5\.0×5\.0\\timeson Mistral\-7B, and5\.1×5\.1\\timeson Qwen2\.5\-7B \(standard deviation 0\.1×\\times\)\. The ratio is not constant inTT: it grows from2\.0×2\.0\\timesatT=100T=100\(mean across backbones\) to11\.8×11\.8\\timesatT=50,000T=50\{,\}000, because LadderEdit’s allocation table has a long tail of low\-rank rungs that absorb additional edits with near\-constant marginal memory, while exact LoRA stores every additional edit at full rank\.

The5\.2×5\.2\\timesheadline atT=10,000T=10\{,\}000grows to approximately12×12\\timesatT=50,000T=50\{,\}000on WikiBigEdit for the same reason\. Cross\-backbone stability atT=50,000T=50\{,\}000is similar:12\.0×12\.0\\timeson LLaMA\-3\-8B,11\.6×11\.6\\timeson Mistral\-7B, and11\.8×11\.8\\timeson Qwen2\.5\-7B\.

## Appendix BAdditional Rank\-Compression Diagnostics

This appendix provides further evidence and explanation for the compression and behavioral audit results discussed in the main text, focusing on the reliability of the method, the role of the audit, and the interpretation of diagnostic categories\.

### B\.1Training and Seed Details

All experiments use frozen backbone models; the only learned object for an incoming edit is its LoRA\-style edit adapter\. We first acquire an exact editΔiE\\Delta\_\{i\}^\{E\}with the same edit\-writing procedure used by the exact LoRA baseline\. LadderEdit is applied after this acquisition step: it does not change the edit writer, the base model weights, the rewrite examples, or the optimizer used to obtain the exact edit\. This keeps the comparison focused on persistent storage and serving\-time representation rather than on a stronger or weaker edit writer\.

For exact LoRA acquisition, each edit is represented by a pair of LoRA factors whose effective update isΔiE\\Delta\_\{i\}^\{E\}\. The rank\-compression diagnostics use an exact raw LoRA rank of 8, corresponding to 5\.05M trainable parameters per exact edit in the LLaMA\-3\-8B diagnostic accounting\. The deployed rank menu isℛ=\{1,2,…,8\}\\mathcal\{R\}=\\\{1,2,\\ldots,8\\\}, matching the exact LoRA acquisition rank\. Candidate sketches at each rung are produced by a single truncated SVD of the effective update, which yields all\|ℛ\|\|\\mathcal\{R\}\|spectral tailsTi​\(r\)T\_\{i\}\(r\)in one pass\. The strict frontier diagnostics in Table[5](https://arxiv.org/html/2610.11160#S4.T5)and Appendix Table[20](https://arxiv.org/html/2610.11160#A6.T20)report the sparse representative subset\{1,2,4\}\\\{1,2,4\\\}for readability; the audited deployment uses the full menu\. The default audited configuration in the main experiments usesτ=0\.05\\tau=0\.05unless a table explicitly varies this value\.

The behavioral audit is run at insertion or rebalancing time\. It evaluates rewrite, generalization, and locality probes, then stores the cheapest rung whose measured contract is acceptable\. If the sketch fails, LadderEdit promotes the edit to a higher rank along the ladder until the contract is met \(orRmaxR\_\{\\max\}is reached\)\. Probe thresholds, audit weights, and the compressibility\-score threshold are selected on held\-out validation probes for each benchmark and then held fixed for the corresponding test streams\.

We use fixed edit streams and shared random seeds across compared methods so that differences reflect the editing policy rather than the sampled order\. The rank\-compression package reports averages over multiple seed runs whenever those runs are available\. Fixed\-rank spectral and denoising diagnostics are averaged over seeds 0, 1, and 2 for CF atN=32,128,512N=32,128,512and for ZsRE atN=128,512N=128,512; ZsRE atN=32N=32includes six available runs\. The budget\-matched strict\-memory frontier in the main text is reported from theN=512N=512, seed\-0 frontier file used for the plotted strict\-frontier analysis, with exact\-cache rows matched to the same normalized memory budget\.

## Appendix CProbe Split Protocol and Validation

This appendix verifies that the audit signal used by LadderEdit does not leak into the evaluation reported in the main tables\. We describe the partition, report inter\-split similarity, and show that running LadderEdit with a leaky \(audit\-equals\-test\) configuration would inflate the reported numbers by a measurable margin — which the strict split avoids\.

## Appendix DWhy Not Standard LoRA Compression?

LadderEdit is methodologically close to several LoRA\-centric techniques but differs from each along a specific axis\. We distinguish three of them here, since reviewers familiar with one of these methods may otherwise read LadderEdit as a re\-application of it\.

#### Unit and criterion of allocation\.

Adaptive\-rank methods such as AdaLoRA allocate rank across the layers or modules of a single adapter using gradient\-derived importance during training\. LadderEdit allocates persistent storage across edits after acquisition, using behavioral contract satisfaction as the criterion\. The decision variable is “how much of editiito retain” rather than “where to spend rank inside one adapter”\. Running AdaLoRA independently for each edit does not solve the lifelong\-editing problem: it still stores one adapter per edit and lacks a stream\-level policy for deciding which edits deserve higher resolution\.

#### Stream\-level policy\.

Multi\-adapter serving systems such as S\-LoRA deploy many independently trained adapters efficiently at inference time but do not decide which adapter directions to persist\. LadderEdit operates one level higher: as the stream of edits accumulates, the residual\-ladder controller spends extra memory only on the behaviorally difficult tail and leaves easy edits at low rank\. Without this stream\-level policy, multi\-adapter serving still stores every adapter at its acquired rank, and the persistent memory grows linearly at the slope set by the edit writer rather than the slope set by the audit\.

#### Independence versus merging\.

Adapter\-merging methods reduce per\-edit storage by combining adapters into one shared representation, which loses edit\-local isolation and reintroduces the plasticity pressure of constant\-memory editors\. LadderEdit keeps one representation per edit; the residual ladder reduces memory without folding edits together\. This is the property that distinguishes LadderEdit from the merging\-based lifelong editors \(MELO, ELDER, MEMoE, LEMoE\) in our baseline set\.

### D\.1Partition statistics

For each edit, the benchmark prompts are split as follows:

Table 11:Per\-edit probe counts after the three\-way split\. Locality probes are drawn from disjoint pools of unrelated subjects between audit and test\.The split is performed once per edit at stream\-construction time using a fixed seed, and is shared across all compared methods so that no method sees a different test set\.

### D\.2Audit\-test independence

Strong support for the split requires that the audit score is not a perfect predictor of the test score\. If it were, the audit would amount to memorising the test set\. We measure two quantities atN=2,000N=2\{,\}000on ZsRE / LLaMA\-3\-8B\.

#### Audit\-test correlation\.

Pearson correlation between per\-edit audit utility and per\-edit test utility isr=0\.62r=0\.62\(p<10−50p<10^\{\-50\}\)\. The correlation is positive \(an edit that passes its audit usually passes its test\) but well below 1, indicating that the test contains genuine held\-out variation rather than re\-scored audit prompts\.

#### Lexical overlap\.

Word\-level Jaccard similarity between audit and test paraphrases of the same edit averages0\.340\.34on ZsRE and0\.290\.29on CounterFact\. As a control, the Jaccard between paraphrases of two random different edits is0\.110\.11on ZsRE\. The within\-edit overlap is higher than the between\-edit overlap, as expected for paraphrases of the same fact, but is far from the trivial near\-1\.0 overlap that audit\-equals\-test would imply\.

### D\.3Leaky versus clean comparison

To quantify the importance of the split, we re\-run LadderEdit with a deliberately leaky configuration in which the audit uses the test paraphrases directly, and compare against the strict split used in the main paper\.

Table 12:Leaky\-versus\-clean comparison atT=1,000T=1\{,\}000on ZsRE / LLaMA\-3\-8B\. The leaky configuration inflates Generalization by 6 points and the average by 4 points\. The numbers reported in the main paper correspond to the strict split\.The 4\-point average gap quantifies how much the strict split costs us in headline numbers\. We report the strict\-split numbers throughout the main paper\. The leaky\-vs\-clean table is the strongest single piece of evidence: it shows the split is real, that it costs measurable utility, and that the main\-paper numbers correspond to the harder configuration\.

### D\.4AdaLoRA Comparison and Scope

AdaLoRA is an important low\-rank adaptation method, but it solves a different allocation problem from LadderEdit\. We do not claim SVD truncation or behavioral audits as new primitives\. The contribution is the per\-edit decision rule that converts edit difficulty into rank assignment, applied to the sequential editing problem rather than the single fine\-tuning problem AdaLoRA targets\. AdaLoRA reallocates a fixed parameter budget among LoRA matrices during training for one adaptation objective; LadderEdit allocates rank across a stream of independent edits, with each edit committed only after passing a behavioral audit at the assigned rank\.

There are two natural ways to apply AdaLoRA in this setting, and neither is equivalent to LadderEdit\. If AdaLoRA is run independently for each edit, it may reduce the rank of each adapter, but the system still stores one adapter per edit and still lacks a stream\-level policy for deciding which edits deserve higher rank\. If AdaLoRA is run as one shared adapter over many edits, it no longer preserves edit\-local isolation; later edits must share capacity with earlier edits, which is the same plasticity pressure faced by constant\-memory editors\. LadderEdit is designed for the middle regime: one representation per edit, but each representation can occupy a cheaper rung if the behavioral audit permits it\.

The main benchmark tables therefore include AdaLoRA as an adaptive\-rank LoRA baseline, using the same edit streams and evaluation probes\. These runs are useful because they test the closest low\-rank alternative: adaptive rank alone does not solve the long\-stream memory allocation problem when the system lacks per\-edit residual decisions and behavioral auditing\. The present diagnostics answer the complementary question: once an exact per\-edit LoRA update is acquired, how much of that update must be persistently stored to preserve the edit contract?

### D\.5Rationale and Conditional Guarantees

This section makes the allocation logic explicit\. The statements below are not safety guarantees for unprobed prompts\. They explain why, under the measured edit\-time utility used in the paper, a residual ladder is a more natural memory allocation than an all\-or\-nothing exact cache\.

#### Coverage versus fidelity\.

The main text gives the coverage\-versus\-fidelity decomposition used to interpret the budget\-matched frontier\. The key point is that exact\-cache allocation reduces memory by reducing coverage, while LadderEdit keeps coverage high by assigning every edit a sketch before spending additional rank on fidelity\. At roughly5\.6×5\.6\\timescompression, an exact cache stores only about 18% of the 512 edits, whereas rank\-1 sketches give every edit a nonzero representation and higher\-rank promotion repairs the hard tail without discarding easy edits\.

#### Stronger drop policies\.

The exact\-cache result in Section 4 uses uniform random retention, which is a weak drop policy\. We tested four stronger policies at the same memory budget \(91 of 512 edits retained\) on ZsRE / LLaMA\-3\-8B\. Uniform random drop reaches 0\.18 pooled Avg; FIFO drop reaches 0\.20; LRU drop reaches 0\.22; learned utility\-scored drop \(a small MLP scoring edits by retrieval frequency and recency\) reaches 0\.35\. Adding a fallback that routes discarded queries to the unedited base model raises learned drop to 0\.41\. LadderEdit reaches 0\.93 at the same memory budget because no edit is discarded\. Evaluating only on retained edits, all drop policies reach 0\.93–0\.94, matching LadderEdit on that subset\. The gap on the full evaluation pool reflects the cost of not preserving the discarded 80%; this is a fundamental limitation of any drop strategy, since information about discarded edits is gone regardless of policy\.

#### Optimal promotion choice for a fixed baseline rank\.

For a fixed rankrr, let

gi​\(r\)=ui​\(ΔiE\)−ui​\(Δi\(r\)\)g\_\{i\}\(r\)=u\_\{i\}\(\\Delta\_\{i\}^\{E\}\)\-u\_\{i\}\(\\Delta\_\{i\}^\{\(r\)\}\)\(10\)be the measured utility gain from promoting editiifrom rankrrto the exact update\. This is the same two\-state simplification used in Eq\.[8](https://arxiv.org/html/2610.11160#S3.E8)\-[9](https://arxiv.org/html/2610.11160#S3.E9); the deployed controller can stop promotion at intermediate rungs\. If all promotions have the same extra memory costPE−PrP\_\{E\}\-P\_\{r\}and the budget permitsKKpromotions, the utility\-maximizing promoted set is theKKedits with the largest positivegi​\(r\)g\_\{i\}\(r\)\. This follows by an exchange argument: if a selected edit has smaller gain than an unselected edit, swapping them increases utility without changing memory\. When promotion costs differ across ranks or layers, the same argument becomes a value\-density rule\.

#### What the diagnostics establish\.

The rank\-compression package supports three claims needed by the method\. First, exact LoRA updates are often over\-resolved: rank\-2 and rank\-4 sketches retain most spectral energy and recover most measured contract behavior\. Second, the hard tail is real: rank\-1 sketches create false retirements, so a single compressed rank is not enough\. Third, the hard tail is small enough for selective rank promotion to be useful: budget\-matched ladder policies retain much higher utility than exact\-cache policies at comparable memory\. These findings justify the ladder design without claiming that spectral compression alone is always safe\.

### D\.6Diagnostic Examples

The most useful examples are those that isolate the decision made by the ladder\. The goal is not only to show that the average improves, but to show why the controller sometimes stores only a sketch and sometimes promotes to higher rank\. Table[13](https://arxiv.org/html/2610.11160#A4.T13)gives representative edit\-level cases from theN=512N=512, seed\-0 rank\-1 diagnostics\. These are index\-level examples from the released diagnostic files, not hand\-picked text prompts\. They cover the four behavioral cases used by the method\.

Table 13:Representative edit\-level diagnostic cases atN=512N=512, seed 0, rank 1\.Δ​U\\Delta Uis the measured utility of the rank\-1 sketch minus exact LoRA\. A positive value means the sketch is better on the audit; a negative value means higher rank is useful\.The compact labels in Table[13](https://arxiv.org/html/2610.11160#A4.T13)are useful for accounting, but the rate\-distortion story is clearest at the prompt level\. Table[16](https://arxiv.org/html/2610.11160#A4.T16)shows four representative generations\. The first two cases show that rank 1 can preserve the exact rewrite behavior on both ZsRE and CF\. The third shows why the ladder must be adaptive: rank 1 captures a coarse entity pattern but rank 2 is needed to recover the specific value\. The fourth shows a compression\-regularized case where the exact edit misses the target in free generation while the low\-rank sketch includes it\.

Table 14:Sensitivity of LadderEdit to the audit thresholdτ\\tauand LoRA rankrron ZsRE / LLaMA\-3\-8B atT=1,000T=1\{,\}000\. Avg\. is the arithmetic mean of Rel\., Gen\., and Loc\. Bold marks the default configuration\. The default\(τ=0\.05,r=8\)\(\\tau\{=\}0\.05,r\{=\}8\)sits at the peak of both sweeps: smallerτ\\tauorrrunderfits the audit, largerτ\\tauorrradmits noisy edits that erode locality\.Table 15:Probe\-coverage stress test on ZsRE / LLaMA\-3\-8B atT=2,000T=2\{,\}000\. We vary the audit probe\-bank size\|𝒫\|\|\\mathcal\{P\}\|and report performance on probe\-covered and unprobed edits\. Unprobed edits are outside the probe bank’s semantic coverage and are handled through nearest\-probe interpolation\.Table 16:Prompt\-level qualitative examples illustrating the main residual\-ladder decisions\. Outputs are shortened only for formatting\. These cases show why LadderEdit allocates storage per edit rather than assigning one fixed rank to the whole stream\.Table 17:Caveat example: exact acquisition can fail under free generation\. This supports the limitation that exact LoRA is an acquisition reference, not an infallible semantic oracle\.CaseExact LoRASketchesInterpretationCF 27, target Russian\. Rewrite prompt: Jean Gaven, speaker ofJean Gaven was the President of the French Senate\. The generalization response asks for more information about the native language\.Rank 1 and rank 2 produce the same incorrect or uninformative behavior\.Exact\-limited\. The edit writer failed before compression, so residual storage cannot guarantee success\.

These examples make three points that the aggregate tables can hide\. First, many edits are sketch\-sufficient even when rank 1 retains only about half of the spectral energy, so spectral energy is informative but not identical to behavioral utility\. Second, false retirements exist on both CF and ZsRE, which is why LadderEdit keeps the behavioral audit rather than relying on spectrum alone\. Third, compression sometimes improves the measured contract, suggesting that some low\-energy directions encode over\-specific or interfering detail\. The method’s rationale is therefore not “low\-rank is always enough\.” It is that a cheap sketch should cover every edit, while residual memory should be reserved for measured hard cases\.

### D\.7Sensitivity, Coverage, and Audit Cost

We evaluate threshold sensitivity, probe coverage, overfitting to the audit set, and insertion\-time overhead\. Table[14](https://arxiv.org/html/2610.11160#A4.T14)varies the audit threshold and LoRA rank on ZsRE / LLaMA\-3\-8B atT=1,000T=1\{,\}000\. The default configuration is not a narrow optimum: the average score changes by less than 0\.03 across the tested thresholds and ranks, while memory changes substantially with rank\. Table[15](https://arxiv.org/html/2610.11160#A4.T15)stress\-tests probe\-bank coverage atT=2,000T=2\{,\}000\. The default probe bank covers 87% of edits and keeps the unprobed\-edit average within 0\.06 of the probed\-edit average\. Lower coverage degrades more clearly, which supports the limitation that the audit depends on probe coverage\.

\(a\)Held\-out probe behavior\(b\)Insertion\-time audit overhead
Figure 6:Additional audit diagnostics\. The held\-out probe plot checks whether threshold selection overfits to the validation probe set\. The audit\-overhead plot decomposes insertion latency and compares LadderEdit with exact LoRA across stream lengths\.Table 18:Insertion latency summary \(A100\-40GB\)\.Insertion latency on an A100\-40GB: exact LoRA goes from 86 ms per edit atT=100T=100to 850 ms per edit atT=50,000T=50\{,\}000because of per\-edit adapter merge overhead, which is linear in the number of stored adapters\. LadderEdit goes from 93 ms to 150 ms over the same range; the audit dominates at smallTT\(about 7 ms per edit\), and merge growth is suppressed by the low\-rank rungs\. LadderEdit is slower per edit atT<2,000T<2\{,\}000and faster atT\>2,000T\>2\{,\}000; atT=50,000T=50\{,\}000LadderEdit is5\.7×5\.7\\timesfaster per edit than exact LoRA\. The cumulative audit time atT=50,000T=50\{,\}000is approximately 32 minutes, paid once at edit acquisition rather than at serving time\.

## Appendix ERank\-Predictor Ablation

The spectral predictor used by LadderEdit combines two factors: the singular\-value tailTi​\(r\)T\_\{i\}\(r\)and the exact\-edit behavioral marginμiE\\mu\_\{i\}^\{E\}\(Eq\.[4](https://arxiv.org/html/2610.11160#S3.E4)\)\. We isolate the contribution of each factor and compare against three reference baselines: a random rank proposal, an audit\-and\-promote loop with no predictor \(*always rank\-1, then promote*\), and an exhaustive\-audit oracle that audits every rank inℛ\\mathcal\{R\}and stores the cheapest passing rung\. Results atT=1,000T=1\{,\}000on ZsRE / LLaMA\-3\-8B are reported in Table[19](https://arxiv.org/html/2610.11160#A5.T19)\.

Table 19:Predictor ablation atT=1,000T=1\{,\}000on ZsRE / LLaMA\-3\-8B\. Utility is the audited behavioral contract score; Audits/edit measures the cost of the proposal at insertion time; Memory is normalized to exact LoRA\. Bold marks the configuration used in the main paper\. The combined spectral\-times\-margin predictor reaches utility within 0\.02 of oracle at4\.6×4\.6\\timesfewer audits per edit, while each factor alone trails by 0\.03–0\.05\. The always\-rank\-1 baseline shows that the audit\-and\-promote loop alone is competitive on utility but pays a higher audit cost without a predictor to localise the proposal\.Three observations support the predictor design\. First, the predictor is necessary: random proposal collapses to 0\.81 utility, well below any informed strategy\. Second, both factors contribute: removing either the spectral tail or the margin reduces utility by 0\.03–0\.05 and increases the audits\-per\-edit cost\. Third, the combined predictor approaches oracle quality without paying the oracle’s audit cost, justifying the calibrated heuristic in Eq\.[4](https://arxiv.org/html/2610.11160#S3.E4)as an effective proposal mechanism\.

### E\.1Denoising Audit and Spectral\-Filter Visual Audit

Thedenoising audittests whether compressed \(low\-rank\) edit representations can match or even exceed the behavioral contract of the full exact LoRA update\. This is important because exact LoRA is treated as an*acquisition reference*, not a behavioral oracle: an uncompressed update may contain nuisance directions that hurt locality or overfit to the edit prompt\. The audit categorizes edits as:

- •Both\-pass: Both the sketch and exact update satisfy the behavioral contract \(“sketch\-sufficient”\)\.
- •Detail\-needed: The sketch fails but the exact passes \(higher rank is required\)\.
- •Compression\-regularized \(extra success\): The sketch passes but the exact fails \(compression removes harmful detail\)\.
- •Both\-fail: Neither passes \(edit is “exact\-limited”\)\.
- •False retirement: An exact\-pass edit that fails after compression\.

Thespectral\-filter visual auditis summarized by the frontier analysis in Figure[4](https://arxiv.org/html/2610.11160#S4.F4), which shows how much edit energy is retained at each rank and how this relates to behavioral contract satisfaction\.

## Appendix FFixed\-Rank Tier Diagnostics

Table[20](https://arxiv.org/html/2610.11160#A6.T20)reports the per\-rank compression statistics \(memory, retained energy, behavioral contract score, and false retirement count\) acrossN∈\{32,128,512\}N\\in\\\{32,128,512\\\}for both datasets, expanding the per\-rank view summarised in the main\-text population structure\. Most edit energy is captured by low\-rank sketches, but a small hard subset requires higher rank; rank\-1 false retirements are substantial, while rank\-4 leaves only 1 to 2 per dataset\.

Table 20:Fixed\-rank tier diagnostics for CF and ZsRE atN=32N=32,128128,512512\. Memory is normalized to exact LoRA\. Retained energy is1−‖ρ‖F2/‖ΔE‖F21\-\\\|\\rho\\\|\_\{F\}^\{2\}/\\\|\\Delta^\{E\}\\\|\_\{F\}^\{2\}\. False retirements count exact\-pass edits that fail after compression\.### F\.1Swap\-Count and Frontier Visuals

Figure[7](https://arxiv.org/html/2610.11160#A6.F7)shows swap\-count plots for ZsRE and CF, visualizing pass\-set transitions as rank increases\. These plots help explain how many edits change status \(e\.g\., from fail to pass\) as more detail is added\.

![Refer to caption](https://arxiv.org/html/2610.11160v1/figures/zsre_swap_counts.png)

![Refer to caption](https://arxiv.org/html/2610.11160v1/figures/counterfact_swap_counts.png)

Figure 7:Swap\-count diagnostics for ZsRE and CF atN=512N=512\. The plots show how many edits change pass/fail status as rank increases, highlighting the difficult subset that requires higher rank\.
### F\.2Interpretation

These diagnostics clarify several practical points about the method:

- •Why not constant memory?LadderEdit reduces the slope of memory growth but does not achieve constant memory, as every edit retains at least a sketch\.
- •Why is exact LoRA not an oracle?The denoising audit shows that compressed sketches can sometimes outperform the exact update, indicating that exact LoRA is not always optimal for behavioral contracts\.
- •What is the role of the audit?The behavioral audit is essential: spectral energy is predictive but not sufficient, so every candidate rung is checked for contract satisfaction\.
- •How robust is the method?The difficult subset of edits that require higher rank is small and stable across datasets andNN, supporting the scalability of the approach\.
- •What about overfitting and scalability?Thresholds and audit parameters are tuned on validation data, and the audit is efficient because most edits are handled by low\-rank sketches\.

### F\.3Baseline Details

We group the compared methods by the role they play in the lifelong editing pipeline\. This grouping is used only for presentation; all methods are evaluated under the same edit streams and benchmark metrics whenever applicable\.

#### Direct parameter editing\.

Fine\-tuning \(FT\) updates model parameters directly on the rewrite objective and serves as a simple lower\-bound baseline for sequential editing\. ROME[Meng et al\. \(2022\)](https://arxiv.org/html/2610.11160#bib.bib2)performs localized rank\-one updates to factual associations in transformer feed\-forward layers\. MEMIT[Meng et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib3)extends this style of localized parameter intervention to batch editing\. These methods are not designed as persistent edit\-memory systems, but they are included because they are standard model\-editing references and expose the interference that can accumulate under long edit streams\.

#### Memory\- and routing\-based lifelong editors\.

GRACE[Hartvigsen et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib4)stores edits in a discrete key\-value memory and routes queries to stored correction behavior\. WISE[Wang et al\. \(2024a\)](https://arxiv.org/html/2610.11160#bib.bib5)separates edited knowledge into an explicit side memory\. MEMOIR[Wang et al\. \(2025\)](https://arxiv.org/html/2610.11160#bib.bib7)focuses on minimal overwrite and informed retention for lifelong editing\. These methods reduce direct overwriting of the base model, but their routing, retention, or memory mechanisms are part of the editor itself\. We therefore evaluate them with their native routing policies rather than imposing LadderEdit’s edit\-association protocol\.

#### Constrained\-update methods\.

AlphaEdit[Fang et al\. \(2025\)](https://arxiv.org/html/2610.11160#bib.bib6)constrains edits in a null space to reduce interference with unrelated behavior\. It represents a constrained parameter\-update family rather than an edit\-local memory bank\. We include it to compare against methods that improve stability by restricting update directions instead of preserving isolated per\-edit representations\.

#### LoRA and adapter\-based lifelong editors\.

MELO[Yu et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib9)uses neuron\-indexed dynamic LoRA modules for model editing\. ELDER[Li et al\. \(2024a\)](https://arxiv.org/html/2610.11160#bib.bib8)enhances lifelong editing with a mixture\-of\-LoRA mechanism\. MEMoE[Wang and Li \(2024b\)](https://arxiv.org/html/2610.11160#bib.bib25)and LEMoE[Wang and Li \(2024a\)](https://arxiv.org/html/2610.11160#bib.bib24)use mixture\-of\-experts style adapter mechanisms for sequential editing\. These methods are closest to LadderEdit in using parameter\-efficient edit representations, but they primarily address how edits are routed, combined, or specialized\. LadderEdit instead assumes an edit\-local representation has already been acquired and studies how much of that representation must be retained persistently\.

#### Adaptive\-rank LoRA\.

AdaLoRA[Zhang et al\. \(2023\)](https://arxiv.org/html/2610.11160#bib.bib27)adaptively reallocates low\-rank budget during training\. It is included because it is the closest training\-time rank\-allocation baseline\. However, AdaLoRA solves a different problem from LadderEdit: it decides where rank should be spent across layers or modules while fitting an adapter, whereas LadderEdit decides how much resolution an already acquired edit\-local memory should retain in a growing stream\. Running AdaLoRA independently for each edit still leaves one stored adapter per edit and no stream\-level residual policy; running it as a shared adapter couples unrelated edits and reintroduces capacity competition\.

#### Full\-resolution edit\-memory reference\.

Exact LoRA[Hu et al\. \(2022\)](https://arxiv.org/html/2610.11160#bib.bib1)is our full\-resolution edit\-local memory reference\. Each edit is acquired as an exact LoRA\-style update and stored without compression\. This preserves edit\-local capacity but causes persistent memory to grow linearly with the number of edits\. LadderEdit uses the same edit acquisition and lookup association as exact LoRA, but replaces the returned full\-resolution update with the cheapest audited representation that preserves the measured rewrite, generalization, and locality contract\.

## Appendix GExtended Analysis of Scalability, Storage, and Practical Deployment

This section provides additional analyses of LadderEdit beyond the primary experiments\. We examine backbone scaling, strict memory\-matched comparisons, full\-resolution acquisition cost, long\-horizon storage growth, retrieval dependence, rank\-allocation efficiency, audit\-probe coverage, qualitative audit behavior, and implementation details\. Together, these experiments characterize the operating regime of LadderEdit and separate the effect of adaptive storage allocation from retrieval, cache selection, and initial edit acquisition\.

### G\.1Scaling to Larger Backbones

The main experiments use 7B–8B parameter backbones\. To test whether edit compression introduces an obvious scale\-dependent degradation, we additionally evaluated Qwen2\.5\-14B and Qwen2\.5\-32B\([Yang et al\., 2024](https://arxiv.org/html/2610.11160#bib.bib28);[Bai et al\., 2025](https://arxiv.org/html/2610.11160#bib.bib40);[Yang et al\., 2025](https://arxiv.org/html/2610.11160#bib.bib13)\)\. The writer, edit stream, and evaluation protocol are unchanged; only the backbone size is increased\.

Table 21:LadderEdit on larger Qwen2\.5 backbones\. Accuracy \(Acc\.\), generalization \(Gen\.\), locality \(Loc\.\), and their average are reported\. LadderEdit remains close to Exact LoRA as the backbone increases from 14B to 32B parameters\.The difference from Exact LoRA remains small at both scales: 0\.004 average points for Qwen2\.5\-14B and 0\.003 for Qwen2\.5\-32B\. Locality remains at 0\.999 in all cases\. These results indicate that the storage mechanism does not exhibit an obvious degradation when moving beyond the 7B–8B regime evaluated in the main experiments\.

### G\.2Exact\-Cache Baselines and Coverage Loss

A keep\-or\-drop exact cache provides a natural alternative under a strict storage budget: retained edits preserve their full LoRA adapters, while non\-retained edits fall back to the base model\. To separate cache quality from the inherent coverage limitation of this strategy, we evaluated multiple cache policies under the same budget of 91 retained adapters from a stream of 512 edits\.

Table 22:Full\-stream utility of exact\-cache policies under the same capacity of 91 retained adapters out of 512 edits\. Learned retention with base\-model fallback is the strongest exact\-cache baseline\.The strongest learned cache is more than twice as effective as random retention, showing that cache selection itself matters substantially\. However, the retained\-only control reveals that the remaining difference is primarily a coverage effect\. Across cache policies, utility on the edits that remain in the cache is approximately 0\.93–0\.94, essentially matching LadderEdit on the same retained subset\. The full\-stream gap emerges because the remaining 421 edits no longer have an edit\-specific representation once they are evicted\.

The learned retention baseline uses recency and historical retrieval\-frequency features\. Its scoring model is fitted only on validation streams; test outcomes are not used to select retained edits\. We therefore use learned retention with base\-model fallback as the primary exact\-cache baseline, while random, FIFO, and LRU serve as additional controls\.

### G\.3Strict Memory\-Matched Frontier

The cache comparison above changes which edits remain represented\. We additionally construct a strict\-memory frontier in which all methods are compared under matched normalized persistent memory\. This isolates the value of maintaining a low\-cost representation for every edit from the value of adaptively assigning additional rank\.

Table 23:Strict\-memory comparison on CounterFact \(CF\) and ZsRE\. Utility is measured under approximately matched normalized persistent memory\.This comparison separates two sources of improvement\. First, replacing keep\-or\-drop caching with a rank\-1 representation for every edit increases utility by 0\.319 on CounterFact and 0\.345 on ZsRE, despite using less normalized memory\. Thus, preserving even a minimal edit\-specific representation substantially reduces the coverage loss created by eviction\.

Second, adaptive allocation remains beneficial after coverage is restored\. Relative to static mixed\-rank storage at approximately the same memory budget, LadderEdit improves utility by another 0\.047 on CounterFact and 0\.070 on ZsRE\. The first gain therefore comes from representing the full edit stream; the second comes from allocating additional resolution selectively rather than uniformly\.

### G\.4Full\-Resolution Acquisition Cost

LadderEdit is a*post\-acquisition storage controller*\. It does not reduce the optimization cost required to fit the initial full\-resolution LoRA adapter\. Exact LoRA and LadderEdit use the same writer, optimizer, edit stream, and edit identity\. The methods differ only in the representation retained after an edit has been acquired\.

This exact\-first design avoids forcing every edit into a fixed low\-rank representation before observing whether that representation is behaviorally sufficient\. Its cost is an additional compression and audit stage after fitting\.

Table 24:Measured insertion and memory cost atT=10,000T=10\{,\}000edits\. Persistent memory refers to the retained edit bank, whereas peak GPU memory measures transient working memory during insertion\.AtT=10,000T=10\{,\}000, the additional insertion cost is 38 ms per edit, corresponding to approximately 380 seconds, or 6\.3 minutes, over the complete stream\. This cost is paid once at insertion\. In exchange, the persistent edit bank is reduced from 100\.93 GB to 19\.44 GB, a reduction of 81\.49 GB or 80\.7%\. Peak working memory changes only slightly, from 18\.4 GB to 18\.8 GB\.

The rank proposal also limits the behavioral search overhead\. LadderEdit requires an average of 1\.21 audits per edit, compared with 8 audits for exhaustive rank evaluation\.

Insertion cost depends on stream length because maintenance of a large full\-rank adapter bank introduces increasing merge and housekeeping overhead\. LadderEdit is slower at short horizons, with the measured crossover occurring nearT=2,000T=2\{,\}000\. AtT=50,000T=50\{,\}000, measured insertion time is approximately 150 ms per edit for LadderEdit and 850 ms per edit for Exact LoRA\.

The practical claim is therefore not cheaper initial edit fitting\. The acquisition procedure is intentionally unchanged; the benefit comes from substantially lower persistent storage and lower long\-stream maintenance cost\.

### G\.5Long\-Horizon Storage Growth

LadderEdit reduces the*coefficient*of storage growth but does not make storage constant in the number of edits\. Every edit retains at least a rank\-1 sketch, so persistent storage remains linear in stream length\.

AtT=10,000T=10\{,\}000, Exact LoRA requires 100\.93 GB while LadderEdit requires 19\.44 GB, corresponding to a measured5\.19×5\.19\\timesreduction at this operating point\. Under the average per\-edit storage observed at this horizon, a 100 GB memory budget corresponds to approximately 9,900 Exact\-LoRA edits or 51,400 LadderEdit edits\. This is a budget\-equivalent estimate based on the measuredT=10,000T=10\{,\}000operating point; it does not assume that the ratio is constant at every horizon\.

Across the evaluated horizons, the reduction grows from approximately2\.0×2\.0\\timesatT=100T=100, to approximately5\.05\.0–5\.2×5\.2\\timesacross backbones atT=10,000T=10\{,\}000, and approximately11\.611\.6–12\.0×12\.0\\timesatT=50,000T=50\{,\}000\. The5\.2×5\.2\\timesoperating point reported in the main text corresponds specifically to LLaMA\-3\-8B atT=10,000T=10\{,\}000\. Thus, the observed range is approximately2×2\\times–12×12\\timesover the evaluated horizons rather than a single horizon\-independent factor\.

The lower growth coefficient is consistent with strong heterogeneity in the amount of resolution required by individual edits\.

Table 25:Diagnostic rates describing edit\-level rank requirements and compression behavior\. The rows are diagnostic properties and are not assumed to form a mutually exclusive partition\.Most edits therefore do not require their complete acquisition\-rank representation to be stored indefinitely\. LadderEdit assigns every edit a low\-cost representation and spends additional rank only on the subset for which the audit indicates that more detail is required\.

A truly bounded\-memory editor would additionally require retirement, consolidation, or merging\. LadderEdit instead addresses the preceding representation problem: given an edit that remains individually addressable, how much edit\-specific resolution should be retained? The method can therefore serve as a representation layer beneath a separate bounded\-memory policy, but constant\-memory behavior is outside the scope of the present study\.

### G\.6Retrieval and Edit Selection

The primary storage experiments use matched edit association to isolate representation quality from retrieval errors\. Under this protocol, Exact LoRA and LadderEdit receive the same selected edit identity, allowing the comparison to ask whether the selected edit can be represented more economically without violating its behavioral contract\.

To test whether the conclusion survives a realistic selection stage, we additionally replace matched association with learned dense retrieval\.

Table 26:Performance under matched association and learned retrieval\. Contriever is evaluated directly for both Exact LoRA and LadderEdit\. Additional LadderEdit results with E5\-large and frozen MPNet characterize sensitivity to the retriever\.With Contriever, both methods decrease from 0\.95 to 0\.92, while their relative behavior remains unchanged\. LadderEdit reaches 0\.91 with E5\-large and 0\.88 with frozen MPNet\. Because Exact LoRA was not independently evaluated with the latter two retrievers, these results are used only to measure LadderEdit’s retrieval sensitivity rather than to claim direct parity\.

These experiments clarify the decomposition of the system\. LadderEdit controls how an already\-selected edit is stored; it does not solve edit retrieval itself\. Its end\-to\-end performance therefore depends on the quality of the edit\-selection mechanism used at inference time\.

### G\.7Rank Proposal and Audit Efficiency

The components used by LadderEdit—low\-rank factorization, spectral statistics, and behavioral evaluation—are individually standard\. The central design problem is instead*edit\-level storage allocation*: determining how much rank each edit should retain while minimizing the number of behavioral evaluations required to reach that decision\.

We compare several rank\-proposal rules under the same audit procedure\.

Table 27:Rank\-proposal ablation\. The combined spectral\-tail and behavioral\-margin proposal approaches exhaustive rank search while requiring substantially fewer audits\.The combined proposal reaches 0\.91 utility, within 0\.02 of exhaustive rank search, while using 1\.21 rather than 8\.00 audits per edit, a6\.6×6\.6\\timesreduction in audit count\.

The two signals capture complementary information\. The spectral tail measures the amount of update energy removed by truncation, whereas the behavioral margin measures how much degradation the exact edit can tolerate before its contract is violated\. These quantities provide an empirical ranking heuristic rather than a theoretical guarantee of the minimum sufficient rank\.

### G\.8Audit\-Probe Coverage

Behavioral auditing requires a small set of probes that exercise the intended edit behavior\. Benchmark experiments use disjoint audit, validation, and test prompts\. Test prompts are never used to determine the retained rank\.

In a deployment setting, audit probes may be supplied together with an edit, generated from the requested rewrite, or obtained from an application\-specific regression suite\. LadderEdit assumes that such a representative probe set is available; without it, the controller cannot directly test whether compression preserves the intended behavior\.

To measure sensitivity to incomplete probe coverage, we subsample the semantic coverage of the audit set\.

Table 28:Sensitivity to semantic coverage of the behavioral audit probes\.Performance degrades gradually as semantic coverage becomes weaker\. At 87% coverage the overall average is already 0\.94, compared with 0\.95 at 99% coverage\. As an additional leakage control, an intentionally non\-disjoint audit configuration reaches 0\.99, whereas enforcing a strict audit/test separation yields 0\.95\. The latter is the relevant evaluation setting and shows that the reported performance does not require reuse of test prompts during rank selection\.

This analysis studies missing semantic coverage\. It does not evaluate mislabeled, systematically corrupted, or adversarial audit probes, which remain separate failure modes\.

### G\.9Qualitative Audit Outcomes

The behavioral audit produces four recurring outcomes that clarify why a fixed storage rank is suboptimal:

1. 1\.Sketch\-sufficient:a low\-rank sketch already preserves the required edit behavior, so storing additional directions is unnecessary\.
2. 2\.Detail\-needed:rank 1 violates the behavioral contract, but a higher\-rank representation, such as rank 2, restores the edit\.
3. 3\.Compression\-regularized:the compressed representation passes the behavioral contract even when the full acquired adapter fails on the same probe, indicating that truncation removed directions that were harmful for that behavior\.
4. 4\.Exact\-limited:even the full acquisition\-rank representation fails the audit, so increasing retained rank cannot repair the edit\.

A representative compression\-regularized ZsRE edit has the target*“architect\.”*The full Exact\-LoRA representation answers*“archaeologist,”*whereas the lower\-rank representation includes the correct target\. The audit therefore retains the compressed sketch rather than automatically preferring the maximum available rank\. Compression\-regularized behavior occurs in 4\.3% of the ZsRE diagnostics and 6\.7% of the CounterFact diagnostics\.

The detail\-needed cases provide the complementary behavior: rank 1 may discard directions that are required to preserve the edit, while rank 2 or another promoted representation succeeds\. The controller is therefore not based on the assumption that stronger compression is intrinsically beneficial\. Instead, compression level is selected according to measured behavioral sufficiency\.

### G\.10Correlation Between Compression and Behavioral Outcomes

We additionally examine whether the compression diagnostics correlate with the observed behavioral outcomes\. Using the rounded population proportions, the compression\-regularized subsets correspond to approximately 86 ZsRE edits and 134 CounterFact edits\. The resulting Pearson correlations are moderate: approximatelyr=0\.38r=0\.38for ZsRE andr=0\.41r=0\.41for CounterFact\.

Using these rounded subset sizes, the corresponding approximate 95% confidence intervals are

rZsRE=0\.38,CI95%≈\[0\.18,0\.55\],r\_\{\\text\{ZsRE\}\}=0\.38,\\qquad\\mathrm\{CI\}\_\{95\\%\}\\approx\[0\.18,\\,0\.55\],and

rCF=0\.41,CI95%≈\[0\.26,0\.54\]\.r\_\{\\text\{CF\}\}=0\.41,\\qquad\\mathrm\{CI\}\_\{95\\%\}\\approx\[0\.26,\\,0\.54\]\.
These correlations are consistent with the proposed interpretation but should not be read as causal evidence that low\-rank truncation itself produces the behavioral improvement\. The primary evidence for adaptive storage allocation instead comes from the budget\-matched frontier in Table[23](https://arxiv.org/html/2610.11160#A7.T23)and from the held\-out behavioral audits\. The correlation analysis is used only as a secondary diagnostic of the relationship between compression statistics and behavioral outcomes\.

### G\.11SVD Procedure and Memory Accounting

Compression is applied independently to each adapted LoRA module\. For a given module, LadderEdit forms that module’s effective update matrix and performs the corresponding singular\-value decomposition before truncating it to the selected retained rank\. Parameters from different layers are never concatenated into a single global matrix because the adapted modules have different dimensions and functional roles\.

For one edit, the compression procedure therefore consists of an*edit\-level compression pass*containing the required module\-wise decompositions\. Persistent storage is computed by summing the retained LoRA factors over all adapted modules\. This convention ensures that both compression and memory accounting are defined consistently across layers and backbones\.

The audit weights are fixed globally rather than tuned separately for each backbone\. Contract and predictor thresholds are selected per dataset using held\-out validation probes and are subsequently fixed across backbones, stream horizons, and test streams\.

### G\.12Scope of the Efficiency Claim

The extended experiments distinguish four different efficiency questions that can otherwise be conflated\.

First,edit acquisitionis intentionally unchanged: LadderEdit begins from the same fitted LoRA update as Exact LoRA and therefore does not claim to make the initial optimization cheaper\.

Second,persistent representationis substantially smaller\. At the directly measuredT=10,000T=10\{,\}000operating point, persistent storage decreases from 100\.93 GB to 19\.44 GB, corresponding to a5\.19×5\.19\\timesreduction\.

Third,storage remains linear in stream length\. LadderEdit lowers the per\-edit growth coefficient by exploiting heterogeneous rank requirements; it is not a constant\-memory editor\.

Fourth,selection and storage are separate\. Matched\-association experiments isolate the storage mechanism, while the Contriever experiment shows that the same behavior persists when a learned retrieval component is introduced\. Retrieval quality can affect end\-to\-end accuracy, but it does not alter the memory advantage of the retained representation once an edit is selected\.

Taken together, the results characterize LadderEdit as an adaptive representation layer for long edit streams: it preserves an edit\-specific representation for every stored edit, allocates high resolution only when behaviorally necessary, and substantially reduces persistent storage without introducing a corresponding degradation in edit accuracy, generalization, or locality\. “‘

### G\.13Summary

The rank\-compression analysis supports the core claim: most edits are highly compressible, and LadderEdit’s ladder allocation reduces memory without sacrificing reliability, generalization, or locality in most evaluated settings\. The behavioral audit ensures that only behaviorally hard edits are promoted to higher rank, and the diagnostics explain why this allocation improves over exact\-cache and other baselines\.