Modality-Decoupled Online Recursive Editing

# Modality-Decoupled Online Recursive Editing
Source: [https://arxiv.org/html/2605.20273](https://arxiv.org/html/2605.20273)
###### Abstract

Online model editing for multimodal large language models \(MLLMs\) requires assimilating a stream of corrections under tight compute and memory budgets\. Yet editors developed for text\-only LLMs often degrade on MLLMs: visually dominant activations skew the statistics that shape updates, causing*cross\-modal conflict*, while sequential writes become entangled in a shared edit space and amplify long\-horizon interference, causing*inter\-edit interference*\. To address these, we proposeM\-ORE, a modality\-decoupled online recursive editor for lifelong MLLM adaptation\. M\-ORE is derived from a unified proximal\-projection formulation and admits a closed\-form update with a Sherman\-Morrison recursion, yielding constant per\-edit overhead\. It maintains module\-wise locality statistics for the text stack and the visual projector to avoid visually dominated update shaping and performs continual updates in a fixed orthogonal low\-rank edit subspace via a Sherman\-Morrison recursion to mitigate long\-horizon interference\. Experiments on multiple MLLM backbones and online editing benchmarks show that our M\-ORE method consistently improves reliability, generality, and locality over strong baselines, while achieving favorable quality\-efficiency scaling\. Our code is publicly available at[https://github\.com/lab\-klc/M\-ORE](https://github.com/lab-klc/M-ORE)\.

Machine Learning, ICML

## 1Introduction

Large Language Models \(LLMs\) have become a foundation of natural language processing\(Touvronet al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib1); Zhaoet al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib2)\)\. Recent Multimodal Large Language Models \(MLLMs\) extend LLMs with visual perception and achieve strong performance on vision\-language tasks\(Linet al\.,[2024](https://arxiv.org/html/2605.20273#bib.bib3); Zhuet al\.,[2026](https://arxiv.org/html/2605.20273#bib.bib54); Lianget al\.,[2026](https://arxiv.org/html/2605.20273#bib.bib55); Shiet al\.,[2026](https://arxiv.org/html/2605.20273#bib.bib56); Luet al\.,[2026](https://arxiv.org/html/2605.20273#bib.bib57)\)\. However, their knowledge is encoded in parameters and thus remains static after training, reducing reliability as real\-world facts evolve\(Jianget al\.,[2026](https://arxiv.org/html/2605.20273#bib.bib58)\)\. Since full retraining is costly,*model editing*provides an efficient alternative by updating knowledge with minimal parameter changes while preserving general capabilities\(Menget al\.,[2022](https://arxiv.org/html/2605.20273#bib.bib6); Fanget al\.,[2025](https://arxiv.org/html/2605.20273#bib.bib7)\)\.

![Refer to caption](https://arxiv.org/html/2605.20273v1/x1.png)Figure 1:Overview of M\-ORE and its contrast to existing online MLLM editing paradigms\. M\-ORE decouples vision/text updates and performs continual writes in a fixed orthogonal low\-rank space\.Recent research in model editing has mainly focused on pure LLMs and achieved notable success\(Mitchellet al\.,[2022a](https://arxiv.org/html/2605.20273#bib.bib18),[b](https://arxiv.org/html/2605.20273#bib.bib11); Fanget al\.,[2025](https://arxiv.org/html/2605.20273#bib.bib7)\)\. However, directly applying text\-oriented editors to MLLMs is often ineffective: the visual modality and its coupling with text introduce pronounced modality discrepancies\(Shiet al\.,[2025](https://arxiv.org/html/2605.20273#bib.bib12); Chenet al\.,[2025b](https://arxiv.org/html/2605.20273#bib.bib5)\)\. Mainstream approaches \(e\.g\., ROME and AlphaEdit\(Menget al\.,[2022](https://arxiv.org/html/2605.20273#bib.bib6); Fanget al\.,[2025](https://arxiv.org/html/2605.20273#bib.bib7)\)\) typically optimize a global objective and implicitly assume that input representations share relatively homogeneous statistical properties; yet in MLLMs, visual representations often exhibit higher energy, thereby dominating the estimation of covariance and inducing*Cross\-modal Conflict*\. Moreover, practical systems require*online*editing under a stream of corrections\(Caoet al\.,[2026](https://arxiv.org/html/2605.20273#bib.bib13)\), where parameter\-modifying methods accumulate*inter\-edit interference*and drift\(Li and Chu,[2024](https://arxiv.org/html/2605.20273#bib.bib15)\), while parameter\-preserving methods incur edit\-growing computation or storage\(Chenet al\.,[2025a](https://arxiv.org/html/2605.20273#bib.bib14)\)\. As a result, existing MLLM editors rarely address both failure modes while retaining*constant\-overhead*online updates\.

To bridge this gap, we first conduct pilot analyses to diagnose the failure modes above\. We observe that*cross\-modal conflict*primarily arises from the mismatch in gradient\-energy scales across modalities and*statistical heterogeneity*, whereas*inter\-edit interference*stems from the competition between newly introduced edits and previously injected edits for representational capacity in the shared parameter space, causing*weight entanglement*\. Figure[1](https://arxiv.org/html/2605.20273#S1.F1)summarizes our high\-level design and contrasts it with prior paradigms\.

Driven by the preceding analysis, we propose Modality\-decoupled Online Recursive Editing \(M\-ORE\), a novel constant\-overhead framework for lifelong adaptation in MLLMs\. M\-ORE maintains separate locality statistics for different edited modules \(text layers vs\. visual projector\) to prevent*cross\-modal conflict*, and performs continual writes in a*fixed orthogonal*low\-rank subspace with a*Sherman\-Morrison*grounded closed\-form recursion\. This design yields efficient online updates by adaptively suppressing updates on heavily\-used coordinates in the fixed edit subspace, thereby reducing long\-range*inter\-edit entanglement*\. Experiments on representative MLLM backbones and online multimodal editing benchmarks demonstrate that M\-ORE achieves a better stability\-plasticity trade\-off than strong baselines under strictO​\(1\)O\(1\)per\-edit overhead\. Our main contributions are summarized as follows:

- •We identify two key failure modes when applying LLM\-based editors to MLLMs in the online setting:*cross\-modal conflict*induced by modality scale mismatch and*inter\-edit interference*induced by entangled sequential updates \(Section[3](https://arxiv.org/html/2605.20273#S3)\)\.
- •We propose M\-ORE, a modality\-decoupled editor with a Sherman–Morrison grounded closed\-form recursion in a fixed orthogonal low\-rank write space, enabling constant\-overhead online editing \(Section[4](https://arxiv.org/html/2605.20273#S4)\)\.
- •We conduct extensive online editing experiments on multiple MLLM backbones and benchmarks, demonstrating that M\-ORE consistently improves the stability\-plasticity trade\-off and achieves a quality\-efficiency scaling compared with strong baselines \(Section[5](https://arxiv.org/html/2605.20273#S5)\)\.

## 2Related Work

### 2\.1Model Editing

##### Model Editing for LLMs\.

LLM editing methods are commonly grouped into*parameter\-modifying*and*parameter\-preserving*paradigms\(Daiet al\.,[2022](https://arxiv.org/html/2605.20273#bib.bib21); Sinitsinet al\.,[2020](https://arxiv.org/html/2605.20273#bib.bib22); Zhanget al\.,[2024](https://arxiv.org/html/2605.20273#bib.bib26); Huanget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib23)\)\. Parameter\-modifying editors directly update internal weights, including \(i\)*locate\-then\-edit*methods such as ROME\(Menget al\.,[2022](https://arxiv.org/html/2605.20273#bib.bib6)\)and MEMIT\(Menget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib16)\), which localize knowledge\-bearing components and apply closed\-form low\-rank updates; AlphaEdit\(Fanget al\.,[2025](https://arxiv.org/html/2605.20273#bib.bib7)\)adds null\-space constraints, and DeltaEdit\(Caoet al\.,[2026](https://arxiv.org/html/2605.20273#bib.bib13)\)analyzes long\-horizon error accumulation; and \(ii\)*meta\-learning*editors such as KE\(De Caoet al\.,[2021](https://arxiv.org/html/2605.20273#bib.bib17)\)and MEND\(Mitchellet al\.,[2022a](https://arxiv.org/html/2605.20273#bib.bib18)\), which train hypernetworks to predict edit updates from gradients or error signals\. Parameter\-preserving approaches avoid changing the backbone, instead using retrieval/in\-context prompting \(e\.g\., IKE\(Zhenget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib37)\)\) or lightweight auxiliary models/modules \(e\.g\., SERAC\(Mitchellet al\.,[2022b](https://arxiv.org/html/2605.20273#bib.bib11)\), GRACE\(Hartvigsenet al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib19)\)\) to locally override behavior\. While effective for text\-only settings, these techniques often degrade in multimodal regimes where visual inputs introduce substantial heterogeneity and noise\(Chenet al\.,[2025b](https://arxiv.org/html/2605.20273#bib.bib5); Yuet al\.,[2024](https://arxiv.org/html/2605.20273#bib.bib24)\)\.

##### Model Editing for MLLMs\.

Editing MLLMs is still emerging\(Chenget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib4)\)\. Early studies mainly adapt LLM\-based editors to multimodal benchmarks\(Heet al\.,[2024](https://arxiv.org/html/2605.20273#bib.bib25); Zhanget al\.,[2024](https://arxiv.org/html/2605.20273#bib.bib26)\); MMEdit systematically evaluates such adaptations and shows that naive transfer struggles with coupled vision\-language representations\(Chenget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib4)\)\. Recent methods start to incorporate multimodal\-specific designs: VisEdit\(Chenet al\.,[2025b](https://arxiv.org/html/2605.20273#bib.bib5)\)uses attribution to identify and modify critical visual units; DualEdit\(Shiet al\.,[2025](https://arxiv.org/html/2605.20273#bib.bib12)\)introduces gated dual\-branch editing; LiveEdit\(Chenet al\.,[2025a](https://arxiv.org/html/2605.20273#bib.bib14)\)targets lifelong editing via expert composition\. Nevertheless, existing approaches typically incur edit\-growing retrieval/composition costs or accumulate interference under sequential weight updates, and a principled framework that simultaneously mitigates*cross\-modal conflict*and*inter\-edit interference*under strict online constraints remains underexplored\(Yaoet al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib27); Guet al\.,[2024](https://arxiv.org/html/2605.20273#bib.bib28)\)\.

### 2\.2Online Editing

##### Online editing for LLMs\.

Online \(or sequential\) editing extends single\-step correction to a stream of updates, requiring models to address the plasticity\-stability dilemma over time\(Mitchellet al\.,[2022b](https://arxiv.org/html/2605.20273#bib.bib11); Jianget al\.,[2025](https://arxiv.org/html/2605.20273#bib.bib29)\)\. In text\-only LLMs, retrieval\-based methods\(Chenet al\.,[2024](https://arxiv.org/html/2605.20273#bib.bib33)\)mitigate catastrophic forgetting by storing edits in external memory, but incur inference latency that grows linearly with the number of edits\(Wanget al\.,[2024b](https://arxiv.org/html/2605.20273#bib.bib30)\)\. Locate\-then\-edit approaches\(Menget al\.,[2022](https://arxiv.org/html/2605.20273#bib.bib6)\)are computationally efficient, yet often degrade severely in sequential settings\.

![Refer to caption](https://arxiv.org/html/2605.20273v1/x2.png)Figure 2:Cross\-modal conflict caused by modality mismatch \(BLIP2\-OPT\)\.Left:log\-variance density indicates higher\-energy visual features than text activations\.Right:attribution analysis shows visually dominated contributions, implying that shared global statistics bias updates toward the visual subspace and weaken textual preservation\. Results for other MLLMs are provided in the Appendix[B\.1](https://arxiv.org/html/2605.20273#A2.SS1)\.
##### Online editing for MLLMs\.

In contrast, online editing for MLLMs remains under\-explored\(Chenet al\.,[2025a](https://arxiv.org/html/2605.20273#bib.bib14)\)and faces additional challenges induced by modality heterogeneity\(Chenget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib4); Chenet al\.,[2025b](https://arxiv.org/html/2605.20273#bib.bib5); Shiet al\.,[2025](https://arxiv.org/html/2605.20273#bib.bib12)\)\. Current solutions largely rely on*parameter\-preserving*\. For example, LiveEdit\(Chenet al\.,[2025a](https://arxiv.org/html/2605.20273#bib.bib14)\)proposes a mixture\-of\-experts framework that generates an edit\-specific LoRA expert for each sample and routes queries at inference time\. However, such approaches incur substantial memory overhead as the number of edits increases\(Shiet al\.,[2025](https://arxiv.org/html/2605.20273#bib.bib12); Thedeet al\.,[2025](https://arxiv.org/html/2605.20273#bib.bib34)\)\. Crucially,*parameter\-modifying*online editing for MLLMs remains largely unexplored\(Liet al\.,[2025](https://arxiv.org/html/2605.20273#bib.bib36); Zhanget al\.,[2025](https://arxiv.org/html/2605.20273#bib.bib31); Denget al\.,[2025](https://arxiv.org/html/2605.20273#bib.bib32)\)\.

## 3Analysis on Online Sequential Editing

### 3\.1Preliminary

##### Multimodal Language Model\.

We consider an MLLMfθf\_\{\\theta\}with a vision encoderℰv\\mathcal\{E\}\_\{v\}, projector𝒫\\mathcal\{P\}, and LLMℳ\\mathcal\{M\}\. Given\(xv,xq\)\(x^\{v\},x^\{q\}\), wherextvx\_\{t\}^\{v\}is the visual input,xtqx\_\{t\}^\{q\}is the text query, and the model autoregressively predictsp​\(y∣xv,xq\)p\(y\\mid x^\{v\},x^\{q\}\)from embeddingsH=\[𝒫​\(ℰv​\(xv\)\),Embed​\(xq\)\]H=\[\\mathcal\{P\}\(\\mathcal\{E\}\_\{v\}\(x^\{v\}\)\),\\mathrm\{Embed\}\(x^\{q\}\)\]\. For transformer layerll, we adopt the standard residual decomposition\(Menget al\.,[2022](https://arxiv.org/html/2605.20273#bib.bib6); Fanget al\.,[2025](https://arxiv.org/html/2605.20273#bib.bib7)\):

hl=hl−1\+al\+ml,ml=Woutl​σ​\(Winl​γ​\(hl−1\+al\)\),\\small h^\{l\}=h^\{l\-1\}\+a^\{l\}\+m^\{l\},\\quad m^\{l\}=W\_\{\\mathrm\{out\}\}^\{l\}\\,\\sigma\\\!\\big\(W\_\{\\mathrm\{in\}\}^\{l\}\\,\\gamma\(h^\{l\-1\}\+a^\{l\}\)\\big\),\(1\)whereala^\{l\}andmlm^\{l\}denote the attention and FFN outputs,γ​\(⋅\)\\gamma\(\\cdot\)is LayerNorm, andσ​\(⋅\)\\sigma\(\\cdot\)is a nonlinearity\. Following the FFN key\-value view\(Gevaet al\.,[2021](https://arxiv.org/html/2605.20273#bib.bib20)\), we define

kl≜σ​\(Winl​γ​\(hl−1\+al\)\),vl≜ml=Woutl​kl\.k^\{l\}\\triangleq\\sigma\\\!\\big\(W\_\{\\mathrm\{in\}\}^\{l\}\\,\\gamma\(h^\{l\-1\}\+a^\{l\}\)\\big\),\\quad v^\{l\}\\triangleq m^\{l\}=W\_\{\\mathrm\{out\}\}^\{l\}k^\{l\}\.\(2\)Many parameter\-modifying editors rewrite knowledge by updatingWoutlW\_\{\\mathrm\{out\}\}^\{l\}; we denote it asWWwhen clear\.

##### Online Editing in MLLMs\.

We study online editing of an MLLMfθf\_\{\\theta\}that maps a multimodal inputx=\(xv,xq\)∈𝒳x=\(x^\{v\},x^\{q\}\)\\in\\mathcal\{X\}to an autoregressive output distribution over text\. Following the FFN key\-value view, we consider*parameter\-modifying*edits that update a selected FFN output matrixWWwhile keeping the remaining parameters fixed, i\.e\.,θ=\(θ0,W\)\\theta=\(\\theta\_\{0\},W\)withθ0\\theta\_\{0\}frozen\. At steptt, an edit request is a target pairet=\(xte,yte\)e\_\{t\}=\(x\_\{t\}^\{e\},y\_\{t\}^\{e\}\)such thatyte≠f\(θ0,Wt−1\)​\(xte\)y\_\{t\}^\{e\}\\neq f\_\{\(\\theta\_\{0\},W\_\{t\-1\}\)\}\(x\_\{t\}^\{e\}\)\. An editorME\\mathrm\{ME\}produces an additive update with bounded per\-edit cost:

Δ​Wt=ME​\(f\(θ0,Wt−1\),xte,yte\),Wt=Wt−1\+Δ​Wt,\\small\\Delta W\_\{t\}=\\mathrm\{ME\}\\\!\\left\(f\_\{\(\\theta\_\{0\},W\_\{t\-1\}\)\},x\_\{t\}^\{e\},y\_\{t\}^\{e\}\\right\),\\;W\_\{t\}=W\_\{t\-1\}\+\\Delta W\_\{t\},\(3\)sofθt=f\(θ0,Wt\)f\_\{\\theta\_\{t\}\}=f\_\{\(\\theta\_\{0\},W\_\{t\}\)\}andWt=W0\+∑i=1tΔ​WiW\_\{t\}=W\_\{0\}\+\\sum\_\{i=1\}^\{t\}\\Delta W\_\{i\}\.

A successful online editor should satisfy three criteria\(Chenget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib4); Chenet al\.,[2025a](https://arxiv.org/html/2605.20273#bib.bib14)\):*Reliability*\(correct on the edited request\),*Generality*\(holds under semantically equivalent text/visual variants\), and*Locality*\(minimal side effects on unrelated inputs\)\. We quantify locality by the distributional shift betweenfθtf\_\{\\theta\_\{t\}\}andfθt−1f\_\{\\theta\_\{t\-1\}\}on an irrelevant set𝒰t\\mathcal\{U\}\_\{t\}:

𝔼x∼𝒰t\[exp\(−KL\(pθt\(⋅\|x\)∥pθt−1\(⋅\|x\)\)\)\],\\small\\mathbb\{E\}\_\{x\\sim\\mathcal\{U\}\_\{t\}\}\\Big\[\\exp\\\!\\big\(\-\\mathrm\{KL\}\\big\(p\_\{\\theta\_\{t\}\}\(\\cdot\|x\)\\,\\\|\\,p\_\{\\theta\_\{t\-1\}\}\(\\cdot\|x\)\\big\)\\big\)\\Big\],\(4\)wherepθ\(⋅\|x\)p\_\{\\theta\}\(\\cdot\|x\)is the next\-token distribution under the same decoding prefix\. Full metric definitions are in Appendix[A\.2](https://arxiv.org/html/2605.20273#A1.SS2)\. We next present pilot analyses to motivate our design\.

![Refer to caption](https://arxiv.org/html/2605.20273v1/x3.png)\(a\)Top\-kkcoordinate overlap\.
![Refer to caption](https://arxiv.org/html/2605.20273v1/x4.png)\(b\)Weight cosine similarity\.
![Refer to caption](https://arxiv.org/html/2605.20273v1/x5.png)\(c\)Entanglement horizon vs\. lag\.
![Refer to caption](https://arxiv.org/html/2605.20273v1/x6.png)\(d\)Sequential forgetting\.

Figure 3:Inter\-edit interference induced by MEND on BLIP2\-OPT over 10 online sequential edits on the E\-IC test set\.

### 3\.2Cross\-modal Conflict

We identify*cross\-modal conflict*as a key failure mode when applying text\-oriented editors to MLLMs: visual representations typically have much larger scale and can dominate the second\-order statistics used for preconditioning or constraints, thereby biasing edits toward the visual subspace111Estimating full cross\-modal covariance isO​\(d2\)O\(d^\{2\}\)and impractical at MLLM scale; we therefore use diagonal variance and layer\-wise energy as simple second\-order proxies, which already reveal the scale mismatch that biases pooled statistics\.\.

Statistical heterogeneity\.We compute diagonal variances of textual hidden states across transformer layers and compare them with the projected visual representations\. Figure[2](https://arxiv.org/html/2605.20273#S2.F2)reveals a clear scale gap: variance magnitudes vary across text layers, while the visual projection consistently stays at a higher scale than early text layers\. As a result, pooling statistics across modalities is prone to visual\-dominated estimates and cross\-modal contamination\.

Energy dominance and propagation\.We further measure layer\-wise output energy averaged over the same edits\. Figure[2](https://arxiv.org/html/2605.20273#S2.F2)shows that the visual projection contributes the largest energy, and the MLP branch increasingly dominates in higher layers; deeper blocks also attend more strongly to salient image regions\. These patterns suggest that strong visual signals are amplified and propagated through shared blocks\. Taken together, these results motivate maintaining*separate*second\-order statistics for different edited modules \(text layers vs\. visual projection\), rather than sharing a single statistic across modalities\.

### 3\.3Inter\-edit Interference

Beyond single\-step reliability,*online*MLLM editing must absorb a stream of updates without overwriting previously injected knowledge\. We observe a common failure mode,*inter\-edit interference*, where successive edits become increasingly entangled in a shared parameter subspace, leading to accumulated drift and forgetting\.

Edits collapse into an “edit core”\.At each steptt, we extract the effective updateΔ​Wt\\Delta W\_\{t\}and compute pairwise overlap of the top\-kkupdated coordinates\. Figure[3\(a\)](https://arxiv.org/html/2605.20273#S3.F3.sf1)shows a clear transition: early edits activate diverse coordinates, whereas later edits repeatedly modify a compact, stable subset, indicating collapse into a shared*edit core*\.

Entanglement is predominantly co\-directional\.We further measure directional coupling via cosine similarity\. Figure[3\(b\)](https://arxiv.org/html/2605.20273#S3.F3.sf2)reveals mostly positive correlations that become strongly aligned in the late\-stage block, suggesting that interference is driven by co\-directional accumulation within the reused edit core rather than push–pull cancelations\. Consistently, Figure[4](https://arxiv.org/html/2605.20273#S3.F4)shows that MLLM hidden\-state distributions drift after only a few edits\.

Long\-range coupling explains forgetting\.Aggregating the above pairwise statistics by lagτ\\tauyields an*entanglement horizon*\(Figure[3\(c\)](https://arxiv.org/html/2605.20273#S3.F3.sf3)\): both overlap and cosine decay slowly and remain non\-trivial for distant edits\. This long\-range coupling provides a parameter\-level explanation for sequential degradation: as new edits continue to reuse and move along the same edit\-core subspace, earlier edits are progressively overwritten, matching the forgetting trends in Figure[3\(d\)](https://arxiv.org/html/2605.20273#S3.F3.sf4)\. Motivated by these observations, sequential updates should be geometrically de\-entangled\. We thus restrict edits to a fixed orthogonal low\-rank write space and apply a recursive rule with an implicit orthogonalization bias, discouraging repeated reuse of heavily\-occupied coordinates and mitigating collapse and drift\.

![Refer to caption](https://arxiv.org/html/2605.20273v1/x7.png)Figure 4:Sequential visualization of MEND\-induced hidden\-state shifts on BLIP2\-OPT across 100 consecutive locality samples\.

## 4Methodology

### 4\.1Unified Proximal Projection Principle

Existing MLLM editors are mostly adapted from*locate\-then\-edit*or*meta\-learning*paradigms, both of which are fragile in the online multimodal regime: reliable localization is costly, while meta\-training is expensive and sensitive to deployment\-time shifts\. We instead adopt a*unified optimization*view: each online edit should \(i\) fit the current request \(reliability/generality\) while \(ii\) minimizing side effects on unrelated inputs \(locality\)\. At steptt, letWt−1W\_\{t\-1\}denote the editable parametersandgt=∇Wt−1ℒeditg\_\{t\}=\\nabla\_\{W\_\{t\-1\}\}\\mathcal\{L\}\_\{\\text\{edit\}\}be the edit gradient,whereℒedit\\mathcal\{L\}\_\{\\text\{edit\}\}denotes the standard editing loss\(See Appendix[A\.3](https://arxiv.org/html/2605.20273#A1.SS3)\)\. We compute the update via a single*proximal projection*:

Δ​Wt=arg⁡minΔ​W⁡‖Δ​W\+η​gt‖F2\+Tr​\(Δ​W​Ct−1​Δ​W⊤\),\\small\\Delta W\_\{t\}=\\arg\\min\_\{\\Delta W\}\\ \\\|\\Delta W\+\\eta g\_\{t\}\\\|\_\{F\}^\{2\}\+\\mathrm\{Tr\}\\\!\\left\(\\Delta W\\,\{\\color\[rgb\]\{0,0,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@gray@stroke\{0\}\\pgfsys@color@gray@fill\{0\}C\_\{t\-1\}\}\\,\\Delta W^\{\\top\}\\right\),\(5\)where the first term is a one\-step descent surrogate for fitting the edit, and the quadratic formCt−1C\_\{t\-1\}penalizes updates along directions that are important for locality\.The locality statisticCt−1C\_\{t\-1\}is accumulated online from step11through stept−1t\-1, using a fixed\-size*step\-only*context setℬs\\mathcal\{B\}\_\{s\}at each stepss\(e\.g\., edit batch and a small locality sample at stepss\):

Ct−1=C0\+∑s=1t−1∑x∈ℬsks​\(x\)​ks​\(x\)⊤,C0=𝟎\.\\small\{\\color\[rgb\]\{0,0,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@gray@stroke\{0\}\\pgfsys@color@gray@fill\{0\}\\small C\_\{t\-1\}=C\_\{0\}\+\\sum\_\{s=1\}^\{t\-1\}\\sum\_\{x\\in\\mathcal\{B\}\_\{s\}\}k\_\{s\}\(x\)k\_\{s\}\(x\)^\{\\top\},\\quad C\_\{0\}=\\mathbf\{0\}\.\}\(6\)whereks​\(x\)k\_\{s\}\(x\)is the FFN key induced by multimodal inputxxat the edited module\. This projection view is closely related to the quadratic target\-matching formulation used by*locate\-then\-edit*methods \(Appendix[C](https://arxiv.org/html/2605.20273#A3)\)\.

### 4\.2Drift\-Free Low\-Rank Coordinate Subspace

MaintainingCt−1C\_\{t\-1\}in the full space is infeasible online\. We restrict edits to a low\-rank write interface with a*time\-invariant*coordinate system\. For each edited layerl∈ℒl\\in\\mathcal\{L\}:

W~\(l\)=W\(l\)\+Δ​W\(l\),Δ​W\(l\)=B\(l\)​A\(l\),\\small\\widetilde\{W\}^\{\(l\)\}=W^\{\(l\)\}\+\\Delta W^\{\(l\)\},\\quad\\Delta W^\{\(l\)\}=B^\{\(l\)\}A^\{\(l\)\},\(7\)whereA\(l\)∈ℝr×dA^\{\(l\)\}\\in\\mathbb\{R\}^\{r\\times d\}defines a fixed write coordinate system andB\(l\)∈ℝdout×rB^\{\(l\)\}\\in\\mathbb\{R\}^\{d\_\{\\text\{out\}\}\\times r\}is updated online\.

##### Frozen orthogonal coordinates\.

We orthogonally initialize and*freeze*A\(l\)A^\{\(l\)\}such that its rows are orthonormal:

A\(l\)​A\(l\)⊤=Ir,A\(l\)​frozen,B\(l\)​updated \(online\)\.\\small A^\{\(l\)\}\{A^\{\(l\)\}\}^\{\\top\}=I\_\{r\},\\quad A^\{\(l\)\}~\\text\{frozen\},\\quad B^\{\(l\)\}~\\text\{updated \(online\)\}\.\(8\)This fixed coordinate system enables us to carry the locality quadratic constraintCt−1C\_\{t\-1\}in Eq\. \([5](https://arxiv.org/html/2605.20273#S4.E5)\) into a stable low\-dimensional statistic, which can be accumulated online\.

##### Steady\-space locality features\.

Given buffer keyskt−1\(l\)​\(x\)∈ℝdk\_\{t\-1\}^\{\(l\)\}\(x\)\\in\\mathbb\{R\}^\{d\}, we project them into the steady coordinates,

zt−1\(l\)​\(x\)=A\(l\)​kt−1\(l\)​\(x\)∈ℝr\.z\_\{t\-1\}^\{\(l\)\}\(x\)=A^\{\(l\)\}k\_\{t\-1\}^\{\(l\)\}\(x\)\\in\\mathbb\{R\}^\{r\}\.\(9\)The full projected second\-order statistic would be∑x∈ℬt−1\(l\)zt−1\(l\)​\(x\)​zt−1\(l\)​\(x\)⊤\\small\\sum\_\{x\\in\\mathcal\{B\}\_\{t\-1\}^\{\(l\)\}\}z\_\{t\-1\}^\{\(l\)\}\(x\)z\_\{t\-1\}^\{\(l\)\}\(x\)^\{\\top\}\. To keep constant overhead, we summarize the step context with a rank\-one sketch obtained by masked pooling over the relevant token positions:z¯t−1\(l\)≜Mean​\(\{zt−1\(l\)​\(x\)∣x∈ℬt−1\(l\)\}\)∈ℝr\.\\bar\{z\}\_\{t\-1\}^\{\(l\)\}\\triangleq\\text\{Mean\}\\big\(\\\{z\_\{t\-1\}^\{\(l\)\}\(x\)\\mid x\\in\\mathcal\{B\}\_\{t\-1\}^\{\(l\)\}\\\}\\big\)\\in\\mathbb\{R\}^\{r\}\.We then maintain a compact steady\-space locality matrix:

St−1\(l\)=S0\(l\)\+∑s=1t−1z¯s\(l\)​\(z¯s\(l\)\)⊤,S0\(l\)=λ​Ir,λ\>0\.\\small\{\\color\[rgb\]\{0,0,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@gray@stroke\{0\}\\pgfsys@color@gray@fill\{0\}S\_\{t\-1\}^\{\(l\)\}=S\_\{0\}^\{\(l\)\}\+\\sum\_\{s=1\}^\{t\-1\}\\bar\{z\}\_\{s\}^\{\(l\)\}\(\\bar\{z\}\_\{s\}^\{\(l\)\}\)^\{\\top\},\\quad S\_\{0\}^\{\(l\)\}=\\lambda I\_\{r\},\\ \\lambda\>0\.\}\(10\)whereλ\\lambdaprovides numerical stability and sets the plasticity–locality trade\-off\. A formal derivation of the induced steady\-space quadratic form is provided in Appendix[D\.1](https://arxiv.org/html/2605.20273#A4.SS1)\.

### 4\.3Steady\-Subspace Recursive Least Squares Update

We now instantiate Eq\. \([5](https://arxiv.org/html/2605.20273#S4.E5)\) in the steady space withB\(l\)B^\{\(l\)\}as the editable variables\. LetGt\(l\)=∇Bt−1\(l\)ℒeditG\_\{t\}^\{\(l\)\}=\\nabla\_\{B^\{\(l\)\}\_\{t\-1\}\}\\mathcal\{L\}\_\{\\text\{edit\}\}be the corresponding gradient at edit steptt\. The current per\-layer update is computed using only the statistics accumulated before steptt:

arg⁡minΔ​B\(l\)⁡‖Δ​B\(l\)\+η​Gt\(l\)‖F2\+Tr​\(Δ​B\(l\)​St−1\(l\)​\(Δ​B\(l\)\)⊤\),\\small\\arg\\min\_\{\\Delta B^\{\(l\)\}\}\\ \\\|\\Delta B^\{\(l\)\}\+\\eta G\_\{t\}^\{\(l\)\}\\\|\_\{F\}^\{2\}\+\\mathrm\{Tr\}\\\!\\Big\(\\Delta B^\{\(l\)\}\\,S\_\{t\-1\}^\{\(l\)\}\\,\(\\Delta B^\{\(l\)\}\)^\{\\top\}\\Big\),\(11\)whose unique minimizer is the closed\-form write:

Δ​Bt\(l\)=−η​Gt\(l\)​\(I\+St−1\(l\)\)−1≜−η​Gt\(l\)​Pt−1\(l\)\.\\small\\Delta B\_\{t\}^\{\(l\)\}=\-\\eta\\,G\_\{t\}^\{\(l\)\}\\big\(I\+S\_\{t\-1\}^\{\(l\)\}\\big\)^\{\-1\}\\triangleq\-\\eta\\,G\_\{t\}^\{\(l\)\}P\_\{t\-1\}^\{\(l\)\}\.\(12\)Here,Pt−1\(l\)≜\(I\+St−1\(l\)\)−1P\_\{t\-1\}^\{\(l\)\}\\triangleq\(I\+S\_\{t\-1\}^\{\(l\)\}\)^\{\-1\}serves as the steady\-space preconditioner for the current edit at steptt\. SinceSt−1\(l\)S\_\{t\-1\}^\{\(l\)\}accumulates outer products of previous steady features,Pt−1\(l\)P\_\{t\-1\}^\{\(l\)\}suppresses updates on coordinates that have been repeatedly activated by past edits, yielding an implicit de\-entangling bias across edits\. After applying the current edit, the current step featurez¯t\(l\)\\bar\{z\}\_\{t\}^\{\(l\)\}is efficiently inserted into the recursive statistic via the Sherman–Morrison lemma, avoiding direct matrix inversion and yielding the preconditioner for the next edit stept\+1t\+1:

Pt\(l\)=Pt−1\(l\)−Pt−1\(l\)​z¯t\(l\)​\(z¯t\(l\)\)⊤​Pt−1\(l\)1\+\(z¯t\(l\)\)⊤​Pt−1\(l\)​z¯t\(l\),P0\(l\)=11\+λ​Ir\.\\small P\_\{t\}^\{\(l\)\}=P\_\{t\-1\}^\{\(l\)\}\-\\frac\{P\_\{t\-1\}^\{\(l\)\}\\bar\{z\}\_\{t\}^\{\(l\)\}\(\\bar\{z\}\_\{t\}^\{\(l\)\}\)^\{\\top\}P\_\{t\-1\}^\{\(l\)\}\}\{1\+\(\\bar\{z\}\_\{t\}^\{\(l\)\}\)^\{\\top\}P\_\{t\-1\}^\{\(l\)\}\\bar\{z\}\_\{t\}^\{\(l\)\}\},\\quad P\_\{0\}^\{\(l\)\}=\\frac\{1\}\{1\+\\lambda\}I\_\{r\}\.\(13\)Thus,Pt−1\(l\)P\_\{t\-1\}^\{\(l\)\}is used to computeΔ​Bt\(l\)\\Delta B\_\{t\}^\{\(l\)\}, while the updatedPt\(l\)P\_\{t\}^\{\(l\)\}is cached and used only from stept\+1t\+1onward\. Each active layer stores onlyPt\(l\)∈ℝr×rP\_\{t\}^\{\(l\)\}\\in\\mathbb\{R\}^\{r\\times r\}and updates it inO​\(r2\)O\(r^\{2\}\)time, yielding constant per\-edit overhead\. Derivations of Eq\. \([12](https://arxiv.org/html/2605.20273#S4.E12)\) and Eq\. \([13](https://arxiv.org/html/2605.20273#S4.E13)\) are in Appendix[D](https://arxiv.org/html/2605.20273#A4)\.

Table 1:Online editing results on E\-VQA and E\-IC for BLIP2\-OPT and LLaVA\-v1\.5 under different edit horizons\. “Rel\.”, “T/M\-Gen\.”, and “T/M\-Loc\.” abbreviate*Reliability*,*Generality*, and*Locality*\(for text/modal evaluations\), respectively\. The subscript of each method \(e\.g\.,1,100\) denotes the number of online edits performed\. Rows shaded in light purple indicate*parameter\-modifying*methods\. Due to space limitations, complete results are provided in Appendix[B\.2](https://arxiv.org/html/2605.20273#A2.SS2)\.Table 2:Ablation study of M\-ORE design choices on E\-IC task for LLaVA\-v1\.5\.Δ\\Deltaindicates the change in Average score\.
### 4\.4Implications of the Steady\-Space Recursion

Steady\-space RLS rule in Eq\. \([12](https://arxiv.org/html/2605.20273#S4.E12)\)–Eq\. \([13](https://arxiv.org/html/2605.20273#S4.E13)\) already implies the key design principles needed for online MLLM editing\.

##### How Does M\-ORE Resolve*Cross\-Modal Conflict*?

Cross\-modal conflict arises when heterogeneous modules \(visual projector vs\. text layers\) are forced to share statistics \(Section[3\.2](https://arxiv.org/html/2605.20273#S3.SS2)\)\.Our formulation avoids this issue by maintaining a separate preconditionerP\(l\)P^\{\(l\)\}for each edited module, so that vision\-side and text\-side statistics are fully decoupled\. High\-energy visual features are therefore confined to the projector statistics, preventing contamination of text\-side updates\.When formingz¯\(l\)\\bar\{z\}^\{\(l\)\}, we further apply modality\-specific masks and pool only the relevant token positions, preventing mixed\-token summaries inside MLLMs\.

##### How Does M\-ORE Mitigate*Inter\-Edit Interference*?

Inter\-edit interference is driven by repeatedly reusing a shared*edit core*\(Section[3\.3](https://arxiv.org/html/2605.20273#S3.SS3)\)\. In our fixed orthogonal write coordinates \(Eq\. \([8](https://arxiv.org/html/2605.20273#S4.E8)\)\), the accumulationSt−1\(l\)=λ​Ir\+∑i≤t−1z¯i\(l\)​\(z¯i\(l\)\)⊤S\_\{t\-1\}^\{\(l\)\}=\\lambda I\_\{r\}\+\\sum\_\{i\\leq\{t\-1\}\}\\bar\{z\}\_\{i\}^\{\(l\)\}\(\\bar\{z\}\_\{i\}^\{\(l\)\}\)^\{\\top\}increases the penalty on frequently used coordinates, and the resulting preconditionerPt−1\(l\)P\_\{t\-1\}^\{\(l\)\}adaptively suppresses writes into those occupied coordinates\. Consequently,Δ​Bt\(l\)=−η​Gt\(l\)​Pt−1\(l\)\\Delta B\_\{t\}^\{\(l\)\}=\-\\eta G\_\{t\}^\{\(l\)\}P\_\{t\-1\}^\{\(l\)\}discourages persistent reuse of the same edit subspace, reducing long\-horizon entanglement with constant overhead\.

## 5Experiments

We conduct experiments to address the following questions:

- •RQ1:How does M\-ORE compare with baselines in online MLLM editing, particularly in alleviating*cross\-modal conflict*and*inter\-edit interference*?
- •RQ2:Does M\-ORE preserve the edited model’s general capabilities on standard generalization evaluations?
- •RQ3:What are the time and space complexities of M\-ORE for each online edit compared to the baselines? Concretely, how does M\-ORE achieve a better quality–efficiency trade\-off under the same settings?
- •RQ4:Can M\-ORE effectively prevent shifts in the distribution of hidden representations after editing?

### 5\.1Experiment Setup

We briefly introduce the datasets, metrics, backbones and baselines,with full experimental setup and orthogonal\-basis implementation details in Appendices[A](https://arxiv.org/html/2605.20273#A1)and[A\.6](https://arxiv.org/html/2605.20273#A1.SS6)\.

##### Datasets & Metrics\.

Following\(Chenget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib4)\), we evaluate online editing on E\-VQA \(Editing VQA\) and E\-IC \(Editing Image Caption\), where E\-IC requires finer\-grained visual grounding\. We report*Reliability*,*Generality*, and*Locality*\(Section[3\.1](https://arxiv.org/html/2605.20273#S3.SS1)\), and further decompose Generality/Locality into text\- and image\-conditioned evaluations\.

##### MLLM Backbones & Baseline Editors\.

We use two representative MLLM backbones with distinct architectures and scales: BLIP2\-OPT \(2\.7B\)\(Liet al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib39)\)and LLaVA\-v1\.5 \(7B\)\(Liuet al\.,[2024](https://arxiv.org/html/2605.20273#bib.bib40)\)\. Since dedicated online MLLM editors remain limited, we follow MMEdit\-style evaluation\(Chenget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib4)\)and adapt widely\-used LLM editors for comparison\. We group baselines into*parameter\-modifying*methods \(FT\-L/FT\-M\(Chenget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib4)\), MEND\(Mitchellet al\.,[2022a](https://arxiv.org/html/2605.20273#bib.bib18)\), AlphaEdit\(Fanget al\.,[2025](https://arxiv.org/html/2605.20273#bib.bib7)\)\) and*parameter\-preserving*methods \(IKE\(Zhenget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib37)\), SERAC\(Mitchellet al\.,[2022b](https://arxiv.org/html/2605.20273#bib.bib11)\), LiveEdit\(Chenet al\.,[2025a](https://arxiv.org/html/2605.20273#bib.bib14)\)\)\.

Table 3:Effect of different write\-space basis initializations on E\-IC with LLaVA\-v1\.5\. We compare Gaussian, Xavier, data\-driven, and the proposed orthogonal initialization under short\- and long\-horizon online editing\. The data\-driven basis is initialized using activation statistics collected from the training set![Refer to caption](https://arxiv.org/html/2605.20273v1/x8.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x9.png)

Figure 5:Inter\-edit interference statistics of M\-ORE on BLIP2\-OPT over 10 online sequential edits\.![Refer to caption](https://arxiv.org/html/2605.20273v1/x10.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x11.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x12.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x13.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x14.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x15.png)

Figure 6:accuracy of the post\-edited LLaVA\-v1\.5 \(7B\) on six tasks used for MLLM general capability testing\.

### 5\.2Comprehensive Performance Comparison \(RQ1\)

##### Main Results\.

Table[1](https://arxiv.org/html/2605.20273#S4.T1)summarizes online editing results on multiple MLLM backbones, where long edit streams expose long\-horizon robustness beyond single\-edit success\. Overall, existing baselines exhibit two recurring failure patterns\.*Parameter\-modifying*methods, such as FT variants and MEND, can achieve strong single\-edit reliability but tend to accumulate drift under sequential edits, leading to severe degradation att=100t\{=\}100\. For example, MEND collapses to nearly zero reliability and generality on both backbones after long\-horizon editing\. AlphaEdit, while effective for text\-only LLMs, remains brittle in MLLMs, likely due to unreliable multimodal localization and second\-order statistics under modality heterogeneity\.*Parameter\-preserving*methods, such as SERAC, IKE, and LiveEdit, better avoid direct weight drift, but still struggle to maintain stable multimodal behavior as the edit horizon grows\.

In contrast, M\-ORE consistently maintains a stronger stability\-plasticity trade\-off under long edit horizons\. Undert=100t\{=\}100edits, M\-ORE100achieves the best average performance across both backbones and both tasks\. On BLIP2\-OPT, M\-ORE100improves over LiveEdit100by\+0\.54\+0\.54average points on E\-VQA and\+1\.89\+1\.89average points on E\-IC\. In particular, it improves E\-IC reliability from74\.6374\.63to83\.1483\.14and text generality from74\.3374\.33to78\.1778\.17, while keeping locality comparable\. On LLaVA\-v1\.5, the gains are more pronounced: M\-ORE100improves the average score over LiveEdit100by\+1\.79\+1\.79points on E\-VQA and\+7\.07\+7\.07points on E\-IC\. For E\-IC, it substantially improves reliability from78\.4978\.49to94\.6494\.64, text generality from78\.7778\.77to93\.2793\.27, and multimodal generality from65\.5065\.50to78\.6678\.66, demonstrating stronger long\-horizon edit retention and generalization\. Moreover, Figure[5](https://arxiv.org/html/2605.20273#S5.F5)shows no late\-stage “edit\-core” collapse for M\-ORE: cross\-step top\-kkoverlaps remain low and cosine couplings stay near zero, indicating more disentangled sequential updates and reduced long\-range drift/forgetting\.For additional experimental results, includingsensitivity analyses,extension to vision\-side editing, andcase studies, please refer to Appendix[B\.4](https://arxiv.org/html/2605.20273#A2.SS4),[B\.5](https://arxiv.org/html/2605.20273#A2.SS5), and[B\.6](https://arxiv.org/html/2605.20273#A2.SS6)\.

##### Ablation Studies\.

We conduct ablation experiments to validate M\-ORE’s key design choices, including the drift\-free orthogonal write space and the constant\-cost sketch/pooling scheme for estimating locality statistics\. Specifically, we consider the following variants:

- •w/o freezingA\(l\)A^\{\(l\)\}: This variant updates the write basisA\(l\)A^\{\(l\)\}online jointly withB\(l\)B^\{\(l\)\}, rather than keepingA\(l\)A^\{\(l\)\}fixed as in Eq\. \([8](https://arxiv.org/html/2605.20273#S4.E8)\)\. It removes the drift\-free coordinate constraint and allows the low\-rank write space to change over the edit stream\.
- •w/o pooling: This variant replaces the rank\-one sketch in Eq\. \([10](https://arxiv.org/html/2605.20273#S4.E10)\) with the full projected statistic∑x∈ℬt−1\(l\)zt−1\(l\)​\(x\)​zt−1\(l\)​\(x\)⊤\\sum\_\{x\\in\\mathcal\{B\}\_\{t\-1\}^\{\(l\)\}\}z\_\{t\-1\}^\{\(l\)\}\(x\)z\_\{t\-1\}^\{\(l\)\}\(x\)^\{\\top\}, and then updatesPt\(l\)P\_\{t\}^\{\(l\)\}\.

Table[2](https://arxiv.org/html/2605.20273#S4.T2)shows that*freezing*the orthogonal basisA\(l\)A^\{\(l\)\}is critical for long\-horizon stability: removing it consistently reduces the average performance, and the degradation grows with the edit horizon, most notably att=100t\{=\}100\. This indicates that a stable coordinate system is necessary to prevent sequential drift and mitigate accumulated inter\-edit interference\. In contrast, removing masked pooling when formingz¯t\(l\)\\bar\{z\}\_\{t\}^\{\(l\)\}does not degrade performance and can slightly improve the average score, suggesting that our rank\-one sketch is a robust constant\-overhead approximation to the full projected statistic rather than a fragile heuristic\.

##### Basis Initialization Analysis\.

To examine the robustness of the steady\-subspace design more explicitly, we compare different initializations of the fixed basisA\(l\)A^\{\(l\)\}\. As shown in Table[3](https://arxiv.org/html/2605.20273#S5.T3), random initializations such as Gaussian and Xavier are substantially less stable, especially under long edit horizons\. Although the data\-driven basis improves over random initialization, it still falls behind the orthogonal basis\. In contrast, the proposed orthogonal initialization achieves the best overall performance at botht=1t\{=\}1andt=100t\{=\}100, demonstrating that an orthonormal and drift\-free coordinate system provides a more reliable low\-rank write space for editing\. We further test five random seeds for the orthogonal basis\. As shown in Table[4](https://arxiv.org/html/2605.20273#S5.T4), M\-ORE remains highly stable across seeds, with average absolute deviations of only0\.070\.07on E\-VQA and0\.130\.13on E\-IC relative to the default seed 42\.

Table 4:Seed sensitivity of the orthogonal basis initialization on LLaVA\-v1\.5 witht=100t\{=\}100online edits\.

### 5\.3General Capability Evaluation \(RQ2\)

To assess whether editing preserves general\-purpose multimodal abilities, we evaluate the post\-edited LLaVA\-v1\.5 on representative categories from the MME benchmark\(Fuet al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib38)\)\. Since MME contains 14 categories, we report six representative ones here and defer the remaining results to Appendix[B\.3](https://arxiv.org/html/2605.20273#A2.SS3)\. The selected categories include:

- •Existence: tests whether a queried object/concept is present in the image\.
- •Count: evaluates counting by verifying the queried number of target objects\.
- •Position: measures spatial understanding of object locations and relative relations\.
- •OCR: assesses text recognition and grounding for image\-based questions\.
- •Commonsense Reasoning: probes image\-grounded commonsense inference beyond literal perception\.
- •Numerical Calculation: evaluates image\-grounded arithmetic based on numbers/formulas in the image\.

Figure[6](https://arxiv.org/html/2605.20273#S5.F6)shows that M\-ORE largely preserves general capabilities after editing, with stability comparable to the unedited model and AlphaEdit\. The advantage is most evident on reasoning\-heavy tasks \(e\.g\., OCR and Numerical Calculation\), where M\-ORE avoids the progressive degradation observed in other baselines\. In contrast, parameter\-modifying baselines suffer clear catastrophic forgetting under long horizons: MEND drops to near\-zero accuracy on Existence/Count after 100 edits, and FT\-M exhibits a steady decline across multiple categories\.

Table 5:Per\-edit computational and memory complexities and their dependence on edit stream lengthtt\. Time/edit accounts for editor\-specific updates and statistic maintenance, excluding the shared forward/backward cost for computing edit gradients\. Herer≪dr\\ll d,\|ℒ\|\|\\mathcal\{L\}\|denotes the number of edited layers, andpmemp\_\{\\text\{mem\}\}denotes the size of an edit\-specific stored item\. We do not include meta\-learning editors since they require an additional training stage\.![Refer to caption](https://arxiv.org/html/2605.20273v1/x16.png)Figure 7:Efficiency metrics of editors on LLaVA\-v1\.5 over the online edit sequencett\.Top:Edit latency \(raw and rolling mean\) \.Bottom:Incremental peak memory \(Δ\\Deltapeak mem\)\.
### 5\.4Online Editing Efficiency Analysis \(RQ3\)

We report the*editor\-specific*overhead per online edit\. Letr≪dr\\ll dbe the steady\-subspace rank,dd/doutd\_\{\\text\{out\}\}the FFN key/input and output dimensions, andℒ\\mathcal\{L\}the edited layers\. With a constant\-size online buffer, M\-ORE hasTM\-ORE≈O​\(\|ℒ\|​dout​r2\)T\_\{\\text\{M\-ORE\}\}\\approx O\\\!\\big\(\|\\mathcal\{L\}\|\\,d\_\{\\text\{out\}\}r^\{2\}\\big\),MM\-ORE≈O​\(\|ℒ\|​dout​r\)M\_\{\\text\{M\-ORE\}\}\\approx O\\\!\\big\(\|\\mathcal\{L\}\|\\,d\_\{\\text\{out\}\}r\\big\)\. Both of them are constant with respect to the edit stream lengthtt\.

Table[5](https://arxiv.org/html/2605.20273#S5.T5)summarizes per\-edit time/memory complexities of representative editors,with derivations provided in Appendix[E](https://arxiv.org/html/2605.20273#A5)\. M\-ORE achieves*constant*per\-edit time and memory without relying on dense second\-order solvers or edit\-growing storage, leading to a favorable long\-horizon quality\-efficiency trade\-off\. In contrast, while naive finetuning is cheaper per edit, it fails to preserve multimodal locality under sequential edits \(Table[1](https://arxiv.org/html/2605.20273#S4.T1)\)\. Beyond the complexity analysis, we examine the*empirical scaling*of editor\-specific overhead along the edit stream\. As shown in Figure[7](https://arxiv.org/html/2605.20273#S5.F7), M\-ORE incurs only a short warmup at the first edit \(one\-off initialization\), after which the per\-edit latency quickly plateaus and stays stable with increasingtt, matching the predictedO​\(1\)O\(1\)time\. Similarly,Δ\\Deltapeak mem remains flat without monotonic growth, indicating bounded editor state under a constant\-size buffer\.

![Refer to caption](https://arxiv.org/html/2605.20273v1/x17.png)\(a\)M\-ORE
![Refer to caption](https://arxiv.org/html/2605.20273v1/x18.png)\(b\)AlphaEdit

Figure 8:Sequential visualization of hidden\-state shifts on LLaVA across 500 consecutive locality samples\.
### 5\.5Representation Shift Analysis \(RQ4\)

We evaluate overfitting by measuring the hidden\-state distribution shift on locality samples after sequential edits\. Since these samples are unrelated to the edited requests, an effective editor should preserve their representations while incorporating new knowledge\. As shown in Figure[8](https://arxiv.org/html/2605.20273#S5.F8), M\-ORE keeps the post\-edit distributions after the 100th and 500th edits close to the pre\-edit distribution, with substantial overlap in both the scatter plots and marginal densities\. This indicates that M\-ORE does not globally distort the representation space under long edit horizons\. AlphaEdit also shows comparable representation stability, but M\-ORE achieves this while maintaining stronger long\-horizon editing performance in Table[1](https://arxiv.org/html/2605.20273#S4.T1)\.

## 6Conclusion

We studied online editing for multimodal large language models and identified two key obstacles that limit existing editors in practice:*cross\-modal conflict*induced by modality\-dependent scale mismatch, and*inter\-edit interference*caused by long\-horizon entanglement under sequential writes\. To address both under strict efficiency constraints, we proposed M\-ORE, a modality\-decoupled online recursive editor derived from a unified proximal\-projection view\. M\-ORE maintains module\-wise locality statistics for the text stack and the visual projector, and performs continual updates in a fixed orthogonal low\-rank edit subspace via a Sherman\-Morrison grounded closed\-form recursion, enabling constant per\-edit compute and memory\. Extensive experiments on representative MLLM backbones and lifelong multimodal editing benchmarks show that M\-ORE consistently improves reliability, generality, and locality while preserving general capabilities, achieving a favorable long\-horizon quality\-efficiency trade\-off\.

## Impact Statement

This work aims to improve the reliability and maintainability of multimodal large language models by enabling efficient, online updates under strict compute and memory budgets\. A practical benefit is that outdated or incorrect multimodal knowledge \(e\.g\., factual errors grounded in images or instructions\) can be corrected without expensive retraining, potentially reducing harmful misinformation and improving downstream system robustness\.

As with all model editing techniques, the same capability could be misused to inject undesired or misleading behaviors into deployed models\. We therefore emphasize the importance of controlled access, auditing, and provenance tracking for edits, as well as evaluating edits under both capability and safety criteria\. Our method is designed to preserve pre\-edit behaviors via locality constraints, but it does not replace broader safety measures such as content filtering, red\-teaming, and policy enforcement\.

We release implementation details to support reproducibility and future research on safe and accountable editing, including better monitoring of edit side effects, stronger verification of edit intent, and standardized benchmarks for multimodal online editing\.

## Acknowledgements

This work was supported in part by National Natural Science Foundation of China \(62476070\), Shenzhen Science and Technology Program \(JCYJ20241202123503005, GXWD20231128103232001, ZDSYS20230626091203008, KQTD20240729102154066\), Department of Science and Technology of Guangdong \(2024A1515011540\), National Key R&D Program of China \(SQ2024YFE0200592\), the Major Key Project of PCL under Grant PCL2025A10 and PCL2024A06, and in part by the Shenzhen Science and Technology Program under Grant RCJC20231211085918010\.

## References

- S\. Bai, Y\. Cai, R\. Chen, K\. Chen, X\. Chen, Z\. Cheng, L\. Deng, W\. Ding, C\. Gao, C\. Ge,et al\.\(2025\)Qwen3\-vl technical report\.arXiv preprint arXiv:2511\.21631\.Cited by:[§B\.1](https://arxiv.org/html/2605.20273#A2.SS1.p2.6)\.
- D\. Cao, Y\. Cai, Y\. Huang, X\. He, R\. Guo, G\. Liu, and G\. Sun \(2026\)On the superimposed noise accumulation problem in sequential knowledge editing of large language models\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 30139–30147\.Cited by:[§1](https://arxiv.org/html/2605.20273#S1.p2.1),[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px1.p1.1)\.
- Q\. Chen, C\. Wang, D\. Wang, T\. Zhang, W\. Li, and X\. He \(2025a\)Lifelong knowledge editing for vision language models with low\-rank mixture\-of\-experts\.InProceedings of the Computer Vision and Pattern Recognition Conference \(CVPR\),pp\. 9455–9466\.Cited by:[2nd item](https://arxiv.org/html/2605.20273#A1.I2.i2.p1.1),[§A\.5](https://arxiv.org/html/2605.20273#A1.SS5.SSS0.Px4.p1.1),[§A\.5](https://arxiv.org/html/2605.20273#A1.SS5.SSS0.Px6.p1.1),[§A\.6](https://arxiv.org/html/2605.20273#A1.SS6.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2605.20273#S1.p2.1),[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px2.p1.1),[§2\.2](https://arxiv.org/html/2605.20273#S2.SS2.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2605.20273#S3.SS1.SSS0.Px2.p2.3),[§5\.1](https://arxiv.org/html/2605.20273#S5.SS1.SSS0.Px2.p1.1)\.
- Q\. Chen, T\. Zhang, X\. He, D\. Li, C\. Wang, L\. Huang, and H\. Xue’ \(2024\)Lifelong knowledge editing for LLMs with retrieval\-augmented continuous prompt learning\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 13565–13580\.Cited by:[§2\.2](https://arxiv.org/html/2605.20273#S2.SS2.SSS0.Px1.p1.1)\.
- Q\. Chen, T\. Zhang, C\. Wang, X\. He, D\. Wang, and T\. Liu \(2025b\)Attribution analysis meets model editing: advancing knowledge correction in vision language models with visedit\.InProceedings of the AAAI Conference on Artificial Intelligence \(AAAI\),pp\. 2168–2176\.Cited by:[§1](https://arxiv.org/html/2605.20273#S1.p2.1),[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px1.p1.1),[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px2.p1.1),[§2\.2](https://arxiv.org/html/2605.20273#S2.SS2.SSS0.Px2.p1.1)\.
- X\. Chen, H\. Fang, T\. Lin, R\. Vedantam, S\. Gupta, P\. Dollar, and C\. L\. Zitnick \(2015\)Microsoft coco captions: data collection and evaluation server\.arXiv preprint arXiv:1504\.00325\.Cited by:[2nd item](https://arxiv.org/html/2605.20273#A1.I1.i2.p1.1)\.
- S\. Cheng, B\. Tian, Q\. Liu, X\. Chen, Y\. Wang, H\. Chen, and N\. Zhang \(2023\)Can we edit multimodal large language models?\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 13877–13888\.Cited by:[1st item](https://arxiv.org/html/2605.20273#A1.I1.i1.p1.1.1),[2nd item](https://arxiv.org/html/2605.20273#A1.I1.i2.p1.1.1),[2nd item](https://arxiv.org/html/2605.20273#A1.I2.i2.p1.1),[§A\.5](https://arxiv.org/html/2605.20273#A1.SS5.SSS0.Px1.p1.1),[§A\.6](https://arxiv.org/html/2605.20273#A1.SS6.SSS0.Px1.p1.1),[§A\.6](https://arxiv.org/html/2605.20273#A1.SS6.SSS0.Px2.p1.2),[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px2.p1.1),[§2\.2](https://arxiv.org/html/2605.20273#S2.SS2.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2605.20273#S3.SS1.SSS0.Px2.p2.3),[§5\.1](https://arxiv.org/html/2605.20273#S5.SS1.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2605.20273#S5.SS1.SSS0.Px2.p1.1)\.
- D\. Dai, L\. Dong, Y\. Hao, Z\. Sui, B\. Chang, and F\. Wei \(2022\)Knowledge neurons in pretrained transformers\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 8493–8502\.Cited by:[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px1.p1.1)\.
- N\. De Cao, W\. Aziz, and I\. Titov \(2021\)Editing factual knowledge in language models\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),Cited by:[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px1.p1.1)\.
- J\. Deng, Z\. Wei, L\. Pang, H\. Ding, H\. Shen, and X\. Cheng \(2025\)Everything is editable: extend knowledge editing to unstructured data in large language models\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§2\.2](https://arxiv.org/html/2605.20273#S2.SS2.SSS0.Px2.p1.1)\.
- J\. Devlin, M\. Chang, K\. Lee, and K\. Toutanova \(2019\)BERT: pre\-training of deep bidirectional transformers for language understanding\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics\(NAACL\),pp\. 4171–4186\.Cited by:[§A\.5](https://arxiv.org/html/2605.20273#A1.SS5.SSS0.Px4.p1.1)\.
- Z\. Du, Y\. Qian, X\. Liu, M\. Ding, J\. Qiu, Z\. Yang, and J\. Tang \(2022\)GLM: general language model pretraining with autoregressive blank infilling\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 320–335\.Cited by:[1st item](https://arxiv.org/html/2605.20273#A1.I1.i3.I1.i1.p1.1)\.
- J\. Fang, H\. Jiang, K\. Wang, Y\. Ma, J\. Shi, X\. Wang, X\. He, and T\. Chua \(2025\)AlphaEdit: null\-space constrained knowledge editing for language models\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§A\.5](https://arxiv.org/html/2605.20273#A1.SS5.SSS0.Px3.p1.1),[§A\.6](https://arxiv.org/html/2605.20273#A1.SS6.SSS0.Px2.p1.2),[§1](https://arxiv.org/html/2605.20273#S1.p1.1),[§1](https://arxiv.org/html/2605.20273#S1.p2.1),[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2605.20273#S3.SS1.SSS0.Px1.p1.10),[§5\.1](https://arxiv.org/html/2605.20273#S5.SS1.SSS0.Px2.p1.1)\.
- C\. Fu, P\. Chen, Y\. Shen, Y\. Qin, M\. Zhang, X\. Lin, J\. Yang, X\. Zheng, K\. Li, X\. Sun, Y\. Wu, R\. Ji, C\. Shan, and R\. He \(2023\)MME: a comprehensive evaluation benchmark for multimodal large language models\.arXiv preprint arXiv:2306\.13394\.Cited by:[§B\.3](https://arxiv.org/html/2605.20273#A2.SS3.p1.1),[§5\.3](https://arxiv.org/html/2605.20273#S5.SS3.p1.1)\.
- M\. Geva, R\. Schuster, J\. Berant, and O\. Levy \(2021\)Transformer feed\-forward layers are key\-value memories\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 5484–5495\.Cited by:[§3\.1](https://arxiv.org/html/2605.20273#S3.SS1.SSS0.Px1.p1.14)\.
- Y\. Goyal, T\. Khot, D\. Summers\-Stay, D\. Batra, and D\. Parikh \(2017\)Making the V in VQA matter: elevating the role of image understanding in visual question answering\.InProceedings of the Computer Vision and Pattern Recognition Conference \(CVPR\),pp\. 6325–6334\.Cited by:[1st item](https://arxiv.org/html/2605.20273#A1.I1.i1.p1.1)\.
- J\. Gu, H\. Xu, J\. Ma, P\. Lu, Z\. Ling, K\. Chang, and N\. Peng \(2024\)Model editing harms general abilities of large language models: regularization to the rescue\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 16801–16819\.Cited by:[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px2.p1.1)\.
- T\. Hartvigsen, S\. Sankaranarayanan, H\. Palangi, Y\. Kim, and M\. Ghassemi \(2023\)Aging with GRACE: lifelong model editing with discrete key\-value adaptors\.InProceedings of the Advances in neural information processing systems \(NeurIPS\),Cited by:[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px1.p1.1)\.
- Y\. He, Z\. Liu, J\. Chen, Z\. Tian, H\. Liu, X\. Chi, R\. Liu, R\. Yuan, Y\. Xing, W\. Wang, J\. Dai, Y\. Zhang, W\. Xue, Q\. Liu, Y\. Guo, and Q\. Chen \(2024\)LLMs meet multimodal generation and editing: a survey\.arXiv preprint arXiv:2405\.19334\.Cited by:[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px2.p1.1)\.
- Z\. Huang, Y\. Shen, X\. Zhang, J\. Zhou, W\. Rong, and Z\. Xiong \(2023\)Transformer\-patcher: one mistake worth one neuron\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px1.p1.1)\.
- H\. Jiang, J\. Fang, T\. Zhang, B\. Bi, A\. Zhang, R\. Wang, T\. Liang, and X\. Wang \(2025\)Neuron\-level sequential editing for large language models\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 16678–16702\.Cited by:[§2\.2](https://arxiv.org/html/2605.20273#S2.SS2.SSS0.Px1.p1.1)\.
- K\. Jiang, Y\. Du, Y\. Ding, Y\. Ren, N\. Jiang, Z\. Gao, Z\. Zheng, L\. Liu, B\. Li, and Q\. Li \(2026\)When large multimodal models confront evolving knowledge: challenges and explorations\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2605.20273#S1.p1.1)\.
- T\. Kwiatkowski, J\. Palomaki, O\. Redfield, M\. Collins, A\. P\. Parikh, C\. Alberti, D\. Epstein, I\. Polosukhin, J\. Devlin, K\. Lee, K\. Toutanova, L\. Jones, M\. Kelcey, M\. Chang, A\. M\. Dai, J\. Uszkoreit, Q\. Le, and S\. Petrov \(2019\)Natural questions: a benchmark for question answering research\.Transactions of the Association for Computational Linguistics,pp\. 452–466\.Cited by:[2nd item](https://arxiv.org/html/2605.20273#A1.I1.i3.I1.i2.p1.1)\.
- J\. Li, D\. Li, S\. Savarese, and S\. C\. H\. Hoi \(2023\)BLIP\-2: bootstrapping language\-image pre\-training with frozen image encoders and large language models\.InProceedings of the International Conference on Machine Learning \(ICML\),pp\. 19730–19742\.Cited by:[2nd item](https://arxiv.org/html/2605.20273#A1.I2.i2.p1.1.1),[§5\.1](https://arxiv.org/html/2605.20273#S5.SS1.SSS0.Px2.p1.1)\.
- Q\. Li and X\. Chu \(2024\)Can we continually edit language models? on the knowledge attenuation in sequential model editing\.InFindings of the Association for Computational Linguistics \(ACL\),pp\. 5438–5455\.Cited by:[§1](https://arxiv.org/html/2605.20273#S1.p2.1)\.
- Z\. Li, H\. Jiang, H\. Chen, B\. Bi, Z\. Zhou, F\. Sun, J\. Fang, and X\. Wang \(2025\)Reinforced lifelong editing for language models\.InProceedings of the International Conference on Machine Learning \(ICML\),Cited by:[§2\.2](https://arxiv.org/html/2605.20273#S2.SS2.SSS0.Px2.p1.1)\.
- G\. Liang, Z\. Wang, J\. Hu, H\. Zhou, Z\. Xue, J\. Zhang, D\. Xu, and Q\. Yu \(2026\)Render\-in\-the\-loop: vector graphics generation via visual self\-feedback\.arXiv preprint arXiv:2604\.20730\.Cited by:[§1](https://arxiv.org/html/2605.20273#S1.p1.1)\.
- B\. Lin, Y\. Ye, B\. Zhu, J\. Cui, M\. Ning, P\. Jin, and L\. Yuan \(2024\)Video\-llava: learning united visual representation by alignment before projection\.InProceedings of the 2024 conference on empirical methods in natural language processing \(EMNLP\),pp\. 5971–5984\.Cited by:[§1](https://arxiv.org/html/2605.20273#S1.p1.1)\.
- H\. Liu, C\. Li, Y\. Li, and Y\. J\. Lee \(2024\)Improved baselines with visual instruction tuning\.InProceedings of the Computer Vision and Pattern Recognition Conference \(CVPR\),pp\. 26286–26296\.Cited by:[1st item](https://arxiv.org/html/2605.20273#A1.I2.i1.p1.1),[1st item](https://arxiv.org/html/2605.20273#A1.I2.i1.p1.1.1),[§5\.1](https://arxiv.org/html/2605.20273#S5.SS1.SSS0.Px2.p1.1)\.
- L\. Lu, X\. Chen, M\. Guo, S\. Li, J\. Wang, and Y\. Shi \(2026\)Chordedit: one\-step low\-energy transport for image editing\.arXiv preprint arXiv:2602\.19083\.Cited by:[§1](https://arxiv.org/html/2605.20273#S1.p1.1)\.
- K\. Marino, M\. Rastegari, A\. Farhadi, and R\. Mottaghi \(2019\)OK\-VQA: A visual question answering benchmark requiring external knowledge\.InProceedings of the Computer Vision and Pattern Recognition Conference \(CVPR\),pp\. 3195–3204\.Cited by:[2nd item](https://arxiv.org/html/2605.20273#A1.I1.i3.I1.i2.p1.1)\.
- K\. Meng, D\. Bau, A\. Andonian, and Y\. Belinkov \(2022\)Locating and editing factual associations in gpt\.InProceedings of the Advances in neural information processing systems \(NeurIPS\),pp\. 17359–17372\.Cited by:[§1](https://arxiv.org/html/2605.20273#S1.p1.1),[§1](https://arxiv.org/html/2605.20273#S1.p2.1),[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px1.p1.1),[§2\.2](https://arxiv.org/html/2605.20273#S2.SS2.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2605.20273#S3.SS1.SSS0.Px1.p1.10)\.
- K\. Meng, A\. S\. Sharma, A\. J\. Andonian, Y\. Belinkov, and D\. Bau \(2023\)Mass\-editing memory in a transformer\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px1.p1.1)\.
- E\. Mitchell, C\. Lin, A\. Bosselut, C\. Finn, and C\. D\. Manning \(2022a\)Fast model editing at scale\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§A\.5](https://arxiv.org/html/2605.20273#A1.SS5.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2605.20273#S1.p2.1),[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2605.20273#S5.SS1.SSS0.Px2.p1.1)\.
- E\. Mitchell, C\. Lin, A\. Bosselut, C\. D\. Manning, and C\. Finn \(2022b\)Memory\-based model editing at scale\.InProceedings of the International Conference on Machine Learning \(ICML\),pp\. 15817–15831\.Cited by:[§A\.5](https://arxiv.org/html/2605.20273#A1.SS5.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2605.20273#S1.p2.1),[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px1.p1.1),[§2\.2](https://arxiv.org/html/2605.20273#S2.SS2.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2605.20273#S5.SS1.SSS0.Px2.p1.1)\.
- OpenAI \(2023\)GPT\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[1st item](https://arxiv.org/html/2605.20273#A1.I2.i1.p1.1)\.
- R\. Rombach, A\. Blattmann, D\. Lorenz, P\. Esser, and B\. Ommer \(2022\)High\-resolution image synthesis with latent diffusion models\.InProceedings of the Computer Vision and Pattern Recognition Conference \(CVPR\),pp\. 10674–10685\.Cited by:[1st item](https://arxiv.org/html/2605.20273#A1.I1.i3.I1.i1.p1.1)\.
- Y\. Shi, Y\. Xie, M\. Guo, L\. Lu, M\. Huang, J\. Wang, Z\. Zhu, B\. Xu, and Z\. Huang \(2026\)MMErroR: a benchmark for erroneous reasoning in vision\-language models\.arXiv preprint arXiv:2601\.03331\.Cited by:[§1](https://arxiv.org/html/2605.20273#S1.p1.1)\.
- Z\. Shi, B\. Wang, C\. Si, Y\. Wu, J\. Kim, and H\. Pfister \(2025\)DualEdit: dual editing for knowledge updating in vision\-language models\.InProceedings of the Second Conference on Language Modeling \(COLM\),Cited by:[§1](https://arxiv.org/html/2605.20273#S1.p2.1),[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px2.p1.1),[§2\.2](https://arxiv.org/html/2605.20273#S2.SS2.SSS0.Px2.p1.1)\.
- A\. Sinitsin, V\. Plokhotnyuk, D\. V\. Pyrkin, S\. Popov, and A\. Babenko \(2020\)Editable neural networks\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px1.p1.1)\.
- L\. Thede, K\. Roth, M\. Bethge, Z\. Akata, and T\. Hartvigsen \(2025\)WikiBigEdit: understanding the limits of lifelong knowledge editing in llms\.InProceedings of the International Conference on Machine Learning \(ICML\),Cited by:[§2\.2](https://arxiv.org/html/2605.20273#S2.SS2.SSS0.Px2.p1.1)\.
- H\. Touvron, T\. Lavril, G\. Izacard, X\. Martinet, M\. Lachaux, T\. Lacroix, B\. Rozière, N\. Goyal, E\. Hambro, F\. Azhar,et al\.\(2023\)Llama: open and efficient foundation language models\.arXiv preprint arXiv:2302\.13971\.Cited by:[§1](https://arxiv.org/html/2605.20273#S1.p1.1)\.
- P\. Wang, S\. Bai, S\. Tan, S\. Wang, Z\. Fan, J\. Bai, K\. Chen, X\. Liu, J\. Wang, W\. Ge,et al\.\(2024a\)Qwen2\-vl: enhancing vision\-language model’s perception of the world at any resolution\.arXiv preprint arXiv:2409\.12191\.Cited by:[§B\.1](https://arxiv.org/html/2605.20273#A2.SS1.p2.6)\.
- P\. Wang, Z\. Li, N\. Zhang, Z\. Xu, Y\. Yao, Y\. Jiang, P\. Xie, F\. Huang, and H\. Chen \(2024b\)WISE: rethinking the knowledge memory for lifelong model editing of large language models\.InProceedings of the Advances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2\.2](https://arxiv.org/html/2605.20273#S2.SS2.SSS0.Px1.p1.1)\.
- Y\. Yao, P\. Wang, B\. Tian, S\. Cheng, Z\. Li, S\. Deng, H\. Chen, and N\. Zhang \(2023\)Editing large language models: problems, methods, and opportunities\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 10222–10240\.Cited by:[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px2.p1.1)\.
- L\. Yu, Q\. Chen, J\. Zhou, and L\. He \(2024\)MELO: enhancing model editing with neuron\-indexed dynamic lora\.InProceedings of the AAAI Conference on Artificial Intelligence \(AAAI\),pp\. 19449–19457\.Cited by:[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px1.p1.1)\.
- N\. Zhang, Y\. Yao, B\. Tian, P\. Wang, S\. Deng, M\. Wang, Z\. Xi, S\. Mao, J\. Zhang, Y\. Ni, S\. Cheng, Z\. Xu, X\. Xu, J\. Gu, Y\. Jiang, P\. Xie, F\. Huang, L\. Liang, Z\. Zhang, X\. Zhu, J\. Zhou, and H\. Chen \(2024\)A comprehensive study of knowledge editing for large language models\.arXiv preprint arXiv:2401\.01286\.Cited by:[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px1.p1.1),[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px2.p1.1)\.
- S\. Zhang, S\. Roller, N\. Goyal, M\. Artetxe, M\. Chen, S\. Chen, C\. Dewan, M\. Diab, X\. Li, X\. V\. Lin, T\. Mihaylov, M\. Ott, S\. Shleifer, K\. Shuster, D\. Simig, P\. S\. Koura, A\. Sridhar, T\. Wang, and L\. Zettlemoyer \(2022\)OPT: open pre\-trained transformer language models\.arXiv preprint arXiv:2205\.01068\.Cited by:[§A\.5](https://arxiv.org/html/2605.20273#A1.SS5.SSS0.Px4.p1.1)\.
- Z\. Zhang, W\. Zhou, J\. Zhao, and H\. Li \(2025\)Robust multimodal large language models against modality conflict\.InProceedings of the International Conference on Machine Learning \(ICML\),Cited by:[§2\.2](https://arxiv.org/html/2605.20273#S2.SS2.SSS0.Px2.p1.1)\.
- W\. X\. Zhao, K\. Zhou, J\. Li, T\. Tang, X\. Wang, Y\. Hou, Y\. Min, B\. Zhang, J\. Zhang, Z\. Dong,et al\.\(2023\)A survey of large language models\.arXiv preprint arXiv:2303\.18223\.Cited by:[§1](https://arxiv.org/html/2605.20273#S1.p1.1)\.
- C\. Zheng, L\. Li, Q\. Dong, Y\. Fan, Z\. Wu, J\. Xu, and B\. Chang \(2023\)Can we edit factual knowledge by in\-context learning?\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 4862–4876\.Cited by:[§A\.5](https://arxiv.org/html/2605.20273#A1.SS5.SSS0.Px5.p1.5),[§2\.1](https://arxiv.org/html/2605.20273#S2.SS1.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2605.20273#S5.SS1.SSS0.Px2.p1.1)\.
- C\. Zhu, Y\. Lin, S\. Chen, Y\. Wang, and J\. Lin \(2026\)Medeyes: learning dynamic visual focus for medical progressive diagnosis\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 13916–13924\.Cited by:[§1](https://arxiv.org/html/2605.20273#S1.p1.1)\.

## Appendix

## Appendix AExperimental Setup

In this section, we provide detailed descriptions of experimental setup, including introduction to datasets, explanation of evaluation metrics and editing objective, discussion of baseline methods and implementation details\.

### A\.1Datasets

- •E\-VQA\(Chenget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib4)\):Designed for rectifying errors in VQA\-v2\(Goyalet al\.,[2017](https://arxiv.org/html/2605.20273#bib.bib42)\), this dataset contains 6,346 training and 2,093 testing samples\. It requires MLLMs to analyze visual content alongside textual questions to produce precise responses\.
- •E\-IC\(Chenget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib4)\):Developed to correct descriptive errors in COCO Caption\(Chenet al\.,[2015](https://arxiv.org/html/2605.20273#bib.bib43)\), it consists of 2,849 training and 1,000 testing instances\. The task demands a comprehensive understanding of images to generate accurate captions\.
- •Evaluation Composition:In addition to the editing samples, the benchmark includes auxiliary samples for evaluating whether edits generalize to equivalent inputs and preserve unrelated knowledge: - –Generality:Textual generality is assessed using semantically rephrased text inputs generated by ChatGLM\(Duet al\.,[2022](https://arxiv.org/html/2605.20273#bib.bib45)\)for E\-VQA and manually written prompt templates for E\-IC\. Visual generality is assessed using reinterpreted images generated by Stable Diffusion 2\.1\(Rombachet al\.,[2022](https://arxiv.org/html/2605.20273#bib.bib44)\)\. - –Locality:Textual locality is evaluated using NQ\(Kwiatkowskiet al\.,[2019](https://arxiv.org/html/2605.20273#bib.bib47)\), and multimodal locality is evaluated using OK\-VQA\(Marinoet al\.,[2019](https://arxiv.org/html/2605.20273#bib.bib46)\), measuring whether unrelated knowledge remains unchanged after editing\.

### A\.2Evaluation Metrics

We consider an online edit stream\{et=\(xtv,xtq,yt\)\}t=1T\\\{e\_\{t\}=\(x\_\{t\}^\{v\},x\_\{t\}^\{q\},y\_\{t\}\)\\\}\_\{t=1\}^\{T\}, wherextvx\_\{t\}^\{v\}is the visual input,xtqx\_\{t\}^\{q\}is the text query, andyty\_\{t\}is the desired edited response\. Letfθtf\_\{\\theta\_\{t\}\}denote the model after applying thett\-th edit\. We use𝕀​\(⋅\)\\mathbb\{I\}\(\\cdot\)as the correctness indicator after answer normalization\.

##### Reliability \(Rel\.\)\.

Reliability measures whether the edited model produces the desired response on the edited request:

Rel=1T​∑t=1T𝕀​\(fθt​\(xtv,xtq\)=yt\)\.\\small\\mathrm\{Rel\}=\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}\\mathbb\{I\}\\\!\\left\(f\_\{\\theta\_\{t\}\}\(x\_\{t\}^\{v\},x\_\{t\}^\{q\}\)=y\_\{t\}\\right\)\.\(14\)

##### Generality \(T\-Gen\. / M\-Gen\.\)\.

Generality evaluates whether the edit generalizes to semantically equivalent variants of the edited request\. For each edit steptt, we define a text\-query neighborhood𝒢tq​\(xtq\)\\mathcal\{G\}\_\{t\}^\{q\}\(x\_\{t\}^\{q\}\), such as paraphrased queries, and a visual neighborhood𝒢tv​\(xtv\)\\mathcal\{G\}\_\{t\}^\{v\}\(x\_\{t\}^\{v\}\), such as benign image transformations or semantically equivalent visual inputs\. We report text\-side and multimodal\-side generality as:

T​\-​Gen\\displaystyle\\small\\mathrm\{T\\text\{\-\}Gen\}=1T​∑t=1T𝔼x~tq∼𝒢tq​\(xtq\)​\[𝕀​\(fθt​\(xtv,x~tq\)=yt\)\],\\displaystyle=\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}\\mathbb\{E\}\_\{\\tilde\{x\}\_\{t\}^\{q\}\\sim\\mathcal\{G\}\_\{t\}^\{q\}\(x\_\{t\}^\{q\}\)\}\\left\[\\mathbb\{I\}\\\!\\left\(f\_\{\\theta\_\{t\}\}\(x\_\{t\}^\{v\},\\tilde\{x\}\_\{t\}^\{q\}\)=y\_\{t\}\\right\)\\right\],\(15\)M​\-​Gen\\displaystyle\\small\\mathrm\{M\\text\{\-\}Gen\}=1T​∑t=1T𝔼x~tv∼𝒢tv​\(xtv\)​\[𝕀​\(fθt​\(x~tv,xtq\)=yt\)\]\.\\displaystyle=\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}\\mathbb\{E\}\_\{\\tilde\{x\}\_\{t\}^\{v\}\\sim\\mathcal\{G\}\_\{t\}^\{v\}\(x\_\{t\}^\{v\}\)\}\\left\[\\mathbb\{I\}\\\!\\left\(f\_\{\\theta\_\{t\}\}\(\\tilde\{x\}\_\{t\}^\{v\},x\_\{t\}^\{q\}\)=y\_\{t\}\\right\)\\right\]\.\(16\)

##### Locality \(T\-Loc\. / M\-Loc\.\)\.

Locality measures the absence of side effects on inputs unrelated to the current edit\. Let𝒰tq\\mathcal\{U\}\_\{t\}^\{q\}denote text\-side locality inputs, which are not semantically related to the edited queryxtqx\_\{t\}^\{q\}, and let𝒰tv\\mathcal\{U\}\_\{t\}^\{v\}denote visual\-side locality inputs, which are not semantically related to the edited visual inputxtvx\_\{t\}^\{v\}\. We quantify locality by the distributional shift between the post\-edit modelfθtf\_\{\\theta\_\{t\}\}and the pre\-edit modelfθt−1f\_\{\\theta\_\{t\-1\}\}using the KL divergence over next\-token distributions:

T​\-​Loc\\displaystyle\\small\\mathrm\{T\\text\{\-\}Loc\}=1T∑t=1T𝔼x~tq∼𝒰tq\[exp\(−KL\(pθt\(⋅\|x~tq\)∥pθt−1\(⋅\|x~tq\)\)\)\],\\displaystyle=\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}\\mathbb\{E\}\_\{\\tilde\{x\}\_\{t\}^\{q\}\\sim\\mathcal\{U\}\_\{t\}^\{q\}\}\\left\[\\exp\\\!\\left\(\-\\mathrm\{KL\}\\\!\\left\(p\_\{\\theta\_\{t\}\}\(\\cdot\|\\tilde\{x\}\_\{t\}^\{q\}\)\\,\\\|\\,p\_\{\\theta\_\{t\-1\}\}\(\\cdot\|\\tilde\{x\}\_\{t\}^\{q\}\)\\right\)\\right\)\\right\],\(17\)M​\-​Loc\\displaystyle\\small\\mathrm\{M\\text\{\-\}Loc\}=1T∑t=1T𝔼\(x~tv,x~tq\)∼𝒰tv\[exp\(−KL\(pθt\(⋅\|x~tv,x~tq\)∥pθt−1\(⋅\|x~tv,x~tq\)\)\)\]\.\\displaystyle=\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}\\mathbb\{E\}\_\{\(\\tilde\{x\}\_\{t\}^\{v\},\\tilde\{x\}\_\{t\}^\{q\}\)\\sim\\mathcal\{U\}\_\{t\}^\{v\}\}\\left\[\\exp\\\!\\left\(\-\\mathrm\{KL\}\\\!\\left\(p\_\{\\theta\_\{t\}\}\(\\cdot\|\\tilde\{x\}\_\{t\}^\{v\},\\tilde\{x\}\_\{t\}^\{q\}\)\\,\\\|\\,p\_\{\\theta\_\{t\-1\}\}\(\\cdot\|\\tilde\{x\}\_\{t\}^\{v\},\\tilde\{x\}\_\{t\}^\{q\}\)\\right\)\\right\)\\right\]\.\(18\)Herepθ\(⋅\|x\)p\_\{\\theta\}\(\\cdot\|x\)is the model’s next\-token distribution under the same decoding prefix, and higher values indicate better locality\.

### A\.3Edit objectiveℒedit\\mathcal\{L\}\_\{\\text\{edit\}\}\.

At edit steptt, we optimize the negative log\-likelihood of the desired edited responses:

ℒeditt=𝔼\(xtv,xtq,yt\)​\[−log⁡pθ​\(yt∣xtv,xtq\)\]\.\\small\\mathcal\{L\}\_\{\\text\{edit\}\}^\{t\}=\\mathbb\{E\}\_\{\(x\_\{t\}^\{v\},x\_\{t\}^\{q\},y\_\{t\}\)\}\\Big\[\-\\log p\_\{\\theta\}\\big\(y\_\{t\}\\mid x\_\{t\}^\{v\},x\_\{t\}^\{q\}\\big\)\\Big\]\.\(19\)

### A\.4MLLM Backbones

We employ several representative Multimodal Large Language Models \(MLLMs\) as our experimental backbones\. Specific model versions are detailed in the following\.

- •LLaVA222[https://huggingface\.co/liuhaotian/llava\-v1\.5\-7b](https://huggingface.co/liuhaotian/llava-v1.5-7b)\(Liuet al\.,[2024](https://arxiv.org/html/2605.20273#bib.bib40)\):LLaVA aligns vision and language by using an MLP\-based visual projector to map image features into the embedding space of LLaMA\(Liuet al\.,[2024](https://arxiv.org/html/2605.20273#bib.bib40)\)\. With GPT\-4\-generated instruction\-following data\(OpenAI,[2023](https://arxiv.org/html/2605.20273#bib.bib48)\), it shows strong fine\-grained visual reasoning and complex instruction following\.
- •BLIP\-2333[https://huggingface\.co/Salesforce/blip2\-opt\-2\.7b](https://huggingface.co/Salesforce/blip2-opt-2.7b)\(Liet al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib39)\):BLIP\-2 introduces a lightweight Q\-Former and a two\-stage pre\-training pipeline to bridge a frozen vision encoder and a frozen LLM\. Following\(Chenget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib4); Chenet al\.,[2025a](https://arxiv.org/html/2605.20273#bib.bib14)\), we adopt the BLIP2\-OPT variant, which prioritizes inference efficiency by compressing visual tokens before interfacing with OPT\.

### A\.5Baseline Editors

##### Fine\-tuning \(FT\)\.

We include two finetuning variants commonly used for MLLM editing: FT\-L and FT\-M\(Chenget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib4)\)\. FT\-L updates \(only\) the last layer of the language transformer, while FT\-M finetunes the visual encoder \(or vision\-side adaptation module\) for each edit sample, following the setup in prior work\(Chenget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib4)\)\.

##### MEND\.

Model Editor Networks with Decomposition\(Mitchellet al\.,[2022a](https://arxiv.org/html/2605.20273#bib.bib18)\)is a hypernetwork\-based editor that learns to predict parameter updates efficiently\. It trains a set of lightweight MLP hypernetworks that take decomposed backpropagated gradients on edit samples as input and output offsets to the target FFN parameters\. After editor\-specific training, these hypernetworks can generate per\-edit updates that satisfy the editing objective with relatively low runtime overhead\.

##### AlphaEdit\.

AlphaEdit\(Fanget al\.,[2025](https://arxiv.org/html/2605.20273#bib.bib7)\)is a plug\-and\-play, optimization\-based editor that achieves targeted updates while explicitly preserving pre\-edit behaviors\. Under a locate\-then\-edit pipeline, it first solves for a task\-specific weight update on a chosen subset \(e\.g\., FFNWoutW\_\{\\text\{out\}\}\), and then projects this update into the null space induced by a set of*retain*keys, enforcing invariance on those activations to control side effects\. This null\-space projection also helps reduce interference across sequential edits by keeping previously protected mappings unchanged\.

##### SERAC\.

Semi\-parametric Editing with a Retrieval\-Augmented Counterfactual\(Mitchellet al\.,[2022b](https://arxiv.org/html/2605.20273#bib.bib11)\)is a memory\-based editing method\. It stores edited samples and trains \(i\) a*scope classifier*to detect whether a query is related to previous edits, and \(ii\) a small*counterfactual model*to produce the modified response when the query falls within the edit scope\. Otherwise, the original model is used for generation\. Following the standard setup\(Chenet al\.,[2025a](https://arxiv.org/html/2605.20273#bib.bib14)\), we instantiate the scope classifier with BERT\(Devlinet al\.,[2019](https://arxiv.org/html/2605.20273#bib.bib50)\)and the counterfactual model with OPT\-125M\(Zhanget al\.,[2022](https://arxiv.org/html/2605.20273#bib.bib51)\)\.

##### IKE\.

In\-Context Knowledge Editing\(Zhenget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib37)\)performs editing by*retrieval\-augmented in\-context prompting*, without directly updating model parameters\. Given a target fact pair\(x∗,y∗\)\(x^\{\\ast\},y^\{\\ast\}\), IKE retrieveskkdemonstrationsC=\{c1,…,ck\}C=\\\{c\_\{1\},\\ldots,c\_\{k\}\\\}from a training set using an unsupervised retriever \(e\.g\., cosine similarity\), and concatenates them as in\-context examples to guide generation\. The demonstrations are ordered by similarity to the target, and the resulting augmented prompt aims to maximizeP​\(y∣x,f,C\)P\(y\\mid x,f,C\)for inputsxxthat fall within the scope of the target prompt\.

##### LiveEdit\.

LiveEdit\(Chenet al\.,[2025a](https://arxiv.org/html/2605.20273#bib.bib14)\)is designed for*lifelong*/*streaming*edits in vision–language models, aiming to maintain edit quality over long horizons\. Instead of repeatedly overwriting shared weights, it stores edits as lightweight low\-rank experts \(a low\-rank MoE\) and performs gated composition at inference, with routing that favors visually and textually relevant experts to mitigate inter\-edit interference\. These mechanisms promote stability and reduce cumulative drift under continuous updates\.

### A\.6Implementation Details

##### Aligned baseline protocols\.

For FT\-L, FT\-M, MEND, SERAC, and LiveEdit, we follow and align with the official editing protocol in LiveEdit/MMEdit\(Chenet al\.,[2025a](https://arxiv.org/html/2605.20273#bib.bib14); Chenget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib4)\), including the same edit\-stream construction, training/evaluation splits, and per\-edit optimization settings\. For IKE, we use the MMEdit\-aligned setup\(Chenget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib4)\)to ensure a fair comparison under identical edit scopes\.

##### AlphaEdit configuration and multimodalK0K\_\{0\}\.

For AlphaEdit, we adopt the hyperparameters recommended in the original paper\(Fanget al\.,[2025](https://arxiv.org/html/2605.20273#bib.bib7)\)\. To estimate the retain key setK0K\_\{0\}for MLLMs, we buildK0K\_\{0\}using samples from E\-VQA and E\-IC\(Chenget al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib4)\), so that both visual and textual knowledge are covered when constructing the null\-space constraint\. We restrict the editable modules to the last seven transformer layers of each MLLM, since these upper layers are the primary contribution layers identified in Section[3\.2](https://arxiv.org/html/2605.20273#S3.SS2)\.

##### M\-ORE configuration\.

We perform online updates with M\-ORE using the closed\-form solver \(Eq\. \([12](https://arxiv.org/html/2605.20273#S4.E12)\)\), without iterative optimization\. To make the setting more challenging, we use a per\-edit batch size of 1 throughout\. We apply model\-specific hyperparameters as follows:

- •LLaVA\-v1\.5\.We set the LoRA rank tor=512r=512withα/r=2\.0\\alpha/r=2\.0\. We use shared hyperparameters for the language and vision components:η=0\.1\\eta=0\.1andλ=2000\\lambda=2000\. We update the last seven transformer layers as well as the visual projection layer\. We initializeA\(l\)A^\{\(l\)\}usingtorch\.nn\.init\.orthogonal\. The online\-updated matrixB\(l\)B^\{\(l\)\}is initialized to zeros, ensuring the pre\-edit model remains unchanged at step 0\.
- •BLIP2\-OPT\.We set the LoRA rank tor=128r=128withα/r=2\.0\\alpha/r=2\.0\. For the language backbone, we useη=0\.06\\eta=0\.06andλ=5000\\lambda=5000; for the visual projection, we useηvis=0\.03\\eta\_\{\\text\{vis\}\}=0\.03andλvis=20000\\lambda\_\{\\text\{vis\}\}=20000\. We update the last seven transformer layers as well as the visual projection layer\. We use the same initialization strategy as above\.

##### Codebase\.

For fairness and reproducibility, we implement all methods \(including M\-ORE and baselines\) on top of the widely\-usedEasyEditediting framework444[https://github\.com/zjunlp/EasyEdit](https://github.com/zjunlp/EasyEdit), and make our modifications within its unified editing/evaluation pipeline\. All experiments are conducted on a single NVIDIA H20 GPU \(96GB\) with NVLink\. We additionally provide our M\-ORE implementation in the[https://github\.com/lab\-klc/M\-ORE](https://github.com/lab-klc/M-ORE)

## Appendix BSupplementary Experiments

### B\.1Cross\-modal Conflict

For the attribution analysis, we sample four instances from E\-IC and evaluate LLaVA\-v1\.5 and BLIP2\-OPT\. We quantify layer\-wise attributions for the self\-attention, MLP, and visual projector modules by aggregating their activation magnitudes across the four representative samples to derive the final contribution profiles\. We additionally compute diagonal activation variances \(log\-variance\) for textual layers and the projected visual representations\. We provide the corresponding cross\-modal conflict results for LLaVA\-v1\.5 in Figure[9](https://arxiv.org/html/2605.20273#A2.F9)\.

To further verify whether the identified cross\-modal conflict persists in more recent MLLM architectures, we additionally examine Qwen2\-VL\-7B\-Instruct\(Wanget al\.,[2024a](https://arxiv.org/html/2605.20273#bib.bib52)\)555[https://huggingface\.co/Qwen/Qwen2\-VL\-7B\-Instruct](https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct)and Qwen3\-VL\-8B\-Instruct\(Baiet al\.,[2025](https://arxiv.org/html/2605.20273#bib.bib53)\)666[https://huggingface\.co/Qwen/Qwen3\-VL\-8B\-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct)\. As shown in Table[6](https://arxiv.org/html/2605.20273#A2.T6), both models exhibit a clear modality\-scale mismatch: the visual projection layer produces substantially larger mean outputs than representative MLP layers\. Specifically, the visual projection output reaches17\.6717\.67on Qwen2\-VL and51\.5151\.51on Qwen3\-VL, while the representative MLP\-layer outputs are only in the ranges of4\.604\.60–11\.6611\.66and10\.0310\.03–25\.5025\.50, respectively\. These results indicate that the cross\-modal conflict is not specific to BLIP2\-OPT or LLaVA\-v1\.5, but remains observable in newer MLLM backbones\.

![Refer to caption](https://arxiv.org/html/2605.20273v1/x19.png)Figure 9:Cross\-modal conflict caused by modality mismatch \(LLaVA\-v1\.5\)\.Left:log\-variance density indicates higher\-energy visual features than text activations\.Right:attribution analysis shows visually dominated contributions, implying that shared global statistics bias updates toward the visual subspace and weaken textual preservation\.Table 6:Modality\-scale mismatch on recent MLLM backbones\. We report the mean output magnitudes of the visual projection layer and representative MLP layers\.
### B\.2Comprehensive Performance Comparison \(RQ1\)

We provide the complete online editing results on E\-VQA and E\-IC for BLIP2\-OPT and LLaVA\-v1\.5 under all edit horizons in Table[11](https://arxiv.org/html/2605.20273#A5.T11), which supplements the partial results reported in the main paper\.

### B\.3General Capability Evaluation \(RQ2\)

MME\(Fuet al\.,[2023](https://arxiv.org/html/2605.20273#bib.bib38)\)contains 14 evaluation categories\. The main paper reports six representative tasks for brevity, while we provide the remaining eight categories here to complete the benchmark:

- •Artwork: evaluates understanding of artistic images and stylized visual content\.
- •Celebrity: tests recognition and reasoning about well\-known public figures in images\.
- •Color: measures color perception and discrimination for queried objects/regions\.
- •Landmark: evaluates recognition of landmarks and related visual reasoning\.
- •Poster: assesses understanding of poster\-like images with dense visual\-textual layouts\.
- •Scene: tests holistic scene understanding and context\-aware reasoning\.
- •Text Translation: evaluates translating text appearing in images into the target language\.
- •Code Reasoning: probes reasoning over code\-like or structured text content presented visually\.

Figure[10](https://arxiv.org/html/2605.20273#A2.F10)summarizes the accuracy trajectories of post\-edited LLaVA\-v1\.5 across these additional tasks under the same evaluation protocol\.

### B\.4Hyperparameter Analysis

##### Sensitivity to rankrr\.

Figure[11](https://arxiv.org/html/2605.20273#A2.F11)studies the LoRA rankr∈\{128,256,512,1024\}r\\in\\\{128,256,512,1024\\\}\. Increasingrrimproves edit capacity and generally benefits multimodal generalization under long horizons, while the gains saturate beyond a moderate rank \(e\.g\.,r=512r\{=\}512\)\. Small ranks tend to underfit multimodal updates, consistent with insufficient write capacity\.

##### Sensitivity to locality regularizationλ\\lambda\.

Figure[12](https://arxiv.org/html/2605.20273#A2.F12)variesλ\\lambdain\{100,1000,2000,3000\}\\\{100,1000,2000,3000\\\}\. Smallλ\\lambdaleads to worse locality and lower average scores, indicating insufficient preservation of pre\-edit behavior, whereas strong regularization can slightly reduce plasticity\. A moderate range yields the best overall stability\-plasticity trade\-off across edit horizons\.

Table 7:Online E\-IC editing results of M\-ORE on recent other MLLM backbones\.

### B\.5Extension to More Intricate Vision\-Language Alignment Architectures

First, we evaluate online E\-IC editing on more recent MLLM backbones, including Qwen2\-VL\-7B\-Instruct and Qwen3\-VL\-8B\-Instruct\. Importantly, we directly reuse the hyperparameter setting from LLaVA\-v1\.5 without architecture\-specific retuning\. As shown in Table[7](https://arxiv.org/html/2605.20273#A2.T7), M\-ORE maintains strong performance across different edit horizons on both newer architectures\. Even aftert=100t\{=\}100edits, M\-ORE achieves average scores of89\.5089\.50on Qwen2\-VL and89\.9789\.97on Qwen3\-VL, indicating that its online recursive editing mechanism remains effective across modern MLLM architectures\.

Second, M\-ORE does not assume that the visual projector is a single homogeneous module\. Its decoupling principle is module\-wise: when a backbone contains multiple vision\-language alignment modules, M\-ORE can assign an independent locality statistic and preconditioner to each edited module\. This confines high\-energy visual features to the corresponding vision\-side statistics and prevents them from contaminating the update geometry of text\-side modules\. To verify this extension beyond the default projector\-plus\-text\-LLM setting, we further evaluate vision\-side parameter editing on LLaVA\-v1\.5 undert=100t\{=\}100edits\. Besides the default setting that edits the projector and text\-LLM modules, we consider two variants:\(1\) editing the vision encoder and projector only, and \(2\) editing the vision encoder, projector, text\-LLM jointly\. For the vision encoder, we edit only its last three layers\.

As shown in Tables[8](https://arxiv.org/html/2605.20273#A2.T8)and[9](https://arxiv.org/html/2605.20273#A2.T9), M\-ORE remains compatible with vision\-side editing\. Editing the vision encoder and projector alone preserves high text locality but yields lower reliability and generality, suggesting that vision\-side updates alone are insufficient to fully absorb multimodal factual corrections\. Jointly editing vision\-side modules and text\-LLM modules improves multimodal generality, especially on E\-VQA, but slightly reduces the overall average compared with the default setting\. These results support our design choice of applying module\-wise decoupled statistics to the projector and text\-LLM modules by default, while also showing that the same principle naturally extends to more intricate alignment architectures with multiple editable vision\-language modules\.

### B\.6Case Study

We provide qualitative case studies on both E\-IC and E\-VQA using LLaVA\-v1\.5, covering challenging edits that require fine\-grained visual grounding \(Figures[13](https://arxiv.org/html/2605.20273#A5.F13)–[16](https://arxiv.org/html/2605.20273#A5.F16)\)\. For each example, we report the target concept, the original prompt, the model’s outputs before/after editing, and a paraphrased prompt to probe text generality\. To visualize grounding and potential side effects, we additionally plot image\-space attribution/attention rollout maps for the pre\-edit model \(Base\) and the post\-edit model at representative layers \(L1,L16,L31\)\. Each group contains an edited sample \(top\) and an unrelated locality sample \(bottom\)\. We summarize the main qualitative findings as follows:

- •Edit success & paraphrase robustness\.M\-ORE consistently corrects the edited samples to the target responses, and the correction remains valid under paraphrased prompts\.
- •Locality preservation\.On the paired locality samples, the post\-edit model largely preserves the pre\-edit predictions, indicating limited side effects\.
- •Grounded attribution shifts\.Attribution maps show that, after editing, deeper layers place more mass on image regions supporting the updated concept\. In contrast, locality samples exhibit spatial patterns close to the pre\-edit baseline, suggesting the edit is grounded rather than inducing spurious global evidence shifts\.
- •Occasional locality flips\.In a small number of locality cases, the decoded output changes after editing\. To better understand the cause of these flips, we compare stable and flipped locality cases in Table[10](https://arxiv.org/html/2605.20273#A2.T10)\. As shown, flipped cases tend to have smaller pre\-edit top1–top2 margins, indicating that*they lie near fragile decoding boundaries and can be affected by small distributional shifts*\. For multimodal locality, flipped cases also show higher image similarity to the edited sample, suggesting that*imperfect visual irrelevance and residual visual coupling may contribute to occasional flips*\. In contrast, the low prompt token\-F1 suggests that these flips are not primarily caused by surface\-level textual overlap\.

Table 8:Vision\-side extension results on E\-VQA with LLaVA\-v1\.5 undert=100t\{=\}100online edits\.Table 9:Vision\-side extension results on E\-IC with LLaVA\-v1\.5 undert=100t\{=\}100online edits\.Table 10:Comparison between stable and flipped locality cases\. Pre top1–top2 margin measures pre\-edit decoding confidence; mean gold shift measures the distributional movement toward the locality reference answer; prompt token\-F1 measures lexical overlap between the edit prompt and locality prompt; image cosine measures visual similarity between the edit image and multimodal locality image\.![Refer to caption](https://arxiv.org/html/2605.20273v1/x20.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x21.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x22.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x23.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x24.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x25.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x26.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x27.png)

Figure 10:Accuracy of the post\-edited LLaVA\-v1\.5 \(7B\) on remaining eight tasks used for MLLM general capability testing\.![Refer to caption](https://arxiv.org/html/2605.20273v1/x28.png)Figure 11:Sensitivity analysis of the LoRA rank\.![Refer to caption](https://arxiv.org/html/2605.20273v1/x29.png)Figure 12:Sensitivity analysis of the regularization weightλ\\lambda\.

## Appendix CRelation to Locate\-then\-Editing Editors

We relate our proximal\-projection update \(Eq\. \([5](https://arxiv.org/html/2605.20273#S4.E5)\)\) to the classical*target\-matching*formulation that enforces\(W\+Δ​W\)​K≈V\(W\+\\Delta W\)K\\approx V, whereKKandVVstack the key and value vectors, respectively\. We establish a first\-order connection for the quadratic objective and explain why our formulation is better suited to online sequential editing\.

##### Target matching\.

LetW∈ℝdout×dW\\in\\mathbb\{R\}^\{d\_\{\\text\{out\}\}\\times d\}, keysK∈ℝd×nK\\in\\mathbb\{R\}^\{d\\times n\}, and desired valuesV∈ℝdout×nV\\in\\mathbb\{R\}^\{d\_\{\\text\{out\}\}\\times n\}\. A standard objective is

Δ​Wtm=arg⁡minΔ​W⁡‖\(W\+Δ​W\)​K−V‖F2\+‖Δ​W‖F2\.\\Delta W\_\{\\text\{tm\}\}=\\arg\\min\_\{\\Delta W\}\\ \\\|\(W\+\\Delta W\)K\-V\\\|\_\{F\}^\{2\}\+\\\|\\Delta W\\\|\_\{F\}^\{2\}\.\(20\)

##### Proximal view\.

Defineℒtm​\(W\)=‖W​K−V‖F2\\mathcal\{L\}\_\{\\text\{tm\}\}\(W\)=\\\|WK\-V\\\|\_\{F\}^\{2\}and let

g≜∇Wℒtm​\(W\)=2​\(W​K−V\)​K⊤,g\\triangleq\\nabla\_\{W\}\\mathcal\{L\}\_\{\\text\{tm\}\}\(W\)=2\(WK\-V\)K^\{\\top\},\(21\)where⟨A,B⟩≜Tr​\(A⊤​B\)\\langle A,B\\rangle\\triangleq\\mathrm\{Tr\}\(A^\{\\top\}B\)denotes the Frobenius inner product\. A gradient\-centered proximal step can be viewed as a special case of Eq\. \([5](https://arxiv.org/html/2605.20273#S4.E5)\) with an isotropic locality geometryC=IC=I:

Δ​Wprox=arg⁡minΔ​W⁡‖Δ​W\+η​g‖F2\+‖Δ​W‖F2=−η2​g\.\\Delta W\_\{\\text\{prox\}\}=\\arg\\min\_\{\\Delta W\}\\ \\\|\\Delta W\+\\eta g\\\|\_\{F\}^\{2\}\+\\\|\\Delta W\\\|\_\{F\}^\{2\}=\-\\frac\{\\eta\}\{2\}\\,g\.\(22\)
###### Lemma C\.1\(First\-order descent and preconditioned form\)\.

Assumeη\>0\\eta\>0\. For the quadratic lossℒtm\\mathcal\{L\}\_\{\\text\{tm\}\}, bothΔ​Wprox\\Delta W\_\{\\text\{prox\}\}andΔ​Wtm\\Delta W\_\{\\text\{tm\}\}are first\-order descent directions, i\.e\.,⟨g,Δ​Wprox⟩<0\\langle g,\\Delta W\_\{\\text\{prox\}\}\\rangle<0and⟨g,Δ​Wtm⟩<0\\langle g,\\Delta W\_\{\\text\{tm\}\}\\rangle<0wheneverg≠0g\\neq 0\. Moreover,Δ​Wtm\\Delta W\_\{\\text\{tm\}\}admits the closed form

Δ​Wtm=−12​g​\(K​K⊤\+I\)−1,\\Delta W\_\{\\text\{tm\}\}=\-\\tfrac\{1\}\{2\}\\,g\\,\(KK^\{\\top\}\+I\)^\{\-1\},\(23\)hence it is a*right\-preconditioned*gradient step\. In particular, ifK​K⊤≈c​IKK^\{\\top\}\\approx cI\(e\.g\., under approximately whitened keys\), thenΔ​Wtm\\Delta W\_\{\\text\{tm\}\}is approximately proportional togg\.

###### Proof\.

For any perturbationΔ​W\\Delta W, a first\-order Taylor expansion gives

ℒtm​\(W\+Δ​W\)=ℒtm​\(W\)\+⟨g,Δ​W⟩\+O​\(‖Δ​W‖F2\)\.\\mathcal\{L\}\_\{\\text\{tm\}\}\(W\+\\Delta W\)=\\mathcal\{L\}\_\{\\text\{tm\}\}\(W\)\+\\langle g,\\Delta W\\rangle\+O\(\\\|\\Delta W\\\|\_\{F\}^\{2\}\)\.From Eq\. \([22](https://arxiv.org/html/2605.20273#A3.E22)\),Δ​Wprox=−α​g\\Delta W\_\{\\text\{prox\}\}=\-\\alpha gwithα=η/2\>0\\alpha=\\eta/2\>0, so

⟨g,Δ​Wprox⟩=−α​‖g‖F2<0when​g≠0\.\\langle g,\\Delta W\_\{\\text\{prox\}\}\\rangle=\-\\alpha\\\|g\\\|\_\{F\}^\{2\}<0\\quad\\text\{when \}g\\neq 0\.
For Eq\. \([20](https://arxiv.org/html/2605.20273#A3.E20)\), the first\-order optimality condition is

2​\(\(W\+Δ​Wtm\)​K−V\)​K⊤\+2​Δ​Wtm=0,2\\big\(\(W\+\\Delta W\_\{\\text\{tm\}\}\)K\-V\\big\)K^\{\\top\}\+2\\Delta W\_\{\\text\{tm\}\}=0,which rearranges to

Δ​Wtm​\(K​K⊤\+I\)=\(V−W​K\)​K⊤,Δ​Wtm=−\(W​K−V\)​K⊤​\(K​K⊤\+I\)−1\.\\Delta W\_\{\\text\{tm\}\}\(KK^\{\\top\}\+I\)=\(V\-WK\)K^\{\\top\},\\;\\Delta W\_\{\\text\{tm\}\}=\-\(WK\-V\)K^\{\\top\}\(KK^\{\\top\}\+I\)^\{\-1\}\.Since\(K​K⊤\+I\)≻0\(KK^\{\\top\}\+I\)\\succ 0is invertible, this yields Eq\. \([23](https://arxiv.org/html/2605.20273#A3.E23)\)\. Finally,

⟨g,Δ​Wtm⟩=−12​Tr​\(g⊤​g​\(K​K⊤\+I\)−1\)<0when​g≠0\.\\langle g,\\Delta W\_\{\\text\{tm\}\}\\rangle=\-\\tfrac\{1\}\{2\}\\,\\mathrm\{Tr\}\\\!\\big\(g^\{\\top\}g\(KK^\{\\top\}\+I\)^\{\-1\}\\big\)<0\\quad\\text\{when \}g\\neq 0\.Since\(K​K⊤\+I\)−1≻0\(KK^\{\\top\}\+I\)^\{\-1\}\\succ 0, the proportionality remark follows from\(K​K⊤\+I\)−1≈1c\+1​I\(KK^\{\\top\}\+I\)^\{\-1\}\\approx\\frac\{1\}\{c\+1\}IwhenK​K⊤≈c​IKK^\{\\top\}\\approx cI\. ∎

###### Proposition C\.2\(Online suitability\)\.

At edit steptt, Eq\. \([5](https://arxiv.org/html/2605.20273#S4.E5)\) explicitly imposes a history\-dependent quadratic locality geometryTr​\(Δ​W​Ct−1​Δ​W⊤\)\\mathrm\{Tr\}\(\\Delta W\\,C\_\{t\-1\}\\,\\Delta W^\{\\top\}\), whereCt−1C\_\{t\-1\}contains only statistics accumulated from previous edit steps\. After the current edit, the step\-only context is inserted into the statistic to obtainCtC\_\{t\}for future edits\. This makes the update naturally suited to online sequential editing: the current write is protected by past locality information, while the current context is cached only for subsequent steps\. In contrast, a one\-shot target\-matching objective of the form Eq\. \([20](https://arxiv.org/html/2605.20273#A3.E20)\) does not encode this cross\-edit geometry unless one augments it with additional preservation terms, e\.g\., aCt−1C\_\{t\-1\}\-weighted quadratic penalty at steptt\. In that case, the resulting update becomes equivalent in spirit to our proximal\-projection formulation\.

## Appendix DDerivations for Proximal Projection and Steady\-Space RLS

We consider Eq\. \([5](https://arxiv.org/html/2605.20273#S4.E5)\) at edit steptt:

minΔ​W⁡‖Δ​W\+η​gt‖F2\+Tr​\(Δ​W​Ct−1​Δ​W⊤\),Ct−1⪰0\.\\min\_\{\\Delta W\}\\ \\\|\\Delta W\+\\eta g\_\{t\}\\\|\_\{F\}^\{2\}\+\\mathrm\{Tr\}\(\\Delta WC\_\{t\-1\}\\Delta W^\{\\top\}\),\\quad C\_\{t\-1\}\\succeq 0\.Using∇Δ​W‖Δ​W\+η​gt‖F2=2​\(Δ​W\+η​gt\)\\nabla\_\{\\Delta W\}\\\|\\Delta W\+\\eta g\_\{t\}\\\|\_\{F\}^\{2\}=2\(\\Delta W\+\\eta g\_\{t\}\)and, sinceCt−1C\_\{t\-1\}is symmetric PSD \(Ct⪰0C\_\{t\}\\succeq 0\),

∇Δ​WTr​\(Δ​W​Ct−1​Δ​W⊤\)=2​Δ​W​Ct−1,\\nabla\_\{\\Delta W\}\\mathrm\{Tr\}\(\\Delta WC\_\{t\-1\}\\Delta W^\{\\top\}\)=2\\Delta WC\_\{t\-1\},setting the gradient to zero yields

\(Δ​W\+η​gt\)\+Δ​W​Ct−1=0⇒Δ​W​\(I\+Ct−1\)=−η​gt⇒Δ​Wt=−η​gt​\(I\+Ct−1\)−1\.\(\\Delta W\+\\eta g\_\{t\}\)\+\\Delta WC\_\{t\-1\}=0\\ \\Rightarrow\\ \\Delta W\(I\+C\_\{t\-1\}\)=\-\\eta g\_\{t\}\\ \\Rightarrow\\ \\Delta W\_\{t\}=\-\\eta\\,g\_\{t\}\(I\+C\_\{t\-1\}\)^\{\-1\}\.SinceI\+Ct−1≻0I\+C\_\{t\-1\}\\succ 0, the minimizer is unique\.

### D\.1From Full\-Space Locality to the Steady Orthogonal Coordinates

Under the low\-rank interfaceΔ​W\(l\)=Δ​B\(l\)​A\(l\)\\Delta W^\{\(l\)\}=\\Delta B^\{\(l\)\}A^\{\(l\)\}, the locality quadratic form used at edit steptttransforms as

Tr​\(Δ​W\(l\)​Ct−1\(l\)​\(Δ​W\(l\)\)⊤\)\\displaystyle\\mathrm\{Tr\}\\\!\\left\(\\Delta W^\{\(l\)\}C\_\{t\-1\}^\{\(l\)\}\(\\Delta W^\{\(l\)\}\)^\{\\top\}\\right\)=Tr​\(Δ​B\(l\)​A\(l\)​Ct−1\(l\)​A\(l\)⊤​\(Δ​B\(l\)\)⊤\)\\displaystyle=\\mathrm\{Tr\}\\\!\\left\(\\Delta B^\{\(l\)\}A^\{\(l\)\}C\_\{t\-1\}^\{\(l\)\}\{A^\{\(l\)\}\}^\{\\top\}\(\\Delta B^\{\(l\)\}\)^\{\\top\}\\right\)=Tr​\(Δ​B\(l\)​C~t−1\(l\)​\(Δ​B\(l\)\)⊤\),\\displaystyle=\\mathrm\{Tr\}\\\!\\left\(\\Delta B^\{\(l\)\}\\widetilde\{C\}\_\{t\-1\}^\{\(l\)\}\(\\Delta B^\{\(l\)\}\)^\{\\top\}\\right\),where

C~t−1\(l\)≜A\(l\)​Ct−1\(l\)​A\(l\)⊤∈ℝr×r\.\\widetilde\{C\}\_\{t\-1\}^\{\(l\)\}\\triangleq A^\{\(l\)\}C\_\{t\-1\}^\{\(l\)\}\{A^\{\(l\)\}\}^\{\\top\}\\in\\mathbb\{R\}^\{r\\times r\}\.Moreover, withzs\(l\)​\(x\)=A\(l\)​ks\(l\)​\(x\)z\_\{s\}^\{\(l\)\}\(x\)=A^\{\(l\)\}k\_\{s\}^\{\(l\)\}\(x\)and

Ct−1\(l\)=∑s=1t−1∑x∈ℬs\(l\)ks\(l\)​\(x\)​ks\(l\)​\(x\)⊤,C\_\{t\-1\}^\{\(l\)\}=\\sum\_\{s=1\}^\{t\-1\}\\sum\_\{x\\in\\mathcal\{B\}\_\{s\}^\{\(l\)\}\}k\_\{s\}^\{\(l\)\}\(x\)\{k\_\{s\}^\{\(l\)\}\(x\)\}^\{\\top\},we have

C~t−1\(l\)=∑s=1t−1∑x∈ℬs\(l\)zs\(l\)​\(x\)​zs\(l\)​\(x\)⊤\.\\widetilde\{C\}\_\{t\-1\}^\{\(l\)\}=\\sum\_\{s=1\}^\{t\-1\}\\sum\_\{x\\in\\mathcal\{B\}\_\{s\}^\{\(l\)\}\}z\_\{s\}^\{\(l\)\}\(x\)\{z\_\{s\}^\{\(l\)\}\(x\)\}^\{\\top\}\.To keep constant per\-edit overhead, we approximate each step\-wise second\-order statistic by a rank\-one sketch using the pooled featurez¯s\(l\)\\bar\{z\}\_\{s\}^\{\(l\)\}and maintainSt−1\(l\)S\_\{t\-1\}^\{\(l\)\}via Eq\. \([10](https://arxiv.org/html/2605.20273#S4.E10)\)\.

### D\.2Closed Form and Sherman–Morrison Recursion in the Steady Space

At edit steptt, the current write uses the steady\-space statistic accumulated before the current step\. Consider Eq\. \([11](https://arxiv.org/html/2605.20273#S4.E11)\):

minΔ​B\(l\)⁡‖Δ​B\(l\)\+η​Gt\(l\)‖F2\+Tr​\(Δ​B\(l\)​St−1\(l\)​\(Δ​B\(l\)\)⊤\)\.\\min\_\{\\Delta B^\{\(l\)\}\}\\ \\\|\\Delta B^\{\(l\)\}\+\\eta G\_\{t\}^\{\(l\)\}\\\|\_\{F\}^\{2\}\+\\mathrm\{Tr\}\\\!\\left\(\\Delta B^\{\(l\)\}S\_\{t\-1\}^\{\(l\)\}\(\\Delta B^\{\(l\)\}\)^\{\\top\}\\right\)\.Taking derivatives gives

2​\(Δ​B\(l\)\+η​Gt\(l\)\)\+2​Δ​B\(l\)​St−1\(l\)=0,2\(\\Delta B^\{\(l\)\}\+\\eta G\_\{t\}^\{\(l\)\}\)\+2\\Delta B^\{\(l\)\}S\_\{t\-1\}^\{\(l\)\}=0,hence

Δ​Bt\(l\)=−η​Gt\(l\)​\(I\+St−1\(l\)\)−1\.\\Delta B\_\{t\}^\{\(l\)\}=\-\\eta\\,G\_\{t\}^\{\(l\)\}\(I\+S\_\{t\-1\}^\{\(l\)\}\)^\{\-1\}\.Define

Pt−1\(l\)≜\(I\+St−1\(l\)\)−1\.P\_\{t\-1\}^\{\(l\)\}\\triangleq\(I\+S\_\{t\-1\}^\{\(l\)\}\)^\{\-1\}\.Then the current write is

Δ​Bt\(l\)=−η​Gt\(l\)​Pt−1\(l\)\.\\Delta B\_\{t\}^\{\(l\)\}=\-\\eta\\,G\_\{t\}^\{\(l\)\}P\_\{t\-1\}^\{\(l\)\}\.
After applying the current write, the current step featurez¯t\(l\)\\bar\{z\}\_\{t\}^\{\(l\)\}is inserted into the recursive statistic:

St\(l\)=St−1\(l\)\+z¯t\(l\)​\(z¯t\(l\)\)⊤\.S\_\{t\}^\{\(l\)\}=S\_\{t\-1\}^\{\(l\)\}\+\\bar\{z\}\_\{t\}^\{\(l\)\}\(\\bar\{z\}\_\{t\}^\{\(l\)\}\)^\{\\top\}\.Therefore,

\(I\+St\(l\)\)=\(I\+St−1\(l\)\)\+z¯t\(l\)​\(z¯t\(l\)\)⊤\.\(I\+S\_\{t\}^\{\(l\)\}\)=\(I\+S\_\{t\-1\}^\{\(l\)\}\)\+\\bar\{z\}\_\{t\}^\{\(l\)\}\(\\bar\{z\}\_\{t\}^\{\(l\)\}\)^\{\\top\}\.Applying the Sherman–Morrison lemma withu=v=z¯t\(l\)u=v=\\bar\{z\}\_\{t\}^\{\(l\)\}yields

Pt\(l\)=Pt−1\(l\)−Pt−1\(l\)​z¯t\(l\)​\(z¯t\(l\)\)⊤​Pt−1\(l\)1\+\(z¯t\(l\)\)⊤​Pt−1\(l\)​z¯t\(l\),P\_\{t\}^\{\(l\)\}=P\_\{t\-1\}^\{\(l\)\}\-\\frac\{P\_\{t\-1\}^\{\(l\)\}\\bar\{z\}\_\{t\}^\{\(l\)\}\(\\bar\{z\}\_\{t\}^\{\(l\)\}\)^\{\\top\}P\_\{t\-1\}^\{\(l\)\}\}\{1\+\(\\bar\{z\}\_\{t\}^\{\(l\)\}\)^\{\\top\}P\_\{t\-1\}^\{\(l\)\}\\bar\{z\}\_\{t\}^\{\(l\)\}\},which is Eq\. \([13](https://arxiv.org/html/2605.20273#S4.E13)\)\. Thus,Pt−1\(l\)P\_\{t\-1\}^\{\(l\)\}is used to compute the current writeΔ​Bt\(l\)\\Delta B\_\{t\}^\{\(l\)\}, while the updatedPt\(l\)P\_\{t\}^\{\(l\)\}is cached for the next edit stept\+1t\+1\. Finally, becauseS0\(l\)=λ​IrS\_\{0\}^\{\(l\)\}=\\lambda I\_\{r\}withλ\>0\\lambda\>0and each update adds a PSD rank\-one matrix,I\+St\(l\)≻0I\+S\_\{t\}^\{\(l\)\}\\succ 0for alltt\. HencePt\(l\)P\_\{t\}^\{\(l\)\}is always well\-defined\. The recursion updates each layer inO​\(r2\)O\(r^\{2\}\)time and stores onlyPt\(l\)∈ℝr×rP\_\{t\}^\{\(l\)\}\\in\\mathbb\{R\}^\{r\\times r\}\.

## Appendix EComplexity Derivations and Details

##### Scope and accounting\.

We analyze the*editor\-specific*overhead per online edit step, i\.e\., \(i\) statistics maintained by the editor and \(ii\) parameter updates applied by the editor\. We*exclude*the forward/backward cost for computing edit gradients, since it is shared by parameter\-modifying editors under the same training objective and hardware settings\. Unless stated otherwise, we assume a constant\-size step\-only bufferℬt\\mathcal\{B\}\_\{t\}\(per layer\) withb≜\|ℬt\|=O​\(1\)b\\triangleq\|\\mathcal\{B\}\_\{t\}\|=O\(1\)\.

### E\.1Notation

Letl∈ℒl\\in\\mathcal\{L\}index the edited FFN layers\. Denote the FFN key/input dimension byddand the FFN output dimension bydoutd\_\{\\text\{out\}\}\. M\-ORE uses a fixed steady\-space rankr≪dr\\ll dand a frozen orthogonal basisA\(l\)∈ℝr×dA^\{\(l\)\}\\in\\mathbb\{R\}^\{r\\times d\}\(Eq\. \([7](https://arxiv.org/html/2605.20273#S4.E7)\) in the main paper\)\. The editable write parameters areBt−1\(l\)∈ℝdout×rB^\{\(l\)\}\_\{t\-1\}\\in\\mathbb\{R\}^\{d\_\{\\text\{out\}\}\\times r\}\. At edit steptt, letkt\(l\)​\(x\)∈ℝdk\_\{t\}^\{\(l\)\}\(x\)\\in\\mathbb\{R\}^\{d\}be the FFN key vector for editing sample inputxx, andGt\(l\)=∇Bt−1\(l\)ℒedit∈ℝdout×rG\_\{t\}^\{\(l\)\}=\\nabla\_\{B^\{\(l\)\}\_\{t\-1\}\}\\mathcal\{L\}\_\{\\text\{edit\}\}\\in\\mathbb\{R\}^\{d\_\{\\text\{out\}\}\\times r\}the edit gradient in the steady space\. The steady\-space statistic isz¯t\(l\)∈ℝr\\bar\{z\}\_\{t\}^\{\(l\)\}\\in\\mathbb\{R\}^\{r\}and the preconditioner isPt\(l\)∈ℝr×rP\_\{t\}^\{\(l\)\}\\in\\mathbb\{R\}^\{r\\times r\}\.

### E\.2M\-ORE: Per\-edit Time Complexity

##### Step A: preconditioned write\.

The layer\-wise write at edit stepttuses the preconditioner accumulated before the current step:

Δ​Bt\(l\)=−η​Gt\(l\)​Pt−1\(l\),\\small\\Delta B\_\{t\}^\{\(l\)\}=\-\\eta\\,G\_\{t\}^\{\(l\)\}P\_\{t\-1\}^\{\(l\)\},\(24\)whereGt\(l\)∈ℝdout×rG\_\{t\}^\{\(l\)\}\\in\\mathbb\{R\}^\{d\_\{\\text\{out\}\}\\times r\}andPt−1\(l\)∈ℝr×rP\_\{t\-1\}^\{\(l\)\}\\in\\mathbb\{R\}^\{r\\times r\}\. Right\-multiplying byPt−1\(l\)P\_\{t\-1\}^\{\(l\)\}costsO​\(dout​r2\)O\(d\_\{\\text\{out\}\}r^\{2\}\)\.

##### Step B: pooled steady\-space statistic\.

After the current write, M\-ORE forms the pooled key for updating the next preconditioner:

k¯t\(l\)≜Meanx∈ℬt\(l\)​\(kt\(l\)​\(x\)\)∈ℝd,\\small\\bar\{k\}\_\{t\}^\{\(l\)\}\\triangleq\\mathrm\{Mean\}\_\{x\\in\\mathcal\{B\}\_\{t\}^\{\(l\)\}\}\\\!\\big\(k\_\{t\}^\{\(l\)\}\(x\)\\big\)\\in\\mathbb\{R\}^\{d\},\(25\)which requires a reduction overbbvectors of lengthdd, i\.e\.,O​\(b​d\)O\(bd\)time\. It then projects the pooled key into the steady subspace,

z¯t\(l\)=A\(l\)​k¯t\(l\)∈ℝr,\\small\\bar\{z\}\_\{t\}^\{\(l\)\}=A^\{\(l\)\}\\bar\{k\}\_\{t\}^\{\(l\)\}\\in\\mathbb\{R\}^\{r\},\(26\)which costsO​\(r​d\)O\(rd\)time sinceA\(l\)∈ℝr×dA^\{\(l\)\}\\in\\mathbb\{R\}^\{r\\times d\}\. Equivalently, by linearity,z¯t\(l\)=Meanx∈ℬt\(l\)​\(A\(l\)​kt\(l\)​\(x\)\)\\bar\{z\}\_\{t\}^\{\(l\)\}=\\mathrm\{Mean\}\_\{x\\in\\mathcal\{B\}\_\{t\}^\{\(l\)\}\}\\big\(A^\{\(l\)\}k\_\{t\}^\{\(l\)\}\(x\)\\big\)\.

##### Step C: Sherman–Morrison update for the next preconditioner\.

M\-ORE inserts the current step featurez¯t\(l\)\\bar\{z\}\_\{t\}^\{\(l\)\}into the recursive statistic via the Sherman–Morrison lemma, yielding the preconditioner for the next edit step:

Pt\(l\)=Pt−1\(l\)−Pt−1\(l\)​z¯t\(l\)​\(z¯t\(l\)\)⊤​Pt−1\(l\)1\+\(z¯t\(l\)\)⊤​Pt−1\(l\)​z¯t\(l\)\.\\small P\_\{t\}^\{\(l\)\}=P\_\{t\-1\}^\{\(l\)\}\-\\frac\{P\_\{t\-1\}^\{\(l\)\}\\bar\{z\}\_\{t\}^\{\(l\)\}\(\\bar\{z\}\_\{t\}^\{\(l\)\}\)^\{\\top\}P\_\{t\-1\}^\{\(l\)\}\}\{1\+\(\\bar\{z\}\_\{t\}^\{\(l\)\}\)^\{\\top\}P\_\{t\-1\}^\{\(l\)\}\\bar\{z\}\_\{t\}^\{\(l\)\}\}\.\(27\)This can be computed via the standard rank\-one routine: computeu=Pt−1\(l\)​z¯t\(l\)u=P\_\{t\-1\}^\{\(l\)\}\\bar\{z\}\_\{t\}^\{\(l\)\}inO​\(r2\)O\(r^\{2\}\)time, compute the scalar denominator inO​\(r\)O\(r\)time, and apply the outer\-product updatePt−1\(l\)−u​u⊤/denP\_\{t\-1\}^\{\(l\)\}\-uu^\{\\top\}/\\mathrm\{den\}inO​\(r2\)O\(r^\{2\}\)time\. Thus, Step C isO​\(r2\)O\(r^\{2\}\)\.

##### Total per\-edit time \(exact and simplified\)\.

Summing Steps A–C for a single layer gives

O​\(dout​r2\+b​d\+r​d\+r2\)\.\\small O\\big\(d\_\{\\text\{out\}\}r^\{2\}\+bd\+rd\+r^\{2\}\\big\)\.\(28\)Aggregating over all edited layers\|ℒ\|\|\\mathcal\{L\}\|yields the exact per\-edit time:

TM\-ORE=O​\(\|ℒ\|​\(dout​r2\+b​d\+r​d\+r2\)\)\.\\small T\_\{\\text\{M\-ORE\}\}=O\\\!\\Big\(\|\\mathcal\{L\}\|\\,\(d\_\{\\text\{out\}\}r^\{2\}\+bd\+rd\+r^\{2\}\)\\Big\)\.\(29\)With a constant\-size buffer \(b=O​\(1\)b=O\(1\)\) andr≪d,doutr\\ll d,d\_\{\\text\{out\}\}, we drop lower\-order terms and obtain:

TM\-ORE=O​\(\|ℒ\|​\(dout​r2\+r​d\+r2\)\)≈O​\(\|ℒ\|​dout​r2\)\.\\small T\_\{\\text\{M\-ORE\}\}=O\\\!\\Big\(\|\\mathcal\{L\}\|\\,\(d\_\{\\text\{out\}\}r^\{2\}\+rd\+r^\{2\}\)\\Big\)\\approx O\\\!\\big\(\|\\mathcal\{L\}\|\\,d\_\{\\text\{out\}\}r^\{2\}\\big\)\.\(30\)

### E\.3M\-ORE: Per\-edit Space Complexity

##### Stored state\.

For each edited layerl∈ℒl\\in\\mathcal\{L\}, M\-ORE stores: \(i\) the frozen low\-rank write basisA\(l\)∈ℝr×dA^\{\(l\)\}\\in\\mathbb\{R\}^\{r\\times d\}, \(ii\) the low\-rank write parametersB\(l\)∈ℝdout×rB^\{\(l\)\}\\in\\mathbb\{R\}^\{d\_\{\\text\{out\}\}\\times r\}, and \(iii\) the steady\-space preconditionerP\(l\)∈ℝr×rP^\{\(l\)\}\\in\\mathbb\{R\}^\{r\\times r\}\. Therefore, the memory cost is

MM\-ORE=O​\(\|ℒ\|​\(r​d\+dout​r\+r2\)\)\.\\small M\_\{\\text\{M\-ORE\}\}=O\\\!\\Big\(\|\\mathcal\{L\}\|\\,\(rd\+d\_\{\\text\{out\}\}r\+r^\{2\}\)\\Big\)\.\(31\)Sincer≪d,doutr\\ll d,d\_\{\\text\{out\}\}andddis typically of the same order asdoutd\_\{\\text\{out\}\}for the edited modules, the lower\-orderr2r^\{2\}term can be ignored, yielding

MM\-ORE≈O​\(\|ℒ\|​r​\(d\+dout\)\)≈O​\(\|ℒ\|​dout​r\)\.\\small M\_\{\\text\{M\-ORE\}\}\\approx O\\\!\\Big\(\|\\mathcal\{L\}\|\\,r\(d\+d\_\{\\text\{out\}\}\)\\Big\)\\approx O\\\!\\big\(\|\\mathcal\{L\}\|\\,d\_\{\\text\{out\}\}r\\big\)\.\(32\)The buffer memory isO​\(b​d\)O\(bd\)per layer if keys are stored explicitly; in our implementationbbis constant and keys can be recomputed on\-the\-fly, so the asymptotic dependence on the edit lengthttremains unchanged\.

##### Independence from edit lengthtt\.

M\-ORE maintains only fixed\-size per\-layer state \(B\(l\)B^\{\(l\)\}andP\(l\)P^\{\(l\)\}\) and does not store edit\-specific experts or retrieval items\. Thus, bothTM\-ORET\_\{\\text\{M\-ORE\}\}andMM\-OREM\_\{\\text\{M\-ORE\}\}areO​\(1\)O\(1\)with respect to the edit stream lengthtt\.

### E\.4Baseline Complexity Derivations

Below, we justify the baseline entries in Table[5](https://arxiv.org/html/2605.20273#S5.T5)\. We again consider editor\-specific overhead; when a method also induces inference\-time costs \(e\.g\., selection/retrieval\), we report those costs explicitly\.

#### E\.4\.1Locate\-then\-edit

Many locate\-then\-edit editors require solving a regularized system involving a dense second\-order statistic \(e\.g\., covariance/Gram matrix\)Σ∈ℝd×d\\Sigma\\in\\mathbb\{R\}^\{d\\times d\}, such as computing a factorization ofΣ\+λ​I\\Sigma\+\\lambda Ior its inverse, and then applying it to obtain a closed\-form update\. A standard dense factorization \(SVD/Cholesky\) hasO​\(d3\)O\(d^\{3\}\)time andO​\(d2\)O\(d^\{2\}\)memory\. If a factorization is precomputed and cached, each edit still applies a dense linear map, which costsO​\(d2\)O\(d^\{2\}\)time, while the stored statistic/factor remainsO​\(d2\)O\(d^\{2\}\)memory\. This yields the table entry:T=O​\(d3\)T=O\(d^\{3\}\)\(orO​\(d2\)O\(d^\{2\}\)apply\) andM=O​\(d2\)M=O\(d^\{2\}\)\.

#### E\.4\.2Null\-space constraint \(SVD on accumulated keysKtK\_\{t\}\)

Null\-space constrained editors maintain constraints w\.r\.t\. accumulated keys\. LetKt=\[k1,…,kt\]∈ℝd×tK\_\{t\}=\[k\_\{1\},\\ldots,k\_\{t\}\]\\in\\mathbb\{R\}^\{d\\times t\}be the matrix of past keys\. Computing an orthogonal complement ofspan​\(Kt\)\\mathrm\{span\}\(K\_\{t\}\)via SVD on the tall matrixKtK\_\{t\}typically costsO​\(d​t2\)O\(d\\,t^\{2\}\)time and requires storingKtK\_\{t\}or its bases, i\.e\.,O​\(d​t\)O\(d\\,t\)memory, ford≫td\\gg t\. Hence both time and memory grow withtt\.

#### E\.4\.3Parameter\-preserving memory/expert methods

These methods store edit\-specific items \(e\.g\., adapters/experts or memory entries\)\. If each edit stores an item of sizepmemp\_\{\\text\{mem\}\}, then memory grows linearly asM=O​\(t​pmem\)M=O\(t\\,p\_\{\\text\{mem\}\}\)\. At inference, selecting amongttitems costsSel​\(t\)\\mathrm\{Sel\}\(t\), commonlyO​\(t\)O\(t\)for brute\-force scan orO​\(log⁡t\)O\(\\log t\)\(or sublinear\) with approximate nearest\-neighbor indexing\. This justifies the table entryT=O​\(Sel​\(t\)\)T=O\(\\mathrm\{Sel\}\(t\)\)andM=O​\(t​pmem\)M=O\(t\\,p\_\{\\text\{mem\}\}\)\.

#### E\.4\.4Naive finetuning \(LoRA rankrr\)

Under the same low\-rank interface \(updatingB\(l\)∈ℝdout×rB^\{\(l\)\}\\in\\mathbb\{R\}^\{d\_\{\\text\{out\}\}\\times r\}with SGD/Adam\) but without the preconditioner recursion, the editor\-specific update is a single stepB\(l\)←B\(l\)−η​Gt\(l\)B^\{\(l\)\}\\leftarrow B^\{\(l\)\}\-\\eta G\_\{t\}^\{\(l\)\}, which costsO​\(dout​r\)O\(d\_\{\\text\{out\}\}r\)time per layer\. Storing the trainable parameters costsO​\(dout​r\)O\(d\_\{\\text\{out\}\}r\)per layer\. Thus, this isT=O​\(\|ℒ\|​dout​r\)T=O\(\|\\mathcal\{L\}\|\\,d\_\{\\text\{out\}\}r\)andM=O​\(\|ℒ\|​dout​r\)M=O\(\|\\mathcal\{L\}\|\\,d\_\{\\text\{out\}\}r\), both independent oftt\.

Table 11:Complete editing results on E\-VQA and E\-IC for BLIP2\-OPT and LLaVA\-v1\.5 under different edit horizons\. “Rel\.”, “T/M\-Gen\.”, and “T/M\-Loc\.” abbreviate*Reliability*,*Generality*, and*Locality*\(for text/modal evaluations\), respectively\. The subscript of each method \(e\.g\.,1,10,100\) denotes the number of online edits performed\. Rows shaded in light purple indicate*parameter\-modifying*methods\.![Refer to caption](https://arxiv.org/html/2605.20273v1/x30.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x31.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x32.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x33.png)

Figure 13:E\-IC case studies \(challenging visual\-understanding edits\)\. Top two panels: Group 1; bottom two panels: Group 2\. In each group, the upper panel is the edit sample and the lower panel is the corresponding locality sample\.![Refer to caption](https://arxiv.org/html/2605.20273v1/x34.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x35.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x36.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x37.png)

Figure 14:E\-IC case studies \(challenging visual\-understanding edits\)\. Top two panels: Group 1; bottom two panels: Group 2\. In each group, the upper panel is the edit sample and the lower panel is the corresponding locality sample\.![Refer to caption](https://arxiv.org/html/2605.20273v1/x38.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x39.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x40.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x41.png)

Figure 15:E\-VQA case studies\. Top two panels: Group 1; bottom two panels: Group 2\. In each group, the upper panel is the edit sample and the lower panel is the corresponding locality sample\.![Refer to caption](https://arxiv.org/html/2605.20273v1/x42.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x43.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x44.png)
![Refer to caption](https://arxiv.org/html/2605.20273v1/x45.png)

Figure 16:E\-VQA case studies\. Top two panels: Group 1; bottom two panels: Group 2\. In each group, the upper panel is the edit sample and the lower panel is the corresponding locality sample\.