RoCo-ACE: Rollout-Conditioned Online Distillation for Retention-Aware Knowledge Injection

# RoCo-ACE: Rollout-Conditioned Online Distillation for Retention-Aware Knowledge Injection
Source: [https://arxiv.org/html/2607.24771](https://arxiv.org/html/2607.24771)
Yan Hong1, Wei Li1, Kedong Xiu2, Jun Lan∗,1, Shuheng Zhou1 Zhongcai Lyu1, Huijia Zhu1, Weiqiang Wang1, Jianfu Zhang∗,3 1Ant Group 2Zhejiang University 3Shanghai Jiao Tong University Correspondence:Jun Lan[<yelan\.lj@antgroup\.com\>](https://arxiv.org/html/2607.24771v1/mailto:[email protected]), Jianfu Zhang[<c\.sis@sjtu\.edu\.cn\>](https://arxiv.org/html/2607.24771v1/mailto:[email protected])

###### Abstract

Knowledge injection updates pretrained MLLMs with new factual or domain\-specific knowledge, but fitting full authoritative answers can cause drift in non\-updated behavior\. Online distillation limits this drift by training on model\-generated rollouts, yet uniform reference\-conditioned distillation gives coarse supervision: it can under\-emphasize reference\-supported rollout tokens and supervise omitted facts only indirectly\. We introduceRoCo\-ACE, a rollout\-conditioned online distillation objective for knowledge injection:RoCouses same\-rollout reference\-free/reference\-conditioned likelihood contrast to reallocate additional distillation weight to reference\-supported rollout tokens, whileACEadds sparse reference\-side anchored correction for authoritative anchors omitted from the rollout, without full\-answer imitation\. Across three knowledge\-injection settings, six retention benchmarks, multiple baselines and base models,RoCo\-ACEachieves the best injected\-knowledge accuracy among compared methods while keeping evaluated retention close to the base model\.

RoCo\-ACE: Rollout\-Conditioned Online Distillation for Retention\-Aware Knowledge Injection

Yan Hong1, Wei Li1, Kedong Xiu2, Jun Lan∗,1, Shuheng Zhou1Zhongcai Lyu1, Huijia Zhu1, Weiqiang Wang1, Jianfu Zhang∗,31Ant Group2Zhejiang University3Shanghai Jiao Tong UniversityCorrespondence:Jun Lan[<yelan\.lj@antgroup\.com\>](https://arxiv.org/html/2607.24771v1/mailto:[email protected]), Jianfu Zhang[<c\.sis@sjtu\.edu\.cn\>](https://arxiv.org/html/2607.24771v1/mailto:[email protected])

## 1Introduction

Multimodal Large Language Models \(MLLMs\) have evolved from vision\-language pretraining and image\-language bootstrapping\(Alayrac et al\.,[2022](https://arxiv.org/html/2607.24771#bib.bib2); Li et al\.,[2023](https://arxiv.org/html/2607.24771#bib.bib17)\)to instruction\-tuned multimodal assistants\(Dai et al\.,[2023](https://arxiv.org/html/2607.24771#bib.bib8); Liu et al\.,[2023](https://arxiv.org/html/2607.24771#bib.bib19); Bai et al\.,[2023](https://arxiv.org/html/2607.24771#bib.bib3)\)\. They now support instruction following, visual understanding, grounding, text\-rich image reasoning, and safety\-sensitive response behavior\(Pi et al\.,[2024](https://arxiv.org/html/2607.24771#bib.bib25); Wu et al\.,[2025](https://arxiv.org/html/2607.24771#bib.bib35)\)\. After deployment, these models often need to absorb new entities, events, domain facts, and visual\-world knowledge\. A useful knowledge\-injection update should improve injected\-knowledge accuracy while limiting drift in non\-updated multimodal and safety behavior, as evaluated by general multimodal benchmarks\(Liu et al\.,[2024a](https://arxiv.org/html/2607.24771#bib.bib20); Chen et al\.,[2024](https://arxiv.org/html/2607.24771#bib.bib7)\)and safety benchmarks\(Liu et al\.,[2024b](https://arxiv.org/html/2607.24771#bib.bib21)\)\. This goal is challenging: fitting new data can interfere with previously learned behavior\(Kirkpatrick et al\.,[2017](https://arxiv.org/html/2607.24771#bib.bib14); Li and Hoiem,[2016](https://arxiv.org/html/2607.24771#bib.bib18)\), and recent MLLM knowledge\-injection studies observe the same injection\-retention trade\-off\(Jiang et al\.,[2025a](https://arxiv.org/html/2607.24771#bib.bib12),[b](https://arxiv.org/html/2607.24771#bib.bib13)\)\.

![Refer to caption](https://arxiv.org/html/2607.24771v1/roco_motivation.png)Figure 1:Motivation ofRoCo\-ACE\. The upper panel illustrates the injection–retention trade\-off among direct fitting, constrained updating, online distillation, and our variants\.RoCo\-ACEimproves this trade\-off by combining same\-rollout likelihood contrast with sparse anchored correction\. The lower panel shows an EVOKE example whereRoCo\-ACErecovers more authoritative anchors than SFT and SDFT\(Shenfeld et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib28)\)while keeping a grounded response style\.Existing update strategies occupy different regions of the injection\-retention trade\-off\. Supervised Fine\-Tuning \(SFT\)\(Ouyang et al\.,[2022](https://arxiv.org/html/2607.24771#bib.bib24)\)directly fits authoritative answers and can improve injected\-knowledge accuracy, but it also encourages full\-answer imitation, including dataset\-specific style and narrow response patterns\. Constrained or parameter\-efficient methods limit drift by restricting where or how the model can change\(Hu et al\.,[2022](https://arxiv.org/html/2607.24771#bib.bib11); Jiang et al\.,[2025b](https://arxiv.org/html/2607.24771#bib.bib13); Luo et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib22)\), but they do not directly localize supervision to the factual units that should change\. The key issue is supervision granularity: knowledge injection often requires updating a few entities, dates, relations, or event details, while many training tokens reflect generic phrasing, non\-target visual details, or answer style\. As illustrated in Figure[1](https://arxiv.org/html/2607.24771#S1.F1), fitting\-oriented methods can improve injection at a substantial retention cost, whereas conservative methods better limit drift in non\-updated behavior but leave some target facts under\-injected\.

To address this injection\-retention trade\-off, we proposeRoCo\-ACE, aRollout\-Conditioned contrastive online distillation objective withAnchoredCross\-Entropy for retention\-oriented knowledge injection\. It trains on model\-generated rollouts instead of fitting the full authoritative answer, keeping updates close to the current model policy\(Agarwal et al\.,[2023](https://arxiv.org/html/2607.24771#bib.bib1); Gu et al\.,[2023](https://arxiv.org/html/2607.24771#bib.bib9); Shenfeld et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib28); Zhao et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib39); Ye et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib37)\)\. However, uniform reference\-conditioned online distillation still lacks fine\-grained supervision: it can allocate gradient mass to generic or unsupported wording, fail to emphasize reference\-supported rollout tokens, and provide only indirect supervision for facts omitted from the current rollout\.

Our key observation is that an on\-policy rollout exposes a useful supervision split for factual updating\. It may contain reference\-supported content already reached on\-policy, generic or unsupported wording, and authoritative anchors that are omitted from or weakly matched by the rollout\.RoCohandles the rollout\-side part by contrasting reference\-free and reference\-conditioned teacher views on the same rollout prefix, then reallocating additional distillation weight to tokens whose likelihood increases under the reference\-conditioned view\.ACEcomplements this signal with sparse anchored cross\-entropy over extracted authoritative facts, assigning larger weights to omitted or weakly matched anchors\. Together, the two objectives inject sparse factual content without turning the full reference answer into a behavior\-cloning target\.

This paper makes the following contributions:

- •We identify supervision granularity as a key issue in online distillation for knowledge injection: uniform rollout distillation can under\-emphasize reference\-supported tokens and only indirectly train facts omitted from the current rollout\.
- •We introduceRoCo\-ACE, a rollout\-conditioned contrastive online distillation objective for knowledge injection\.RoCouses same\-rollout likelihood contrast to reallocate additional distillation weight to reference\-supported rollout tokens, whileACEadds sparse anchored correction for omitted or weakly matched authoritative facts\.
- •Experiments on three knowledge\-injection settings and six retention benchmarks show thatRoCo\-ACEimproves injected\-knowledge accuracy while keeping evaluated retention close to online\-distillation baselines and the base model\.

## 2Related Work

### 2\.1Update\-Space Constraints for Retention

A direct way to inject knowledge is to fit authoritative answers, but full\-answer fitting can shift non\-target behavior\. To limit such update\-induced drift, prior work often constrains where or how the model can change\. Adam\-NSCL\(Wang et al\.,[2021](https://arxiv.org/html/2607.24771#bib.bib33)\)updates networks in the null space of feature covariance, and LoRA\-Null\(Tang et al\.,[2025](https://arxiv.org/html/2607.24771#bib.bib30)\)applies a related idea to low\-rank adaptation by choosing initialization spaces that better retain pretrained behavior\. KORE\(Jiang et al\.,[2025b](https://arxiv.org/html/2607.24771#bib.bib13)\)studies multimodal knowledge injection directly and combines knowledge\-oriented augmentation with constraints for limiting drift in multimodal updates\. Parameter\-efficient and forgetting\-aware methods, including LoRA\(Hu et al\.,[2022](https://arxiv.org/html/2607.24771#bib.bib11)\), KeepLoRA\(Luo et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib22)\), MoELoRA\(Luo et al\.,[2024](https://arxiv.org/html/2607.24771#bib.bib23)\), and SEFE\(Chen et al\.,[2025](https://arxiv.org/html/2607.24771#bib.bib5)\), restrict or organize trainable parameters through low\-rank adapters, residual\-gradient updates, expertized modules, or forgetting\-aware constraints\. Related MLLM continual\-learning work, such as MLLM\-CL\(Zhao et al\.,[2025](https://arxiv.org/html/2607.24771#bib.bib38)\)and LiLoRA\(Che et al\.,[2025](https://arxiv.org/html/2607.24771#bib.bib4)\), further studies sequential visual\-instruction updates over heterogeneous task streams\.

These methods are retention\-oriented mainly through constraints on the editable parameter or update space\.RoCo\-ACEis complementary: rather than constraining the update space, it improves supervision granularity over rollout tokens and authoritative anchors\.

### 2\.2Online Distillation for Model Updating

Knowledge distillation\(Hinton et al\.,[2015](https://arxiv.org/html/2607.24771#bib.bib10)\)transfers a teacher distribution to a student\. Online or on\-policy variants, such as Generalized Knowledge Distillation\(Agarwal et al\.,[2023](https://arxiv.org/html/2607.24771#bib.bib1)\)and MiniLLM\(Gu et al\.,[2023](https://arxiv.org/html/2607.24771#bib.bib9)\), distill on student\-generated outputs rather than only fixed target responses, keeping supervision close to the current student policy\. Recent work further studies richer teacher contexts, reasoning compression, and policy\-optimization variants\(Zhao et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib39); Ye et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib37); Sang et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib27); Yang et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib36); Li et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib16)\)\.Song and Zheng \([2026](https://arxiv.org/html/2607.24771#bib.bib29)\)provides a recent survey\. SDFT\(Shenfeld et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib28)\)is especially related, as it uses self\-distillation to stabilize continual updates and reduce forgetting\.

Online distillation is attractive for knowledge injection because it avoids directly fitting the full authoritative answer\. However, uniform reference\-conditioned rollout distillation still provides coarse supervision: it can spend gradients on generic or unsupported wording, under\-emphasize reference\-supported rollout tokens, and supervise omitted authoritative facts only indirectly\.RoCo\-ACEimproves this granularity by combining same\-rollout likelihood contrast for reference\-supported rollout tokens with sparse reference\-side correction for authoritative anchors omitted from or weakly matched by the rollout\.

![Refer to caption](https://arxiv.org/html/2607.24771v1/roco_ace_method.png)Figure 2:Overview ofRoCo\-ACE\. The student samples an on\-policy rollout that mixes generic or non\-target visual spans, reference\-supported factual spans, and omitted authoritative anchors\.RoCoscores the same rollout under reference\-free and reference\-conditioned teacher views, then uses their likelihood contrast to assign larger distillation weights to reference\-supported rollout tokens\.ACEcomplements this rollout\-side signal with sparse anchored cross\-entropy for authoritative anchors omitted from or weakly matched by the rollout\.

## 3Method

### 3\.1Overview

We consider a knowledge\-injection dataset𝒟=\{\(xi,ai,Ii\)\}i=1N\\mathcal\{D\}=\\\{\(x\_\{i\},a\_\{i\},I\_\{i\}\)\\\}\_\{i=1\}^\{N\}, wherexix\_\{i\}is a user prompt,aia\_\{i\}is an authoritative reference answer that specifies the target factual update, andIiI\_\{i\}is an optional image\. Our goal is to adapt a base MLLMpθ0p\_\{\\theta\_\{0\}\}into a student modelpθp\_\{\\theta\}that incorporates the injected facts into its responses while limiting drift in non\-updated behavior\. Figure[2](https://arxiv.org/html/2607.24771#S2.F2)gives an overview of theRoCo\-ACEtraining objective\. For each example, the student samples an on\-policy rolloutyi=\(yi,1,…,yi,Ti\)∼pθ\(⋅∣xi,Ii\)y\_\{i\}=\(y\_\{i,1\},\\ldots,y\_\{i,T\_\{i\}\}\)\\sim p\_\{\\theta\}\(\\cdot\\mid x\_\{i\},I\_\{i\}\)\. This rollout exposes a supervision split for factual updating: it may contain reference\-supported factual spans already reached on\-policy, generic or unsupported wording and non\-target visual details, and authoritative anchors that are omitted from or weakly matched by the rollout\.RoCohandles the rollout\-side signal by scoring the same rollout prefixes with reference\-free and reference\-conditioned teacher views\. It uses their likelihood contrast to reallocate additional distillation weight to reference\-supported rollout tokens, while generic wording and non\-target visual details receive mainly the floor distillation signal\.ACEhandles the omission case with sparse anchored cross\-entropy over extracted authoritative facts, assigning larger weights to anchors omitted from or weakly matched by the rollout\. Together,RoCoandACEinject sparse factual content without treating the full reference answer as a token\-level imitation target\.

### 3\.2RoCo: Rollout\-Conditioned Contrastive Online Distillation

Online distillation\(Shenfeld et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib28); Zhao et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib39)\)is well suited to retention\-oriented knowledge injection because it trains on the student’s own rollout instead of fitting the full authoritative answer\. This keeps the update close to the current student policy, but uniform reference\-conditioned distillation still assigns loss to every valid rollout token, even though only a small subset carries target facts\. A student rollout often mixes reference\-supported factual content with generic or unsupported wording\. In Figure[2](https://arxiv.org/html/2607.24771#S2.F2), for example, the rollout contains reference\-supported spans such as “Princess Anne”, “hospitalized”, and “Gatcombe Park”, but also includes non\-target visual details such as “dark green hat” and “formal coat”\.RoCoaddresses this supervision\-granularity problem by comparing reference\-free and reference\-conditioned teacher likelihoods under the same rollout prefix\. It reallocates additional distillation weight to tokens whose likelihood increases when the authoritative reference is available, while generic wording, non\-target visual details, and answer style receive mainly the floor distillation signal\. To make this selective reallocation explicit,RoComeasures how much the authoritative reference changes the likelihood assigned to each on\-policy token under the same rollout prefix\. Letqϕq\_\{\\phi\}denote the teacher model, and letqϕ\(⋅∣⋅\)q\_\{\\phi\}\(\\cdot\\mid\\cdot\)denote its next\-token probability distribution over the vocabulary\. For each rollout prefixyi,<ty\_\{i,<t\}, we evaluate two matched teacher views:

qbasei,t​\(v\)\\displaystyle q\_\{\\mathrm\{base\}\}^\{i,t\}\(v\)=qϕ​\(v∣xi,Ii,yi,<t\),\\displaystyle=q\_\{\\phi\}\(v\\mid x\_\{i\},I\_\{i\},y\_\{i,<t\}\),\(1\)qrefi,t​\(v\)\\displaystyle q\_\{\\mathrm\{ref\}\}^\{i,t\}\(v\)=qϕ​\(v∣xi,Ii,ai,yi,<t\)\.\\displaystyle=q\_\{\\phi\}\(v\\mid x\_\{i\},I\_\{i\},a\_\{i\},y\_\{i,<t\}\)\.The subscriptbase\\mathrm\{base\}denotes the reference\-free view, andref\\mathrm\{ref\}denotes the reference\-conditioned view\. Both views score the same student\-generated prefix and share the same architecture and parameters\. They differ only in whether the authoritative referenceaia\_\{i\}is included in the prompt context\. In our online setting,qϕq\_\{\\phi\}is an exponential\-moving\-average teacher synchronized from the student, keeping teacher scoring close to the current policy while avoiding a degenerate self\-target at the current optimization step\. For each generated tokenyi,ty\_\{i,t\}, the RoCo contrast is defined as the log\-likelihood ratio between the two views:

Δi,t=log⁡qrefi,t​\(yi,t\)−log⁡qbasei,t​\(yi,t\)\.\\Delta\_\{i,t\}=\\log q\_\{\\mathrm\{ref\}\}^\{i,t\}\(y\_\{i,t\}\)\-\\log q\_\{\\mathrm\{base\}\}^\{i,t\}\(y\_\{i,t\}\)\.\(2\)A high value ofqrefi,t​\(yi,t\)q\_\{\\mathrm\{ref\}\}^\{i,t\}\(y\_\{i,t\}\)alone may simply reflect a generic token that is already likely under the original prompt, whereas a positiveΔi,t\\Delta\_\{i,t\}suggests additional support induced by the authoritative reference\. We therefore use the contrast to estimate reference\-induced support for rollout tokens: the token is generated on\-policy, and the reference\-conditioned view makes it more likely under the same teacher and prefix\. Let𝒱i\\mathcal\{V\}\_\{i\}denote valid completion\-token positions after removing padding and ignored tokens\. The contrast is converted into dense token weights:

wi,t=w0\+βΔ​max⁡\(0,Δi,t−τ\),t∈𝒱i,w\_\{i,t\}=w\_\{0\}\+\\beta\_\{\\Delta\}\\max\(0,\\Delta\_\{i,t\}\-\\tau\),\\quad t\\in\\mathcal\{V\}\_\{i\},\(3\)wherew0w\_\{0\}is a floor weight,βΔ\\beta\_\{\\Delta\}scales the contrast signal, andτ\\tausuppresses small log\-likelihood differences\. The floor term keeps a stable online\-distillation signal over the rollout, while the contrast term reallocates additional gradient mass to tokens whose likelihood becomes sufficiently larger under the reference\-conditioned view\. When a detected factual expression is split into multiple subword tokens, we aggregate the token weights within the expression and broadcast the resulting span score back to its tokens\. For simplicity, we still denote the resulting weights bywi,tw\_\{i,t\}\. This span\-aware smoothing reduces subword\-level noise for names, dates, numbers, and multilingual expressions\.

The resulting distillation objective is

ℒRoCo\(i\)\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{RoCo\}\}^\{\(i\)\}=∑t∈𝒱iwi,t​di,t∑t∈𝒱iwi,t\+ϵ,\\displaystyle=\\frac\{\\sum\_\{t\\in\\mathcal\{V\}\_\{i\}\}w\_\{i,t\}d\_\{i,t\}\}\{\\sum\_\{t\\in\\mathcal\{V\}\_\{i\}\}w\_\{i,t\}\+\\epsilon\},\(4\)di,t\\displaystyle d\_\{i,t\}=DKL\(qrefi,t\(⋅\)∥pθ\(⋅∣xi,Ii,yi,<t\)\)\.\\displaystyle=D\_\{\\mathrm\{KL\}\}\\\!\\left\(q\_\{\\mathrm\{ref\}\}^\{i,t\}\(\\cdot\)\\\|p\_\{\\theta\}\(\\cdot\\mid x\_\{i\},I\_\{i\},y\_\{i,<t\}\)\\right\)\.The denominator makes the objective a weighted average over valid rollout tokens, so RoCo changes the relative distribution of distillation weight rather than simply changing the overall loss scale\. The reference\-conditioned teacher provides the distributional target, while the reference\-versus\-base contrast determines where additional distillation weight should concentrate\. Thus,RoCostrengthens reference\-supported content already reached on\-policy, while keeping training anchored to the student’s own completion and down\-weighting generic wording, non\-target visual details, and answer style\.

### 3\.3ACE: Anchored Cross\-Entropy

The on\-policy design also defines a coverage boundary forRoCo: its loss is evaluated on the valid rollout positions𝒱i\\mathcal\{V\}\_\{i\}, so its strongest supervision comes from content that the student already reaches in its own completion\. When a target fact is absent from the rollout, reference\-conditioned distillation may still influence nearby continuation distributions and help future rollouts move toward that fact, but the current rollout provides only indirect supervision for the omitted anchor\.ACEaddresses this remaining coverage gap by applying explicit supervised correction to extracted authoritative anchors, with larger weights for anchors that are missing or weakly covered in the rollout\.

Letai=\(ai,1,…,ai,Li\)a\_\{i\}=\(a\_\{i,1\},\\ldots,a\_\{i,L\_\{i\}\}\)denote the tokenized authoritative answer\. We segmentaia\_\{i\}into factual spans𝒮i\\mathcal\{S\}\_\{i\}\. In our implementation, spans are constructed from entity\-like strings, dates, numbers, quoted titles, and compact factual phrases after text normalization\. The same normalization is used when comparing reference spans with rollout text\. For each spans∈𝒮is\\in\\mathcal\{S\}\_\{i\}, letρi​\(s,yi\)∈\[0,1\]\\rho\_\{i\}\(s,y\_\{i\}\)\\in\[0,1\]be its coverage score in the student rollout, where11denotes full coverage and0denotes absence\. We assign each span an anchor weight

ui,s=u0\+βmiss​\(1−ρi​\(s,yi\)\),u\_\{i,s\}=u\_\{0\}\+\\beta\_\{\\mathrm\{miss\}\}\\big\(1\-\\rho\_\{i\}\(s,y\_\{i\}\)\\big\),\(5\)whereu0u\_\{0\}is a base weight for extracted factual anchors andβmiss\\beta\_\{\\mathrm\{miss\}\}controls the strength of missing\- or weak\-coverage correction\. Letℐi​\(s\)⊆\{1,…,Li\}\\mathcal\{I\}\_\{i\}\(s\)\\subseteq\\\{1,\\ldots,L\_\{i\}\\\}denote the reference\-token positions covered by spanss, and let𝒜i=⋃s∈𝒮iℐi​\(s\)\\mathcal\{A\}\_\{i\}=\\bigcup\_\{s\\in\\mathcal\{S\}\_\{i\}\}\\mathcal\{I\}\_\{i\}\(s\)be the set of anchor\-token positions in the reference answer\. We broadcast each span weight to its covered reference tokens\. For overlapping spans, the token\-level anchor weight is

ui,k=maxs:k∈ℐi​\(s\)⁡ui,s,k∈𝒜i\.u\_\{i,k\}=\\max\_\{s:\\,k\\in\\mathcal\{I\}\_\{i\}\(s\)\}u\_\{i,s\},\\quad k\\in\\mathcal\{A\}\_\{i\}\.\(6\)The ACE loss for exampleiiis then

ℒACE\(i\)=−∑k∈𝒜iui,k​log⁡pθ​\(ai,k∣xi,Ii,ai,<k\)∑k∈𝒜iui,k\+ϵ\.\\mathcal\{L\}\_\{\\mathrm\{ACE\}\}^\{\(i\)\}=\-\\frac\{\\sum\_\{k\\in\\mathcal\{A\}\_\{i\}\}u\_\{i,k\}\\log p\_\{\\theta\}\(a\_\{i,k\}\\mid x\_\{i\},I\_\{i\},a\_\{i,<k\}\)\}\{\\sum\_\{k\\in\\mathcal\{A\}\_\{i\}\}u\_\{i,k\}\+\\epsilon\}\.\(7\)Only tokens in𝒜i\\mathcal\{A\}\_\{i\}receive supervised loss, and non\-anchor reference tokens serve only as teacher\-forced context\. Thus,ACEis sparser than ordinary SFT even whenu0\>0u\_\{0\}\>0for extracted anchors\. It complementsRoCoby adding reference\-side correction for target facts that are absent or weakly covered in the rollout, whileRoCoreallocates distillation over student\-reachable rollout positions\.

### 3\.4Overall Objective

For a mini\-batchℬ\\mathcal\{B\}, the final training objective combines rollout\-side distillation and reference\-side anchor correction:

ℒ=1\|ℬ\|​∑i∈ℬ\(λRoCo​ℒRoCo\(i\)\+λACE​ℒACE\(i\)\),\\mathcal\{L\}=\\frac\{1\}\{\|\\mathcal\{B\}\|\}\\sum\_\{i\\in\\mathcal\{B\}\}\\left\(\\lambda\_\{\\mathrm\{RoCo\}\}\\mathcal\{L\}\_\{\\mathrm\{RoCo\}\}^\{\(i\)\}\+\\lambda\_\{\\mathrm\{ACE\}\}\\mathcal\{L\}\_\{\\mathrm\{ACE\}\}^\{\(i\)\}\\right\),\(8\)whereλRoCo≥0\\lambda\_\{\\mathrm\{RoCo\}\}\\geq 0andλACE≥0\\lambda\_\{\\mathrm\{ACE\}\}\\geq 0are loss coefficients\. Here,ℒRoCo\(i\)\\mathcal\{L\}\_\{\\mathrm\{RoCo\}\}^\{\(i\)\}is evaluated on the valid student\-rollout positions𝒱i\\mathcal\{V\}\_\{i\}, andℒACE\(i\)\\mathcal\{L\}\_\{\\mathrm\{ACE\}\}^\{\(i\)\}is evaluated only on the reference anchor\-token set𝒜i\\mathcal\{A\}\_\{i\}\. Together, these two losses provide complementary supervision:RoCoreallocates distillation over reference\-aligned reachable rollout positions, whileACEadds sparse reference\-side correction for missing or weakly covered anchors, enabling targeted factual injection without full\-reference imitation\.

## 4Experiments

### 4\.1Settings

#### Benchmarks\.

We evaluate knowledge injection along two axes: injected\-knowledge acquisition and evaluated retention\. For acquisition, we use EVOKE\(Jiang et al\.,[2025a](https://arxiv.org/html/2607.24771#bib.bib12)\), which contains image\-grounded entity and news updates with authoritative reference answers, and VP/Sci from MLLM\-CL\(Zhao et al\.,[2025](https://arxiv.org/html/2607.24771#bib.bib38)\), which test broader visual\-perception and science\-domain update settings\. Appendix[H](https://arxiv.org/html/2607.24771#A8)provides detailed benchmark descriptions\. For retention, we evaluate every trained model on TreeBench\(Wang et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib31)\), VStar\(Wu and Xie,[2024](https://arxiv.org/html/2607.24771#bib.bib34)\), MathVision\(Wang et al\.,[2024](https://arxiv.org/html/2607.24771#bib.bib32)\), MMStar\(Chen et al\.,[2024](https://arxiv.org/html/2607.24771#bib.bib7)\), BabyVision\(Chen et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib6)\), and MM\-SafetyBench\(Liu et al\.,[2024b](https://arxiv.org/html/2607.24771#bib.bib21)\), abbreviated as MM\-Safety in tables\. These benchmarks cover visual grounding, fine\-grained perception, multimodal mathematical reasoning, general multimodal understanding, core visual primitives, and multimodal safety behavior; we report their arithmetic mean as Ret\. Avg\.

#### Evaluation metrics\.

We report accuracy for injected\-knowledge acquisition and evaluated retention\. Open\-ended injection responses are judged against the authoritative answer by an LLM judge, which focuses on factual consistency\-including entity identity, event identity, date, number, location, and relation\. Benchmark\-specific tasks follow their standard answer formats and normalization rules\. For multiple\-choice tasks, we extract the predicted option from the model response\. Appendix[I](https://arxiv.org/html/2607.24771#A9)provides details of the judge model, answer extraction, and normalization protocol\.

#### Models and baselines\.

The main experiments useQwen3\-VL\-8Bas the base model; Qwen3\-VL\-30B results are reported in Appendix[D](https://arxiv.org/html/2607.24771#A4)\. We compare direct fitting baselines \(LoRA\(Hu et al\.,[2022](https://arxiv.org/html/2607.24771#bib.bib11)\)and SFT\(Ouyang et al\.,[2022](https://arxiv.org/html/2607.24771#bib.bib24)\)\), reference\-conditioned online distillation \(SDFT with authoritative reference\(Shenfeld et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib28)\)\), constrained\-update baselines \(KORE\(Jiang et al\.,[2025b](https://arxiv.org/html/2607.24771#bib.bib13)\), KeepLoRA\(Luo et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib22)\), MoELoRA\(Luo et al\.,[2024](https://arxiv.org/html/2607.24771#bib.bib23)\), and SEFE\(Chen et al\.,[2025](https://arxiv.org/html/2607.24771#bib.bib5)\)\), and our variants,RoCoandRoCo\-ACE\. Unless otherwise stated, all methods use the same injected examples, training budget, image preprocessing, and evaluation protocol; only the update rule or supervision objective changes\. KORE is the only exception, as it additionally uses auxiliary general data to estimate its retention constraint\. Implementation details and hyperparameters are provided in Appendix[K](https://arxiv.org/html/2607.24771#A11)\.

Table 1:Baseline comparison across three knowledge\-injection settings\. Inj\. Acc\. reports injected\-knowledge accuracy, and Ret\. Avg\. averages the six held\-out retention benchmarks\. Higher is better\. Bold and bold\-italic mark the best and second\-best scores\.

### 4\.2Baseline Comparison

Figure[3](https://arxiv.org/html/2607.24771#S4.F3)visualizes the EVOKE injection–retention trade\-off, and Table[1](https://arxiv.org/html/2607.24771#S4.T1)summarizes Inj\. Acc\. and Ret\. Avg\. across EVOKE, VP, and Sci\. Full per\-benchmark retention values and VP/Sci radar plots are provided in Appendix[B](https://arxiv.org/html/2607.24771#A2)\. Representative EVOKE responses are shown in Appendix[C](https://arxiv.org/html/2607.24771#A3)\. Overall, Table[1](https://arxiv.org/html/2607.24771#S4.T1)shows that existing baselines tend to favor either injection or retention, whileRoCo\-ACEimproves injected\-knowledge accuracy with limited drift in evaluated retention\.

Fitting\-oriented baselines prioritize injection\.LoRA and SFT directly optimize authoritative answers, making them natural baselines for injected\-knowledge acquisition\. However, full\-answer supervision also fits generic wording, answer style, and non\-target behavior rather than only target factual anchors\. In Table[1](https://arxiv.org/html/2607.24771#S4.T1), SFT improves EVOKE Inj\. Acc\. from 16\.4 to 20\.1, but Ret\. Avg\. drops from 57\.2 to 31\.8\. LoRA has less severe drift, reaching 18\.6 Inj\. Acc\. and 45\.2 Ret\. Avg\. The same trend appears on VP and Sci, where both direct\-fitting baselines improve Inj\. Acc\. over the base model but reduce Ret\. Avg\. These results show that full\-answer fitting provides a strong injection signal, but is not ideal for retention\-oriented knowledge injection\.

![Refer to caption](https://arxiv.org/html/2607.24771v1/radar_evoke_all.png)Figure 3:Radar comparison on EVOKE\. The red axis reports injected\-knowledge accuracy, and the other axes report evaluated retention on six held\-out multimodal and safety benchmarks\. Scores are normalized for visualization; exact values are reported in Appendix[B](https://arxiv.org/html/2607.24771#A2)\.RoCo\-ACEimproves injected\-knowledge accuracy while keeping evaluated retention close to online\-distillation and retention\-oriented baselines\.Constrained methods can favor stronger injection\.KORE is the strongest non\-ours method for Inj\. Acc\. in all three settings, reaching 25\.2 on EVOKE, 76\.4 on VP, and 86\.7 on Sci in Table[1](https://arxiv.org/html/2607.24771#S4.T1)\. This suggests that knowledge\-injection\-specific augmentation and constraints can strengthen target acquisition\. The cost is evaluated retention: KORE obtains Ret\. Avg\. scores of 47\.0, 52\.5, and 48\.3 on EVOKE, VP, and Sci, respectively, lower than the base model and online\-distillation baselines\. KORE therefore represents the injection\-dominant side of the trade\-off\.

Retention\-oriented baselines limit drift but under\-inject\.SDFT \(w/ ref\.\) and KeepLoRA better maintain Ret\. Avg\. but are weaker on Inj\. Acc\. On EVOKE, SDFT keeps Ret\. Avg\. at 55\.1, close to the base model’s 57\.2, but reaches only 20\.9 Inj\. Acc\. KeepLoRA reaches 56\.6 Ret\. Avg\. but only 17\.3 Inj\. Acc\. MoELoRA and SEFE show intermediate behavior across the three settings\. These results are consistent with the limitation of conservative updates or uniform rollout distillation: they can limit evaluated retention drift, but leave some sparse target facts under\-injected\.

RoCo\-ACE improves the trade\-off\.RoCoimproves over the SDFT reference point by replacing uniform rollout distillation with same\-rollout likelihood contrast\. On EVOKE, it increases Inj\. Acc\. from 20\.9 to 23\.7 and Ret\. Avg\. from 55\.1 to 56\.6\. AddingACEfurther supplies sparse correction for authoritative anchors omitted from or weakly matched by the rollout\.RoCo\-ACEachieves the best EVOKE Inj\. Acc\. among compared methods, 27\.6, while keeping Ret\. Avg\. at 56\.5, only 0\.7 below the base model\. Compared with KORE, it improves EVOKE Inj\. Acc\. by 2\.4 points and Ret\. Avg\. by 9\.5 points\. Compared with SFT, it improves them by 7\.5 and 24\.7 points\. The same pattern holds on VP and Sci in Table[1](https://arxiv.org/html/2607.24771#S4.T1)\.RoCo\-ACEobtains the best Inj\. Acc\. on both settings, 77\.8 and 87\.5, while keeping Ret\. Avg\. at 56\.2 and 56\.3, close to SDFT \(w/ ref\.\) and higher than KORE\. Overall, the results support the central design:RoCoreweights reference\-supported rollout tokens through same\-rollout likelihood contrast, whileACEadds sparse reference\-side correction without full\-answer imitation\.

Table 2:Ablation results for Qwen3\-VL\-8B with full retention metrics\. EVOKE measures injected\-knowledge accuracy\. Retention columns report held\-out general multimodal and safety accuracy, and Ret\. Avg\. averages TreeBench, VStar, MathVision, MMStar, BabyVision, and MM\-Safety\. Higher values are better\. Bold and bold\-italic mark the best and second\-best scores in each column\.
### 4\.3Ablation Study

Table[2](https://arxiv.org/html/2607.24771#S4.T2)isolates the main design choices\. SDFT \(w/ ref\.\) andRoCouse the same rollout, reference\-conditioned teacher target, optimizer, and training data\. The key difference is whether rollout tokens receive uniform weights or weights from same\-rollout likelihood contrast\.RoCoimproves EVOKE Inj\. Acc\. from 20\.9 to 23\.7 and Ret\. Avg\. from 55\.1 to 56\.6, showing that reweighting reference\-supported rollout tokens improves over uniform reference\-conditioned distillation\.

We further include teacher\-side controls\. First, a fixed\-teacher variant disables EMA teacher updates while keeping the sameRoCo\-ACEobjective\. Its EVOKE Inj\. Acc\. drops from 27\.6 to 25\.6, while Ret\. Avg\. remains close to the EMA\-teacher setting\. This suggests that a fixed teacher can still keep evaluated retention stable, but provides a weaker injection signal than an EMA teacher that tracks the current student during online distillation\.

Second, a mismatched\-reference control feeds the reference\-conditioned teacher a shuffled authoritative answer from another training example, while keeping the student rollout, original prompt, teacher, optimizer, and matched ACE anchors unchanged\. This preserves the reference pathway and context length but removes semantic alignment between the input and the teacher\-side reference\. EVOKE Inj\. Acc\. drops to 18\.5, indicating that matched reference content is important for the RoCo likelihood contrast\. Ret\. Avg\. remains high, and injection does not collapse to the base level because ACE still provides sparse correction for omitted or weakly matched authoritative anchors, while the rollout\-distillation path remains anchored to the original prompt\.

ACEaddresses the complementary omission case\. SDFT\+ACE reaches 25\.9 EVOKE Inj\. Acc\., showing that sparse anchored correction helps even without same\-rollout likelihood contrast\. ACE only reaches 25\.1 EVOKE Inj\. Acc\. but lowers Ret\. Avg\. to 53\.6, suggesting that anchor correction alone improves injection but lacks the rollout\-distillation signal needed to limit drift\. The fullRoCo\-ACEobjective combines rollout\-side reweighting with reference\-side anchor correction, reaching the best EVOKE Inj\. Acc\. of 27\.6 while keeping Ret\. Avg\. at 56\.5\.

The full\-reference CE controls show why anchor\-level correction matters\. Here, SFT applies uniform CE over the complete authoritative answer, while SDFT\+SFT andRoCo\+SFT add the same full\-reference CE term to the corresponding online\-distillation objectives\. SDFT\+SFT andRoCo\+SFT improve injection less effectively than their ACE counterparts and lower Ret\. Avg\. to 53\.0 and 54\.4, respectively\. This shows that the gains do not come merely from adding ordinary supervised CE\. Anchor\-level correction is more favorable than full\-answer token\-level imitation because it restricts supervised loss to extracted factual spans rather than generic reference wording or answer style\.

### 4\.4Qualitative Example

Figure[4](https://arxiv.org/html/2607.24771#S4.F4)shows an EVOKE example from the training logs\. Additional cases are provided in Appendix[C](https://arxiv.org/html/2607.24771#A3)\. The rollout reaches the right visual and topical neighborhood, mentioning Amsterdam, NDSM, street art, and graffiti, but omits the target museum identity and several concrete facts\.RoCoassigns larger weights to these reference\-supported rollout tokens, whileACEadds sparse anchored correction for omitted authoritative anchors such as “STRAAT Museum”, “October 9, 2020”, and “86,000 sq ft”\. This example illustrates the intended split: rollout\-side reweighting for content already reached on\-policy, and reference\-side correction for omitted facts\.

PromptCould you explain the details of the museum featured in the image?![Refer to caption](https://arxiv.org/html/2607.24771v1/straat_museum_wiki.png)

Student rolloutThe model identifies a different Amsterdam museum and mixes unsupported history about a warehouse and cooperative organizations\. It nevertheless stays close to the visual category by mentioning Amsterdam, NDSM, street art, graffiti, and a cultural venue\.

Authoritative referenceThe STRAAT Museum is an Amsterdam NDSM street\-art and graffiti museum opened on October 9, 2020\. Its 86,000 sq ft space contains over 180 works from more than 170 artists, includes a mezzanine gallery, and features exterior murals such as Eduardo Kobra’s 2016 Anne Frank depiction\.

RoCo signalHigh\-weight rollout spans include“museum in Amsterdam”,“NDSM”,“street art”, and“graffiti”\. These partial tokens preserve useful visual and topical context\.

ACE correctionMissing anchors include“STRAAT Museum”,“October 9, 2020”,“86,000 sq ft”,“over 180 works”,“more than 170 artists”,“mezzanine gallery”, and“Eduardo Kobra”\.

Figure 4:Representative EVOKE example illustrating the complementarity betweenRoCoreweighting andACEanchored correction\.RoCoemphasizes reference\-supported rollout tokens already reached on\-policy, whileACEsupplies sparse correction for authoritative anchors omitted from the rollout\.

## 5Conclusion

We presentedRoCo\-ACE, a rollout\-conditioned contrastive online distillation objective for knowledge injection\.RoCouses same\-rollout reference\-free/reference\-conditioned likelihood contrast to reweight reference\-supported rollout tokens, whileACEadds sparse anchored correction for authoritative anchors omitted from or weakly matched by the rollout\. Together, the two objectives inject sparse factual content without fitting the full reference answer as a token\-level imitation target\. Experiments across three knowledge\-injection settings and multiple retention benchmarks show improved injected\-knowledge accuracy while keeping evaluated retention close to the base model\.

## Limitations

RoCo\-ACEis designed for knowledge\-injection settings where each update is accompanied by an authoritative natural\-language reference\. Its supervision is organized around student rollout tokens and extracted factual anchors, making it most directly suited to entity, event, domain, and visual\-world updates whose target facts can be expressed as compact text spans\. In label\-only settings, very short answers, or cases where the relevant evidence is mostly implicit in the image, the available anchor signal becomes weaker\.

Although images are included in both student and teacher contexts, the current objective does not use region\-level, OCR\-level, or object\-level evidence annotations to explicitly ground each corrected anchor\. A natural extension is to tie anchored factual correction more directly to visual evidence through image\-ablation contrast, OCR/entity grounding, or region\-level supervision\. Finally, retention is evaluated empirically on held\-out multimodal and safety benchmarks; the method is not intended to provide formal guarantees over all non\-updated capabilities or deployment distributions\.

## References

- Agarwal et al\. \(2023\)Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos, Matthieu Geist, and Olivier Bachem\. 2023\.[On\-policy distillation of language models: Learning from self\-generated mistakes](https://arxiv.org/abs/2306.13649)\.*arXiv preprint arXiv:2306\.13649*\.
- Alayrac et al\. \(2022\)Jean\-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, and 8 others\. 2022\.[Flamingo: A visual language model for few\-shot learning](https://proceedings.neurips.cc/paper_files/paper/2022/hash/960a172bc7fbf0177ccccbb411a7d800-Abstract-Conference.html)\.In*Advances in Neural Information Processing Systems*, volume 35, pages 23716–23736\.
- Bai et al\. \(2023\)Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou\. 2023\.[Qwen\-VL: A versatile vision\-language model for understanding, localization, text reading, and beyond](https://arxiv.org/abs/2308.12966)\.*arXiv preprint arXiv:2308\.12966*\.
- Che et al\. \(2025\)Chang Che, Ziqi Wang, Pengwan Yang, Qi Wang, Hui Ma, and Zenglin Shi\. 2025\.[Lora in lora: Towards parameter\-efficient architecture expansion for continual visual instruction tuning](https://arxiv.org/abs/2508.06202)\.*arXiv preprint arXiv:2508\.06202*\.
- Chen et al\. \(2025\)Jinpeng Chen, Runmin Cong, Yuzhi Zhao, Hongzheng Yang, Guangneng Hu, Horace Ho Shing Ip, and Sam Kwong\. 2025\.[Sefe: Superficial and essential forgetting eliminator for multimodal continual instruction tuning](https://arxiv.org/abs/2505.02486)\.*arXiv preprint arXiv:2505\.02486*\.
- Chen et al\. \(2026\)Liang Chen, Weichu Xie, Yiyan Liang, Hongfeng He, Hans Zhao, Zhibo Yang, Zhiqi Huang, Haoning Wu, Haoyu Lu, Y\. charles, Yiping Bao, Yuantao Fan, Guopeng Li, Haiyang Shen, Xuanzhong Chen, Wendong Xu, Shuzheng Si, Zefan Cai, Wenhao Chai, and 10 others\. 2026\.[BabyVision: Visual reasoning beyond language](https://arxiv.org/abs/2601.06521)\.*Preprint*, arXiv:2601\.06521\.
- Chen et al\. \(2024\)Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, and Feng Zhao\. 2024\.[Are we on the right way for evaluating large vision\-language models?](https://arxiv.org/abs/2403.20330)In*Advances in Neural Information Processing Systems*\.
- Dai et al\. \(2023\)Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N\. Fung, and Steven C\. H\. Hoi\. 2023\.[InstructBLIP: Towards general\-purpose vision\-language models with instruction tuning](https://proceedings.neurips.cc/paper_files/paper/2023/hash/9a6a435e75419a836fe47ab6793623e6-Abstract-Conference.html)\.In*Advances in Neural Information Processing Systems*, volume 36\.
- Gu et al\. \(2023\)Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang\. 2023\.[Minillm: Knowledge distillation of large language models](https://arxiv.org/abs/2306.08543)\.*arXiv preprint arXiv:2306\.08543*\.
- Hinton et al\. \(2015\)Geoffrey Hinton, Oriol Vinyals, and Jeff Dean\. 2015\.[Distilling the knowledge in a neural network](https://arxiv.org/abs/1503.02531)\.In*NIPS Deep Learning and Representation Learning Workshop*\.
- Hu et al\. \(2022\)Edward J\. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen\-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen\. 2022\.[LoRA: Low\-rank adaptation of large language models](https://openreview.net/forum?id=nZeVKeeFYf9)\.In*International Conference on Learning Representations*\.
- Jiang et al\. \(2025a\)Kailin Jiang, Yuntao Du, Yukai Ding, Yuchen Ren, Ning Jiang, Zhi Gao, Zilong Zheng, Lei Liu, Bin Li, and Qing Li\. 2025a\.[When large multimodal models confront evolving knowledge: Challenges and pathways](https://arxiv.org/abs/2505.24449)\.*arXiv preprint arXiv:2505\.24449*\.
- Jiang et al\. \(2025b\)Kailin Jiang, Hongbo Jiang, Ning Jiang, Zhi Gao, Jinhe Bi, Yuchen Ren, Bin Li, Yuntao Du, Lei Liu, and Qing Li\. 2025b\.[Kore: Enhancing knowledge injection for large multimodal models via knowledge\-oriented augmentations and constraints](https://arxiv.org/abs/2510.19316)\.*arXiv preprint arXiv:2510\.19316*\.
- Kirkpatrick et al\. \(2017\)James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A\. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska\-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell\. 2017\.[Overcoming catastrophic forgetting in neural networks](https://doi.org/10.1073/pnas.1611835114)\.*Proceedings of the National Academy of Sciences*, 114\(13\):3521–3526\.
- Li et al\. \(2024\)Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li\. 2024\.[LLaVA\-OneVision: Easy visual task transfer](https://arxiv.org/abs/2408.03326)\.*arXiv preprint arXiv:2408\.03326*\.
- Li et al\. \(2026\)Gengsheng Li, Tianyu Yang, Junfeng Fang, Mingyang Song, Mao Zheng, Haiyun Guo, Dan Zhang, Jinqiao Wang, and Tat\-Seng Chua\. 2026\.[Unifying group\-relative and self\-distillation policy optimization via sample routing](https://arxiv.org/abs/2604.02288)\.*arXiv preprint arXiv:2604\.02288*\.
- Li et al\. \(2023\)Junnan Li, Dongxu Li, Silvio Savarese, and Steven C\. H\. Hoi\. 2023\.[BLIP\-2: Bootstrapping language\-image pre\-training with frozen image encoders and large language models](https://proceedings.mlr.press/v202/li23q.html)\.In*Proceedings of the 40th International Conference on Machine Learning*, pages 19730–19742\.
- Li and Hoiem \(2016\)Zhizhong Li and Derek Hoiem\. 2016\.[Learning without forgetting](https://doi.org/10.1007/978-3-319-46493-0_37)\.In*Proceedings of the European Conference on Computer Vision*, pages 614–629\. Springer\.
- Liu et al\. \(2023\)Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee\. 2023\.[Visual instruction tuning](https://proceedings.neurips.cc/paper_files/paper/2023/hash/6dcf277ea32ce3288914faf369fe6de0-Abstract-Conference.html)\.In*Advances in Neural Information Processing Systems*, volume 36\.
- Liu et al\. \(2024a\)Haowei Liu, Xi Zhang, Haiyang Xu, Yaya Shi, Chaoya Jiang, Ming Yan, Ji Zhang, Fei Huang, Chunfeng Yuan, Bing Li, and Weiming Hu\. 2024a\.[MIBench: Evaluating multimodal large language models over multiple images](https://doi.org/10.18653/v1/2024.emnlp-main.1250)\.In*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*, pages 22417–22428, Miami, Florida, USA\. Association for Computational Linguistics\.
- Liu et al\. \(2024b\)Xin Liu, Yichen Zhu, Jindong Gu, Yunshi Lan, Chao Yang, and Yu Qiao\. 2024b\.[MM\-SafetyBench: A benchmark for safety evaluation of multimodal large language models](https://doi.org/10.1007/978-3-031-72992-8_22)\.In*European Conference on Computer Vision*, pages 386–403\. Springer\.
- Luo et al\. \(2026\)Mao\-Lin Luo, Zi\-Hao Zhou, Yi\-Lin Zhang, Yuanyu Wan, Min\-Ling Zhang, and Tong Wei\. 2026\.[Keeplora: Continual learning with residual gradient adaptation](https://openreview.net/forum?id=T3Vc5fkTzV)\.In*The Fourteenth International Conference on Learning Representations*\.
- Luo et al\. \(2024\)Tongxu Luo, Jiahe Lei, Fangyu Lei, Weihao Liu, Shizhu He, Jun Zhao, and Kang Liu\. 2024\.[Moelora: Contrastive learning guided mixture of experts on parameter\-efficient fine\-tuning for large language models](https://arxiv.org/abs/2402.12851)\.*arXiv preprint arXiv:2402\.12851*\.
- Ouyang et al\. \(2022\)Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe\. 2022\.[Training language models to follow instructions with human feedback](https://proceedings.neurips.cc/paper_files/paper/2022/hash/b1efde53be364a73914f58805a001731-Abstract-Conference.html)\.In*Advances in Neural Information Processing Systems*, volume 35, pages 27730–27744\.
- Pi et al\. \(2024\)Renjie Pi, Tianyang Han, Jianshu Zhang, Yueqi Xie, Rui Pan, Qing Lian, Hanze Dong, Jipeng Zhang, and Tong Zhang\. 2024\.[MLLM\-protector: Ensuring MLLM’s safety without hurting performance](https://doi.org/10.18653/v1/2024.emnlp-main.895)\.In*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*, pages 16012–16027, Miami, Florida, USA\. Association for Computational Linguistics\.
- Qwen Team \(2025\)Qwen Team\. 2025\.[Qwen3 technical report](https://arxiv.org/abs/2505.09388)\.*arXiv preprint arXiv:2505\.09388*\.
- Sang et al\. \(2026\)Hejian Sang, Yuanda Xu, Zhengze Zhou, Ran He, Zhipeng Wang, and Jiachen Sun\. 2026\.[Crisp: Compressed reasoning via iterative self\-policy distillation](https://arxiv.org/abs/2603.05433)\.*arXiv preprint arXiv:2603\.05433*\.
- Shenfeld et al\. \(2026\)Idan Shenfeld, Mehul Damani, Jonas Hübotter, and Pulkit Agrawal\. 2026\.[Self\-distillation enables continual learning](https://arxiv.org/abs/2601.19897)\.*arXiv preprint arXiv:2601\.19897*\.
- Song and Zheng \(2026\)Mingyang Song and Mao Zheng\. 2026\.[A survey of on\-policy distillation for large language models](https://arxiv.org/abs/2604.00626)\.*arXiv preprint arXiv:2604\.00626*\.
- Tang et al\. \(2025\)Pengwei Tang, Xiaolin Hu, Yong Liu, Lizhong Ding, Dongjie Zhang, Xing Wu, and Debing Zhang\. 2025\.[Put the space of lora initialization to the extreme to preserve pre\-trained knowledge](https://arxiv.org/abs/2503.02659)\.*arXiv preprint arXiv:2503\.02659*\.
- Wang et al\. \(2026\)Haochen Wang, Xiangtai Li, Zilong Huang, Anran Wang, Jiacong Wang, Tao Zhang, Jiani Zheng, Sule Bai, Zijian Kang, Jiashi Feng, Zhuochen Wang, and Zhaoxiang Zhang\. 2026\.[Traceable evidence enhanced visual grounded reasoning: Evaluation and method](https://openreview.net/forum?id=zwMpimSZEq)\.In*The Fourteenth International Conference on Learning Representations*\.
- Wang et al\. \(2024\)Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Mingjie Zhan, and Hongsheng Li\. 2024\.[Measuring multimodal mathematical reasoning with the math\-vision dataset](https://arxiv.org/abs/2402.14804)\.*arXiv preprint arXiv:2402\.14804*\.
- Wang et al\. \(2021\)Shipeng Wang, Xiaorong Li, Jian Sun, and Zongben Xu\. 2021\.[Training networks in null space of feature covariance for continual learning](https://openaccess.thecvf.com/content/CVPR2021/html/Wang_Training_Networks_in_Null_Space_of_Feature_Covariance_for_Continual_CVPR_2021_paper.html)\.In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 184–193\.
- Wu and Xie \(2024\)Penghao Wu and Saining Xie\. 2024\.[V\*: Guided visual search as a core mechanism in multimodal llms](https://openaccess.thecvf.com/content/CVPR2024/html/Wu_V_Guided_Visual_Search_as_a_Core_Mechanism_in_Multimodal_CVPR_2024_paper.html)\.In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 13084–13094\.
- Wu et al\. \(2025\)Weixiang Wu, Dongxu Wang, Renze Xu, Xinnan Ma, Zhen Zeng, Xiaoshuai Guo, Gao Huang, and Yinghui Wang\. 2025\.[Automating steering for safe multimodal large language models](https://aclanthology.org/2025.emnlp-main.41/)\.In*Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing*\.
- Yang et al\. \(2026\)Chenxu Yang, Chuanyu Qin, Qingyi Si, Minghui Chen, Naibin Gu, Dingyu Yao, Zheng Lin, Weiping Wang, Jiaqi Wang, and Nan Duan\. 2026\.[Self\-distilled rlvr](https://arxiv.org/abs/2604.03128)\.*arXiv preprint arXiv:2604\.03128*\.
- Ye et al\. \(2026\)Tianzhu Ye, Li Dong, Xun Wu, Shaohan Huang, and Furu Wei\. 2026\.[On\-policy context distillation for language models](https://arxiv.org/abs/2602.12275)\.*arXiv preprint arXiv:2602\.12275*\.
- Zhao et al\. \(2025\)Hongbo Zhao, Fei Zhu, Haiyang Guo, Meng Wang, Rundong Wang, Gaofeng Meng, and Zhaoxiang Zhang\. 2025\.[Mllm\-cl: Continual learning for multimodal large language models](https://arxiv.org/abs/2506.05453)\.*arXiv preprint arXiv:2506\.05453*\.
- Zhao et al\. \(2026\)Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover\. 2026\.[Self\-distilled reasoner: On\-policy self\-distillation for large language models](https://arxiv.org/abs/2601.18734)\.*arXiv preprint arXiv:2601\.18734*\.

## Appendix AAnchor Extraction and Supervision\-Weight Diagnostics

Table 3:Token\-allocation diagnostics on EVOKE training rollouts\. The diagnostic pool consists of valid student\-rollout tokens plus missing reference\-anchor tokens\. The three categories are mutually exclusive and sum to one, showing how RoCo\-ACE allocates RoCo and ACE supervision across reference\-supported rollout content, generic rollout content, and missing target anchors\. Percentages are computed from logged RoCo\-ACE training intervals under the paper’s default anchor weighting\.This section describes the lightweight span extraction used byACEand RoCo span smoothing, and reports a supervision\-weight diagnostic that connects these extracted units to the claims in the main paper\. The diagnostic is not intended to estimate final gradient norms or normalized loss mass\. Instead, it shows how training units are assigned to the RoCo contrast term, the RoCo floor term, and reference\-side ACE correction\.

#### Span extraction\.

Span construction is a tokenizer\-level procedure and does not rely on external NER or parsing tools\. We decode valid response tokens, remove special tokens, punctuation\-only tokens, and common function words, and group contiguous content tokens containing CJK characters, Latin letters, or digits into spans\. A span is kept if it contains at least two visible characters or any digit, and we cap each span by a maximum token length\. This simple extractor is designed to capture compact factual expressions such as entity names, dates, numbers, titles, locations, and short noun phrases\.

#### Anchor matching for ACE\.

For ACE, the same extractor is applied to the tokenized authoritative answer\. Each reference span is normalized by lowercasing and retaining only alphanumeric and CJK characters; the student rollout is normalized in the same way\. For each extracted reference spanss, we compute an anchor match scoremi​\(s,yi\)∈\[0,1\]m\_\{i\}\(s,y\_\{i\}\)\\in\[0,1\]against the rollout\. In our default implementation,mi​\(s,yi\)=1m\_\{i\}\(s,y\_\{i\}\)=1when the normalized span string appears in the normalized rollout and0otherwise; the same notation also supports partial\-match variants\. Thus, matched anchors receive the base anchor weight, while omitted or weakly matched anchors receive larger weights\. The ACE token set𝒜i\\mathcal\{A\}\_\{i\}is the union of reference\-token positions covered by the extracted anchors\. Tokens outside𝒜i\\mathcal\{A\}\_\{i\}serve only as teacher\-forced context and do not receive supervised CE loss, making ACE narrower than full\-reference SFT\.

#### Span smoothing for RoCo\.

The same extractor is also used for RoCo span\-aware smoothing on the rollout side\. After preliminary RoCo weights are computed from the same\-rollout likelihood contrast, we aggregate token weights within each extracted rollout span and broadcast the span score back to all tokens in that span\. For notation, the resulting smoothed weights are still denoted bywi,tw\_\{i,t\}\. This reduces subword\-level noise for names, dates, numbers, multilingual expressions, and other compact factual units without requiring external entity taggers\. The diagnostic in Table[3](https://arxiv.org/html/2607.24771#A1.T3)classifies rollout tokens by their pre\-smoothing contrast activation, so it reflects the source of the RoCo weight signal before span\-level broadcasting\.

#### Supervision\-weight diagnostic\.

Table[3](https://arxiv.org/html/2607.24771#A1.T3)reports a token\-level diagnostic on EVOKE training rollouts\. The diagnostic pool contains valid rollout tokens and omitted authoritative\-anchor tokens\. These units correspond to the three supervision cases used byRoCo\-ACE: rollout tokens receiving the RoCo contrast term, rollout tokens receiving mainly the RoCo floor term, and reference\-side anchors receiving ACE omission correction\.

For rollout tokens, Table[3](https://arxiv.org/html/2607.24771#A1.T3)uses the RoCo likelihood contrast as the diagnostic rule\. Tokens withΔi,t\>τ\\Delta\_\{i,t\}\>\\tauare counted as reference\-supported rollout tokens; they account for 30\.8% of the diagnostic pool and receive the RoCo floor plus the additional contrast term\. Tokens withΔi,t≤τ\\Delta\_\{i,t\}\\leq\\tauare counted as generic or non\-activated rollout tokens; they account for 57\.3% and mainly receive the online\-distillation floor\. For reference\-side units, omitted authoritative\-anchor tokens account for 11\.9% and receive larger ACE weights through the omission term1−ρi​\(s,yi\)1\-\\rho\_\{i\}\(s,y\_\{i\}\)\.

This distribution supports the intended supervision split\. Uniform reference\-conditioned rollout distillation would place its rollout\-side loss over all valid rollout tokens, most of which are not contrast\-activated in this diagnostic\. RoCo instead reallocates additional distillation weight to reference\-supported rollout tokens, while ACE supplies sparse correction for omitted authoritative anchors that are not available as generated rollout tokens in the current step\. Because Table[3](https://arxiv.org/html/2607.24771#A1.T3)reports token\-count categories rather than normalized loss mass, it should be interpreted as a weight\-assignment diagnostic rather than an exact gradient\-budget decomposition\.

## Appendix BFull Baseline Comparison Results

![[Uncaptioned image]](https://arxiv.org/html/2607.24771v1/radar_vp_all.png)

Figure 5:Radar comparison on VP, showing target knowledge injection and general capability retention across six multimodal benchmarks\.
![[Uncaptioned image]](https://arxiv.org/html/2607.24771v1/radar_sci_all.png)

Figure 6:Radar comparison on Sci, showing target knowledge injection and general capability retention across six multimodal benchmarks\.
Table[4](https://arxiv.org/html/2607.24771#A2.T4)reports the full baseline comparison on EVOKE using Qwen3\-VL\-8B as the base model\. In addition to the injected\-knowledge accuracy, we include all individual retention benchmarks to show how each method affects general multimodal capabilities after knowledge injection\. This table complements the main\-text discussion by making the injection–retention trade\-off explicit across direct fine\-tuning, constrained\-update baselines, online distillation, and our RoCo\-ACE variants\.

Table 4:Main baseline comparison on Qwen3\-VL\-8B\. The knowledge\-injection column reports LLM\-judge accuracy on EVOKE\. Retention benchmark columns report accuracy on held\-out general multimodal and safety benchmarks, and Ret\. Avg\. averages TreeBench, VStar, MathVision, MMStar, BabyVision, and MM\-Safety\. Higher values are better; bold and bold\-italic mark the best and second\-best scores in each column\.Table[5](https://arxiv.org/html/2607.24771#A2.T5)further evaluates the same set of methods on the VP and Sci knowledge\-injection benchmarks from MLLM\-CL\. VP focuses on ability\-oriented injection, while Sci focuses on domain\-oriented knowledge injection\. We keep the retention benchmark suite unchanged across EVOKE, VP, and Sci, which allows a consistent comparison of whether each method can acquire new knowledge while preserving general capabilities\. Across these additional settings, RoCo\-ACE maintains a favorable balance between injected\-knowledge accuracy and retention performance\.

Table 5:Baseline comparison on the MLLM\-CL VP and Sci knowledge\-injection benchmarks\(Zhao et al\.,[2025](https://arxiv.org/html/2607.24771#bib.bib38)\)with full retention metrics\. Acc\. reports injected\-knowledge accuracy for the corresponding dataset\. Retention columns report accuracy after training on that dataset, and Ret\. Avg\. averages TreeBench, VStar, MathVision, MMStar, BabyVision, and MM\-Safety\. Higher values are better; bold and bold\-italic mark the best and second\-best scores in each column\.Figures[5](https://arxiv.org/html/2607.24771#A2.F5)and[6](https://arxiv.org/html/2607.24771#A2.F6)provide the corresponding radar visualizations for VP and Sci\. We move these plots to the appendix because the main text summarizes the three datasets in Table[1](https://arxiv.org/html/2607.24771#S4.T1)using injected\-knowledge accuracy and Ret\. Avg\.

## Appendix CQualitative Analysis

Table 6:Qualitative EVOKE examples from jointRoCo\-ACEtraining\. We retain three non\-duplicate cases that cover entity under\-injection, sports\-event correction, and collaborator substitution\. Across these cases,RoCoemphasizes useful content already present in the rollout, while ACE supplies authoritative facts that the rollout omits or contradicts\.Case C: person entity with sports, medical, and family factsQuestionCan you describe the human depicted in the image?![[Uncaptioned image]](https://arxiv.org/html/2607.24771v1/lida_maria_manthopoulou_wiki.png)Gold factsLida\-Maria Manthopoulou, born onJune 14, 2005, is aGreek para\-athletewho competed in the2024 Summer Paralympicsand wonsilver in the 100 metres T38\. She representsElpides Thessaloniki, was diagnosed withmultiple sclerosisbefore the Paralympics, and is connected to actorAias Manthopoulosand other public family members\.SFT / direct fittingCapturesLida\-Maria Manthopoulou, Greek para\-athlete identity, and a 2024 Paralympics association, but omits most event, medal, club, health, and family facts\.KORE / constrained updateAdds thesilver medaland100 metres T38event, but only briefly mentions background and omits most medical and family details\.SDFT / online distillationKeeps natural person\-description style, but avoids firm details about theParalympic event, medal, diagnosis, club, and family lineage\.RoCo\-ACE / rollout\-conditioned anchorsRecovers the person identity,Greek para\-athlete,June 14, 2005,2024 Summer Paralympics,silver in the 100 metres T38,Elpides Thessaloniki, and theAias Manthopoulosfamily link, while leaving out some extended family details\.Table 7:Baseline\-family behavior examples \(continued\)\.Table[6](https://arxiv.org/html/2607.24771#A3.T6)presents three non\-duplicate EVOKE cases selected from the jointRoCo\-ACEtraining logs\. The color scheme follows the method overview: orange boxes summarize the student rollout, green boxes show the strongestRoCorollout signals, and red boxes show facts corrected by Anchored Cross\-Entropy\. Together, these examples show howRoCoemphasizes useful on\-policy content while ACE supplies missing or contradicted reference facts\.

The baseline\-family examples in Appendix[C](https://arxiv.org/html/2607.24771#A3)further illustrate how different method families behave on sparse authoritative references\. Direct fitting usually captures only a few salient facts, constrained\-update methods cover a moderate subset, online distillation preserves natural response style but under\-injects, andRoCo\-ACErecovers more anchors without copying the full reference\.

## Appendix DSupplementary Results on Qwen3\-VL\-30B

Table 8:Supplementary Qwen3\-VL\-30B comparison\. EVOKE is evaluated with the same LLM\-judge protocol as the main table\. Retention columns report held\-out general multimodal and safety accuracy, and Ret\. Avg\. averages TreeBench, VStar, MathVision, MMStar, BabyVision, and MM\-Safety\. Higher values are better; bold and bold\-italic mark the best and second\-best scores in each column\.Table 9:Supplementary component ablations on Qwen3\-VL\-30B with full retention metrics\. EVOKE measures injected\-knowledge accuracy\. Retention columns report held\-out general multimodal and safety accuracy, and Ret\. Avg\. averages TreeBench, VStar, MathVision, MMStar, BabyVision, and MM\-Safety\. Higher values are better; bold and bold\-italic mark the best and second\-best scores in each column\.Table[8](https://arxiv.org/html/2607.24771#A4.T8)keeps the benchmark columns and baseline groups consistent with the main Qwen3\-VL\-8B table to test whether the injection\-retention trends persist at a larger model scale\. The 30B results show the same qualitative pattern: direct fitting improves injected knowledge but sacrifices retention, constrained\-update methods are more conservative, andRoCo\-ACEgives the strongest injected\-knowledge result while keeping retention close to or above the base model\.

Table[9](https://arxiv.org/html/2607.24771#A4.T9)reports component\-level Qwen3\-VL\-30B ablations with every retention benchmark\. The larger model follows the same trend as Qwen3\-VL\-8B: full\-answer SFT controls improve EVOKE less efficiently and reduce retention, ACE improves target injection, and combining it with RoCo gives the best injected\-knowledge accuracy among the compared variants while keeping the retention average close to the base model\.

## Appendix ESupplementary Results on InternVL3\.5\-8B

Table 10:InternVL3\.5\-8B component comparison on EVOKE\. EVOKE measures injected\-knowledge accuracy\. Retention columns report held\-out multimodal and safety benchmark accuracy, and Ret\. Avg\. averages TreeBench, VStar, MathVision, MMStar, BabyVision, and MM\-Safety\. Higher values are better; bold and bold\-italic mark the best and second\-best scores in each column\.Table[10](https://arxiv.org/html/2607.24771#A5.T10)reports an EVOKE component comparison on InternVL3\.5\-8B, testing whether the component\-level trend ofRoCo\-ACEtransfers beyond the Qwen3\-VL model family\. The comparison follows the same protocol as the Qwen3\-VL ablations: the base model is compared withRoCo, ACE\-only training, and the fullRoCo\-ACEobjective on EVOKE, while using the same six held\-out retention benchmarks\.

The results are consistent with the intended component behavior\.RoCoraises EVOKE Inj\. Acc\. from 11\.1 to 17\.8 while keeping Ret\. Avg\. nearly unchanged relative to the base model\. ACE\-only training provides a stronger injection signal, reaching 19\.2 EVOKE Inj\. Acc\., but lowers Ret\. Avg\. to 46\.7, reflecting the cost of anchor correction without rollout\-side distillation\. The fullRoCo\-ACEobjective gives the best EVOKE Inj\. Acc\. of 20\.9 while keeping Ret\. Avg\. at 49\.3, close to the base model’s 49\.5\. This supports the same conclusion as the Qwen3\-VL experiments:RoCohelps limit evaluated retention drift, whileACEsupplies sparse correction for omitted authoritative anchors\.

## Appendix FHyperparameter Sensitivity

Table 11:Hyperparameter sensitivity grid forRoCo\-ACEon EVOKE\. EVOKE measures injected\-knowledge accuracy, and Ret\. Avg\. averages TreeBench, VStar, MathVision, MMStar, BabyVision, and MM\-Safety\. The RoCo\-only and default rows reuse the corresponding results from Table[4](https://arxiv.org/html/2607.24771#A2.T4); bold and bold\-italic mark the best and second\-best scores within each hyperparameter group\.Table[11](https://arxiv.org/html/2607.24771#A6.T11)reports sensitivity analysis for the three hyperparameters that directly control the two components ofRoCo\-ACE\. The ACE loss weightλACE\\lambda\_\{\\mathrm\{ACE\}\}controls how strongly missing or weakly covered authoritative anchors are supervised\. The missing\-span multiplierβmiss\\beta\_\{\\mathrm\{miss\}\}controls how much more weight is assigned to anchors absent from the student rollout\. The RoCo contrast marginτ\\taucontrols how much the reference\-conditioned teacher likelihood must exceed the base\-teacher likelihood before a rollout token receives stronger distillation weight\.

TheλACE\\lambda\_\{\\mathrm\{ACE\}\}rows show the role of anchored correction\. SettingλACE=0\\lambda\_\{\\mathrm\{ACE\}\}=0reduces the method to RoCo\-only, which keeps Ret\. Avg\. high but lowers EVOKE accuracy to 23\.7\. The defaultλACE=0\.2\\lambda\_\{\\mathrm\{ACE\}\}=0\.2improves EVOKE to 27\.6 with almost no retention loss, while the more aggressive settingλACE=1\.0\\lambda\_\{\\mathrm\{ACE\}\}=1\.0slightly increases EVOKE but drops Ret\. Avg\. to 52\.5\. This suggests that ACE should remain a targeted correction term rather than dominate the online\-distillation objective\.

Theβmiss\\beta\_\{\\mathrm\{miss\}\}andτ\\taurows show the same balance\. Moderate missing\-span emphasis under\-corrects omitted facts, while overly large missing\-span emphasis pressures retention\. For the RoCo margin,τ=0\.2\\tau=0\.2gives the best injection\-retention balance among the tested settings: a smaller margin is too permissive and a larger margin is too conservative\. Overall, the default configuration is selected because it achieves strong injected\-knowledge accuracy without turning training into full\-reference imitation\.

## Appendix GAdditional Discussion on Retention Scope and Reference Quality

RoCo\-ACEdoes not explicitly guarantee preservation of every general capability\. Its preservation mechanism is indirect: it reduces irrelevant updates during injected\-data training\. This distinction matters because many regressions are not caused by learning a new fact itself, but by repeatedly training on long, narrow, stylistically similar answers\. The method reduces that source of regression by making the update selective with respect to factual support from the reference\.

The method also depends on reference quality\. If the authoritative answer is noisy, overly broad, or stylistically biased, the RoCo contrast can amplify the wrong signal and ACE can reinforce incorrect spans\. Span\-level diagnostics are therefore important: the system should log which generated spans receive high contrast weights and which reference spans receive anchor correction\. Such diagnostics make failure modes visible and help distinguish genuine factual injection from style imitation\. The open\-ended injection metric also relies on an LLM judge, so judge\-agreement or human\-audit results should be reported when the benchmark setting allows them\.

## Appendix HBenchmark Details

#### Knowledge injection benchmarks\.

EVOKE\(Jiang et al\.,[2025a](https://arxiv.org/html/2607.24771#bib.bib12)\)contains image\-grounded entity and news updates with authoritative reference answers\. This setting is deliberately different from rich\-demonstration continual\-learning settings studied by SDFT\-style methods: the reference information is often short, sparse, and localized to a small number of factual spans, making uniform online distillation an incomplete injection signal\. To test whether the method generalizes beyond time\-sensitive news and entity updates, we additionally include two MLLM\-CL benchmarks\(Zhao et al\.,[2025](https://arxiv.org/html/2607.24771#bib.bib38)\): VP from the ability continual\-learning suite and Sci from the domain continual\-learning suite\. MLLM\-CL organizes ability data as non\-IID tasks that introduce new visual capabilities, and domain data as IID tasks drawn from mainstream visual domains\. In our setting, VP evaluates capability\-oriented visual perception injection, while Sci evaluates science\-domain knowledge injection\.

We follow the original data splits released with each benchmark\. EVOKE uses its provided entity/news knowledge\-injection split with authoritative reference answers, and VP and Sci use the corresponding MLLM\-CL training and test splits without additional resampling\. All compared methods are trained and evaluated on the same split for each benchmark; no retention benchmark examples are mixed into the injection training set\.

#### Retention benchmarks\.

TreeBench\(Wang et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib31)\)evaluates traceable visual grounded reasoning and evidence localization\. VStar, following V\*Bench\(Wu and Xie,[2024](https://arxiv.org/html/2607.24771#bib.bib34)\), evaluates fine\-grained visual search in high\-resolution and cluttered scenes\. MathVision\(Wang et al\.,[2024](https://arxiv.org/html/2607.24771#bib.bib32)\)measures multimodal mathematical reasoning over visual problems\. MMStar\(Chen et al\.,[2024](https://arxiv.org/html/2607.24771#bib.bib7)\)evaluates vision\-indispensable multimodal understanding with reduced language\-only shortcuts and data leakage\. BabyVision\(Chen et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib6)\)probes core visual abilities that should be solved with minimal reliance on language priors\. MM\-SafetyBench\(Liu et al\.,[2024b](https://arxiv.org/html/2607.24771#bib.bib21)\)evaluates multimodal safety behavior under image\-conditioned unsafe instructions\. Together, these benchmarks test whether knowledge injection preserves visual grounding, fine\-grained perception, mathematical reasoning, general multimodal understanding, early visual primitives, and safety behavior\.

## Appendix IEvaluation Protocol

We use accuracy as the primary metric\. For open\-ended injected\-knowledge examples, we use Qwen3\-30B\-A3B\-Instruct\-2507 as the LLM judge\(Qwen Team,[2025](https://arxiv.org/html/2607.24771#bib.bib26)\)\. The judge compares each model response against the authoritative answer and returns a binary correctness label\. It is instructed to focus on factual consistency with the reference, including entity identity, event identity, date, number, location, relation, and other task\-specific facts, rather than surface wording\. We report the resulting LLM\-judge accuracy over the evaluation set\.

For general capability benchmarks, we follow the benchmark\-specific answer format and compute accuracy after answer normalization\. For multiple\-choice tasks, the predicted option is extracted from the model response\. When the output is ambiguous or free\-form, the same LLM\-judge protocol is used to decide whether it matches the ground\-truth answer\. We report injected\-knowledge accuracy together with general benchmark accuracy, since a method is only useful if it improves injected facts without causing substantial capability degradation\.

## Appendix JModel and Baseline Details

The main experiments useQwen3\-VL\-8Bas the base model because it exposes realistic retention failures while keeping the full baseline suite computationally tractable\. We includeQwen3\-VL\-30Bexperiments in the supplementary material to check whether the observed trends persist at a larger scale\. Unless otherwise stated, the vision tower is frozen, text and projector\-side trainable components follow the corresponding baseline configuration, andRoCouses one on\-policy generation per prompt with a synchronized reference teacher\.

#### Direct fitting baselines\.

LoRA\(Hu et al\.,[2022](https://arxiv.org/html/2607.24771#bib.bib11)\)is the parameter\-efficient fitting baseline, while SFT\(Ouyang et al\.,[2022](https://arxiv.org/html/2607.24771#bib.bib24)\)directly fits authoritative answers and follows the standard supervised instruction\-tuning paradigm\. These baselines test how much injected knowledge can be acquired through direct answer fitting and how much retention is lost under ordinary supervised updates\.

#### Reference\-conditioned self\-distillation baseline\.

SDFT \(w/ authoritative reference\)\(Shenfeld et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib28)\)uses the same on\-policy student rollout, synchronized teacher, and reference\-conditioned teacher prompt asRoCo, but applies a uniform distillation loss over all valid rollout tokens\. This is the main online\-distillation reference point because every injected example provides an authoritative answer\. A no\-reference vanilla KD baseline is not included in the main comparison: without the authoritative reference, the teacher cannot observe the injected facts and the experiment mainly measures policy preservation rather than knowledge acquisition\.

#### Constrained\-update baselines\.

KORE\(Jiang et al\.,[2025b](https://arxiv.org/html/2607.24771#bib.bib13)\)represents knowledge\-injection\-specific constrained updating and is expected to be strong on target injection, but prior results also show that strong injection does not uniformly preserve every general capability dimension\. Following the KORE protocol, we sample auxiliary general examples from LLaVA\-OneVision\(Li et al\.,[2024](https://arxiv.org/html/2607.24771#bib.bib15)\)to estimate the general representation space used by its retention constraint; these auxiliary examples are not used by our method or by other baselines\. KeepLoRA\(Luo et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib22)\), MoELoRA\(Luo et al\.,[2024](https://arxiv.org/html/2607.24771#bib.bib23)\), and SEFE\(Chen et al\.,[2025](https://arxiv.org/html/2607.24771#bib.bib5)\)represent parameter\-space constrained adaptation methods that reduce forgetting through residual\-gradient updates, expertized LoRA modules, or forgetting\-aware constraints\. Among them, KeepLoRA\(Luo et al\.,[2026](https://arxiv.org/html/2607.24771#bib.bib22)\)is a retention\-oriented baseline that can preserve general behavior more conservatively but may provide weaker injection when the new knowledge is sparse\.

#### Our variants\.

The main variants areRoCoandRoCo\-ACE\. The former isolates rollout\-conditioned contrastive online distillation over student rollouts, while the latter combines it with missing\-span anchored supervision\. Additional component controls, including SDFT\+SFT, SDFT\+ACE,RoCo\+SFT, and ACE only, are reported in the ablation tables\.

#### Implementation form of baselines\.

For Qwen3\-VL\-8B, SFT, SDFT, andRoCovariants use full language\-side updating with the vision tower and multimodal aligner frozen\. LoRA, KeepLoRA, MoELoRA, SEFE, and KORE are implemented as adapter\-based or constrained\-update baselines following their method definitions, using the same rank, target\-module, and training\-budget settings unless the method requires its own constraint or two\-stage procedure\. For Qwen3\-VL\-30B, the non\-adapter methods likewise use full language\-side updating with frozen vision and aligner modules, while adapter\-defined methods remain adapter\-based\. This keeps the comparison aligned with the intended update mechanism of each baseline rather than converting all methods into a single adapter form\.

## Appendix KTraining Configuration and Hyperparameters

Unless otherwise stated, all primary experiments are run for five epochs with learning rate1×10−51\\times 10^\{\-5\}, warmup ratio0\.050\.05, bfloat16 precision, FlashAttention, DeepSpeed ZeRO\-3, per\-device batch size11, and gradient accumulation88\. We set the maximum text sequence length to15361536and the maximum generated completion length to10241024\. For image inputs, we set the minimum and maximum pixel budgets to65,53665\{,\}536and3,379,2003\{,\}379\{,\}200, corresponding to an image\-token range of6464to33003300\. The vision tower and multimodal aligner are frozen\. Full\-model updating is used for SFT, SDFT, andRoCovariants, while adapter\-based baselines use LoRA\-style updates with rank1616, alpha3232, dropout0\.050\.05, and all linear layers as target modules\.

For all online distillation variants, the student samples one on\-policy completion per prompt \(num\_generations=1\) with temperature1\.01\.0, top\-p=1\.0p=1\.0, and one return sequence\. The teacher uses the same base model architecture and the same Qwen3\-VL chat template as the student\. The student prompt contains only the target user question and the optional image\. The reference\-conditioned teacher prompt replaces the user message with a reference\-augmented instruction of the form: target question, authoritative reference answer, and an instruction to generate a new answer that is factually consistent with the reference\. The base teacher prompt uses the original student input without the authoritative reference\. The teacher never generates the training completion in our main setting; it only scores the student’s rollout under these two prompt contexts\. The teacher is synchronized with the student by an exponential\-moving\-average update after every optimization step, so the teacher tracks the current policy while remaining a stable scoring model\.

#### Teacher prompt template\.

For English reference\-conditioned teacher scoring, we use the following template, where\{question\}is the original user query and\{gold\_response\}is the authoritative reference answer:

Target Question:\{question\}Response to the target question:\{gold\_response\}The assistant answer above is the authoritative reference for this training example\.Generate a new answer that stays fully consistent with the same entity/event identity and factual claims\.Use the image only to add visible grounding details and natural wording\.Do not contradict, negate, or correct the reference answer\.Do not claim that the image is unrelated, blank

ForRoCo, we set the distillation loss weightλRoCo=1\.0\\lambda\_\{\\mathrm\{RoCo\}\}=1\.0, the floor token weightw0=1\.0w\_\{0\}=1\.0, the reference\-contrast scaleβΔ=1\.0\\beta\_\{\\Delta\}=1\.0, and the contrast marginτ=0\.2\\tau=0\.2\. Dense token weighting, span aggregation, and weight normalization are enabled\. We do not use top\-mmtruncation or any auxiliary gating for rollout\-conditioned contrastive tokens\. The synchronized teacher is updated after every optimization step with exponential moving average decay0\.9990\.999, corresponding to an effective update coefficient of0\.0010\.001from the current student\.

For Anchored Cross\-Entropy, we set the objective weightλACE=0\.2\\lambda\_\{\\mathrm\{ACE\}\}=0\.2unless otherwise stated\. The base weight inside extracted anchor spans isu0=1\.0u\_\{0\}=1\.0, and the missing\-span weight isβmiss=5\.0\\beta\_\{\\mathrm\{miss\}\}=5\.0; reference tokens outside extracted factual anchors are not included in the ACE loss\. Missing\-span weighting and span aggregation are enabled for ACE variants\. For SDFT \(w/ ref\.\), we keep the same student rollout and reference\-conditioned teacher\-scoring pipeline asRoCo, but replace the rollout\-conditioned contrastive weights with uniform weights and setλACE=0\\lambda\_\{\\mathrm\{ACE\}\}=0\. For hybrid SDFT\+SFT andRoCo\+SFT variants, the supervised loss weight is set to1\.01\.0over the full reference answer\.

Table 12:Estimated runtime comparison on the EVOKE setting with Qwen3\-VL\-8B, three training epochs, and 16 PPU accelerators\. Training time is reported as wall\-clock hours from representative runs\. Inference latency is similar across methods because the deployed model sizes are comparable; we observe roughly one second per query under the same inference setup\.
#### Runtime cost\.

Table[12](https://arxiv.org/html/2607.24771#A11.T12)reports approximate wall\-clock training and inference cost on EVOKE with Qwen3\-VL\-8B, three training epochs, and 16 PPU accelerators\.RoCo\-ACEis slower than direct fitting and adapter\-only baselines because it performs online rollout scoring with reference\-conditioned and base\-teacher contexts, but its cost remains close to SDFT since both use the same online\-distillation backbone\. At inference time, all methods use model weights of comparable scale, so their per\-query latency remains approximately the same under the same decoding setup\.