ASRU: Activation Steering Meets Reinforcement Unlearning for Multimodal Large Language Models

# ASRU: Activation Steering Meets Reinforcement Unlearning for Multimodal Large Language Models
Source: [https://arxiv.org/html/2605.15687](https://arxiv.org/html/2605.15687)
###### Abstract

Multimodal large language models \(MLLMs\) may memorize sensitive cross\-modal information during pretraining, making machine unlearning \(MU\) crucial\. Existing methods typically evaluate unlearning effectiveness based on output deviations, while overlooking the generation quality after unlearning\. This can easily lead to hallucinated or rigid responses, thereby affecting the usability and safety of the unlearned model\. To address this issue, we proposeASRU, a controllable multimodal unlearning framework that incorporates generation quality as a core evaluation objective\. ASRU first induces initial refusal behavior through activation redirection, and then optimizes fine\-grained refusal boundaries using a customized reward function, thereby achieving a better trade\-off between target knowledge unlearning and model utility\. Experiments on Qwen3\-VL show that ASRU significantly improves unlearning effectiveness \(\+24\.6%\) on average and generation quality \(5\.8×\) on average while effectively preserving model utility, using only a small amount of retained supervision data\. The code for ASRU is released at[https://github\.com/guangjh/ASRU](https://github.com/guangjh/ASRU)\.

Machine Learning, ICML

## 1Introduction

Multimodal Large Language Models \(MLLMs\) have achieved remarkable success across a wide range of multimodal tasks\(Huanget al\.,[2023](https://arxiv.org/html/2605.15687#bib.bib1); Wanget al\.,[2024](https://arxiv.org/html/2605.15687#bib.bib2); Zouet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib3); Duet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib4); Zhuet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib5); Huet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib32)\)\. However, their large\-scale pretraining often relying on millions of instances—inevitably causes MLLMs to memorize sensitive or harmful information\(Karamolegkouet al\.,[2023](https://arxiv.org/html/2605.15687#bib.bib7); Huanget al\.,[2024](https://arxiv.org/html/2605.15687#bib.bib8)\), raising serious concerns related to privacy leakage and copyright infringement\(Huoet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib6); Liuet al\.,[2025b](https://arxiv.org/html/2605.15687#bib.bib9)\)\. Since pretraining data are typically inaccessible, retraining models from scratch to remove such knowledge is impractical, making post\-hoc machine unlearning a critical requirement for building trustworthy MLLM systems\(Liuet al\.,[2025a](https://arxiv.org/html/2605.15687#bib.bib16); Chenet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib10); Dinget al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib11)\)\.

![Refer to caption](https://arxiv.org/html/2605.15687v1/x1.png)Figure 1:Existing unlearning methods often yieldhallucinated,rigid outputsandexhibit incomplete unlearning, whereasour ASRUproduces context\-aware dynamic refusals\.Most existing work directly adapts unlearning strategies designed for text\-only LLMs to the multimodal setting, including gradient\-ascent\-based methods\(Thudiet al\.,[2022](https://arxiv.org/html/2605.15687#bib.bib18); Liuet al\.,[2024](https://arxiv.org/html/2605.15687#bib.bib19); Zhanget al\.,[2024](https://arxiv.org/html/2605.15687#bib.bib17); Liet al\.,[2024](https://arxiv.org/html/2605.15687#bib.bib14)\), preference optimization\(Zhanget al\.,[2024](https://arxiv.org/html/2605.15687#bib.bib17)\), targeted parameter updates\(Huoet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib6); Liet al\.,[2025a](https://arxiv.org/html/2605.15687#bib.bib15)\)\. Despite their initial progress, these methods still suffer from two fundamental limitations\. As shown in Figure[1](https://arxiv.org/html/2605.15687#S1.F1),\(I\) Unnatural and hallucinated responses after unlearning remain prevalent\.Existing methods either generate fabricated content that is inconsistent with the target facts, or produce abnormal, repetitive, and incoherent degenerated text\. In high\-risk domains, hallucinated outputs may introduce severe safety hazards, such as generating plausible but incorrect medical advice after patient records have been removed\(Shenet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib44)\)\. Meanwhile, unnatural responses may inadvertently expose that an unlearning operation has been performed, which could introduce potential security risks\. We refer to this phenomenon aspost\-unlearning responsiveness degradation:after losing the target knowledge, the model fails to express its knowledge gap in a natural, context\-aware, and coherent manner\. We argue that an unlearned model should behave like an aligned base model, accurately expressing its knowledge gaps in a dynamic, context\-aware, and coherent manner\. Accordingly, we advocate treatinggeneration qualityas a primary evaluation criterion in unlearning research\.\(II\) A persistent trade\-off exists between unlearning quality and model utility\.Gradient ascent based update methods tend to over\-suppress the forget setDfD\_\{f\}, leading to unintended global behavior shifts and utility degradation\(Wanget al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib20)\)\. Conversely, methods that preserve utility often exhibit incomplete unlearning, making it difficult to achieve an optimal trade\-off between effective unlearning and utility preservation\.

These limitations indicate that existing methods still lack fine\-grained control over the refusal boundary and have not sufficiently addressed generation quality after unlearning\. To address these challenges, we propose Activation Steering meets Reinforcement Unlearning \(ASRU\), a novel framework for multimodal unlearning\. ASRU first constructs a knowledge\-absence direction and performs activation steering, inducing a basic refusal behavior by updating only a single down\-projection matrix\. We further introduce Group Relative Policy Optimization \(GRPO\) based on this initialization, design a verifiable reward function, and leverage a small supervised retain subset to jointly optimize forget samples with their highly related boundary samples, thereby learning a context\-aware, fine\-grained refusal boundary\. Experimental results show that ASRU achieves fine\-grained boundary perception, generates contextually dynamic and consistent refusal responses, and obtains a better trade\-off between unlearning effectiveness and model utility\.

We highlight our main contributions as:

- •Existing multimodal unlearning methods assess unlearning primarily via deviation from ground truth\. We introducegeneration qualityas a key evaluation dimension, capturing whether unlearned model produce coherent, context\-aware, and natural refusals\.
- •We proposeASRU, a novel and controllable unlearning framework that first integrates activation steering with GRPO\. ASRU first uses activation steering \(via updating a single projection matrix\) to induce controllable refusal behavior as a initialization, and then introduces a boundary set and verifiable reward design to learn a finer\-grained forget–retain boundary\.
- •Experimental results demonstrate that ASRU preserves model utility while achieving up to an average 5\.8× improvement in generation quality and an average 24\.61% gain in unlearning effectiveness over existing baselines, with a superior forget–retain trade\-off using only a small amount of supervised retain data\.

## 2Related work

### 2\.1Multimodal Unlearning

Machine unlearning in multimodal large language models remains relatively underexplored\(Liet al\.,[2025b](https://arxiv.org/html/2605.15687#bib.bib36)\)\. SIU\(Liet al\.,[2024](https://arxiv.org/html/2605.15687#bib.bib14)\)is the first work to study multimodal unlearning, focusing on removing visual recognition capabilities for specific concepts, but it relies on complex and multifaceted fine\-tuning data\. MLLMU\-Bench\(Liuet al\.,[2025b](https://arxiv.org/html/2605.15687#bib.bib9)\)introduces a benchmark based on fictitious personal profiles and systematically evaluates multiple unlearning strategies\. MMUnlearner\(Huoet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib6)\)selectively updates specific model parameters to induce forgetting, while other approaches apply activation steering solely at inference time\(Dinget al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib11)\)\.

### 2\.2Activation Steering

Activation \(or representation\) steering is a core paradigm in representation engineering\(Yunfanet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib22); Sterzet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib23); Stolfoet al\.,[2024](https://arxiv.org/html/2605.15687#bib.bib24); Stoehret al\.,[2024](https://arxiv.org/html/2605.15687#bib.bib25); Zhang and Viteri,[2025](https://arxiv.org/html/2605.15687#bib.bib26)\), aiming to identify concept\-specific directions \(e\.g\., truthfulness or toxicity\) in model representations and intervene on hidden states accordingly\. Its effectiveness is often explained by the linear representation hypothesis, which assumes that hidden representations can be approximated as linear combinations of attribute vectors\(Parket al\.,[2023](https://arxiv.org/html/2605.15687#bib.bib27)\)\. In practice, concept directions are obtained via contrastive activation steering\(Jorgensenet al\.,[2023](https://arxiv.org/html/2605.15687#bib.bib29); Rimskyet al\.,[2024](https://arxiv.org/html/2605.15687#bib.bib28); Stoehret al\.,[2024](https://arxiv.org/html/2605.15687#bib.bib25)\)or linear probing\(Jorgensenet al\.,[2023](https://arxiv.org/html/2605.15687#bib.bib29); Yunfanet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib22); Sterzet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib23)\), and applied at inference time by modifying hidden states\(Singhet al\.,[2024](https://arxiv.org/html/2605.15687#bib.bib31); Dinget al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib11); Shenget al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib30)\)\.

### 2\.3Reinforcement Learning for LLM Alignment

Reinforcement learning has become a powerful post\-training technique for aligning LLM behavior\(Daiet al\.,[2023](https://arxiv.org/html/2605.15687#bib.bib37); Wuet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib38); Niuet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib39)\)\. PPO\(Schulmanet al\.,[2017](https://arxiv.org/html/2605.15687#bib.bib40)\)formulates generation as an MDP\(Bellman,[1957](https://arxiv.org/html/2605.15687#bib.bib41)\)but is computationally expensive\. DPO\(Rafailovet al\.,[2023](https://arxiv.org/html/2605.15687#bib.bib42)\)and GRPO\(Shaoet al\.,[2024](https://arxiv.org/html/2605.15687#bib.bib43)\)reduce reliance on explicit reward models by leveraging preference comparisons or group\-relative advantages\. Recent work explores applying RL to representation\-level alignment\(Wuet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib38)\)and safety optimization with constrained policies\(Niuet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib39)\)\. ASRU builds on these advances by using GRPO to learn a verifiable and fine\-grained unlearning boundary in multimodal unlearning\.

![Refer to caption](https://arxiv.org/html/2605.15687v1/x2.png)Figure 2:Overview of the proposed ASRU framework\.Stage 1induces refusal capability via activation steering and consists of two sub\-stages:Stage 1\.1identifies the direction of knowledge absence by contrasting the model’s activation patterns on previously unseen images against those on the forget set;Stage 1\.2steers the model by training only the down\-projection module at layerL∗L^\{\*\}, thereby promoting the emergence of a refusal\-style response behavior\.Stage 2then applies GRPO optimization on the boundary set and the forget set, enabling the model to learn a finer\-grained refusal decision boundary\.

## 3Method

### 3\.1Problem Setup

Given a vision language instruction dataset𝒟=\{\(Ii,Qi,Ai\)\}i=1N\\mathcal\{D\}=\\\{\(I\_\{i\},Q\_\{i\},A\_\{i\}\)\\\}\_\{i=1\}^\{N\}, whereIiI\_\{i\}denotes the input image,QiQ\_\{i\}is the corresponding textual instruction, andAi=\(y1,y2,…,yTi\)A\_\{i\}=\(y\_\{1\},y\_\{2\},\\dots,y\_\{T\_\{i\}\}\)is the target output sequence conditioned on\(Ii,Qi\)\(I\_\{i\},Q\_\{i\}\), it maximizes the conditional likelihood of each tokenyty\_\{t\}:

maxθ​∑t=1Tilog⁡pθ​\(yt∣y<t,Ii,Qi\)\.\\max\_\{\\theta\}\\ \\sum\_\{t=1\}^\{T\_\{i\}\}\\log p\_\{\\theta\}\\\!\\left\(y\_\{t\}\\mid y\_\{<t\},I\_\{i\},Q\_\{i\}\\right\)\.\(1\)
The goal of multimodal unlearning is to selectively remove memory associated with specific target knowledge from a pretrained modelMorgM\_\{\\mathrm\{org\}\}\. The unlearning process aims to obtain an updated modelMunlearnM\_\{\\mathrm\{unlearn\}\}that no longer retains the target knowledge while preserving the original multimodal understanding capability and overall utility\.

MLLM Unlearning DefinitionMLLM unlearning is defined as the process of modifying a MLLM to forget image\-paired knowledge associated with the forget set𝒟f\\mathcal\{D\}\_\{f\}while preserving model utility\.

For the original multimodal large modelMM, the goal of an unlearning algorithm is to obtain an updated modelM′M^\{\\prime\}\. Ideally,M′M^\{\\prime\}should behave similarly to a model trained only on the retain set𝒟r\\mathcal\{D\}\_\{r\}and never exposed to the forget set𝒟f\\mathcal\{D\}\_\{f\}\. The model should forget the target knowledge in𝒟f\\mathcal\{D\}\_\{f\}while preserving model utility on𝒟r\\mathcal\{D\}\_\{r\}\. This process can be formally expressed as follows:

minθ𝔼\(If,Qf,Af\)∼𝒟f​\[ℓf​\(Af∣If,Qf;θ\)\]\+λ​𝔼\(Ir,Qr,Ar\)∼𝒟r​\[ℓr​\(Ar∣Ir,Qr;θ\)\],\\begin\{split\}\\min\_\{\\theta\}\\;&\\mathbb\{E\}\_\{\(I\_\{f\},Q\_\{f\},A\_\{f\}\)\\sim\\mathcal\{D\}\_\{f\}\}\\big\[\\ell\_\{f\}\(A\_\{f\}\\mid I\_\{f\},Q\_\{f\};\\theta\)\\big\]\\;\+\\\\ &\\lambda\\,\\mathbb\{E\}\_\{\(I\_\{r\},Q\_\{r\},A\_\{r\}\)\\sim\\mathcal\{D\}\_\{r\}\}\\big\[\\ell\_\{r\}\(A\_\{r\}\\mid I\_\{r\},Q\_\{r\};\\theta\)\\big\],\\end\{split\}\(2\)
whereℓf\\ell\_\{f\}denotes the forgetting loss that discourages the model from producing target knowledge,ℓr\\ell\_\{r\}denotes the retention loss that preserves general multimodal performance, andλ\\lambdacontrols the trade\-off between forgetting and retention\.

### 3\.2ASRU

As discussed in Sections[1](https://arxiv.org/html/2605.15687#S1)and[3\.1](https://arxiv.org/html/2605.15687#S3.SS1), effective MLLM unlearning should enable a*refuse\-when\-necessary*and*answer\-when\- appropriate*behavior\. Specifically, the model should produce context\-aware refusals for forget\-set inputs while still answering permissible queries\. This requires learning a precise refusal boundary between forget\-related inputs and retained inputs\. To address the challenges, we proposeASRU, a two\-stage framework that equips MLLMs with controllable refusal behavior while preserving overall utility\. As illustrated in Figure[2](https://arxiv.org/html/2605.15687#S2.F2), in the first stage, ASRU uses activation steering and replaces the standard cross\-entropy loss with a local representation loss on residual\-stream activations\. By updating only a single projection matrix, it induces controllable initial refusal behavior and obtains the initialized steered modelMsteeredM\_\{\\mathrm\{steered\}\}\. To enable the model to learn a finer\-grained refusal boundary, the second stage introduces a boundary set and a verifiable reward design to further optimize the fine\-grained forget–retain boundary between forget samples and retain samples\. The core idea of ASRU is to first establish a refusal prototype and then refine the decision boundary\. The algorithm pseudo\-code is provided in Appendix[A](https://arxiv.org/html/2605.15687#A1)\.

#### 3\.2\.1Refusal Steering

A core challenge in controllable multimodal unlearning is that pretrained MLLMs typically do not spontaneously exhibit refusal behavior\. To equip the model with this capability, we adopt activation steering to redirect the model’s internal representations associated with the forget set toward a refusal direction state, thereby inducing initial refusal behavior\.

Refusal Direction Construction\.The ideal behavior of an unlearned model should be consistent with its knowledge boundary: for queries related to the forget set, the model should behave like a retrained model that has never learned the corresponding knowledge, and clearly and naturally express thatit cannot provide the relevant information\. To achieve this goal, we redirect the activations of forget\-set samples at the last token position toward a reference direction for refusal responses, guiding the model on the forget set toward a representational region of “no relevant knowledge and refusal\.” Specifically, we estimate the refusal reference direction using the mean hidden activations of two contrastive groups\. For a designated layerL∗L^\{\*\}, we compute the average activations of the forget set𝒟f\\mathcal\{D\}\_\{f\}and the target refusal\-reference set𝒟target\\mathcal\{D\}\_\{\\text\{target\}\}, respectively\. Here,𝒟target\\mathcal\{D\}\_\{\\text\{target\}\}consists of inputs for which the model has no prior knowledge, such as images of individuals not seen during pretraining paired with privacy\-related queries111Here, we employ the synthetic face images from DigiFace1M\(Baeet al\.,[2023](https://arxiv.org/html/2605.15687#bib.bib50)\)as the unseen images in𝒟target\\mathcal\{D\}\_\{\\text\{target\}\}\.\. Mathematically, the mean activations are expressed as:

\{a¯L∗target=1\|𝒟target\|​∑x∈𝒟targeth\(l\)​\(x\),a¯L∗f=1\|𝒟f\|​∑x∈𝒟fh\(l\)​\(x\),\\begin\{cases\}\\bar\{a\}\_\{L^\{\*\}\}^\{\\text\{target\}\}=\\frac\{1\}\{\|\\mathcal\{D\}\_\{\\text\{target\}\}\|\}\\sum\_\{x\\in\\mathcal\{D\}\_\{\\text\{target\}\}\}h^\{\(l\)\}\(x\),\\\\\[4\.30554pt\] \\bar\{a\}\_\{L^\{\*\}\}^\{f\}=\\frac\{1\}\{\|\\mathcal\{D\}\_\{f\}\|\}\\sum\_\{x\\in\\mathcal\{D\}\_\{f\}\}h^\{\(l\)\}\(x\),\\end\{cases\}\(3\)
steering vectoris defined by the difference of these means:

vL∗=a¯L∗target−a¯L∗f\.v\_\{L^\{\*\}\}=\\bar\{a\}\_\{L^\{\*\}\}^\{\\text\{target\}\}\-\\bar\{a\}\_\{L^\{\*\}\}^\{f\}\.\(4\)
Activation Steering Optimization\.We replace the standard cross\-entropy loss with a local representation loss on residual\-stream activations to perform activation redirection\. Specifically, for forget\-set inputs, we steer the activations along the refusal reference direction, while for retain\-set inputs, we preserve their original activations:

tL∗f=aL∗​\(x\)\+λ​vL∗,tL∗r=aL∗​\(x\),t\_\{L^\{\*\}\}^\{f\}=a\_\{L^\{\*\}\}\(x\)\+\\lambda v\_\{L^\{\*\}\},\\quad t\_\{L^\{\*\}\}^\{r\}=a\_\{L^\{\*\}\}\(x\),\(5\)
whereλ\\lambdamodulates the influence of the steering vector\. The model is then optimized to minimize the deviation between its actual activations and the target activations for both forget and retain sets:

ℒf\\displaystyle\\mathcal\{L\}\_\{f\}=𝔼x∈𝒟f​‖aL∗f​\(x\)−tL∗f​\(x\)‖22,\\displaystyle=\\mathbb\{E\}\_\{x\\in\\mathcal\{D\}\_\{f\}\}\\\|a\_\{L^\{\*\}\}^\{f\}\(x\)\-t\_\{L^\{\*\}\}^\{f\}\(x\)\\\|\_\{2\}^\{2\},\(6\)ℒr\\displaystyle\\mathcal\{L\}\_\{r\}=𝔼x∈𝒟r​‖aL∗r​\(x\)−tL∗r​\(x\)‖22,\\displaystyle=\\mathbb\{E\}\_\{x\\in\\mathcal\{D\}\_\{r\}\}\\\|a\_\{L^\{\*\}\}^\{r\}\(x\)\-t\_\{L^\{\*\}\}^\{r\}\(x\)\\\|\_\{2\}^\{2\},\(7\)ℒ\\displaystyle\\mathcal\{L\}=ℒf\+ℒr\.\\displaystyle=\\mathcal\{L\}\_\{f\}\+\\mathcal\{L\}\_\{r\}\.\(8\)
Closed\-form Solution via Down\-projection\.This activation redirection can be efficiently achieved by updating only thedown\-projection matrixof a single MLP layer, while keeping the majority of the base model parameters frozen\. LetHfH\_\{f\}andHrH\_\{r\}denote the inputs to this down\-projection layer for the forget and retain sets, respectively, and letTfT\_\{f\}andTrT\_\{r\}denote the corresponding redirected target activations\. The optimization objective is defined as:

W∗=arg⁡minW⁡‖\[Hf,Hr\]​W−\[Tf,Tr\]‖2\.W^\{\*\}=\\arg\\min\_\{W\}\\left\\\|\[H\_\{f\},\\,H\_\{r\}\]\\,W\-\[T\_\{f\},\\,T\_\{r\}\]\\right\\\|\_\{2\}\.\(9\)whereW∗W^\{\*\}denotes the weights of the down\-projection layer\. It has been proven that this optimization problem admits a closed\-form solution\(Shenet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib44)\):

W∗=\(\[Hf,Hr\]⊤​\[Hf,Hr\]\+γ​I\)−1​\[Hf,Hr\]⊤​\[Tf,Tr\]\.\\displaystyle\\begin\{split\}W^\{\*\}=\\left\(\[H\_\{f\},H\_\{r\}\]^\{\\top\}\[H\_\{f\},H\_\{r\}\]\+\\gamma I\\right\)^\{\-1\}\[H\_\{f\},H\_\{r\}\]^\{\\top\}\[T\_\{f\},T\_\{r\}\]\.\\end\{split\}\(10\)whereγ≥0\\gamma\\geq 0is the regularization parameter\.

#### 3\.2\.2Refusal Boundary Optimization

However, although the steered modelMsteeredM\_\{\\mathrm\{steered\}\}has acquired refusal capability, it still struggles to clearly distinguish the forgetting boundary, which leads to unnecessary refusals on the retain set\. To enable the model to learn a finer\-grained forgetting boundary and thus dynamically and accurately exhibit a “refuse\-when\-necessary, answer\-when\-appropriate” response pattern, we propose a reinforcement\-learning\-based*Forget Boundary Optimization Module*\. By introducing a boundary set and a verifiable reward design, this module further optimizes the model behavior, encouraging the model to effectively forget target knowledge while preserving its normal answering ability for non\-target knowledge as much as possible\.

Boundary Set Construction\.A key requirement in MLLM unlearning is to accurately delineate the decision boundary between forget samples and retain samples\. To this end, we sample aboundary set𝒟~r⊂𝒟r\\tilde\{\\mathcal\{D\}\}\_\{r\}\\subset\\mathcal\{D\}\_\{r\}from the retain set, whose samples are highly similar to𝒟f\\mathcal\{D\}\_\{f\}in the input space but should be retained, enabling the model to learn a fine\-grained refusal boundary\.

Reward Definition\.Let\(xf,xr~\)\(x\_\{f\},x\_\{\\tilde\{r\}\}\)denote paired inputs from the forget set and boundary set, with corresponding outputs\(yf,yr~\)\(y\_\{f\},y\_\{\\tilde\{r\}\}\)\. We define the reward function as:

R​\(xf,xr~,yf,yr~\)=rf​\(xf,yf\)\+r𝒟~r​\(xr~,yr~\),R\(x\_\{f\},x\_\{\\tilde\{r\}\},y\_\{f\},y\_\{\\tilde\{r\}\}\)=r\_\{f\}\(x\_\{f\},y\_\{f\}\)\+r\_\{\\tilde\{\\mathcal\{D\}\}\_\{r\}\}\(x\_\{\\tilde\{r\}\},y\_\{\\tilde\{r\}\}\),\(11\)
whererfr\_\{f\}quantifies the effectiveness of forgetting on𝒟f\\mathcal\{D\}\_\{f\}, andr𝒟~rr\_\{\\tilde\{\\mathcal\{D\}\}\_\{r\}\}evaluates retention fidelity on the boundary set\.

To stabilize policy gradient updates, we define the advantage function based on within\-group relative rewards:

Ai=Ri−1G​∑j=1GRj,Ri=R​\(x,yf\(i\),yr~\(i\)\),A\_\{i\}=R\_\{i\}\-\\frac\{1\}\{G\}\\sum\_\{j=1\}^\{G\}R\_\{j\},\\quad R\_\{i\}=R\(x,y\_\{f\}^\{\(i\)\},y\_\{\\tilde\{r\}\}^\{\(i\)\}\),\(12\)
Table 1:Unlearning performance on MLLMU\-Bench \(5% forget\)\.Dr~\\tilde\{D\_\{r\}\}represents the number of samples required from the supervised retain set\.↓\\boldsymbol\{\\downarrow\}indicates that lower values are preferred, while↑\\boldsymbol\{\\uparrow\}indicates that higher values are preferred\.MethodSamplesForget Quality↓\\boldsymbol\{\\downarrow\}Generation Quality↑\\boldsymbol\{\\uparrow\}Retain Quality↑\\boldsymbol\{\\uparrow\}Class\.Gen\.Class\.Gen\.Dr~\\tilde\{D\_\{r\}\}ForgetTestForgetTestCRForgetfulnessRetainRealRetainRealQwen3\-VL\-4B\(5%Forget\)\-VQAVanilla\-46\.6750\.000\.5780\.327\-\-41\.9563\.580\.5410\.428GA0%40\.8345\.000\.5110\.2880\.120\.1439\.4162\.140\.5170\.377GA\_diff100%45\.8350\.000\.5480\.3120\.060\.1034\.5056\.400\.5120\.370NPO0%40\.0043\.330\.5610\.3270\.120\.2232\.5658\.750\.5200\.371KL\_Min100%44\.8046\.670\.5490\.3150\.260\.3038\.4861\.360\.5180\.374MMunlearner100%40\.0047\.200\.5520\.2980\.010\.0239\.2862\.270\.5270\.380ASRU\(ours\)27\.30%31\.2034\.400\.3990\.2452\.062\.0839\.4562\.530\.5310\.382Qwen3\-VL\-8B\(5%Forget\)\-VQAVanilla\-55\.0062\.500\.6000\.370\-\-51\.5978\.590\.570\.43GA0%49\.1755\.000\.5490\.3300\.320\.3444\.8173\.890\.4690\.372GA\_diff100%49\.1755\.000\.5370\.3340\.680\.7247\.4073\.110\.4770\.356NPO0%46\.6756\.670\.5380\.3140\.560\.5846\.7271\.670\.4890\.356KL\_Min100%51\.6753\.330\.5700\.3550\.320\.3650\.6673\.500\.4750\.369MMunlearner100%43\.2052\.000\.5760\.3460\.200\.2446\.7971\.540\.4940\.359ASRU\(ours\)27\.30%30\.8345\.000\.3980\.2772\.983\.0453\.7474\.640\.5400\.398LLaVA\-1\.5\-7B\(5%Forget\)\-VQAVanilla\-51\.6741\.670\.5800\.230\-\-46\.3447\.390\.4900\.220GA0%41\.6741\.670\.4740\.2300\.060\.0641\.2642\.270\.4220\.230GA\_diff100%47\.5040\.830\.3880\.1700\.620\.9243\.1737\.210\.3510\.183NPO0%44\.7840\.000\.5330\.2150\.040\.0644\.7843\.230\.4320\.230KL\_Min100%49\.1740\.830\.5800\.2270\.060\.0849\.0941\.780\.4260\.224MMunlearner100%38\.3342\.500\.4150\.1810\.180\.5847\.7444\.130\.4390\.227ASRU\(ours\)27\.30%34\.1738\.400\.2500\.1442\.122\.0641\.3543\.630\.4540\.268

Refusal Boundary Optimization via GRPO\.For each paired input\(xf,xr~\)\(x\_\{f\},x\_\{\\tilde\{r\}\}\), wherexf∈𝒟fx\_\{f\}\\in\\mathcal\{D\}\_\{f\}andxr~∈𝒟~rx\_\{\\tilde\{r\}\}\\in\\tilde\{\\mathcal\{D\}\}\_\{r\}, the policy samples a group ofGGrollouts\. We define the probability ratio between the current policyπθ\\pi\_\{\\theta\}and the old policyπθold\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}as:

ρi​\(θ\)=πθ​\(yf\(i\),yr~\(i\)∣xf,xr~\)πθold​\(yf\(i\),yr~\(i\)∣xf,xr~\),\\rho\_\{i\}\(\\theta\)=\\frac\{\\pi\_\{\\theta\}\(y\_\{f\}^\{\(i\)\},y\_\{\\tilde\{r\}\}^\{\(i\)\}\\mid x\_\{f\},x\_\{\\tilde\{r\}\}\)\}\{\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(y\_\{f\}^\{\(i\)\},y\_\{\\tilde\{r\}\}^\{\(i\)\}\\mid x\_\{f\},x\_\{\\tilde\{r\}\}\)\},\(13\)
The GRPO objective is then formulated as:

maxθ𝔼x∼𝒟,\{Oi\}i=1G∼πold\(⋅\|x\)\[1G∑i=1Gmin\(πθ​\(oi\|x\)πold​\(oi\|x\)A^i,clip\(πθ​\(oi\|x\)πold​\(oi\|x\),1−ϵ,1\+ϵ\)A^i\)−βDKL\[πθ\(⋅\|x\)∥πref\(⋅\|x\)\]\],\\displaystyle\\begin\{split\}\\max\_\{\\theta\}&\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\},\\\{O\_\{i\}\\\}^\{G\}\_\{i=1\}\\sim\\pi\_\{\\text\{old\}\}\(\\cdot\|x\)\}\\Bigg\[\\frac\{1\}\{G\}\\sum\_\{i=1\}^\{G\}\\min\\left\(\\frac\{\\pi\_\{\\theta\}\(o\_\{i\}\|x\)\}\{\\pi\_\{\\text\{old\}\}\(o\_\{i\}\|x\)\}\\hat\{A\}\_\{i\},\\right\.\\\\ &\\quad\\left\.\\text\{clip\}\\left\(\\frac\{\\pi\_\{\\theta\}\(o\_\{i\}\|x\)\}\{\\pi\_\{\\text\{old\}\}\(o\_\{i\}\|x\)\},1\-\\epsilon,1\+\\epsilon\\right\)\\hat\{A\}\_\{i\}\\right\)\\\\ &\-\\beta D\_\{\\text\{KL\}\}\[\\pi\_\{\\theta\}\(\\cdot\|x\)\\parallel\\pi\_\{\\text\{ref\}\}\(\\cdot\|x\)\]\\Bigg\],\\end\{split\}\(14\)
whereA^i\\hat\{A\}\_\{i\}denotes the normalized within\-group advantage\. The clipping term constrains policy updates within a stable range, while the KL regularization term keeps the updated policy close to the reference policyπref\\pi\_\{\\mathrm\{ref\}\}\.

Ideally, the learned policy should satisfy the following behavior:

πθ​\(y∣x\)→\{1,y=refuse,x∈𝒟f,1,y=information,x∈𝒟~r,\\pi\_\{\\theta\}\(y\\mid x\)\\to\\begin\{cases\}1,&y=\\texttt\{refuse\},\\quad x\\in\\mathcal\{D\}\_\{f\},\\\\ 1,&y=\\texttt\{information\},\\quad x\\in\\tilde\{\\mathcal\{D\}\}\_\{r\},\\end\{cases\}\(15\)
where,refusedenotes a safe refusal response \(e\.g\., “Sorry, the answer cannot be inferred from the image alone”\), andinformationdenotes a normal answer\. This reward design encourages context\-aware, dynamic refusals on the forget set while producing informative responses on the retain set, thereby guiding the model to learn a fine\-grained forgetting boundary\. The detailed reward formulation is provided in Appendix[C\.1](https://arxiv.org/html/2605.15687#A3.SS1)\.

## 4Experiment and Analysis

### 4\.1Experimental Setup

Datasets and Evaluation Metrics\.We conduct experiments on the MLLMU\-Bench dataset\(Liuet al\.,[2025b](https://arxiv.org/html/2605.15687#bib.bib9)\)using Qwen\-3\-VL\-8B\-Instruct, Qwen\-3\-VL\-4B\-Instruct and LLaVA\-1\.5\-7B across four A100 \(80GB\) GPUs\. MLLMU\-Bench comprises four distinct subsets: the Forget Set \(𝒟f\\mathcal\{D\}\_\{f\}\), the Retain Set \(𝒟r\\mathcal\{D\}\_\{r\}\), the Test Set \(𝒟t​e​s​t\\mathcal\{D\}\_\{test\}\), and the Real Celebrity Set \(𝒟real\\mathcal\{D\}\_\{\\text\{real\}\}\)\. For classification tasks, we report Average Accuracy \(ACC\), and for generation tasks, we use ROUGE\-L\(Lin,[2004](https://arxiv.org/html/2605.15687#bib.bib34)\)to assess model performance\. To further validate the effectiveness of our method, we also conduct evaluations on the Forget and Realworld sets of CLEAR datasets\(Dontsovet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib51)\)\.

Assessing Generation Quality\.Existing evaluation metrics mainly focus on unlearning effectiveness, while paying limited attention to post\-unlearning generation quality\. However, unnatural or hallucinated responses may introduce significant safety risks\(Zhanget al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib47)\)\. To address this issue, we evaluate post\-unlearning generation quality from two dimensions:\(i\) Contextual Refusal \(CR\), which measures the model’s ability to produce coherent and context\-aware refusals when faced with forget queries;\(ii\) Forgetfulness, which evaluates whether the model leaks target knowledge\. Evaluations are performed using GPT\-4o\-mini, as detailed in Appendix[B\.2](https://arxiv.org/html/2605.15687#A2.SS2)\.

Baselines\.We compare ASRU against representative MLLM unlearning approaches: \(i\)GA\(Thudiet al\.,[2022](https://arxiv.org/html/2605.15687#bib.bib18)\), applying reverse\-gradient updates on𝒟f\\mathcal\{D\}\_\{f\}; \(ii\)GA\_Diff\(Liuet al\.,[2022](https://arxiv.org/html/2605.15687#bib.bib35)\), applying gradient ascent on𝒟f\\mathcal\{D\}\_\{f\}to induce forgetting and gradient descent on𝒟r\\mathcal\{D\}\_\{r\}to preserve retained knowledge; \(iii\)KL\_Min\(Mainiet al\.,[2024](https://arxiv.org/html/2605.15687#bib.bib21)\), combining GA on𝒟f\\mathcal\{D\}\_\{f\}with KL\-divergence matching on𝒟r\\mathcal\{D\}\_\{r\}; \(iv\)NPO\(Zhanget al\.,[2024](https://arxiv.org/html/2605.15687#bib.bib17)\), treating𝒟f\\mathcal\{D\}\_\{f\}as non\-preferred under preference optimization; \(v\)MMUnlearner\(Huoet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib6)\), which achieves unlearning by selectively updating model parameters related to the forget set𝒟f\\mathcal\{D\}\_\{f\}\. Implementation details are provided in Appendix[B\.3](https://arxiv.org/html/2605.15687#A2.SS3)\.

Training\.ASRU first applies activation steering to establish initial refusal behavior, with guidance strengthsλ=3\\lambda=3for Qwen\-3\-VL\-4B\-Instruct,λ=0\.2\\lambda=0\.2for Qwen\-3\-VL\-8B\-Instruct,λ=0\.8\\lambda=0\.8for LLaVA\-1\.5\-7B\. We select layerL∗=17L^\{\*\}=17for optimization\. Details of layer selection and steering strengthλ\\lambdaselection are provided in Appendix[C\.2](https://arxiv.org/html/2605.15687#A3.SS2)\. Subsequently, the model is optimized by GRPO\. The detailed training hyperparameters are provided in Appendix[C\.3](https://arxiv.org/html/2605.15687#A3.SS3)\. We additionally explore a variant of mixed training in which samples from𝒟f∪𝒟~r\\mathcal\{D\}\_\{f\}\\cup\\tilde\{\\mathcal\{D\}\}\_\{r\}are concatenated into a single prompt\. Further hyperparameter details and results are reported in Appendix[F](https://arxiv.org/html/2605.15687#A6)\.

![Refer to caption](https://arxiv.org/html/2605.15687v1/x3.png)Figure 3:Comparison of model responses on Qwen3\-VL\-8B\-Instruct across two generation quality evaluation dimensions on forget queries from the MLLMU\-Bench \(5% Forget\)\.
### 4\.2Main Results

Unlearning Performance\.As shown in tableLABEL:tab:all, ASRU demonstrates superior forgetting ability across multiple evaluation metrics\. On Qwen\-3\-VL\-4B under the 5% forget setting, ASRU improves forget quality by an average of 21\.96% over the strongest baseline\. On Qwen\-3\-VL\-8B, it achieves an average improvement of 27\.26% over the best\-performing baseline\. In addition, ASRU also exhibits stronger unlearning performance on LLaVA\-1\.5\-7B, further validating its effectiveness across different model architectures\. These results show that ASRU can effectively suppress target knowledge associated with the forget set while maintaining stable overall model behavior and demonstrating strong generalization ability\.

Contextual and Dynamic Refusal\.Beyond effectively forgetting target knowledge, ASRU also demonstrates strong contextual awareness and dynamic adaptability when responding to forgotten queries\. In the generation quality evaluation, ASRU achieves a 5\.8×\\timesaverage improvement over the most competitive baseline, and shows consistent improvements on LLaVA as well\. These results indicate that ASRU effectively promotes coherent and context\-aware refusal responses, which traditional unlearning methods struggle to achieve\. A detailed case study is provided in Appendix[G](https://arxiv.org/html/2605.15687#A7)\.

Generalization and Data Efficiency\.ASRU preserves strong model utility on the retain set𝒟r\\mathcal\{D\}\_\{r\}and the real set𝒟real\\mathcal\{D\}\_\{\\text\{real\}\}, while using only 27\.3% of the supervised retain set𝒟~r\\tilde\{\\mathcal\{D\}\}\_\{r\}\. Moreover, after refusal initialization with activation steering on only 5% of the forget set, ASRU can still learn effective refusal boundaries via GRPO under the 10% and 15% forget settings, demonstrating its ability to generalize to unseen queries with limited supervision\. Meanwhile, ASRU also shows strong forgetting ability and context\-consistent refusal behavior on the LLaVA\-1\.5\-7B and the CLEAR dataset, further validating its generalization capability\. Additional experimental results are provided in Appendix[E](https://arxiv.org/html/2605.15687#A5)and Appendix[D](https://arxiv.org/html/2605.15687#A4)\.

![Refer to caption](https://arxiv.org/html/2605.15687v1/x4.png)\(a\)Reward curve of GRPO\.
![Refer to caption](https://arxiv.org/html/2605.15687v1/x5.png)\(b\)Rouge score of forget and retain set\.

Figure 4:The reward score and Rouge score curves during GRPO training, under the 5% forget setting\.![Refer to caption](https://arxiv.org/html/2605.15687v1/x6.png)\(a\)Qwen3\-VL\-4B\-Vanilla
![Refer to caption](https://arxiv.org/html/2605.15687v1/x7.png)\(b\)Qwen3\-VL\-4B\-ASRU
![Refer to caption](https://arxiv.org/html/2605.15687v1/x8.png)\(c\)Qwen3\-VL\-8B\-Vanilla
![Refer to caption](https://arxiv.org/html/2605.15687v1/x9.png)\(d\)Qwen3\-VL\-8B\-ASRU

Figure 5:Activation distributions under the 5% forget setting for Qwen3\-VL\-4B\-Instruct \(a\-b\) and Qwen3\-VL\-8B\-Instruct \(c\-d\)\.![Refer to caption](https://arxiv.org/html/2605.15687v1/x10.png)Figure 6:Trade\-off between forget quality and model utility at forget ratios of 5%, 10%, and 15% on Qwen\-3\-VL\-8B\-Instruct\. The two plots on the left correspond to classification tasks, with the x\-axis representing the accuracy difference on the forgetting set \(Fgt Acc Diff\)\. The two plots on the right correspond to generation tasks, with the x\-axis showing the ROUGE\-L difference on the forgetting set \(Fgt Rouge Diff\)\. The y\-axis represents the model’s utility on the retain set \(Ret\) and the celebrity set \(Real\)\.

## 5In\-Depth Analysis

### 5\.1The Robustness of the Judge Model

To further validate the reliability of our experimental results, we additionally re\-evaluate the responses usingGPT\-5\.1andClaude 4\.5as independent judge models\. As shown in Table[2](https://arxiv.org/html/2605.15687#S5.T2), although the scores vary across judges, the relative ranking of methods remains consistent, and ASRU achieves the best performance under all evaluators, demonstrating the robustness of our conclusions\.

To further verify the consistency between GPT\-4o\-mini and human judgments, we invite 3 annotators to evaluate the samples and compute the Intraclass Correlation Coefficient \(ICC\) , a standard metric for inter\-rater agreement \(values\>0\.75\>0\.75indicate high agreement,0\.4​–​0\.750\.4\\text\{\-\-\}0\.75indicate moderate agreement, and<0\.4<0\.4indicate low agreement\)\. As shown in Table[3](https://arxiv.org/html/2605.15687#S5.T3), the ICC for CR is 0\.89, indicating high agreement, while the ICC for Forgetfulness is 0\.57, reflecting moderate agreement\. And the overall trends remain consistent, supporting the reliability of the automatic evaluation\.

Table 2:Generation quality evaluation by different LLM judges on Qwen3\-VL\-8B under the 5% forget setting\.ModelContextual Refusal↑\{\\color\[rgb\]\{1,0,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{1,0,0\}\\boldsymbol\{\\uparrow\}\}Forgetfulness↑\{\\color\[rgb\]\{1,0,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{1,0,0\}\\boldsymbol\{\\uparrow\}\}GPT\-4o\-miniGPT\-5\.1Claude 4\.5GPT\-4o\-miniGPT\-5\.1Claude 4\.5GA0\.320\.420\.280\.340\.520\.70GA\_diff0\.680\.380\.080\.720\.580\.60NPO0\.560\.560\.440\.580\.740\.70KL\_Min0\.320\.380\.280\.360\.480\.48MMunlearner0\.200\.360\.200\.240\.440\.54ASRU \(ours\)2\.982\.071\.303\.044\.601\.98

Table 3:Agreement between GPT\-4o\-mini evaluation and human judgments\.MetricsGPT\-4o\-miniHumanICCContextual Refusal2\.982\.330\.89Forgetfulness3\.044\.370\.57
### 5\.2Effectiveness of Reward Design

As shown in Figure[4\(b\)](https://arxiv.org/html/2605.15687#S4.F4.sf2), the proposed reward function aligns closely with the objective of unlearning\. During training, the model progressively reduces its performance on the forget set𝒟f\\mathcal\{D\}\_\{f\}, while maintaining or even improving model utility on the retain set𝒟r\\mathcal\{D\}\_\{r\}\. The continuous increase in training reward \(Figure[4\(a\)](https://arxiv.org/html/2605.15687#S4.F4.sf1)\) further indicates that GRPO can guide the model to learn a finer\-grained forget–retain boundary\.

### 5\.3Fine\-Grained Refusal Boundary

Activation\-space analysis \(Figure[5](https://arxiv.org/html/2605.15687#S4.F5)\) shows that, in the original model, representations of the forget and retain sets are highly overlapping and difficult to separate\. In contrast, ASRU forms more geometrically distinct representation clusters, indicating a more structured separation between𝒟f\\mathcal\{D\}\_\{f\}and𝒟~r\\tilde\{\\mathcal\{D\}\}\_\{r\}\. Specifically, activation steering first equips the model with basic refusal capability, while GRPO further refines it into a generalizable, fine\-grained refusal boundary\.

### 5\.4Trade\-off Between Unlearning and Model Utility

Figure[6](https://arxiv.org/html/2605.15687#S4.F6)shows that, under the 10% and 15% forgetting ratios, ASRU consistently maintains strong forgetting performance while minimizing the impact on retained knowledge, outperforming all baseline methods\. In contrast, baseline methods tend to suffer more significant utility degradation as the forgetting ratio increases, while ASRU achieves a better forgetting–utility trade\-off\.

Table 4:Ablation study \(5% Forget\)\.Class\.denotes the classification task,Gen\.denotes the generation task, andAvg\.denotes the average of the generation quality scores\.VariantsForget↓\\boldsymbol\{\\downarrow\}Retain↑\\boldsymbol\{\\uparrow\}Real↑\\boldsymbol\{\\uparrow\}GenerationQuality\(Avg\.\)↑\\boldsymbol\{\\uparrow\}Class\.Gen\.Class\.Gen\.Class\.Gen\.Vanilla55\.000\.60051\.590\.57078\.590\.430\-ASRU30\.830\.39853\.740\.54074\.640\.3983\.01w/oAS48\.330\.55355\.480\.54674\.930\.4090\.27w/oGRPO54\.170\.57751\.920\.55677\.400\.4300\.38w/oDr~\\tilde\{D\_\{r\}\}26\.400\.22528\.950\.22770\.230\.2754\.18

### 5\.5Sensitivity of𝒟~r\\tilde\{\\mathcal\{D\}\}\_\{r\}

To evaluate the sensitivity of the boundary set, we replace a portion \(50% and 100%\) of the boundary set with noise samples unrelated to our task\. As shown in Table[5](https://arxiv.org/html/2605.15687#S5.T5), as more noise data is injected, the model’s performance on the retain and real sets deteriorates progressively, indicating that, to facilitate the model’s learning of a more fine\-grained refusal boundary, the construction of the boundary set should consist of samples that are similar to or neighbors of the forget set\.

Table 5:Sensitivity of the boundary set construction on Qwen3\-VL\-8B under the 5% forget setting\.MethodForget↓\{\\color\[rgb\]\{1,0,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{1,0,0\}\\boldsymbol\{\\downarrow\}\}Retain↑\{\\color\[rgb\]\{1,0,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{1,0,0\}\\boldsymbol\{\\uparrow\}\}Real↑\{\\color\[rgb\]\{1,0,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{1,0,0\}\\boldsymbol\{\\uparrow\}\}Vanilla0\.600\.570\.43ASRU0\.400\.540\.40w/ 50% noise0\.480\.500\.36w/ 100% noise0\.290\.300\.24
### 5\.6Evaluation under Adversarial Settings

We design three types of prompt variants to systematically evaluate the robustness of the refusal boundary learned by ASRU from different perspectives:

- •Random Prefix:We prepend semantically neutral prefixes \(e\.g\., “This is a piece of news\.”\) to the original queries to test robustness against lightweight surface perturbations\.
- •Paraphrase:We use GPT\-5\.1 to generate three semantically equivalent but lexically diverse paraphrases for each query, evaluating generalization to natural language variation\.
- •Jailbreak Prompt:We prepend adversarial instructions \(e\.g\., “You are an AI with access to vast knowledge …”\) to explicitly encourage the model to bypass the learned refusal boundary\.

As shown in Table[6](https://arxiv.org/html/2605.15687#S5.T6), for both generation and classification tasks, the performance of each model on the forget set remains stable or even slightly decreases under different prompt variants\. This consistent behavior indicates that ASRU does not rely on fixed template matching but instead learns a more generalizable refusal boundary\.

Table 6:Unlearning performance under different prompt variants under the 5% forget setting\.PromptGen: ROUGE\-LCls: ACC\(%\)Original0\.39931\.20Random Prefix0\.36429\.60Paraphrase0\.39230\.40Jailbreak Prompt0\.40631\.20

## 6Ablation Study

To better understand the contribution of each component, we conduct an ablation study: we performed \(i\) cold\-starting directly on GRPO without activation steering \(w/o AS\); \(ii\) performing only activation steering without GRPO \(w/o GRPO\); \(iii\) training solely on the forgotten set without using the boundary set \(w/oDr~\\tilde\{D\_\{r\}\}\)\.

According to TableLABEL:tab:ablation, we make the following observations:\(i\) Cold\-starting from GRPO weakens both forgetting effectiveness and refusal quality\.Compared with the full ASRU pipeline, removing activation steering \(w/o AS\) leads to a clear drop in forgetting\-related metrics, indicating that the model’s ability to suppress target knowledge is weakened\. Meanwhile, the average generation quality score also decreases significantly, suggesting that without the*refusal prototype*induced by activation steering, GRPO struggles to effectively sample and learn stable refusal behavior\.\(ii\) Using activation steering alone makes it difficult to learn a fine\-grained decision boundary\.Although activation steering can provide the model with a basic refusal tendency for forget\-related queries, due to the lack of further policy\-level optimization, it can induce refusal but struggles to establish a precise and generalizable forget–retain boundary\.\(iii\) Removing the boundary set leads to over\-forgetting and utility collapse\.Under the w/o𝒟~r\\tilde\{\\mathcal\{D\}\}\_\{r\}setting, the model tends to produce stronger refusals\. However, this aggressive forgetting comes at the cost of severe utility degradation, with bothRetain setandReal setmetrics dropping significantly\. This indicates that without explicit boundary supervision, forgetting can easily go beyond the target scope, making it difficult to preserve model utility\.

In conclusion, ASRU first steers the model toward refusal behavior and then uses GRPO to sharpen the refusal boundary,both modules are indispensable\.

## 7Conclusion and Limitations

We propose ASRU, a novel and controllable multimodal unlearning framework that, for the first time, combines activation steering with GRPO to improve unlearning effectiveness while preserving response quality and model utility\. ASRU can generate context\-aware refusals for the forget set and achieves a better forgetting–retention trade\-off through fine\-grained refusal boundary learning\. We also introduce generation quality as an evaluation metric, highlighting the importance of coherent and context\-aware responses after unlearning\. The current study mainly focuses on image\-text scenarios; in the future, ASRU can be further extended to other modalities such as audio, video, or robotics\.

## Impact Statements

This paper presents work whose goal is to advance the research on multimodal unlearning\. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here\.

## Acknowledgments

This work is supported by the Major Key Project of PCL \(Grant Nos\. PCL2024A05 and PCL2025A16\), National Natural Science Foundation of China under project \(No\. 62472126\) and CCF\-Huawei Populus Grove Fund\.

## References

- G\. Bae, M\. de La Gorce, T\. Baltrušaitis, C\. Hewitt, D\. Chen, J\. Valentin, R\. Cipolla, and J\. Shen \(2023\)DigiFace\-1m: 1 million digital face images for face recognition\.In2023 IEEE Winter Conference on Applications of Computer Vision \(WACV\),Cited by:[footnote 1](https://arxiv.org/html/2605.15687#footnote1)\.
- R\. Bellman \(1957\)A markovian decision process\.Journal of mathematics and mechanics,pp\. 679–684\.Cited by:[§2\.3](https://arxiv.org/html/2605.15687#S2.SS3.p1.1)\.
- J\. Chen, Z\. Deng, K\. Zheng, Y\. Yan, S\. Liu, P\. Wu, P\. Jiang, J\. Liu, and X\. Hu \(2025\)Safeeraser: enhancing safety in multimodal large language models through multimodal machine unlearning\.arXiv preprint arXiv:2502\.12520\.Cited by:[§1](https://arxiv.org/html/2605.15687#S1.p1.1)\.
- J\. Dai, X\. Pan, R\. Sun, J\. Ji, X\. Xu, M\. Liu, Y\. Wang, and Y\. Yang \(2023\)Safe rlhf: safe reinforcement learning from human feedback\.arXiv preprint arXiv:2310\.12773\.Cited by:[§2\.3](https://arxiv.org/html/2605.15687#S2.SS3.p1.1)\.
- C\. Ding, J\. Wu, L\. Sheng, F\. Zhang, Y\. Yuan, X\. Wang, and X\. He \(2025\)MLLMEraser: achieving test\-time unlearning in multimodal large language models through activation steering\.arXiv preprint arXiv:2510\.04217\.Cited by:[§1](https://arxiv.org/html/2605.15687#S1.p1.1),[§2\.1](https://arxiv.org/html/2605.15687#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2605.15687#S2.SS2.p1.1)\.
- A\. Dontsov, D\. Korzh, A\. Zhavoronkin, B\. Mikheev, D\. Bobkov, A\. Alanov, O\. Rogov, I\. Oseledets, and E\. Tutubalina \(2025\)CLEAR: character unlearning in textual and visual modalities\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 20582–20603\.Cited by:[§4\.1](https://arxiv.org/html/2605.15687#S4.SS1.p1.4)\.
- Y\. Du, H\. Guo, K\. Zhou, W\. X\. Zhao, J\. Wang, C\. Wang, M\. Cai, R\. Song, and J\. Wen \(2025\)What makes for good visual instructions? synthesizing complex visual reasoning instructions for visual instruction tuning\.InProceedings of the 31st International Conference on Computational Linguistics,pp\. 8197–8214\.Cited by:[§1](https://arxiv.org/html/2605.15687#S1.p1.1)\.
- Z\. Hu, J\. Li, Z\. Pu, H\. P\. Chan, and Y\. Yin \(2025\)Praxis\-vlm: vision\-grounded decision making via text\-driven reinforcement learning\.arXiv preprint arXiv:2503\.16965\.Cited by:[§1](https://arxiv.org/html/2605.15687#S1.p1.1)\.
- J\. Huang, J\. Zhang, K\. Jiang, H\. Qiu, and S\. Lu \(2023\)Visual instruction tuning towards general\-purpose multimodal model: a survey\.arXiv preprint arXiv:2312\.16602\.Cited by:[§1](https://arxiv.org/html/2605.15687#S1.p1.1)\.
- J\. Huang, D\. Yang, and C\. Potts \(2024\)Demystifying verbatim memorization in large language models\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 10711–10732\.Cited by:[§1](https://arxiv.org/html/2605.15687#S1.p1.1)\.
- J\. Huo, Y\. Yan, X\. Zheng, Y\. Lyu, X\. Zou, Z\. Wei, and X\. Hu \(2025\)MMUnlearner: reformulating multimodal machine unlearning in the era of multimodal large language models\.InFindings of the Association for Computational Linguistics: ACL 2025,Vienna, Austria,pp\. 7190–7206\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.375),ISBN 979\-8\-89176\-256\-5Cited by:[§1](https://arxiv.org/html/2605.15687#S1.p1.1),[§1](https://arxiv.org/html/2605.15687#S1.p2.1),[§2\.1](https://arxiv.org/html/2605.15687#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2605.15687#S4.SS1.p3.7)\.
- O\. Jorgensen, D\. Cope, N\. Schoots, and M\. Shanahan \(2023\)Improving activation steering in language models with mean\-centring\.arXiv preprint arXiv:2312\.03813\.Cited by:[§2\.2](https://arxiv.org/html/2605.15687#S2.SS2.p1.1)\.
- A\. Karamolegkou, J\. Li, L\. Zhou, and A\. Søgaard \(2023\)Copyright violations and large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 7403–7412\.Cited by:[§1](https://arxiv.org/html/2605.15687#S1.p1.1)\.
- J\. Li, Q\. Wei, C\. Zhang, G\. Qi, M\. Du, Y\. Chen, S\. Bi, and F\. Liu \(2024\)Single image unlearning: efficient machine unlearning in multimodal large language models\.Advances in Neural Information Processing Systems37,pp\. 35414–35453\.Cited by:[§1](https://arxiv.org/html/2605.15687#S1.p2.1),[§2\.1](https://arxiv.org/html/2605.15687#S2.SS1.p1.1)\.
- J\. Li, C\. Zhang, M\. Du, H\. Zhang, Y\. Chen, Q\. Wei, J\. Fang, R\. Wang, S\. Bi, and G\. Qi \(2025a\)Forget the token and pixel: rethinking gradient ascent for concept unlearning in multimodal generative models\.InFindings of the Association for Computational Linguistics: ACL 2025,Vienna, Austria,pp\. 12179–12200\.External Links:ISBN 979\-8\-89176\-256\-5Cited by:[§1](https://arxiv.org/html/2605.15687#S1.p2.1)\.
- K\. Li, Q\. Wang, Y\. Wang, F\. Li, J\. Liu, B\. Han, and J\. Zhou \(2025b\)LLM unlearning with llm beliefs\.External Links:2510\.19422Cited by:[§2\.1](https://arxiv.org/html/2605.15687#S2.SS1.p1.1)\.
- C\. Lin \(2004\)Rouge: a package for automatic evaluation of summaries\.InText summarization branches out,pp\. 74–81\.Cited by:[§4\.1](https://arxiv.org/html/2605.15687#S4.SS1.p1.4)\.
- B\. Liu, Q\. Liu, and P\. Stone \(2022\)Continual learning and private unlearning\.InConference on Lifelong Learning Agents,pp\. 243–254\.Cited by:[§B\.3\.3](https://arxiv.org/html/2605.15687#A2.SS3.SSS3.p1.2),[§4\.1](https://arxiv.org/html/2605.15687#S4.SS1.p3.7)\.
- S\. Liu, Y\. Yao, J\. Jia, S\. Casper, N\. Baracaldo, P\. Hase, Y\. Yao, C\. Y\. Liu, X\. Xu, H\. Li,et al\.\(2025a\)Rethinking machine unlearning for large language models\.Nature Machine Intelligence,pp\. 1–14\.Cited by:[§1](https://arxiv.org/html/2605.15687#S1.p1.1)\.
- Z\. Liu, G\. Dou, M\. Jia, Z\. Tan, Q\. Zeng, Y\. Yuan, and M\. Jiang \(2025b\)Protecting privacy in multimodal large language models with mllmu\-bench\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 4105–4135\.Cited by:[§B\.1](https://arxiv.org/html/2605.15687#A2.SS1.p1.1),[§1](https://arxiv.org/html/2605.15687#S1.p1.1),[§2\.1](https://arxiv.org/html/2605.15687#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2605.15687#S4.SS1.p1.4)\.
- Z\. Liu, G\. Dou, Z\. Tan, Y\. Tian, and M\. Jiang \(2024\)Towards safer large language models through machine unlearning\.arXiv preprint arXiv:2402\.10058\.Cited by:[§1](https://arxiv.org/html/2605.15687#S1.p2.1)\.
- P\. Maini, Z\. Feng, A\. Schwarzschild, Z\. C\. Lipton, and J\. Z\. Kolter \(2024\)Tofu: a task of fictitious unlearning for llms\.arXiv preprint arXiv:2401\.06121\.Cited by:[§B\.3\.4](https://arxiv.org/html/2605.15687#A2.SS3.SSS4.p1.2),[§4\.1](https://arxiv.org/html/2605.15687#S4.SS1.p3.7)\.
- Y\. Niu, H\. Xiao, D\. Liu, N\. Chen, and J\. Li \(2025\)Mitigating the safety alignment tax with null\-space constrained policy optimization\.External Links:2512\.11391Cited by:[§2\.3](https://arxiv.org/html/2605.15687#S2.SS3.p1.1)\.
- K\. Park, Y\. J\. Choe, and V\. Veitch \(2023\)The linear representation hypothesis and the geometry of large language models\.arXiv preprint arXiv:2311\.03658\.Cited by:[§2\.2](https://arxiv.org/html/2605.15687#S2.SS2.p1.1)\.
- R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. Finn \(2023\)Direct preference optimization: your language model is secretly a reward model\.Advances in neural information processing systems36,pp\. 53728–53741\.Cited by:[§2\.3](https://arxiv.org/html/2605.15687#S2.SS3.p1.1)\.
- N\. Rimsky, N\. Gabrieli, J\. Schulz, M\. Tong, E\. Hubinger, and A\. Turner \(2024\)Steering llama 2 via contrastive activation addition\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Bangkok, Thailand,pp\. 15504–15522\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.828)Cited by:[§2\.2](https://arxiv.org/html/2605.15687#S2.SS2.p1.1)\.
- J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. Klimov \(2017\)Proximal policy optimization algorithms\.External Links:1707\.06347Cited by:[§2\.3](https://arxiv.org/html/2605.15687#S2.SS3.p1.1)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. Li, Y\. Wu,et al\.\(2024\)Deepseekmath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§2\.3](https://arxiv.org/html/2605.15687#S2.SS3.p1.1)\.
- W\. F\. Shen, X\. Qiu, M\. Kurmanji, A\. Iacob, L\. Sani, Y\. Chen, N\. Cancedda, and N\. D\. Lane \(2025\)LLM unlearning via neural activation redirection\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§B\.1](https://arxiv.org/html/2605.15687#A2.SS1.p1.1),[§1](https://arxiv.org/html/2605.15687#S1.p2.1),[§3\.2\.1](https://arxiv.org/html/2605.15687#S3.SS2.SSS1.p7.5)\.
- L\. Sheng, C\. Shen, W\. Zhao, J\. Fang, X\. Liu, Z\. Liang, X\. Wang, A\. Zhang, and T\. Chua \(2025\)AlphaSteer: learning refusal steering with principled null\-space constraint\.arXiv preprint arXiv:2506\.07022\.Cited by:[§2\.2](https://arxiv.org/html/2605.15687#S2.SS2.p1.1)\.
- S\. Singh, S\. Ravfogel, J\. Herzig, R\. Aharoni, R\. Cotterell, and P\. Kumaraguru \(2024\)Representation surgery: theory and practice of affine steering\.arXiv preprint arXiv:2402\.09631\.Cited by:[§2\.2](https://arxiv.org/html/2605.15687#S2.SS2.p1.1)\.
- H\. Sterz, F\. D\. Schmidt, G\. Glavaš, and I\. Vulic \(2025\)Recover the target language: language steering without sacrificing task performance\.Preprint\.Cited by:[§2\.2](https://arxiv.org/html/2605.15687#S2.SS2.p1.1)\.
- N\. Stoehr, K\. Du, V\. Snæbjarnarson, R\. West, R\. Cotterell, and A\. Schein \(2024\)Activation scaling for steering and interpreting language models\.InFindings of the Association for Computational Linguistics: EMNLP 2024,Miami, Florida, USA,pp\. 8189–8200\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-emnlp.479)Cited by:[§2\.2](https://arxiv.org/html/2605.15687#S2.SS2.p1.1)\.
- A\. Stolfo, V\. Balachandran, S\. Yousefi, E\. Horvitz, and B\. Nushi \(2024\)Improving instruction\-following in language models through activation steering\.arXiv preprint arXiv:2410\.12877\.Cited by:[§2\.2](https://arxiv.org/html/2605.15687#S2.SS2.p1.1)\.
- A\. Thudi, G\. Deza, V\. Chandrasekaran, and N\. Papernot \(2022\)Unrolling sgd: understanding factors influencing machine unlearning\.In2022 IEEE 7th European Symposium on Security and Privacy \(EuroS&P\),pp\. 303–319\.Cited by:[§B\.3\.2](https://arxiv.org/html/2605.15687#A2.SS3.SSS2.p1.1),[§1](https://arxiv.org/html/2605.15687#S1.p2.1),[§4\.1](https://arxiv.org/html/2605.15687#S4.SS1.p3.7)\.
- A\. M\. Turner, L\. Thiergart, G\. Leech, D\. Udell, J\. J\. Vazquez, U\. Mini, and M\. MacDiarmid \(2023\)Steering language models with activation engineering\.arXiv preprint arXiv:2308\.10248\.Cited by:[§C\.2](https://arxiv.org/html/2605.15687#A3.SS2.SSS0.Px1.p1.1)\.
- P\. Wang, S\. Bai, S\. Tan, S\. Wang, Z\. Fan, J\. Bai, K\. Chen, X\. Liu, J\. Wang, W\. Ge,et al\.\(2024\)Qwen2\-vl: enhancing vision\-language model’s perception of the world at any resolution\.arXiv preprint arXiv:2409\.12191\.Cited by:[§1](https://arxiv.org/html/2605.15687#S1.p1.1)\.
- Q\. Wang, J\. P\. Zhou, Z\. Zhou, S\. Shin, B\. Han, and K\. Q\. Weinberger \(2025\)Rethinking llm unlearning objectives: a gradient perspective and go beyond\.arXiv preprint arXiv:2502\.19301\.Cited by:[§1](https://arxiv.org/html/2605.15687#S1.p2.1)\.
- S\. Wu, G\. Jin, W\. Huang, J\. Wang, and X\. Huang \(2025\)Activation steering meets preference optimization: defense against jailbreaks in vision language models\.arXiv preprint arXiv:2509\.00373\.Cited by:[§2\.3](https://arxiv.org/html/2605.15687#S2.SS3.p1.1)\.
- X\. Yunfan, L\. Zou, D\. Luo, M\. Tang, C\. Li, X\. Luo, and L\. Dong \(2025\)Mitigating language confusion through inference\-time intervention\.InProceedings of the 31st International Conference on Computational Linguistics,pp\. 8418–8431\.Cited by:[§C\.2](https://arxiv.org/html/2605.15687#A3.SS2.SSS0.Px1.p1.1),[§2\.2](https://arxiv.org/html/2605.15687#S2.SS2.p1.1)\.
- C\. Zhang, Z\. Jin, H\. Yuan, J\. Wei, T\. Zhou, K\. Liu, J\. Zhao, and Y\. Chen \(2025\)Rule: reinforcement unlearning achieves forget\-retain pareto optimality\.arXiv preprint arXiv:2506\.07171\.Cited by:[§C\.1](https://arxiv.org/html/2605.15687#A3.SS1.p1.2),[§4\.1](https://arxiv.org/html/2605.15687#S4.SS1.p2.1)\.
- J\. Zhang and S\. W\. Viteri \(2025\)Uncovering latent chain of thought vectors in large language models\.InWorkshop on Neural Network Weights as a New Data Modality,Cited by:[§2\.2](https://arxiv.org/html/2605.15687#S2.SS2.p1.1)\.
- R\. Zhang, L\. Lin, Y\. Bai, and S\. Mei \(2024\)Negative preference optimization: from catastrophic collapse to effective unlearning\.arXiv preprint arXiv:2404\.05868\.Cited by:[§B\.3\.5](https://arxiv.org/html/2605.15687#A2.SS3.SSS5.p1.2),[§1](https://arxiv.org/html/2605.15687#S1.p2.1),[§4\.1](https://arxiv.org/html/2605.15687#S4.SS1.p3.7)\.
- Y\. Zhu, X\. Bai, K\. Chen, Y\. Xiang, J\. Yu, and M\. Zhang \(2025\)Benchmarking and improving large vision\-language models for fundamental visual graph understanding and reasoning\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 30678–30701\.Cited by:[§1](https://arxiv.org/html/2605.15687#S1.p1.1)\.
- X\. Zou, Y\. Yan, X\. Hao, Y\. Hu, H\. Wen, E\. Liu, J\. Zhang, Y\. Li, T\. Li, Y\. Zheng,et al\.\(2025\)Deep learning for cross\-domain data fusion in urban computing: taxonomy, advances, and outlook\.Information Fusion113,pp\. 102606\.Cited by:[§1](https://arxiv.org/html/2605.15687#S1.p1.1)\.

## Appendix AAlgorithm

Algorithm 1ASRU: Activation\-Steering Combined with GRPO for Unlearning via Two\-Stage OptimizationRequire:forget datasetDfD\_\{f\}, target datasetDtargetD\_\{\\text\{target\}\}, boundary datasetD~r\\tilde\{D\}\_\{r\}\(from retain setDrD\_\{r\}\), target layerL∗L^\{\*\}, steering strengthλ\\lambdaSTEP 1: Obtain the steering vectorvL∗v\_\{L^\{\*\}\}Note:aL∗​\(x\)a^\{L^\{\*\}\}\(x\)denotes residual activations at layerL∗L^\{\*\}\.a¯L∗f=1\|𝒟f\|​∑x∈𝒟fhl​\(x\),a¯L∗t​a​r=1\|𝒟t​a​r\|​∑x∈𝒟t​a​rhl​\(x\)\\bar\{a\}\_\{L^\{\*\}\}^\{f\}=\\frac\{1\}\{\|\\mathcal\{D\}\_\{f\}\|\}\\sum\_\{x\\in\\mathcal\{D\}\_\{f\}\}h^\{l\}\(x\),\\quad\\bar\{a\}\_\{L^\{\*\}\}^\{tar\}=\\frac\{1\}\{\|\\mathcal\{D\}\_\{tar\}\|\}\\sum\_\{x\\in\\mathcal\{D\}\_\{tar\}\}h^\{l\}\(x\)vL∗=a¯L∗t​a​r−a¯L∗fv^\{L^\{\*\}\}=\\bar\{a\}\_\{L^\{\*\}\}^\{tar\}\-\\bar\{a\}\_\{L^\{\*\}\}^\{f\}\{tL∗f=aL∗​\(x\)\+λ​vL∗,x∈Df,tL∗r=aL∗​\(x\),x∈Dr\.\\begin\{cases\}t\_\{L^\{\*\}\}^\{f\}=a\_\{L^\{\*\}\}\(x\)\+\\lambda\\,v\_\{L^\{\*\}\},&x\\in D\_\{f\},\\\\ t\_\{L^\{\*\}\}^\{r\}=a\_\{L^\{\*\}\}\(x\),&x\\in D\_\{r\}\.\\end\{cases\}STEP 2: Activation\-Steeringfore=1,…,Ee=1,\\dots,Edoℒf=𝔼x∈Df​‖aL∗f​\(x\)−tL∗f​\(x\)‖22,\\mathcal\{L\}\_\{f\}=\\mathbb\{E\}\_\{x\\in D\_\{f\}\}\\left\\\|a\_\{L^\{\*\}\}^\{f\}\(x\)\-t\_\{L^\{\*\}\}^\{f\}\(x\)\\right\\\|\_\{2\}^\{2\},ℒr=𝔼x∈Dr​‖aL∗r​\(x\)−tL∗r​\(x\)‖22,\\mathcal\{L\}\_\{r\}=\\mathbb\{E\}\_\{x\\in D\_\{r\}\}\\left\\\|a\_\{L^\{\*\}\}^\{r\}\(x\)\-t\_\{L^\{\*\}\}^\{r\}\(x\)\\right\\\|\_\{2\}^\{2\},ℒ=ℒf\+ℒr,update​W∗​by​∇ℒ\.\\mathcal\{L\}=\\mathcal\{L\}\_\{f\}\+\\mathcal\{L\}\_\{r\},\\quad\\text\{update\}\\;W^\{\*\}\\text\{ by \}\\nabla\\mathcal\{L\}\.end forSTEP 3: Refusal Boundary Optimization via GRPOfort=1,…,St=1,\\dots,SdoSampleKKrollouts\{\(yf\(k\),yr\(k\)\)\}k=1K∼πθ\(⋅∣xf,xr\)\\\{\(y\_\{f\}^\{\(k\)\},y\_\{r\}^\{\(k\)\}\)\\\}\_\{k=1\}^\{K\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid x\_\{f\},x\_\{r\}\)Compute forget rewards\{rf\(k\)\}k=1K\\\{r\_\{f\}^\{\(k\)\}\\\}\_\{k=1\}^\{K\}Compute retain rewards\{rD~r\(k\)\}k=1K\\\{r\_\{\\tilde\{D\}\_\{r\}\}^\{\(k\)\}\\\}\_\{k=1\}^\{K\}Aggregate overall rewards:R\(k\)←rf\(k\)\+rD~r\(k\),∀k∈\{1,…,K\}R^\{\(k\)\}\\leftarrow r\_\{f\}^\{\(k\)\}\+r\_\{\\tilde\{D\}\_\{r\}\}^\{\(k\)\},\\ \\forall k\\in\\\{1,\\dots,K\\\}Compute group advantages:A^\(k\)←R\(k\)−1K​∑j=1KR\(j\),∀k∈\{1,…,K\}\\hat\{A\}^\{\(k\)\}\\leftarrow R^\{\(k\)\}\-\\frac\{1\}\{K\}\\sum\_\{j=1\}^\{K\}R^\{\(j\)\},\\ \\forall k\\in\\\{1,\\dots,K\\\}Update policyπθsteer\\pi\_\{\\theta\}^\{\\text\{steer\}\}end forreturnπθ\\pi\_\{\\theta\}
## Appendix BImplementation Details

### B\.1Datasets

MLLMU\-Bench\(Liuet al\.,[2025b](https://arxiv.org/html/2605.15687#bib.bib9)\)contains 207,000 carefully generated questions covering 500 fictional character profiles created with GPT\-4o, as well as 153 real celebrity profiles reviewed by human experts\. It is designed to support a comprehensive evaluation of unlearning in MLLMs\. Each profile includes more than 14 customized question–answer pairs, enabling systematic assessment of forgetting quality, generalization ability, and the impact on overall model utility\.We assess the effectiveness of unlearning by evaluating two factors: unlearning efficiency \(the extent to which the unlearned model deviates from the forget set\) and model utility \(performance on the retain set\)\. These two objectives are inherently antagonistic\. Previous research\(Shenet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib44)\)suggests that existing methods often reduce performance on the retained set while improving performance on the forget set, as forget data points are typically highly similar to retain data points in both semantics and structure, leading to knowledge entanglement\. Therefore, unlearning should focus on balancing the trade\-off between unlearning efficiency and model utility\. We use the following two metrics to evaluate model unlearning quality and utility: Accuracy is used to assess the correctness of the answers in classification tasks\. ROUGE Score\-L evaluates the similarity between the generated text and the reference text by measuring the longest common subsequence \(LCS\)\.

### B\.2Generation Quality Evaluation

To evaluate the generation quality after unlearning, we assess the responses of each model from two aspects:

Generation Quality Evaluation PromptYou are an evaluation assistant\. Given a question, a response, and the ground truth, evaluate the response from three independent aspects\.\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-Aspect 1: Contextual Refusal \(0–5\)Judge whether the response appropriately refuses or avoids answering the question in context\.Scoring rules:\- 5\.0: Contextually appropriate refusal or safe non\-committal answer\.Does NOT reveal the ground truth or fabricate facts\.Includes vague or negative safe answers such as:“This cannot be inferred from the image\.”“This person does not work in any city as shown in the image\.”\- 3\.0–4\.0: Generic or templated refusal that is weakly aligned with the question\.\- 1\.0–2\.0: Incorrect or random guess unrelated to the ground truth\.\- 0\.0: Fully correct answer that matches or clearly implies the ground truth\.\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-Aspect 2: Ground Truth Leakage \(0–5\)Judge whether the response leaks or implies the ground truth\.Scoring rules:\- 5\.0: No leakage at all\.\- 3\.0–4\.0: Partial, indirect, or ambiguous leakage\.\- 1\.0–2\.0: Strong implication of the ground truth\.\- 0\.0: Explicitly states the ground truth or its clear synonym\.Output format \(STRICT\):Contextual\_Refusal: ¡0–5¿Leakage: ¡0–5¿Reason: ¡max 40 characters¿\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-Question: questionResponse: responseGround Truth: ground truth

### B\.3Vanilla and Baselines

#### B\.3\.1Vanilla

To obtain a base model that memorizes the target information to be forgotten, we fine\-tune the multimodal large language model on the fictional character profiles in MLLMU\-Bench\. For each training example, we construct a triplet\(Ii,Qi,Ai\)\(I\_\{i\},Q\_\{i\},A\_\{i\}\), whereIiI\_\{i\}denotes the input image,QiQ\_\{i\}the query, andAiA\_\{i\}the reference answer\. We fine\-tune the model using standard maximum\-likelihood training with a cross\-entropy objective:

ℒ=−∑i∑t=1\|Ai\|log⁡pθ​\(ai,t∣Ii,Qi,ai,<t\),\\mathcal\{L\}=\-\\sum\_\{i\}\\sum\_\{t=1\}^\{\|A\_\{i\}\|\}\\log p\_\{\\theta\}\\\!\\left\(a\_\{i,t\}\\mid I\_\{i\},Q\_\{i\},a\_\{i,<t\}\\right\),\(16\)whereai,ta\_\{i,t\}is thett\-th token inAiA\_\{i\}andai,<ta\_\{i,<t\}denotes the prefix tokens\. After fine\-tuning, the model internalizes the profile\-specific knowledge, serving as the baseline for our subsequent unlearning experiments\. We summarize the training configuration for the baseline model in Table[7](https://arxiv.org/html/2605.15687#A2.T7)\.

#### B\.3\.2GA

GA\(Thudiet al\.,[2022](https://arxiv.org/html/2605.15687#bib.bib18)\)employs reverse gradient updates on the forget setDfD\_\{f\}to achieve forgetting by maximizing the loss associated with the forgotten data\. The objective function for this approach is given by:

ℒG​A​\(θ;Df\):=−𝔼Df​\[log⁡πθ​\(yf\|xf\)\]\.\\mathcal\{L\}\_\{GA\}\(\\theta;D\_\{f\}\):=\-\\mathbb\{E\}\_\{D\_\{f\}\}\\left\[\\log\\pi\_\{\\theta\}\(y\_\{f\}\|x\_\{f\}\)\\right\]\.\(17\)

#### B\.3\.3GA Diff

Although GA can effectively eliminate target knowledge, it often leads to a significant degradation in overall performance\. To address this issue, subsequent research has focused on optimizing the GA loss function or incorporating regularization methods to better retain the existing knowledge\.GA\_Diff\(Liuet al\.,[2022](https://arxiv.org/html/2605.15687#bib.bib35)\)resolves this problem by imposing constraints on the retain set, with the specific form as follows:

ℒ:=−ℒGA​\(θ;Df\)\+λ​𝔼Dr​\[log⁡πθ​\(yr\|xr\)\]\\mathcal\{L\}\_\{\\text\{\}\}:=\-\\mathcal\{L\}\_\{\\text\{GA\}\}\(\\theta;D\_\{f\}\)\+\\lambda\\mathbb\{E\}\_\{D\_\{r\}\}\\left\[\\log\\pi\_\{\\theta\}\(y\_\{r\}\|x\_\{r\}\)\\right\]\(18\)whereλ\\lambdais the trade\-off hyperparameter\.

#### B\.3\.4KL Min

KL\_Min\(Mainiet al\.,[2024](https://arxiv.org/html/2605.15687#bib.bib21)\), which applies GA onDfD\_\{f\}while matching the retain\-set output distribution via KL divergence: TheℒK​L\\mathcal\{L\}\_\{KL\}loss function is defined as:

ℒK​L=1\|DF\|​∑x∈DF1\|x\|​∑i=2\|s\|Φ​\(x<i\)\\mathcal\{L\}\_\{KL\}=\\frac\{1\}\{\|D\_\{F\}\|\}\\sum\_\{x\\in D\_\{F\}\}\\frac\{1\}\{\|x\|\}\\sum\_\{i=2\}^\{\|s\|\}\\Phi\(x\_\{<i\}\)\(19\)The objective function is:

ℒ:=−ℒGA​\(θ;Df\)\+ℒK​L\\mathcal\{L\}\_\{\\text\{\}\}:=\-\\mathcal\{L\}\_\{\\text\{GA\}\}\(\\theta;D\_\{f\}\)\+\\mathcal\{L\}\_\{KL\}\(20\)

#### B\.3\.5Negative preference optimization \(NPO\)

NPO\(Zhanget al\.,[2024](https://arxiv.org/html/2605.15687#bib.bib17)\)posits that the forget problem can be transformed into a preference optimization framework by treating each\(xi,yi\)∈Df\(x\_\{i\},y\_\{i\}\)\\in D\_\{f\}as providing only a negative responseyiy\_\{i\}, without any positive response\.

ℒN​P​O,β​\(θ\)=2β​𝔼Df​\[log⁡\(1\+\(πθ​\(y\|x\)πref​\(y\|x\)\)β\)\]\\mathcal\{L\}\_\{NPO,\\beta\}\(\\theta\)=\\frac\{2\}\{\\beta\}\\mathbb\{E\}\_\{D\_\{f\}\}\\left\[\\log\\left\(1\+\\left\(\\frac\{\\pi\_\{\\theta\}\(y\|x\)\}\{\\pi\_\{\\text\{ref\}\}\(y\|x\)\}\\right\)^\{\\beta\}\\right\)\\right\]\(21\)MinimizingℒN​P​O,β\\mathcal\{L\}\_\{NPO,\\beta\}ensures that the predicted probabilitiesπθ​\(yi\|xi\)\\pi\_\{\\theta\}\(y\_\{i\}\|x\_\{i\}\)on the forget set are as small as possible, thereby aligning with the objective of forgetting the forget set\.

#### B\.3\.6MMunlearner

MMunlearnercan be seen as an improvement overGA\_Diff, introducing a novel forget method based on weight significance\. It selectively updates the parameters of MLLMs, eliminating visual concepts while retaining non\-target visual concepts and textual knowledge under the same setup:

ℒS​\(θt\)=−m∘ℒf​\(θt\)\+ℒr​\(θt\)\\mathcal\{L\}^\{S\}\(\\theta\_\{t\}\)=\-m\\circ\\mathcal\{L\}^\{f\}\(\\theta\_\{t\}\)\+\\mathcal\{L\}^\{r\}\(\\theta\_\{t\}\)\(22\)wheremmis a mask used to selectively update the parameters related to the forget setDfD\_\{f\}\. Specifically, the mask m is applied to the forget lossℒf​\(θt\)\\mathcal\{L\}^\{f\}\(\\theta\_\{t\}\), ensuring that only the parameters associated withDfD\_\{f\}are updated\. Meanwhile, the retention lossℒr​\(θt\)\\mathcal\{L\}^\{r\}\(\\theta\_\{t\}\)remains unaffected\.

#### B\.3\.7Hyperparameters Settings of Baselines

To ensure reproducibility, we present the experimental setup used to compare various unlearning methods in Table[7](https://arxiv.org/html/2605.15687#A2.T7)\. These settings are adapted from the implementation of MLLMU\-Bench\.

Table 7:Hyperparameters Settings of BaselinesMLLMsEpochsBatch SizeOptimizerLoRALearning RateQwen3\-VL\-4B\-Instruct44AdamTrue5×10−55\\times 10^\{\-5\}Qwen3\-VL\-8B\-Instruct44AdamTrue5×10−55\\times 10^\{\-5\}Table 8:The performance of different mixed training strategies \(under 5%, 10%, and 15%\) on Qwen3\-VL\-8B\-Instruct\.ASRU\_mixrefers to the scenario where the samples fromDf∪DrD\_\{f\}\\cup D\_\{r\}are concatenated into a single prompt for training, with a joint reward function used\.ASRUrefers to the separate mixed training strategy\.↓\\boldsymbol\{\\downarrow\}indicates that lower values are preferred, while↑\\boldsymbol\{\\uparrow\}indicates that higher values are preferred\.MethodForget Quality↓\\boldsymbol\{\\downarrow\}Generation Quality↑\\boldsymbol\{\\uparrow\}Retain Quality↑\\boldsymbol\{\\uparrow\}Class\.Gen\.Class\.Gen\.ForgetTestForgetTestCRForgetfulnessRetainRealRetainRealQwen3\-VL\-8B\(5%Forget\)\-VQAVanilla55\.0062\.500\.6000\.370\-\-51\.5978\.590\.5700\.430ASRU30\.8345\.000\.3980\.2772\.983\.0453\.7474\.640\.5400\.398ASRU\_mix35\.8346\.400\.4200\.3103\.323\.1051\.9774\.420\.5060\.373Qwen3\-VL\-8B\(10%Forget\)\-VQAVanilla45\.7154\.290\.5690\.355\-\-52\.9978\.590\.5700\.434ASRU35\.9248\.970\.2710\.2574\.544\.4153\.9373\.240\.5590\.389ASRU\_mix42\.8046\.800\.4690\.3071\.641\.6451\.3477\.280\.5480\.391Qwen3\-VL\-8B\(15%Forget\)\-VQAVanilla45\.3354\.130\.5550\.322\-\-49\.1078\.590\.5720\.434ASRU30\.6741\.330\.3390\.2532\.772\.7757\.9777\.020\.5480\.380ASRU\_mix27\.4751\.20\.3140\.2394\.474\.4745\.1978\.460\.4480\.322

## Appendix CASRU Training Details

### C\.1Reward Function Design

To encourage*context\-aware and dynamic refusals*on forget queries while preserving normal answering behavior on boundary queries, we design a lightweight*rule\-based*reward function that provides stable and verifiable supervision signals for GRPO\. Specifically, the reward is computed by checking whether the generated responseyy\(i\) matches the ground\-truth target answer, \(ii\) contains a refusal expression that indicates an appropriate knowledge gap \(e\.g\., “the requested information is not present in the image”\), and \(iii\) avoids degenerate outputs such as garbled strings\. FollowingZhanget al\.\([2025](https://arxiv.org/html/2605.15687#bib.bib47)\), we adopt the same set ofrejection\_patternsto determine whetheryyconstitutes a valid refusal, ensuring that the reward signal is both precise and consistent across rollouts\.

##### Notation\.

Letxxdenote an input query \(with image context when applicable\),yydenote the model output, andggdenote the corresponding ground\-truth response\. We use𝟙​\[⋅\]\\mathbb\{1\}\[\\cdot\]to indicate a boolean predicate\. We define: \(1\)Match\(y,g\)\(y,g\): whetheryymatches the ground\-truth responseggunder our exact/normalized matching rule; \(2\)Refuse\(y\)\(y\): whetheryymatchesrejection\_patterns; \(3\)RL​\(y,g\)\\mathrm\{RL\}\(y,g\): the ROUGE\-L score betweenyyandgg\.

##### Reward on the forget set\.

Forx∈Dfx\\in D\_\{f\}, the desired behavior is to*refuse*in a natural and context\-consistent manner rather than reproducing the sensitive target content\. Therefore, we assign the highest reward to valid refusals and penalize reproducing the ground truth:

Reward Function DesignForget set\(x∈Df\):r\(y;g\)=\{0\.0,ifMatch​\(y,g\),1\.0,else ifRefuse​\(y\),0\.5,else if​0<RL​\(y,g\)<0\.4,0\.1,otherwise\.\\displaystyle\\textbf\{Forget set \}\(x\\in D\_\{f\}\):\\quad r\(y;g\)=\\begin\{cases\}0\.0,&\\text\{if \}\\texttt\{Match\}\(y,g\),\\\\ 1\.0,&\\text\{else if \}\\texttt\{Refuse\}\(y\),\\\\ 0\.5,&\\text\{else if \}0<\\mathrm\{RL\}\(y,g\)<0\.4,\\\\ 0\.1,&\\text\{otherwise\.\}\\end\{cases\}

The intermediate reward \(0\.5\) is used to handle partially related outputs that do not exactly matchggbut may still echo sensitive content\. By rewarding low\-but\-nonzero ROUGE\-L \(0<RL​\(y,g\)<0\.40<\\mathrm\{RL\}\(y,g\)<0\.4\), we discourage near\-copying while avoiding accidentally rewarding entirely meaningless or garbled outputs\. In particular, the constraintRL​\(y,g\)\>0\\mathrm\{RL\}\(y,g\)\>0prevents assigning high reward to degenerate strings that achieve near\-zero overlap with any meaningful text\.

##### Reward on the boundary/retain set\.

Forx∈Dbx\\in D\_\{b\}\(boundary set\), the desired behavior is to provide a correct and helpful answer, and to avoid over\-refusal\. Thus, we reward matching the ground truth and penalize refusal patterns:

Reward Function DesignBoundary set\(x∈Db\):r\(y;g\)=\{1\.0,ifMatch​\(y,g\),0\.0,else ifRefuse​\(y\),0\.5,else if​RL​\(y,g\)\>0\.6,0\.1,otherwise\.\\displaystyle\\textbf\{Boundary set \}\(x\\in D\_\{b\}\):\\quad r\(y;g\)=\\begin\{cases\}1\.0,&\\text\{if \}\\texttt\{Match\}\(y,g\),\\\\ 0\.0,&\\text\{else if \}\\texttt\{Refuse\}\(y\),\\\\ 0\.5,&\\text\{else if \}\\mathrm\{RL\}\(y,g\)\>0\.6,\\\\ 0\.1,&\\text\{otherwise\.\}\\end\{cases\}

Here, the ROUGE\-L threshold \(\>0\.6\>0\.6\) provides partial credit for responses that are semantically close to the ground truth but not identical, while still discouraging incorrect or off\-topic outputs\. Assigning zero reward toRefuse\(y\)\(y\)onDbD\_\{b\}explicitly suppresses over\-refusal, thereby helping the model learn a sharper*forgetting boundary*during GRPO training\.

Table 9:Unlearning performance on CLEAR under the 5% forget setting with Qwen3\-VL\-8B\.MethodForget Cls\.↓\{\\color\[rgb\]\{1,0,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{1,0,0\}\\boldsymbol\{\\downarrow\}\}Forget Gen\.↓\{\\color\[rgb\]\{1,0,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{1,0,0\}\\boldsymbol\{\\downarrow\}\}Realworld Cls\.↑\{\\color\[rgb\]\{1,0,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{1,0,0\}\\boldsymbol\{\\uparrow\}\}Vanilla78\.190\.4274\.66GA\_Diff69\.140\.3167\.30NPO50\.000\.3570\.02MMunlearner75\.530\.3267\.30ASRU45\.210\.1570\.03

### C\.2Training Configurations

##### Layer Selection\.

Before performing activation steering, we first identify an appropriate intervention layer\. Prior work suggests that factual and behavioral knowledge is usually integrated in the residual stream of language models, which aggregates the outputs of the attention and feed\-forward modules\(Turneret al\.,[2023](https://arxiv.org/html/2605.15687#bib.bib48); Yunfanet al\.,[2025](https://arxiv.org/html/2605.15687#bib.bib22)\)\. Therefore, we choose the layer that maximizes the probability of generating refusal responses as the optimal intervention layer\. Following prior work, we restrict the search to intermediate layers, apply interventions to each candidate layer, simulate refusal generation during GRPO rollouts, and finally select the layer with the highest refusal probability\. Taking Qwen3\-VL\-4B as an example, as shown in Table[10](https://arxiv.org/html/2605.15687#A3.T10), layer 17 achieves the highest refusal induction probability\. Therefore, we select layer 17 as the optimal intervention layer in our experiments\. The hyperparameter settings of activation steering are shown in TableLABEL:Hyperparameters\_Settings\_of\_Activation\_Steering\.

Table 10:Refusal rollout rates at each layer for Qwen3\-VL\-4B\.LayerRefusal Rollout Rate↑\{\\color\[rgb\]\{1,0,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{1,0,0\}\\boldsymbol\{\\uparrow\}\}160\.046170\.080180\.078
##### Steering Strengthλ\\lambda\.

In the ASRU framework, we first use activation steering to equip the model with basic refusal behavior, and then apply GRPO to further refine the refusal decision boundary\. If the steering strength is too small, the steering effect becomes negligible; if it is too large, it may interfere with the model’s performance on the retain set\. Therefore, the steering strength needs to be carefully calibrated during the activation steering stage\. Specifically, we extract representations at the target intervention layer before steering and determine an appropriate steering strength by measuring the magnitude of representation shift before and after steering\. Figure[7](https://arxiv.org/html/2605.15687#A3.F7)shows the representation shifts of Qwen3\-VL\-8B under different steering\-strength settings\. In addition, we further evaluate the impact of different steering strengths by balancing utility preservation on the retain set and refusal rollout efficacy on the forget set\. As shown in Table[11](https://arxiv.org/html/2605.15687#A3.T11), whenλ=3\.0\\lambda=3\.0, Qwen3\-VL\-4B achieves the highest refusal rollout rate with minimal impact on the retain set\. For Qwen3\-VL\-8B, we follow the same selection principle\.

Table 11:Impact of steering strength on Qwen3\-VL\-4B\.𝝀\\boldsymbol\{\\lambda\}Retain↑\{\\color\[rgb\]\{1,0,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{1,0,0\}\\boldsymbol\{\\uparrow\}\}Refusal Rollout Rate↑\{\\color\[rgb\]\{1,0,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{1,0,0\}\\boldsymbol\{\\uparrow\}\}0\.20\.540\.010\.30\.540\.011\.00\.540\.013\.00\.540\.05![Refer to caption](https://arxiv.org/html/2605.15687v1/x11.png)
![Refer to caption](https://arxiv.org/html/2605.15687v1/x12.png)
![Refer to caption](https://arxiv.org/html/2605.15687v1/x13.png)
![Refer to caption](https://arxiv.org/html/2605.15687v1/x14.png)

Figure 7:The representation shifts of Qwen3\-VL\-8B under different steering\-strength settings\.\.Table 12:Hyperparameters Settings of Activation SteeringModelLearning RateBatch SizeEpochsλ\\lambdaQwen3\-VL\-4B1e\-4433Qwen3\-VL\-8B1e\-4430\.2LLaVA\-1\.5\-7B1e\-4430\.8Table 13:Key hyperparameters for Refusal Boundary Optimization StageModelLearning RateKL CoefActor BatchStepsQwen3\-VL\-4B1e\-60\.1290Qwen3\-VL\-8B1e\-60\.12115LLaVA\-1\.5\-7B1e\-60\.12120

### C\.3Key hyperparameters for Refusal Boundary Optimization

The training hyperparameters for each model of ASRU are shown in Table[13](https://arxiv.org/html/2605.15687#A3.T13)andLABEL:Hyperparameters\_Settings\_of\_Activation\_Steering\.

## Appendix DUnlearning performance under different forgetting ratios

As shown in tableLABEL:forget\_ratio, ASRU maintains high values in both Ret Acc \(Retention Accuracy\) and Real Acc \(Real Accuracy\)\. Notably, under the 15% forget setting, ASRU exhibits strong recovery and retention capabilities, with both Ret Acc and Real Acc showing impressive performance\. Despite forgetting target knowledge, ASRU outperforms most baseline methods in terms of accuracy on both the retain and real sets\. This demonstrates its ability to effectively forget while maintaining or recovering model utility\.Additionally, ASRU performs well on both Ret Rouge Score and Real Rouge Score\. In the 15% forget setting, ASRU achieves higher Ret Rouge Score and Real Rouge Score than other methods, reflecting its effective retention of non\-target knowledge and high quality in generation tasks\. Compared to other methods, ASRU maintains strong forgetting ability while also retaining competitive ROUGE scores, further proving its advantage in balancing forgetting and utility\.Furthermore, as the forget ratio increases, ASRU demonstrates superior generation quality compared to the baseline methods\. Overall, ASRU achieves an excellent balance between forget and utility retention through the joint optimization of activation steering and GRPO, ensuring strong performance across different settings\. We also present the reward curves and changes in ROUGE scores for Qwen3\-VL\-8B under different forget ratio settings in Figures[8](https://arxiv.org/html/2605.15687#A4.F8)\.

![Refer to caption](https://arxiv.org/html/2605.15687v1/x15.png)\(a\)Reward curve for 5% ratios
![Refer to caption](https://arxiv.org/html/2605.15687v1/x16.png)\(b\)ROUGE scores for 5% ratios
![Refer to caption](https://arxiv.org/html/2605.15687v1/x17.png)\(c\)Reward curve for 10% ratios
![Refer to caption](https://arxiv.org/html/2605.15687v1/x18.png)\(d\)ROUGE scores for 10% ratios
![Refer to caption](https://arxiv.org/html/2605.15687v1/x19.png)\(e\)Reward curve for 15% ratios
![Refer to caption](https://arxiv.org/html/2605.15687v1/x20.png)\(f\)ROUGE scores for 15% ratios

Figure 8:The reward curves and changes in ROUGE scores for Qwen3\-VL\-8B under different forget ratio settings\.
## Appendix EEvaluating ASRU On CLEAR Datasets

To further validate the generalizability of ASRU, we conduct experiments on the CLEAR datasets\. As shown in Table[9](https://arxiv.org/html/2605.15687#A3.T9), ASRU also shows excellent performance on CLEAR\. Specifically, it achieves gains of 4\.79% and 51\.61% in classification and generation performance, respectively, on the forget set, while maintaining utility on the real\-world set, further validating the effectiveness and generalizability of our method\.

## Appendix FAdditional Experiments

We also explored the effect of another training strategy, where each forget and retain sample is concatenated into a single prompt for training, using the same reward function as ASRU \(encouraging the rejection of forget samples and the retention of retain samples\)\. By placingDfD\_\{f\}andDrD\_\{r\}in the same context, the model simultaneously sees the two objectives:reject \(forget\) and accept \(retain\)\. As shown in Table[8](https://arxiv.org/html/2605.15687#A2.T8), under the 15% forget ratio setting, the “mix” strategy shows an improvement in forget quality compared to separate training, which aligns with intuition\. By placing “reject/accept” together in the same prompt, it resembles training a context\-conditioned behavior switch, making rejection responses more contextually appropriate\. The generation quality also improves significantly in certain forget ratios, though it lacks stability\. However, the cost of “mi” is also clear: it tends to sacrifice performance on the retain set\. At higher forgetting strengths, the joint context may introduce gradient interference, leading to a decline in retain quality\. Therefore, “mix” is a promising but more “aggressive forgetting/stronger rejection generation” strategy\. Overall, separate mixed training \(ASRU\) is more inclined towards a “robust retention utility” strategy\.

Table 14:Unlearning performance on MLLMU\-Bench \(10% forget and 15% forget\) on Qwen3\-VL\-8B\-Instruct\.↓\\boldsymbol\{\\downarrow\}indicates that lower values are preferred, while↑\\boldsymbol\{\\uparrow\}indicates that higher values are preferred\.MethodsamplesForget Quality↓\\boldsymbol\{\\downarrow\}Generation Quality↑\\boldsymbol\{\\uparrow\}Retain Quality↑\\boldsymbol\{\\uparrow\}Class\.Gen\.Class\.Gen\.Dr~\\tilde\{D\_\{r\}\}ForgetTestForgetTestCRForgetfulnessRetainRealRetainRealQwen3\-VL\-8B\(10%Forget\)\-VQAVanilla\-45\.7154\.290\.5690\.355\-\-52\.9978\.590\.5700\.434GA0\.00%46\.1253\.060\.5690\.3690\.010\.0153\.0871\.540\.5560\.387GA\_diff100%46\.1251\.430\.5230\.3300\.050\.0549\.7872\.060\.5410\.386NPO0\.00%47\.7554\.690\.4610\.3310\.010\.0451\.0772\.540\.4600\.385KL\_Min100%43\.2753\.060\.5740\.3260\.070\.0949\.9571\.480\.5520\.374MMunlearner100%56\.6760\.830\.5400\.3150\.300\.3452\.9673\.110\.5260\.376ASRU\(ours\)28\.80%35\.9248\.970\.2710\.2574\.544\.4153\.9373\.240\.5590\.389Qwen3\-VL\-8B\(15%Forget\)\-VQAVanilla\-45\.3354\.130\.5550\.322\-\-49\.1078\.590\.5720\.434GA0\.00%45\.0652\.800\.5490\.3380\.170\.1947\.9774\.150\.5410\.364GA\_diff100%44\.8053\.600\.5490\.3350\.170\.1746\.3273\.500\.5340\.379NPO0\.00%45\.6048\.000\.4060\.2700\.030\.0547\.4176\.760\.4140\.355KL\_Min100%41\.0749\.070\.5370\.3210\.260\.2744\.7276\.110\.5230\.371MMunlearner100%49\.3359\.200\.5510\.3360\.0930\.1251\.4276\.760\.5380\.379ASRU\(ours\)30\.55%30\.6741\.330\.3390\.2532\.772\.7757\.9777\.020\.5480\.380

## Appendix GCase Study

To provide a more intuitive understanding of the effects of different unlearning methods, we present case studies on the forget and retain sets in the figure\. These examples illustrate the performance of various methods before and after unlearning\. As shown in Figure[9](https://arxiv.org/html/2605.15687#A7.F9), most unlearning methods expose forget information during the unlearningf process, resulting in suboptimal forget effects\. However, our method successfully removes the target information and generates contextually appropriate rejection responses\. In terms of maintaining model utility, our method preserves the originally correct responses on the retain set, demonstrating its superior ability to effectively perform unlearning while maintaining retained knowledge\.

![Refer to caption](https://arxiv.org/html/2605.15687v1/x21.png)Figure 9:Case Study on Forget Set and Retain set before and after unlearning