Stochastic Meta-Unlearning: Bridging Language Backbone and Multimodal Unlearning

# Stochastic Meta-Unlearning: Bridging Language Backbone and Multimodal Unlearning
Source: [https://arxiv.org/html/2607.18615](https://arxiv.org/html/2607.18615)
Zijie Liu1, Jinhao Duan1, Gaowen Liu2, Sijia Liu3, Tianlong Chen1 1UNC at Chapel Hill,2Cisco Research,3Michigan State University tianlong@cs\.unc\.edu

###### Abstract

Machine unlearning for vision\-language models \(VLMs\) remains underexplored\. Unlike language models, VLMs combine a language backbone with visual components, which makes unlearning more complex\. There is a surprising phenomenon when moving from single\-modality unlearning to VLM unlearning: a target forgotten by the standalone language backbone can still be recovered when image information is given to the full VLM\. This shows that text\-only feedback is not enough for reliable VLM unlearning\. Motivated by this observation, we propose Stochastic Meta\-Unlearning \(SMU\), a bilevel framework that uses VLM\-level feedback to learn an unlearning\-ready initialization\. In the inner loop, SMU applies a few unlearning steps to the language backbone using text data\. In the outer loop, SMU recomposes the updated backbone with the frozen VLM and evaluates forgetting and utility at the VLM level\. This design makes the unlearning update aware of the final multimodal behavior, while still keeping the update local to the language backbone\. Experiments on two VLMs, two multimodal meme datasets, and three baselines show that SMU achieves the best overall forget\-retain trade\-off\. Compared with the strongest baseline for each metric, SMU reduces average Forget accuracy by 10\.52 points and improves average Retain and Test accuracy by 20\.10 and 17\.01 points, respectively\. More importantly, SMU also transfers to new forgetting targets and to different meta\-test unlearning methods\. These results suggest that VLM\-level feedback can make language\-backbone unlearning more reliable and more transferable for VLMs\.

## 1Introduction

Machine unlearning for vision\-language models \(VLMs\) is both practically important and underexplored\. VLMs are increasingly deployed in sensitive settings, such as privacy\-sensitive assistance, safety\-critical applications, and personalized services, where the ability to selectively remove specific knowledge or behaviors after training is essential\(Radfordet al\.,[2021](https://arxiv.org/html/2607.18615#bib.bib18); Liuet al\.,[2023](https://arxiv.org/html/2607.18615#bib.bib14); Wanget al\.,[2025b](https://arxiv.org/html/2607.18615#bib.bib20); Liet al\.,[2025](https://arxiv.org/html/2607.18615#bib.bib19)\)\. Yet unlearning in VLMs is substantially more complex than in standalone language models, because a VLM couples a visual encoder, a projection module, and a language backbone into a tightly interdependent system\(Alayracet al\.,[2022](https://arxiv.org/html/2607.18615#bib.bib1); Liet al\.,[2023](https://arxiv.org/html/2607.18615#bib.bib13); Liuet al\.,[2023](https://arxiv.org/html/2607.18615#bib.bib14); Zhanget al\.,[2024b](https://arxiv.org/html/2607.18615#bib.bib15)\)\. Edits to any single component can propagate across modalities in unpredictable ways, making targeted forgetting far harder to achieve\.

Existing work addresses this challenge by optimizing the composed VLM directly\. This route provides full multimodal supervision and has been applied to visual concept removal, privacy protection, safety alignment, sequential deletion, and modality\-aware unlearning\(Liet al\.,[2024a](https://arxiv.org/html/2607.18615#bib.bib39); Dontsovet al\.,[2025](https://arxiv.org/html/2607.18615#bib.bib40); Liuet al\.,[2025b](https://arxiv.org/html/2607.18615#bib.bib41); Maet al\.,[2024](https://arxiv.org/html/2607.18615#bib.bib42); Kawakamiet al\.,[2025](https://arxiv.org/html/2607.18615#bib.bib43); Huoet al\.,[2025](https://arxiv.org/html/2607.18615#bib.bib44); Liuet al\.,[2025c](https://arxiv.org/html/2607.18615#bib.bib45); Chenet al\.,[2025](https://arxiv.org/html/2607.18615#bib.bib46)\)\. However, because the visual encoder, projector, and language backbone must remain aligned, updates targeting one behavior can inadvertently degrade unrelated capabilities, a side effect we confirm experimentally\.

A natural alternative is to unlearn only the language backbone and recompose it with the frozen visual modules\. This approach is appealing: it updates only the language backbone, keeps the visual encoder and projector intact, and can directly reuse existing language\-model unlearning objectives\(Petroniet al\.,[2019](https://arxiv.org/html/2607.18615#bib.bib12); Jianget al\.,[2020](https://arxiv.org/html/2607.18615#bib.bib9); Wallatet al\.,[2020](https://arxiv.org/html/2607.18615#bib.bib8)\)\. However, unlearning is optimized and verified on the standalone language model, with no guarantee that forgetting holds after the backbone is recomposed with visual modules\. Prior work shows that behavior suppressed in one modality can remain accessible through another\(Huoet al\.,[2025](https://arxiv.org/html/2607.18615#bib.bib44); Wanget al\.,[2025a](https://arxiv.org/html/2607.18615#bib.bib2)\), and we observe the same failure in Fig\.[1](https://arxiv.org/html/2607.18615#S1.F1): a backbone that forgets the target under text\-only queries can recover it when image information is provided at VLM inference time\. To our knowledge, no prior work has successfully made backbone\-side unlearning hold under multimodal recomposition\.

These observations expose a fundamental mismatch\. For utility preservation, the intervention should be local to the language backbone\. For correctness, forgetting must be verified in the fully composed VLM\. A single\-level objective cannot capture both requirements: if defined only on the backbone, the update is blind to multimodal recovery; if defined on the full VLM, the update risks disturbing multimodal utility\.

Key Challenge\.How can we preserve the utility of language\-backbone unlearning while ensuring forgetting holds in the fully composed VLM?

To address this challenge, we formulate VLM unlearning as a bilevel meta\-optimization problem\. Theinner loopperforms the unlearning operation used at test time: it appliesKKsteps of language\-backbone unlearning on unimodal text queries\. Theouter loopreinserts the updated backbone into the frozen VLM and evaluates forget and retain losses on both text\-only and image\-conditioned queries\. The meta\-gradient is backpropagated through the inner unlearning steps, so the visual modules are never updated but are used to assess whether backbone\-level forgetting persists after multimodal recomposition\.

![Refer to caption](https://arxiv.org/html/2607.18615v1/x1.png)Figure 1:Text\-side unlearning alone does not guarantee multimodal forgetting\.Across Qwen and InternVL on Covid and Hateful Memes, LM\-side unlearning can reduce Forget Text accuracy, but the target often remains recoverable under image\-conditioned VLM queries\.Based on this formulation, we proposeStochastic Meta\-Unlearning\(SMU\)\. SMU meta\-learns a language backbone initialization such that, afterKKsteps of unimodal unlearning from that initialization, the recomposed VLM forgets the target concept under both text\-only and image\-conditioned queries while retaining performance on unrelated tasks\. To improve generalization across diverse forgetting targets, SMU stochastically samples forgetting tasks during meta\-training, exposing the initialization to a distribution of unlearning scenarios rather than a fixed target\. At meta\-test time, SMU applies only backbone\-side unlearning, but the initialization has been optimized with VLM\-level multimodal feedback\.

We evaluate SMU on two VLMs and two multimodal meme benchmarks, comparing against three direct VLM unlearning baselines under both single\-modality and multimodal evaluation\. SMU achieves the strongest overall forget\-retain trade\-off: compared with the strongest baseline per metric, SMU reduces average Forget accuracy by 10\.52 points while improving average Retain and Test accuracy by 20\.10 and 17\.01 points, respectively\. We further show that SMU generalizes to unseen forgetting targets and remains robust when the meta\-test unlearning method changes\. Our contributions are summarized as follows:

- •We identify a component\-system mismatch in VLM unlearning and proposeStochastic Meta\-Unlearning\(SMU\), a bilevel framework that addresses it: the inner loop performs language\-backbone unlearning on unimodal queries, while the outer loop uses the recomposed VLM to provide multimodal forget\-retain feedback, ensuring that backbone\-level forgetting holds after recomposition\.
- •We show that the stochastic task sampling in SMU addresses the generalization problem in VLM unlearning: by exposing the backbone initialization to a distribution of forgetting tasks during meta\-training, SMU transfers to unseen forgetting targets rather than overfitting to a fixed unlearning scenario\.
- •We empirically show that SMU improves the forget\-retain trade\-off over direct VLM unlearning baselines, generalizes to new forgetting targets, and remains robust when the meta\-test unlearning method changes\.

## 2Related Works

Machine Unlearning in Classical Models and LLMs\.

Machine unlearning was first formulated as removing the influence of selected training data without retraining from scratch, and later developed into practical approximate deletion schemes\(Cao and Yang,[2015](https://arxiv.org/html/2607.18615#bib.bib16); Bourtouleet al\.,[2021](https://arxiv.org/html/2607.18615#bib.bib17); Neelet al\.,[2021](https://arxiv.org/html/2607.18615#bib.bib21)\)\. Empirical work on deep unlearning established that forget quality, utility preservation, and repeated\-request robustness are distinct desiderata that must be evaluated separately\(Kurmanjiet al\.,[2023](https://arxiv.org/html/2607.18615#bib.bib22)\)\. In language models, subsequent studies addressed post\-hoc removal of private, copyrighted, or harmful knowledge through privacy\-oriented erasure, targeted content removal, and systematic benchmarks\(Janget al\.,[2023](https://arxiv.org/html/2607.18615#bib.bib23); Eldan and Russinovich,[2024](https://arxiv.org/html/2607.18615#bib.bib24); Yao and Xu,[2024](https://arxiv.org/html/2607.18615#bib.bib25); Yaoet al\.,[2024](https://arxiv.org/html/2607.18615#bib.bib26)\)\. TOFU, RWKU, and MUSE further showed that LLM unlearning must be evaluated beyond exact forget prompts, covering entity knowledge, privacy leakage, adversarial probing, and sequential requests\(Mainiet al\.,[2024](https://arxiv.org/html/2607.18615#bib.bib27); Caoet al\.,[2024](https://arxiv.org/html/2607.18615#bib.bib28); Shiet al\.,[2024](https://arxiv.org/html/2607.18615#bib.bib29)\)\. Optimization\-focused methods such as NPO, ULD, ECO, SPUL, and SOUL improve the forget–retain trade\-off but still study forgetting within a fixed model and procedure\(Zhanget al\.,[2024a](https://arxiv.org/html/2607.18615#bib.bib30); Jiet al\.,[2024](https://arxiv.org/html/2607.18615#bib.bib31); Liuet al\.,[2024](https://arxiv.org/html/2607.18615#bib.bib32); Bhailaet al\.,[2025](https://arxiv.org/html/2607.18615#bib.bib33); Jiaet al\.,[2024](https://arxiv.org/html/2607.18615#bib.bib34)\)\. Analyses of brittleness and reversibility motivate treating transfer as a first\-class objective rather than assuming that forgetting under one setup generalizes to another\(Liuet al\.,[2025a](https://arxiv.org/html/2607.18615#bib.bib35); Yuanet al\.,[2024](https://arxiv.org/html/2607.18615#bib.bib36); Doshi and Stickland,[2024](https://arxiv.org/html/2607.18615#bib.bib37); Huet al\.,[2024](https://arxiv.org/html/2607.18615#bib.bib38)\)\.

Multimodal and Vision\-Language Unlearning\.Recent work extends unlearning to VLMs through new benchmarks and architecture\-specific methods\(Liet al\.,[2024a](https://arxiv.org/html/2607.18615#bib.bib39); Dontsovet al\.,[2025](https://arxiv.org/html/2607.18615#bib.bib40); Liuet al\.,[2025b](https://arxiv.org/html/2607.18615#bib.bib41); Maet al\.,[2024](https://arxiv.org/html/2607.18615#bib.bib42)\)\. SIU shows that forgetting a visual concept requires carefully constructed multimodal objectives rather than direct reuse of text\-only recipes\(Liet al\.,[2024a](https://arxiv.org/html/2607.18615#bib.bib39)\)\. CLEAR, MLLMU\-Bench, FIUBench, and PULSE show that multimodal unlearning is harder to evaluate because target knowledge is distributed across images, text, and alignment behavior, and robustness under adversarial or sequential settings is often weak\(Dontsovet al\.,[2025](https://arxiv.org/html/2607.18615#bib.bib40); Liuet al\.,[2025b](https://arxiv.org/html/2607.18615#bib.bib41); Maet al\.,[2024](https://arxiv.org/html/2607.18615#bib.bib42); Kawakamiet al\.,[2025](https://arxiv.org/html/2607.18615#bib.bib43)\)\. Recent methods address this by directly updating the multimodal model via geometry\-constrained optimization, modality\-aware neuron pruning, or safety\-oriented decoupling losses\(Huoet al\.,[2025](https://arxiv.org/html/2607.18615#bib.bib44); Liuet al\.,[2025c](https://arxiv.org/html/2607.18615#bib.bib45); Chenet al\.,[2025](https://arxiv.org/html/2607.18615#bib.bib46)\)\. Privacy\-centered evaluations confirm that forgetting in MLLMs must be checked from both unimodal and multimodal views under stronger attack protocols\(Patilet al\.,[2025](https://arxiv.org/html/2607.18615#bib.bib47); Liuet al\.,[2025b](https://arxiv.org/html/2607.18615#bib.bib41); Maet al\.,[2024](https://arxiv.org/html/2607.18615#bib.bib42)\)\. In contrast, our method meta\-trains the LM backbone so that a small number of text\-side unlearning steps transfers into the desired multimodal forget/retain behavior after composition with a frozen VLM\.

Transferable Unlearning and Meta\-Learning\.A smaller but directly relevant literature examines whether forgetting can transfer across data domains, task configurations, or downstream deployments\(Sepahvandet al\.,[2024](https://arxiv.org/html/2607.18615#bib.bib48); Luet al\.,[2024](https://arxiv.org/html/2607.18615#bib.bib49)\)\. Meta\-learning provides the algorithmic template for optimizing parameters that become useful after a few gradient updates\(Finnet al\.,[2017](https://arxiv.org/html/2607.18615#bib.bib50); Nicholet al\.,[2018](https://arxiv.org/html/2607.18615#bib.bib51)\), an idea that has recently entered unlearning through learning\-to\-unlearn and meta\-unlearning formulations that prepare the pre\-update model for more effective or relearning\-robust forgetting\(Huanget al\.,[2024](https://arxiv.org/html/2607.18615#bib.bib52); Gaoet al\.,[2025](https://arxiv.org/html/2607.18615#bib.bib53)\)\. Related settings such as unlearning personal data from a single image further underscore the need for models prepared for post\-hoc forgetting under restricted forget\-data access\(De Minet al\.,[2024](https://arxiv.org/html/2607.18615#bib.bib54)\)\. Our method brings these ideas into the LM\-to\-VLM setting: the inner loop performs text unlearning on the LM, while the outer loop evaluates multimodal forgetting and retention after the updated LM is plugged into a frozen VLM, learning an unlearning\-ready initialization for transferable multimodal forgetting\.

## 3Problem Setup

Model notation\.LetfθTf\_\{\\theta\_\{T\}\}denote the language backbone of a VLM, whereθT\\theta\_\{T\}denotes the trainable language\-backbone parameters or trainable adapters\. Letϕ\\phidenote the remaining frozen VLM components, including the visual encoder, projection module, and multimodal input interface\. For a multimodal inputxmm=\(v,t\)x^\{\\mathrm\{mm\}\}=\(v,t\)with imagevvand text prompttt, the composed VLM is written asFθT,ϕ​\(xmm\)=FθT,ϕ​\(v,t\)F\_\{\\theta\_\{T\},\\phi\}\(x^\{\\mathrm\{mm\}\}\)=F\_\{\\theta\_\{T\},\\phi\}\(v,t\)\. For a text\-only prompttt, we writefθT​\(t\)f\_\{\\theta\_\{T\}\}\(t\)for the standalone language\-backbone behavior\. If the same prompt is evaluated through the VLM interface without an image, we writeFθT,ϕ​\(∅,t\)F\_\{\\theta\_\{T\},\\phi\}\(\\varnothing,t\)\. This notation separates the updated language component,θT\\theta\_\{T\}, from the frozen VLM components used for multimodal recomposition,ϕ\\phi\.

Unlearning task\.An unlearning request specifies a targetzzto remove while preserving non\-target behavior\. We write the corresponding task asτz=\(Dftext,z,Drtext,Dfmm,z,Drmm\)\\tau\_\{z\}=\(D\_\{f\}^\{\\mathrm\{text\},z\},D\_\{r\}^\{\\mathrm\{text\}\},D\_\{f\}^\{\\mathrm\{mm\},z\},D\_\{r\}^\{\\mathrm\{mm\}\}\), whereDfD\_\{f\}contains forget examples associated with targetzz,DrD\_\{r\}contains retain examples, and the superscripts indicate text\-only or multimodal data\. When the target is clear, we omitzzfor simplicity\. The forget sets are used to optimize or evaluate target removal, while the retain sets are used to optimize or evaluate preservation of non\-target utility\.

We distinguish the support data used for unlearning from the held\-out data used for evaluation\. Text\-side unlearning uses support sets such asDf,suptext,zD\_\{f,\\mathrm\{sup\}\}^\{\\mathrm\{text\},z\}and optionallyDr,suptextD\_\{r,\\mathrm\{sup\}\}^\{\\mathrm\{text\}\}, while VLM\-side unlearning usesDf,supmm,zD\_\{f,\\mathrm\{sup\}\}^\{\\mathrm\{mm\},z\}and optionallyDr,supmmD\_\{r,\\mathrm\{sup\}\}^\{\\mathrm\{mm\}\}\. Evaluation is performed on disjoint held\-out setsDevaltext=\(Df,testtext,z,Dr,testtext,Dg,testtext\)D\_\{\\mathrm\{eval\}\}^\{\\mathrm\{text\}\}=\(D\_\{f,\\mathrm\{test\}\}^\{\\mathrm\{text\},z\},D\_\{r,\\mathrm\{test\}\}^\{\\mathrm\{text\}\},D\_\{g,\\mathrm\{test\}\}^\{\\mathrm\{text\}\}\)andDevalmm=\(Df,testmm,z,Dr,testmm,Dg,testmm\)D\_\{\\mathrm\{eval\}\}^\{\\mathrm\{mm\}\}=\(D\_\{f,\\mathrm\{test\}\}^\{\\mathrm\{mm\},z\},D\_\{r,\\mathrm\{test\}\}^\{\\mathrm\{mm\}\},D\_\{g,\\mathrm\{test\}\}^\{\\mathrm\{mm\}\}\), whereDfD\_\{f\}measures target removal,DrD\_\{r\}measures retain utility, andDgD\_\{g\}denotes the general held\-out test split\.

Text\-side unlearning\.Given an unlearning methodm∈ℳm\\in\\mathcal\{M\}, text\-side unlearning updates only the language backbone\. Starting fromθT,0=θT\\theta\_\{T,0\}=\\theta\_\{T\}, it appliesKKupdatesθT,k=Um,text​\(θT,k−1;Df,suptext,z,Dr,suptext\)\\theta\_\{T,k\}=U\_\{m,\\mathrm\{text\}\}\(\\theta\_\{T,k\-1\};D\_\{f,\\mathrm\{sup\}\}^\{\\mathrm\{text\},z\},D\_\{r,\\mathrm\{sup\}\}^\{\\mathrm\{text\}\}\)fork=1,…,Kk=1,\\ldots,K, producingθT,K=Um,textK​\(θT;Df,suptext,z,Dr,suptext\)\\theta\_\{T,K\}=U\_\{m,\\mathrm\{text\}\}^\{K\}\(\\theta\_\{T\};D\_\{f,\\mathrm\{sup\}\}^\{\\mathrm\{text\},z\},D\_\{r,\\mathrm\{sup\}\}^\{\\mathrm\{text\}\}\)\. Some unlearning methods use only forget examples; in that case,Dr,suptextD\_\{r,\\mathrm\{sup\}\}^\{\\mathrm\{text\}\}is omitted\. The updated backbone can be evaluated alone asfθT,Kf\_\{\\theta\_\{T,K\}\}or reinserted into the frozen VLM and evaluated asFθT,K,ϕF\_\{\\theta\_\{T,K\},\\phi\}\. This procedure is computationally simple because the unlearning updates do not require multimodal forward or backward passes, but the objective itself is not optimized through the composed VLM and therefore may not guarantee forgetting under image\-conditioned queries\.

VLM\-side unlearning\.VLM\-side unlearning computes the unlearning loss through the composed VLM\. In our setting, the non\-language VLM componentsϕ\\phiremain fixed, and only the language\-backbone parametersθT\\theta\_\{T\}are updated\. Starting fromθT,0vlm=θT\\theta\_\{T,0\}^\{\\mathrm\{vlm\}\}=\\theta\_\{T\}, VLM\-side unlearning appliesθT,kvlm=Um,vlm​\(θT,k−1vlm;ϕ,Df,supmm,z,Dr,supmm\)\\theta\_\{T,k\}^\{\\mathrm\{vlm\}\}=U\_\{m,\\mathrm\{vlm\}\}\(\\theta\_\{T,k\-1\}^\{\\mathrm\{vlm\}\};\\phi,D\_\{f,\\mathrm\{sup\}\}^\{\\mathrm\{mm\},z\},D\_\{r,\\mathrm\{sup\}\}^\{\\mathrm\{mm\}\}\)fork=1,…,Kk=1,\\ldots,K, yieldingθT,Kvlm=Um,vlmK​\(θT;ϕ,Df,supmm,z,Dr,supmm\)\\theta\_\{T,K\}^\{\\mathrm\{vlm\}\}=U\_\{m,\\mathrm\{vlm\}\}^\{K\}\(\\theta\_\{T\};\\phi,D\_\{f,\\mathrm\{sup\}\}^\{\\mathrm\{mm\},z\},D\_\{r,\\mathrm\{sup\}\}^\{\\mathrm\{mm\}\}\)\. The resulting model isFθT,Kvlm,ϕF\_\{\\theta\_\{T,K\}^\{\\mathrm\{vlm\}\},\\phi\}\. The difference from text\-side unlearning is not thatϕ\\phiis updated, but that the unlearning objective is evaluated throughFθT,ϕF\_\{\\theta\_\{T\},\\phi\}on multimodal inputs, which requires VLM\-level computation at each unlearning step\.

Evaluation settings\.After any unlearning method produces an updated backboneθ¯T\\bar\{\\theta\}\_\{T\}, we evaluate it in two settings\.Text→\\rightarrowTextevaluates the standalone backbonefθ¯Tf\_\{\\bar\{\\theta\}\_\{T\}\}onDevaltextD\_\{\\mathrm\{eval\}\}^\{\\mathrm\{text\}\}, whereDf,testtext,zD\_\{f,\\mathrm\{test\}\}^\{\\mathrm\{text\},z\}measures whether the target is removed under language\-only prompts,Dr,testtextD\_\{r,\\mathrm\{test\}\}^\{\\mathrm\{text\}\}measures retained language utility, andDg,testtextD\_\{g,\\mathrm\{test\}\}^\{\\mathrm\{text\}\}measures general text\-side performance\.Text→\\rightarrowMultievaluates the recomposed VLMFθ¯T,ϕF\_\{\\bar\{\\theta\}\_\{T\},\\phi\}onDevalmmD\_\{\\mathrm\{eval\}\}^\{\\mathrm\{mm\}\}, whereDf,testmm,zD\_\{f,\\mathrm\{test\}\}^\{\\mathrm\{mm\},z\}measures whether the target remains accessible when image information is provided,Dr,testmmD\_\{r,\\mathrm\{test\}\}^\{\\mathrm\{mm\}\}measures retained vision\-language utility, andDg,testmmD\_\{g,\\mathrm\{test\}\}^\{\\mathrm\{mm\}\}measures general multimodal performance\. Lower Forget accuracy indicates stronger removal, while higher Retain and Test accuracy indicates better utility preservation\. This distinction motivates the SMU design in Fig\.[2](https://arxiv.org/html/2607.18615#S4.F2): SMU performs the unlearning update on the language backbone, then recomposes the updated backbone with the frozen VLM components and uses VLM\-level forget\-retain feedback to meta\-update the initialization\.

## 4Stochastic Meta\-Unlearning

We proposeStochastic Meta\-Unlearning\(SMU\), a bilevel framework that aligns language\-backbone unlearning with VLM\-level forgetting\(Figure[2](https://arxiv.org/html/2607.18615#S4.F2)\. The inner loop performs the actual unlearning operation on the language backbone using only text\-side data\. The outer loop then recomposes the updated backbone with the frozen VLM components and evaluates the resulting VLM on multimodal forget and retain queries\. The outer loss is backpropagated through the inner unlearning steps to update the pre\-unlearning backbone initialization\. Throughout this section,θT\\theta\_\{T\}denotes the trainable parameters of the language backbone, or the trainable adapters attached to it\. We useϕ\\phito denote the frozen non\-language VLM components, including the visual encoder, projection module, and multimodal input interface\. The standalone language backbone is written asfθTf\_\{\\theta\_\{T\}\}, and the recomposed VLM is written asFθT,ϕF\_\{\\theta\_\{T\},\\phi\}\. During inner\-loop unlearning,θT,k\\theta\_\{T,k\}denotes the backbone parameters after thekk\-th unlearning step, andθT⋆\\theta\_\{T\}^\{\\star\}denotes the meta\-learned initialization returned by SMU\.

### 4\.1Meta\-Unlearning Setup

![Refer to caption](https://arxiv.org/html/2607.18615v1/x2.png)Figure 2:SMU aligns text\-side unlearning with VLM\-level forgetting\.The inner loop updates only the language backbone through text\-side unlearning, while the outer loop recomposes the updated backbone with frozen visual components and uses VLM\-level feedback to meta\-update the initialization\.Each meta\-training episode samples an environmente=\(z,m,c\)∼p​\(ℰ\)e=\(z,m,c\)\\sim p\(\\mathcal\{E\}\), wherezzis the forgetting target,mmis the inner\-loop unlearning operator, andccdenotes the query condition\. For each environment, SMU separates the data into an inner support batch and outer query batches\. The inner support batch contains text\-side examples for applying the unlearning update:Bintext,z=\(Bf,intext,z,Br,intext\)B\_\{\\mathrm\{in\}\}^\{\\mathrm\{text\},z\}=\(B\_\{f,\\mathrm\{in\}\}^\{\\mathrm\{text\},z\},B\_\{r,\\mathrm\{in\}\}^\{\\mathrm\{text\}\}\), where the retain batch is optional and is omitted for unlearning operators that only use forget examples\. The outer query batches are used only for meta\-optimization and evaluation after recomposition\. They include a multimodal forget batchBfmm,z,cB\_\{f\}^\{\\mathrm\{mm\},z,c\}, a multimodal retain batchBrmm,cB\_\{r\}^\{\\mathrm\{mm\},c\}, and optionally text query batchesBf,outtext,zB\_\{f,\\mathrm\{out\}\}^\{\\mathrm\{text\},z\}andBr,outtextB\_\{r,\\mathrm\{out\}\}^\{\\mathrm\{text\}\}\. The goal is to learn an initializationθT\\theta\_\{T\}such that a small number of text\-side unlearning steps produces an updated backbone that forgets the target after VLM recomposition while preserving non\-target utility\.

### 4\.2Meta\-Unlearning

We first describe the base meta\-unlearning objective for a fixed environmente=\(z,m,c\)e=\(z,m,c\), wherezzis the forgetting target,mmis the language\-backbone unlearning operator, andccdenotes the query condition used for outer\-loop evaluation\.

Inner update\.Given an environmente=\(z,m,c\)e=\(z,m,c\), the inner loop starts fromθT,0=θT\\theta\_\{T,0\}=\\theta\_\{T\}and appliesKKlanguage\-backbone unlearning steps:

θT,k\+1=θT,k−α​∇θT,kℒinm​\(fθT,k;Bf,intext,z,Br,intext\),k=0,…,K−1\.\\theta\_\{T,k\+1\}=\\theta\_\{T,k\}\-\\alpha\\nabla\_\{\\theta\_\{T,k\}\}\\mathcal\{L\}^\{m\}\_\{\\mathrm\{in\}\}\\left\(f\_\{\\theta\_\{T,k\}\};B\_\{f,\\mathrm\{in\}\}^\{\\mathrm\{text\},z\},B\_\{r,\\mathrm\{in\}\}^\{\\mathrm\{text\}\}\\right\),\\quad k=0,\\ldots,K\-1\.\(1\)Here,α\\alphais the inner\-loop learning rate, andℒinm\\mathcal\{L\}^\{m\}\_\{\\mathrm\{in\}\}is the text\-side unlearning loss induced by operatormm\. Some operators use both forget and retain examples, while others use only the forget batch; in the latter case,Br,intextB\_\{r,\\mathrm\{in\}\}^\{\\mathrm\{text\}\}is omitted\. We absorb the sign convention for forgetting intoℒinm\\mathcal\{L\}^\{m\}\_\{\\mathrm\{in\}\}, so every inner step is written as gradient descent\. We denote the result of theKKinner steps by

θT,K=Um,textK​\(θT;Bf,intext,z,Br,intext\)\.\\theta\_\{T,K\}=U\_\{m,\\mathrm\{text\}\}^\{K\}\\left\(\\theta\_\{T\};B\_\{f,\\mathrm\{in\}\}^\{\\mathrm\{text\},z\},B\_\{r,\\mathrm\{in\}\}^\{\\mathrm\{text\}\}\\right\)\.The inner update uses only text\-side data and updates only the trainable language\-backbone parameters\. It does not require image inputs or backpropagation through the visual encoder or projection module\.

Outer feedback\.After the inner update, SMU recomposes the updated backbone with the frozen VLM components and evaluatesFθT,K,ϕF\_\{\\theta\_\{T,K\},\\phi\}\. For each environmentee, the outer objective is

ℒouter​\(θT;e\)=\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{outer\}\}\(\\theta\_\{T\};e\)=λfmm​ℒfmm​\(FθT,K,ϕ;Bfmm,z,c\)\+λrmm​ℒrmm​\(FθT,K,ϕ;Brmm,c\)\\displaystyle\\lambda\_\{f\}^\{\\mathrm\{mm\}\}\\mathcal\{L\}\_\{f\}^\{\\mathrm\{mm\}\}\\left\(F\_\{\\theta\_\{T,K\},\\phi\};B\_\{f\}^\{\\mathrm\{mm\},z,c\}\\right\)\+\\lambda\_\{r\}^\{\\mathrm\{mm\}\}\\mathcal\{L\}\_\{r\}^\{\\mathrm\{mm\}\}\\left\(F\_\{\\theta\_\{T,K\},\\phi\};B\_\{r\}^\{\\mathrm\{mm\},c\}\\right\)\(2\)\+λftext​ℒftext​\(fθT,K;Bf,outtext,z\)\+λrtext​ℒrtext​\(fθT,K;Br,outtext\)\.\\displaystyle\+\\lambda\_\{f\}^\{\\mathrm\{text\}\}\\mathcal\{L\}\_\{f\}^\{\\mathrm\{text\}\}\\left\(f\_\{\\theta\_\{T,K\}\};B\_\{f,\\mathrm\{out\}\}^\{\\mathrm\{text\},z\}\\right\)\+\\lambda\_\{r\}^\{\\mathrm\{text\}\}\\mathcal\{L\}\_\{r\}^\{\\mathrm\{text\}\}\\left\(f\_\{\\theta\_\{T,K\}\};B\_\{r,\\mathrm\{out\}\}^\{\\mathrm\{text\}\}\\right\)\.The forget losses encourage removal of the target behavior, and the retain losses preserve non\-target utility\. If text\-side outer supervision is not used, we setλftext=λrtext=0\\lambda\_\{f\}^\{\\mathrm\{text\}\}=\\lambda\_\{r\}^\{\\mathrm\{text\}\}=0\. The multimodal terms are the key part of the meta\-unlearning objective: although the inner update is performed on the language backbone, the outer loss evaluates the updated backbone inside the recomposed VLM\.

Meta\-update\.The meta\-update is

θT←θT−β​dd​θT​ℒouter​\(θT;e\),\\theta\_\{T\}\\leftarrow\\theta\_\{T\}\-\\beta\\frac\{d\}\{d\\theta\_\{T\}\}\\mathcal\{L\}\_\{\\mathrm\{outer\}\}\(\\theta\_\{T\};e\),\(3\)whereβ\\betais the meta\-learning rate\. This derivative is a meta\-gradient becauseθT,K\\theta\_\{T,K\}depends onθT\\theta\_\{T\}through theKKinner unlearning steps\. During this update, gradients may flow through the frozen VLM computation graph to updateθT\\theta\_\{T\}, butϕ\\phiitself is never updated\. In this way, meta\-unlearning learns an initialization whose backbone\-side unlearning steps are judged by VLM\-level feedback\.

### 4\.3Meta\-Test Unlearning

After meta\-training, SMU returns an unlearning\-ready initializationθT⋆\\theta\_\{T\}^\{\\star\}\. At meta\-test time, given a new deletion requestztestz\_\{\\mathrm\{test\}\}, a text\-side support forget setDf,suptext,ztestD\_\{f,\\mathrm\{sup\}\}^\{\\mathrm\{text\},z\_\{\\mathrm\{test\}\}\}, an optional retain support setDr,suptextD\_\{r,\\mathrm\{sup\}\}^\{\\mathrm\{text\}\}, and a meta\-test unlearning operatormtestm\_\{\\mathrm\{test\}\}, we perform only language\-backbone unlearning:

θT,Ktesttest=Umtest,textKtest​\(θT⋆;Df,suptext,ztest,Dr,suptext\)\.\\theta\_\{T,K\_\{\\mathrm\{test\}\}\}^\{\\mathrm\{test\}\}=U\_\{m\_\{\\mathrm\{test\}\},\\mathrm\{text\}\}^\{K\_\{\\mathrm\{test\}\}\}\\left\(\\theta\_\{T\}^\{\\star\};D\_\{f,\\mathrm\{sup\}\}^\{\\mathrm\{text\},z\_\{\\mathrm\{test\}\}\},D\_\{r,\\mathrm\{sup\}\}^\{\\mathrm\{text\}\}\\right\)\.The final deployed model is the recomposed VLMFθT,Ktesttest,ϕF\_\{\\theta\_\{T,K\_\{\\mathrm\{test\}\}\}^\{\\mathrm\{test\}\},\\phi\}\. We then evaluate this model on held\-out text\-only and multimodal test sets\. Here, meta\-test refers to the unlearning stage after meta\-training, where the deletion target or the unlearning operator may differ from those used during meta\-training\. Algorithm[2](https://arxiv.org/html/2607.18615#alg2)in Appendix[A](https://arxiv.org/html/2607.18615#A1)summarizes this procedure\.

### 4\.4Stochastic Meta\-Unlearning

![Refer to caption](https://arxiv.org/html/2607.18615v1/x3.png)Figure 3:Cluster\-based splits produce coherent forget targets\.We visualize PCA\-projected training examples for Covid Memes and Hateful Memes; colors denote clusters, from which target clusters are selected as forget sets\.The base meta\-unlearning objective can overfit to a fixed deletion setting\. For example, it may learn an initialization that works for one forgetting target, one unlearning operator, or one query condition, but does not transfer when any of them changes\. This is undesirable for VLM unlearning, because real deletion requests may involve new targets, and the unlearning operator used after initialization may also differ from the one used during meta\-training\.

To improve generalization, SMU treats the environmente=\(z,m,c\)e=\(z,m,c\)as a random variable sampled from an environment distributionp​\(ℰ\)p\(\\mathcal\{E\}\)\. The targetzzcontrols what should be forgotten, the operatormmcontrols how the language backbone is updated in the inner loop, and the conditioncccontrols how the recomposed VLM is queried in the outer loop\. SMU therefore minimizes the expected outer lossminθT⁡𝔼e∼p​\(ℰ\)​\[ℒouter​\(θT;e\)\]\\min\_\{\\theta\_\{T\}\}\\mathbb\{E\}\_\{e\\sim p\(\\mathcal\{E\}\)\}\[\\mathcal\{L\}\_\{\\mathrm\{outer\}\}\(\\theta\_\{T\};e\)\]\. In practice, each meta\-update samplesSSenvironments\{es\}s=1S\\\{e\_\{s\}\\\}\_\{s=1\}^\{S\}and optimizes the empirical meta\-lossℒSMU\(S\)​\(θT\)=1S​∑s=1Sℒouter​\(θT;es\)\\mathcal\{L\}\_\{\\mathrm\{SMU\}\}^\{\(S\)\}\(\\theta\_\{T\}\)=\\frac\{1\}\{S\}\\sum\_\{s=1\}^\{S\}\\mathcal\{L\}\_\{\\mathrm\{outer\}\}\(\\theta\_\{T\};e\_\{s\}\), wherees∼p​\(ℰ\)e\_\{s\}\\sim p\(\\mathcal\{E\}\)\. The meta\-update isθT←θT−β​dd​θT​ℒSMU\(S\)​\(θT\)\\theta\_\{T\}\\leftarrow\\theta\_\{T\}\-\\beta\\frac\{d\}\{d\\theta\_\{T\}\}\\mathcal\{L\}\_\{\\mathrm\{SMU\}\}^\{\(S\)\}\(\\theta\_\{T\}\)\. This stochastic design is what enables SMU to target transferability\. Sampling differentzzencourages transfer to new forgetting targets; sampling differentmmencourages robustness to different unlearning operators; and sampling differentccencourages robustness across query conditions\. Deterministic MU is a special case: whenp​\(ℰ\)p\(\\mathcal\{E\}\)places all its mass on a single fixed environment, SMU reduces to the fixed\-environment meta\-unlearning objective in Sec\.[4\.2](https://arxiv.org/html/2607.18615#S4.SS2)\. Algorithm[1](https://arxiv.org/html/2607.18615#alg1)in Appendix[A](https://arxiv.org/html/2607.18615#A1)summarizes the full stochastic meta\-training procedure\.

## 5Experiments

In this section, we demonstrate the advantages of SMU from three perspectives: \(1\) whether SMU improves the forget\-retain trade\-off over direct VLM unlearning baselines; \(2\) whether SMU can generalize when the forget target changes at meta\-test time; and \(3\) whether SMU remains robust when the meta\-test unlearning operator changes under a fixed forget target\. Together, these experiments test whether SMU provides a more effective and robust unlearning strategy for modular VLMs\. We include additional image\-side unlearning results in Appendix[B](https://arxiv.org/html/2607.18615#A2)\.

Models, Datasets and Baselines\.We evaluate our method using Qwen2\.5\-VL\-7B\-Instruct \(Qwen25\) and InternVL3\-8B\-hf \(InternVL3\) on two multimodal meme benchmarks: Hateful Memes\(Hateful\)Kielaet al\.\([2020](https://arxiv.org/html/2607.18615#bib.bib6)\)and COVID Memes\(COVID\)Cuoet al\.\([2022](https://arxiv.org/html/2607.18615#bib.bib7)\)\. Each example contains both textual and visual information, which allows us to test whether unlearning performed only on text transfers to multimodal evaluation after the language backbone is inserted back into the VLM\. We compare against standard unlearning objectives, including GradDiffYaoet al\.\([2023](https://arxiv.org/html/2607.18615#bib.bib4)\),NPOZhanget al\.\([2024a](https://arxiv.org/html/2607.18615#bib.bib30)\), and RMULiet al\.\([2024b](https://arxiv.org/html/2607.18615#bib.bib5)\)\.

![Refer to caption](https://arxiv.org/html/2607.18615v1/x4.png)Figure 4:SMU transfers to new forgetting targets\.When the meta\-test forget target differs from the meta\-training target, SMU usually preserves low Forget accuracy while maintaining Retain and Test utility across models and datasets\.Forget and retain split construction\.For each dataset, we construct the forget set using a cluster\-based procedure\. We first represent the training examples in a feature space and apply PCA to the training representations\. We then cluster the PCA\-projected training examples and select the target cluster as the forget set𝒟f\\mathcal\{D\}\_\{f\}\. All remaining training examples are used as the retain set𝒟r\\mathcal\{D\}\_\{r\}\. This construction gives a semantically coherent forget set while keeping the retain set disjoint from the examples targeted for removal\.

Training and unlearning protocol\.All methods start from the same fine\-tuned model\. Specifically, we first fine\-tune the model on the full training set𝒟f∪𝒟r\\mathcal\{D\}\_\{f\}\\cup\\mathcal\{D\}\_\{r\}\. Unlearning is then applied to this fine\-tuned checkpoint\. For our method, the inner unlearning update is performed only on text\-side data from the language backbone\. After unlearning, the updated backbone is plugged back into the frozen VLM, and we evaluate both text\-only and multimodal behavior\.

### 5\.1Main Results

RQ1: Does SMU outperform direct VLM\-side unlearning?

Table 1:Unlearning performance on Hateful Memes and Covid Memes with VLM baselines\. Results are accuracy \(%\)\.Text→\\rightarrowText evaluates text\-only forgetting, and Text→\\rightarrowMulti evaluates forgetting after VLM recomposition\. Lower Forget is better \(↓\\downarrow\), and higher Retain/Test is better \(↑\\uparrow\)\. SMU achieves the strongest overall forget\-retain trade\-off across models and datasets\.We first ask whether directly optimizing the VLM is the best strategy for unlearning\. Direct VLM\-side unlearning is a natural baseline because its loss is computed on the composed multimodal model\. However, it does not separate the component being updated from the system being evaluated, so it can remove the target behavior while also damaging retain and test utility\.

![Refer to caption](https://arxiv.org/html/2607.18615v1/x5.png)Figure 5:SMU transfers across meta\-test unlearning operators\.After meta\-training with GradDiff, SMU remains effective when the meta\-test operator is replaced by RMU or NPO, maintaining low Forget accuracy and high Retain/Test accuracy\.SMU takes a different approach: it performs the inner unlearning update on the language backbone while using the recomposed VLM for outer\-loop forget\-retain feedback\. We compare SMU against three direct VLM\-side baselines \(GradDiff, NPO, RMU\) on Hateful Memes and Covid Memes with two backbones \(Qwen, InternVL\), reporting both text\-only \(Text→\\rightarrowText\) and multimodal \(Text→\\rightarrowMulti\) evaluation\. Lower Forget accuracy indicates stronger unlearning; higher Retain and Test accuracy indicates better utility preservation\.

Table[1](https://arxiv.org/html/2607.18615#S5.T1)shows that SMU achieves the best overall forget\-retain trade\-off\. Averaged over all datasets, models, and evaluation settings, SMU obtains the lowest Forget accuracy \(9\.16%9\.16\\%\) while maintaining the highest Retain and Test accuracy \(75\.23%75\.23\\%and68\.45%68\.45\\%\)\. The direct baselines are less balanced\. GradDiff sometimes reaches lower Forget accuracy, e\.g\.,2\.02%2\.02\\%and0\.87%0\.87\\%on Hateful Memes with Qwen, but its Retain/Test accuracy collapses to32\.09%/41\.20%32\.09\\%/41\.20\\%and31\.16%/40\.20%31\.16\\%/40\.20\\%; on Covid Memes with Qwen under Text→\\rightarrowMulti, GradDiff drives all three metrics to0\.00%0\.00\\%\. NPO and RMU preserve utility better than GradDiff in some settings, but leave substantially higher Forget accuracy: on Hateful Memes with InternVL under Text→\\rightarrowMulti, both reach73\.30%73\.30\\%and77\.06%77\.06\\%Forget, while SMU reduces it to18\.20%18\.20\\%with the highest Retain/Test accuracy of83\.40%/74\.10%83\.40\\%/74\.10\\%\.

These results support the bilevel design of SMU\. VLM\-level feedback during meta\-training aligns the backbone\-side unlearning trajectory with final multimodal behavior, enabling SMU to remove target content more consistently while preserving non\-target utility across both datasets and backbones\.

RQ2: Does SMU transfer to new forgetting targets?We next study whether SMU can handle a new deletion request at meta\-test time\. This setting tests target transferability: the forgetting target used during meta\-training is different from the target used during meta\-test\. A method that only works for the meta\-training target may overfit to a fixed deletion request, rather than learning a reusable pre\-unlearning initialization\.

![Refer to caption](https://arxiv.org/html/2607.18615v1/x6.png)Figure 6:Operator transfer improves with meta\-training\.Evaluating SMU at different checkpoints shows that Forget accuracy is non\-monotonic, while later checkpoints achieve stronger forgetting without large Retain/Test degradation\.To evaluate this, we meta\-train SMU on one target and then change the forget target at meta\-test time\. Starting from the learned initialization, we perform the same text\-side unlearning procedure on the new target, insert the updated backbone into the VLM, and evaluate both text\-only and multimodal performance\. Lower forget accuracy indicates stronger target removal, while higher retain and test accuracy indicates better utility preservation\. Fig\.[4](https://arxiv.org/html/2607.18615#S5.F4)shows that SMU generally transfers to new forgetting targets\. Across most model\-dataset settings, the meta\-test target achieves low forget accuracy while retaining useful text and multimodal performance\. For example, on InternVL, SMU keeps forget accuracy low on both Hateful Memes and Covid Memes after the target changes, while retain and test accuracy remain stable\. On Qwen, SMU also transfers well on Hateful Memes\. The main harder case is Covid under multimodal evaluation, where forget accuracy increases after target transfer, suggesting that some target shifts are more difficult when image\-conditioned evidence is involved\. Overall, the results show that SMU is not only optimized for one fixed forgetting target\. It learns an initialization that can be adapted to new deletion requests through text\-side unlearning\.

RQ3: Does SMU transfer across unlearning operators under a fixed target?We test whether the learned initialization is tied to the inner\-loop operator used during meta\-training\. SMU is meta\-trained with GradDiff as the inner\-loop operator; at meta\-test time, we replace it with RMU or NPO while keeping the forget target fixed, then evaluate both text\-only and multimodal performance\. Fig\.[5](https://arxiv.org/html/2607.18615#S5.F5)shows that SMU remains effective under operator substitution\. On both Hateful Memes and Covid Memes, switching to RMU or NPO at meta\-test time still yields low Forget Text/MM while Retain and Test accuracy stay high\. The learned initialization therefore acts as a transferable starting point rather than one fitted to a specific inner\-loop update rule\. Fig\.[6](https://arxiv.org/html/2607.18615#S5.F6)shows that this transferability emerges gradually\. Forget accuracy is non\-monotonic during meta\-training, first rising then falling, while retain and test accuracy remain stable throughout\. Reliable operator transfer thus requires sufficient meta\-training; it is not a property of the base model initialization\.

## 6Conclusion

We identify a component\-system mismatch in VLM unlearning: the update is applied to the language backbone, while forgetting must hold after the backbone is recomposed with the visual components\. To address this mismatch, we proposeStochastic Meta\-Unlearning\(SMU\), a bilevel framework that performs language\-backbone unlearning in the inner loop and uses VLM\-level forget\-retain feedback in the outer loop to learn an unlearning\-ready initialization\. Experiments show that SMU improves the forget\-retain trade\-off over direct VLM\-side unlearning baselines, transfers to new forgetting targets, and remains effective when the meta\-test unlearning operator changes\.

## References

- \[1\]J\. Alayrac, J\. Donahue, P\. Luc, A\. Miech, I\. Barr, Y\. Hasson, K\. Lenc, A\. Mensch, K\. Millican, M\. Reynolds,et al\.\(2022\)Flamingo: a visual language model for few\-shot learning\.Advances in neural information processing systems35,pp\. 23716–23736\.Cited by:[§1](https://arxiv.org/html/2607.18615#S1.p1.1)\.
- \[2\]\(2025\)Soft prompting for unlearning in large language models\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 4046–4056\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1)\.
- \[3\]L\. Bourtoule, V\. Chandrasekaran, C\. A\. Choquette\-Choo, H\. Jia, A\. Travers, B\. Zhang, D\. Lie, and N\. Papernot\(2021\)Machine unlearning\.In2021 IEEE symposium on security and privacy \(SP\),pp\. 141–159\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1)\.
- \[4\]P\. Cao, C\. Wang, Z\. He, H\. Yuan, J\. Li, Y\. Chen, K\. Liu, J\. Zhao,et al\.\(2024\)Rwku: benchmarking real\-world knowledge unlearning for large language models\.Advances in Neural Information Processing Systems37,pp\. 98213–98263\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1)\.
- \[5\]Y\. Cao and J\. Yang\(2015\)Towards making systems forget with machine unlearning\.In2015 IEEE symposium on security and privacy,pp\. 463–480\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1)\.
- \[6\]J\. Chen, Z\. Deng, K\. Zheng, Y\. Yan, S\. Liu, P\. Wu, P\. Jiang, J\. Liu, and X\. Hu\(2025\)Safeeraser: enhancing safety in multimodal large language models through multimodal machine unlearning\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 14194–14224\.Cited by:[§1](https://arxiv.org/html/2607.18615#S1.p2.1),[§2](https://arxiv.org/html/2607.18615#S2.p3.1)\.
- \[7\]K\. Cuo, W\. Zhao, V\. Vishwamitra, Z\. Zhao, H\. Hu,et al\.\(2022\)Understanding the generalizability of hateful memes detection models against covid\-19\-related hateful memes\.InInternational Conference on Machine Learning and Applications,Cited by:[§5](https://arxiv.org/html/2607.18615#S5.p2.1)\.
- \[8\]T\. De Min, M\. Mancini, S\. Lathuilière, S\. Roy, and E\. Ricci\(2024\)Unlearning personal data from a single image\.arXiv preprint arXiv:2407\.12069\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p4.1)\.
- \[9\]A\. Dontsov, D\. Korzh, A\. Zhavoronkin, B\. Mikheev, D\. Bobkov, A\. Alanov, O\. Rogov, I\. Oseledets, and E\. Tutubalina\(2025\)Clear: character unlearning in textual and visual modalities\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 20582–20603\.Cited by:[§1](https://arxiv.org/html/2607.18615#S1.p2.1),[§2](https://arxiv.org/html/2607.18615#S2.p3.1)\.
- \[10\]J\. Doshi and A\. C\. Stickland\(2024\)Does unlearning truly unlearn? a black box evaluation of llm unlearning methods\.arXiv preprint arXiv:2411\.12103\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1)\.
- \[11\]R\. Eldan and M\. Russinovich\(2024\)Who’s harry potter? approximate unlearning in llms, 2023\.URL https://arxiv\. org/abs/2310\.022381\(2\),pp\. 8\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1)\.
- \[12\]C\. Finn, P\. Abbeel, and S\. Levine\(2017\)Model\-agnostic meta\-learning for fast adaptation of deep networks\.InInternational conference on machine learning,pp\. 1126–1135\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p4.1)\.
- \[13\]H\. Gao, T\. Pang, C\. Du, T\. Hu, Z\. Deng, and M\. Lin\(2025\)Meta\-unlearning on diffusion models: preventing relearning unlearned concepts\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 2131–2141\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p4.1)\.
- \[14\]S\. Hu, Y\. Fu, Z\. S\. Wu, and V\. Smith\(2024\)Unlearning or obfuscating? jogging the memory of unlearned llms via benign relearning\.arXiv preprint arXiv:2406\.13356\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1)\.
- \[15\]M\. H\. Huang, L\. G\. Foo, and J\. Liu\(2024\)Learning to unlearn for robust machine unlearning\.InEuropean conference on computer vision,pp\. 202–219\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p4.1)\.
- \[16\]J\. Huo, Y\. Yan, X\. Zheng, Y\. Lyu, X\. Zou, Z\. Wei, and X\. Hu\(2025\)Mmunlearner: reformulating multimodal machine unlearning in the era of multimodal large language models\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 7190–7206\.Cited by:[§1](https://arxiv.org/html/2607.18615#S1.p2.1),[§1](https://arxiv.org/html/2607.18615#S1.p3.1),[§2](https://arxiv.org/html/2607.18615#S2.p3.1)\.
- \[17\]J\. Jang, D\. Yoon, S\. Yang, S\. Cha, M\. Lee, L\. Logeswaran, and M\. Seo\(2023\)Knowledge unlearning for mitigating privacy risks in language models\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 14389–14408\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1)\.
- \[18\]J\. Ji, Y\. Liu, Y\. Zhang, G\. Liu, R\. R\. Kompella, S\. Liu, and S\. Chang\(2024\)Reversing the forget\-retain objectives: an efficient llm unlearning framework from logit difference\.Advances in Neural Information Processing Systems37,pp\. 12581–12611\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1)\.
- \[19\]J\. Jia, Y\. Zhang, Y\. Zhang, J\. Liu, B\. Runwal, J\. Diffenderfer, B\. Kailkhura, and S\. Liu\(2024\)Soul: unlocking the power of second\-order optimization for llm unlearning\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 4276–4292\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1)\.
- \[20\]Z\. Jiang, F\. F\. Xu, J\. Araki, and G\. Neubig\(2020\)How can we know what language models know?\.Transactions of the Association for Computational Linguistics8,pp\. 423–438\.Cited by:[§1](https://arxiv.org/html/2607.18615#S1.p3.1)\.
- \[21\]T\. Kawakami, K\. Egashira, A\. Miyai, G\. Irie, and K\. Aizawa\(2025\)Pulse: practical evaluation scenarios for large multimodal model unlearning\.arXiv preprint arXiv:2507\.01271\.Cited by:[§1](https://arxiv.org/html/2607.18615#S1.p2.1),[§2](https://arxiv.org/html/2607.18615#S2.p3.1)\.
- \[22\]D\. Kiela, H\. Firooz, A\. Mohan, V\. Goswami, A\. Singh, P\. Ringshia, and D\. Testuggine\(2020\)The hateful memes challenge: detecting hate speech in multimodal memes\.Advances in neural information processing systems33,pp\. 2611–2624\.Cited by:[§5](https://arxiv.org/html/2607.18615#S5.p2.1)\.
- \[23\]M\. Kurmanji, P\. Triantafillou, J\. Hayes, and E\. Triantafillou\(2023\)Towards unbounded machine unlearning\.Advances in neural information processing systems36,pp\. 1957–1987\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1)\.
- \[24\]J\. Li, Q\. Wei, C\. Zhang, G\. Qi, M\. Du, Y\. Chen, S\. Bi, and F\. Liu\(2024\)Single image unlearning: efficient machine unlearning in multimodal large language models\.Advances in Neural Information Processing Systems37,pp\. 35414–35453\.Cited by:[§1](https://arxiv.org/html/2607.18615#S1.p2.1),[§2](https://arxiv.org/html/2607.18615#S2.p3.1)\.
- \[25\]J\. Li, D\. Li, S\. Savarese, and S\. Hoi\(2023\)Blip\-2: bootstrapping language\-image pre\-training with frozen image encoders and large language models\.InInternational conference on machine learning,pp\. 19730–19742\.Cited by:[§1](https://arxiv.org/html/2607.18615#S1.p1.1)\.
- \[26\]N\. Li, A\. Pan, A\. Gopal, S\. Yue, D\. Berrios, A\. Gatti, J\. D\. Li, A\. Dombrowski, S\. Goel, G\. Mukobi, N\. Helm\-Burger, R\. Lababidi, L\. Justen, A\. B\. Liu, M\. Chen, I\. Barrass, O\. Zhang, X\. Zhu, R\. Tamirisa, B\. Bharathi, A\. Herbert\-Voss, C\. B\. Breuer, A\. Zou, M\. Mazeika, Z\. Wang, P\. Oswal, W\. Lin, A\. A\. Hunt, J\. Tienken\-Harder, K\. Y\. Shih, K\. Talley, J\. Guan, I\. Steneker, D\. Campbell, B\. Jokubaitis, S\. Basart, S\. Fitz, P\. Kumaraguru, K\. K\. Karmakar, U\. Tupakula, V\. Varadharajan, Y\. Shoshitaishvili, J\. Ba, K\. M\. Esvelt, A\. Wang, and D\. Hendrycks\(2024\)The WMDP benchmark: measuring and reducing malicious use with unlearning\.InICML,Cited by:[§5](https://arxiv.org/html/2607.18615#S5.p2.1)\.
- \[27\]Q\. Li, J\. Geng, D\. Zhu, F\. Cai, C\. Lyu, and F\. Karray\(2025\)Sauce: selective concept unlearning in vision\-language models with sparse autoencoders\.arXiv preprint arXiv:2503\.14530\.Cited by:[§1](https://arxiv.org/html/2607.18615#S1.p1.1)\.
- \[28\]C\. Y\. Liu, Y\. Wang, J\. Flanigan, and Y\. Liu\(2024\)Large language model unlearning via embedding\-corrupted prompts\.Advances in Neural Information Processing Systems37,pp\. 118198–118266\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1)\.
- \[29\]H\. Liu, C\. Li, Q\. Wu, and Y\. J\. Lee\(2023\)Visual instruction tuning\.Advances in neural information processing systems36,pp\. 34892–34916\.Cited by:[§1](https://arxiv.org/html/2607.18615#S1.p1.1)\.
- \[30\]S\. Liu, Y\. Yao, J\. Jia, S\. Casper, N\. Baracaldo, P\. Hase, Y\. Yao, C\. Y\. Liu, X\. Xu, H\. Li,et al\.\(2025\)Rethinking machine unlearning for large language models\.Nature Machine Intelligence7\(2\),pp\. 181–194\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1)\.
- \[31\]Z\. Liu, G\. Dou, M\. Jia, Z\. Tan, Q\. Zeng, Y\. Yuan, and M\. Jiang\(2025\)Protecting privacy in multimodal large language models with mllmu\-bench\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 4105–4135\.Cited by:[§1](https://arxiv.org/html/2607.18615#S1.p2.1),[§2](https://arxiv.org/html/2607.18615#S2.p3.1)\.
- \[32\]Z\. Liu, G\. Dou, X\. Yuan, C\. Zhang, Z\. Tan, and M\. Jiang\(2025\)Modality\-aware neuron pruning for unlearning in multimodal large language models\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 5913–5933\.Cited by:[§1](https://arxiv.org/html/2607.18615#S1.p2.1),[§2](https://arxiv.org/html/2607.18615#S2.p3.1)\.
- \[33\]H\. Lu, M\. Isonuma, J\. Mori, and I\. Sakata\(2024\)Towards transfer unlearning: empirical evidence of cross\-domain bias mitigation\.arXiv preprint arXiv:2407\.16951\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p4.1)\.
- \[34\]Y\. Ma, J\. Wang, F\. Wang, S\. Ma, J\. Li, J\. Pan, X\. Li, F\. Huang, L\. Sun, B\. Li,et al\.\(2024\)Benchmarking vision language model unlearning via fictitious facial identity dataset\.arXiv preprint arXiv:2411\.03554\.Cited by:[§1](https://arxiv.org/html/2607.18615#S1.p2.1),[§2](https://arxiv.org/html/2607.18615#S2.p3.1)\.
- \[35\]P\. Maini, Z\. Feng, A\. Schwarzschild, Z\. C\. Lipton, and J\. Z\. Kolter\(2024\)Tofu: a task of fictitious unlearning for llms\.arXiv preprint arXiv:2401\.06121\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1)\.
- \[36\]S\. Neel, A\. Roth, and S\. Sharifi\-Malvajerdi\(2021\)Descent\-to\-delete: gradient\-based methods for machine unlearning\.InAlgorithmic Learning Theory,pp\. 931–962\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1)\.
- \[37\]A\. Nichol, J\. Achiam, and J\. Schulman\(2018\)On first\-order meta\-learning algorithms\.arXiv preprint arXiv:1803\.02999\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p4.1)\.
- \[38\]V\. Patil, Y\. Sung, P\. Hase, J\. Peng, T\. Chen, and M\. Bansal\(2025\)Unlearning sensitive information in multimodal llms: benchmark and attack\-defense evaluation\.arXiv preprint arXiv:2505\.01456\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p3.1)\.
- \[39\]F\. Petroni, T\. Rocktäschel, S\. Riedel, P\. Lewis, A\. Bakhtin, Y\. Wu, and A\. Miller\(2019\)Language models as knowledge bases?\.InProceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing \(EMNLP\-IJCNLP\),pp\. 2463–2473\.Cited by:[§1](https://arxiv.org/html/2607.18615#S1.p3.1)\.
- \[40\]A\. Radford, J\. W\. Kim, C\. Hallacy, A\. Ramesh, G\. Goh, S\. Agarwal, G\. Sastry, A\. Askell, P\. Mishkin, J\. Clark,et al\.\(2021\)Learning transferable visual models from natural language supervision\.InInternational conference on machine learning,pp\. 8748–8763\.Cited by:[§1](https://arxiv.org/html/2607.18615#S1.p1.1)\.
- \[41\]N\. M\. Sepahvand, V\. Dumoulin, E\. Triantafillou, and G\. K\. Dziugaite\(2024\)Data selection for transfer unlearning\.arXiv preprint arXiv:2405\.10425\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p4.1)\.
- \[42\]W\. Shi, J\. Lee, Y\. Huang, S\. Malladi, J\. Zhao, A\. Holtzman, D\. Liu, L\. Zettlemoyer, N\. A\. Smith, and C\. Zhang\(2024\)Muse: machine unlearning six\-way evaluation for language models\.arXiv preprint arXiv:2407\.06460\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1)\.
- \[43\]J\. Wallat, J\. Singh, and A\. Anand\(2020\)BERTnesia: investigating the capture and forgetting of knowledge in bert\.InProceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP,pp\. 174–183\.Cited by:[§1](https://arxiv.org/html/2607.18615#S1.p3.1)\.
- \[44\]C\. Wang, Y\. Li, X\. Feng, C\. Chen, X\. Zheng, and J\. Yin\(2025\)UMU\-bench: closing the modality gap in multimodal unlearning evaluation\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track,Cited by:[§1](https://arxiv.org/html/2607.18615#S1.p3.1)\.
- \[45\]X\. Wang, X\. Yi, X\. Xie, and J\. Jia\(2025\)Specify privacy yourself: assessing inference\-time personalized privacy preservation ability of large vision\-language models\.InProceedings of the 33rd ACM International Conference on Multimedia,pp\. 12304–12313\.Cited by:[§1](https://arxiv.org/html/2607.18615#S1.p1.1)\.
- \[46\]J\. Yao, E\. Chien, M\. Du, X\. Niu, T\. Wang, Z\. Cheng, and X\. Yue\(2024\)Machine unlearning of pre\-trained large language models\.InProceedings of the 62nd annual meeting of the association for computational linguistics \(volume 1: Long papers\),pp\. 8403–8419\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1)\.
- \[47\]Y\. Yao, X\. Xu, and Y\. Liu\(2023\)Large language model unlearning\.arXiv preprint arXiv:2310\.10683\.Cited by:[§5](https://arxiv.org/html/2607.18615#S5.p2.1)\.
- \[48\]Y\. Yao and X\. Xu\(2024\)Large language model unlearning\.Advances in Neural Information Processing Systems37,pp\. 105425–105475\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1)\.
- \[49\]X\. Yuan, T\. Pang, C\. Du, K\. Chen, W\. Zhang, and M\. Lin\(2024\)A closer look at machine unlearning for large language models\.arXiv preprint arXiv:2410\.08109\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1)\.
- \[50\]R\. Zhang, L\. Lin, Y\. Bai, and S\. Mei\(2024\)Negative preference optimization: from catastrophic collapse to effective unlearning\.arXiv preprint arXiv:2404\.05868\.Cited by:[§2](https://arxiv.org/html/2607.18615#S2.p2.1),[§5](https://arxiv.org/html/2607.18615#S5.p2.1)\.
- \[51\]Y\. Zhang, J\. Wu, W\. Li, B\. Li, Z\. Ma, Z\. Liu, and C\. Li\(2024\)Llava\-video: video instruction tuning with synthetic data\.arXiv preprint arXiv:2410\.02713\.Cited by:[§1](https://arxiv.org/html/2607.18615#S1.p1.1)\.

## Appendix AAlgorithms

For completeness, Algorithms[1](https://arxiv.org/html/2607.18615#alg1)and[2](https://arxiv.org/html/2607.18615#alg2)provide pseudocode for stochastic meta\-training and meta\-test unlearning, respectively\. They are deferred from the main paper because the corresponding updates are already specified by Eqs\. \([1](https://arxiv.org/html/2607.18615#S4.E1)\)–\([3](https://arxiv.org/html/2607.18615#S4.E3)\)\.

## Appendix BAdditional Results: Image\-Side Unlearning

RQ4: Is image\-side unlearning a reliable intervention point?Since VLMs are modular, a natural alternative to language\-backbone unlearning is to apply the update to the image\-side components instead\. We test this directly and evaluate both image\-side and multimodal performance\.

Table 2:Image\-side unlearning is unstable\.Results are accuracy \(%\)\. Lower Forget is better \(↓\\downarrow\), and higher Retain/Test is better \(↑\\uparrow\)\.Table[2](https://arxiv.org/html/2607.18615#A2.T2)shows that image\-side unlearning yields an unstable forget\-retain trade\-off\. GradDiff can strongly reduce forget accuracy but severely damages retain and test utility, most clearly on Covid Memes, where multimodal retain and test accuracy collapse to0\.00%0\.00\\%\. NPO and RMU preserve more utility but consistently leave high forget accuracy, especially on Hateful Memes\. The image side is therefore not a reliable standalone intervention point: it either fails to remove the target behavior or does so by destroying general utility\. This instability supports our design choice of placing the unlearning update on the language backbone with VLM\-level feedback guiding the forget\-retain objective\.

## Appendix CImplementation Details

### C\.1Unlearning Objectives

We use three unlearning objectives as baselines and as possible inner\-loop operators in SMU: GradDiff, NPO, and RMU\. LetCEθT​\(D\)\\mathrm\{CE\}\_\{\\theta\_\{T\}\}\(D\)denote the sequence negative log\-likelihood on a batchDD, and letθTref\\theta\_\{T\}^\{\\mathrm\{ref\}\}denote a frozen reference backbone\. Unless otherwise stated, each unlearning objective has the form

ℒ=γ​ℒf\+α​ℒr,\\mathcal\{L\}=\\gamma\\mathcal\{L\}\_\{f\}\+\\alpha\\mathcal\{L\}\_\{r\},whereℒf\\mathcal\{L\}\_\{f\}is the forget loss andℒr\\mathcal\{L\}\_\{r\}is the retain loss\.

#### GradDiff\.

GradDiff performs gradient ascent on the forget set by using

ℒf=−CEθT​\(Df\)\.\\mathcal\{L\}\_\{f\}=\-\\mathrm\{CE\}\_\{\\theta\_\{T\}\}\(D\_\{f\}\)\.The retain loss is either the standard NLL lossℒr=CEθT​\(Dr\)\\mathcal\{L\}\_\{r\}=\\mathrm\{CE\}\_\{\\theta\_\{T\}\}\(D\_\{r\}\)or a KL loss against the frozen reference model\.

#### NPO\.

NPO uses a one\-sided DPO\-style loss on forget examples\. LetℓθT​\(x\)\\ell\_\{\\theta\_\{T\}\}\(x\)be the per\-example sequence NLL under the current model andℓref​\(x\)\\ell\_\{\\mathrm\{ref\}\}\(x\)be the NLL under the frozen reference model\. The forget loss is

ℒf=−2βNPO​𝔼xf​\[log⁡σ​\(βNPO​\(ℓθT​\(xf\)−ℓref​\(xf\)\)\)\]\.\\mathcal\{L\}\_\{f\}=\-\\frac\{2\}\{\\beta\_\{\\mathrm\{NPO\}\}\}\\mathbb\{E\}\_\{x\_\{f\}\}\\left\[\\log\\sigma\\left\(\\beta\_\{\\mathrm\{NPO\}\}\(\\ell\_\{\\theta\_\{T\}\}\(x\_\{f\}\)\-\\ell\_\{\\mathrm\{ref\}\}\(x\_\{f\}\)\)\\right\)\\right\]\.The retain loss follows the same NLL or KL choice as GradDiff\.

#### RMU\.

RMU matches hidden activations on forget examples to a random control vector\. LethθT​\(x\)h\_\{\\theta\_\{T\}\}\(x\)be the hidden activation from the selected transformer block, and letccbe a normalized random control vector\. For activationsa,b∈ℝB×T×da,b\\in\\mathbb\{R\}^\{B\\times T\\times d\}and token maskmm, define

𝒜​\(a,b,m\)=1B​∑i=1B1max⁡\(1,∑tmi​t\)​∑tmi​t​\(1d​∑j=1d\(ai​t​j−bi​t​j\)2\)\.\\mathcal\{A\}\(a,b,m\)=\\frac\{1\}\{B\}\\sum\_\{i=1\}^\{B\}\\frac\{1\}\{\\max\(1,\\sum\_\{t\}m\_\{it\}\)\}\\sum\_\{t\}m\_\{it\}\\left\(\\frac\{1\}\{d\}\\sum\_\{j=1\}^\{d\}\(a\_\{itj\}\-b\_\{itj\}\)^\{2\}\\right\)\.The RMU forget loss isℒf=𝒜​\(hθT​\(xf\),c,mf\)\\mathcal\{L\}\_\{f\}=\\mathcal\{A\}\(h\_\{\\theta\_\{T\}\}\(x\_\{f\}\),c,m\_\{f\}\)\. For retention, we either use the NLL/KL retain loss above or an activation matching lossℒr=𝒜​\(hθT​\(xr\),hθTref​\(xr\),mr\)\\mathcal\{L\}\_\{r\}=\\mathcal\{A\}\(h\_\{\\theta\_\{T\}\}\(x\_\{r\}\),h\_\{\\theta\_\{T\}^\{\\mathrm\{ref\}\}\}\(x\_\{r\}\),m\_\{r\}\)\.

#### SMU objective\.

In SMU, the inner loop applies one of the above objectives using text\-side support data\. AfterKKinner updates, the updated backbone is recomposed with the frozen VLM components, and the outer loss is computed on multimodal forget and retain batches:

ℒmeta=λf​ℒfmm\+λr​ℒrmm\.\\mathcal\{L\}\_\{\\mathrm\{meta\}\}=\\lambda\_\{f\}\\mathcal\{L\}\_\{f\}^\{\\mathrm\{mm\}\}\+\\lambda\_\{r\}\\mathcal\{L\}\_\{r\}^\{\\mathrm\{mm\}\}\.

### C\.2Dataset Construction

Forget and retain sets are constructed by clustering training examples\. We encode meme text usingsentence\-transformers/all\-MiniLM\-L6\-v2; if this encoder is unavailable, we use TF\-IDF features with unigrams and bigrams\. We then run KMeans withk=10k=10and random seed 42\. The forget set is the union of selected clusters, and the retain set contains the remaining clusters\. Some transfer experiments additionally exclude specific clusters from the retain set to avoid overlap with held\-out target clusters\. The same split construction is used for all methods\.

### C\.3Prompting and Evaluation

For Hateful Memes, the model predicts whether a meme is hateful or not hateful\. For Covid Memes, the model predicts one of three labels: not harmful, somewhat harmful, or very harmful\. We evaluate both text\-only and multimodal settings\.

Algorithm 1Stochastic Meta\-Unlearning1:Language\-backbone initialization

θT\\theta\_\{T\}
2:Frozen VLM components

ϕ\\phi
3:Environment distribution

p​\(ℰ\)p\(\\mathcal\{E\}\)
4:Inner\-loop steps

KK
5:Number of sampled environments

SS
6:Step sizes

α,β\\alpha,\\beta
7:Loss weights

λ\\lambda
8:whilenot convergeddo

9:Initialize meta\-loss

ℒSMU←0\\mathcal\{L\}\_\{\\mathrm\{SMU\}\}\\leftarrow 0
10:for

s=1s=1to

SSdo

11:Sample environment

es=\(zs,ms,cs\)∼p​\(ℰ\)e\_\{s\}=\(z\_\{s\},m\_\{s\},c\_\{s\}\)\\sim p\(\\mathcal\{E\}\)
12:Sample inner text batch

\(Bf,intext,zs,Br,intext\)\(B\_\{f,\\mathrm\{in\}\}^\{\\mathrm\{text\},z\_\{s\}\},B\_\{r,\\mathrm\{in\}\}^\{\\mathrm\{text\}\}\)
13:Set

θT,s,0←θT\\theta\_\{T,s,0\}\\leftarrow\\theta\_\{T\}
14:for

k=0k=0to

K−1K\-1do

15:Compute language\-backbone unlearning loss

ℒinms​\(fθT,s,k;Bf,intext,zs,Br,intext\)\\mathcal\{L\}^\{m\_\{s\}\}\_\{\\mathrm\{in\}\}\\left\(f\_\{\\theta\_\{T,s,k\}\};B\_\{f,\\mathrm\{in\}\}^\{\\mathrm\{text\},z\_\{s\}\},B\_\{r,\\mathrm\{in\}\}^\{\\mathrm\{text\}\}\\right\)
16:Update language backbone

θT,s,k\+1=θT,s,k−α​∇θT,s,kℒinms​\(fθT,s,k;Bf,intext,zs,Br,intext\)\\theta\_\{T,s,k\+1\}=\\theta\_\{T,s,k\}\-\\alpha\\nabla\_\{\\theta\_\{T,s,k\}\}\\mathcal\{L\}^\{m\_\{s\}\}\_\{\\mathrm\{in\}\}\\left\(f\_\{\\theta\_\{T,s,k\}\};B\_\{f,\\mathrm\{in\}\}^\{\\mathrm\{text\},z\_\{s\}\},B\_\{r,\\mathrm\{in\}\}^\{\\mathrm\{text\}\}\\right\)
17:endfor

18:Recompose updated backbone with frozen VLM:

FθT,s,K,ϕF\_\{\\theta\_\{T,s,K\},\\phi\}
19:Sample outer forget and retain batches under condition

csc\_\{s\}
20:Compute outer loss

ℒouter\(s\)=ℒouter​\(θT;es\)\\mathcal\{L\}\_\{\\mathrm\{outer\}\}^\{\(s\)\}=\\mathcal\{L\}\_\{\\mathrm\{outer\}\}\(\\theta\_\{T\};e\_\{s\}\)using Eq\. \([2](https://arxiv.org/html/2607.18615#S4.E2)\)

21:Accumulate

ℒSMU←ℒSMU\+1S​ℒouter\(s\)\\mathcal\{L\}\_\{\\mathrm\{SMU\}\}\\leftarrow\\mathcal\{L\}\_\{\\mathrm\{SMU\}\}\+\\frac\{1\}\{S\}\\mathcal\{L\}\_\{\\mathrm\{outer\}\}^\{\(s\)\}
22:endfor

23:Meta\-update initialization

θT←θT−β​∇θTℒSMU\\theta\_\{T\}\\leftarrow\\theta\_\{T\}\-\\beta\\nabla\_\{\\theta\_\{T\}\}\\mathcal\{L\}\_\{\\mathrm\{SMU\}\}
24:endwhile

25:returnmeta\-learned initialization

θT⋆←θT\\theta\_\{T\}^\{\\star\}\\leftarrow\\theta\_\{T\}

For text\-only evaluation, the prompt contains only the meme text\. For multimodal evaluation, the prompt includes both the image and the meme text\. We parse Hateful Memes predictions by first checking for class labels 0 or 1 and then falling back to string matching for “not hateful” and “hateful”\. For Covid Memes, we use normalized substring matching against the three class names\. Invalid predictions are treated as unparsable outputs\. The main paper reports accuracy; our evaluation code also computes macro precision, macro recall, and macro F1\.

### C\.4Training Hyperparameters

All models are fine\-tuned before unlearning\. Fine\-tuning uses LoRA with rank 8, learning rate1×10−41\\times 10^\{\-4\}, cosine scheduling, warmup ratio 0\.1, batch size 1, gradient accumulation 8, bf16 precision, cutoff length 1024, and 3 training epochs\.

For direct unlearning baselines, we use AdamW, learning rate5×10−55\\times 10^\{\-5\}, batch size 2, one epoch, gradient clipping at 1\.0, and bfloat16 model loading\. Unless otherwise specified, we setγ=1\.0\\gamma=1\.0,α=1\.0\\alpha=1\.0, and use NLL retain loss\. For NPO, we useβNPO=1\.0\\beta\_\{\\mathrm\{NPO\}\}=1\.0in the direct baseline\. For RMU, the default steering coefficient is 20\.0\.

For meta\-unlearning, we use inner\-loop step size1×10−51\\times 10^\{\-5\}, meta step size1×10−51\\times 10^\{\-5\},K=4K=4inner steps, one inner epoch, five outer epochs, and gradient clipping at 1\.0\. The default outer weights areλf=λr=1\.0\\lambda\_\{f\}=\\lambda\_\{r\}=1\.0\.

### C\.5Compute

Experiments are run on NVIDIA A100\-SXM4\-80GB GPUs\. Most training and evaluation commands use one GPU per run\. The implementation uses bf16 when supported; otherwise it falls back to fp16\. For meta\-unlearning, the optimizer is 8\-bit AdamW when available and AdamW otherwise\. We do not report a full project\-level compute estimate because total runtime depends on the selected model, dataset split, and number of evaluated checkpoints\.

## Appendix DLimitations

Algorithm 2Meta\-Test Unlearning1:Meta\-learned initialization

θT⋆\\theta\_\{T\}^\{\\star\}
2:Frozen VLM components

ϕ\\phi
3:Meta\-test deletion target

ztestz\_\{\\mathrm\{test\}\}
4:Text support forget set

Df,suptext,ztestD\_\{f,\\mathrm\{sup\}\}^\{\\mathrm\{text\},z\_\{\\mathrm\{test\}\}\}
5:Optional text retain set

Dr,suptextD\_\{r,\\mathrm\{sup\}\}^\{\\mathrm\{text\}\}
6:Meta\-test unlearning operator

mtestm\_\{\\mathrm\{test\}\}
7:Inner\-loop steps

KtestK\_\{\\mathrm\{test\}\}and step size

α\\alpha
8:Set

θT,0test←θT⋆\\theta\_\{T,0\}^\{\\mathrm\{test\}\}\\leftarrow\\theta\_\{T\}^\{\\star\}
9:for

k=0k=0to

Ktest−1K\_\{\\mathrm\{test\}\}\-1do

10:Compute language\-backbone unlearning loss

ℒkmtest=ℒinmtest​\(fθT,ktest;Df,suptext,ztest,Dr,suptext\)\\mathcal\{L\}^\{m\_\{\\mathrm\{test\}\}\}\_\{k\}=\\mathcal\{L\}^\{m\_\{\\mathrm\{test\}\}\}\_\{\\mathrm\{in\}\}\\left\(f\_\{\\theta\_\{T,k\}^\{\\mathrm\{test\}\}\};D\_\{f,\\mathrm\{sup\}\}^\{\\mathrm\{text\},z\_\{\\mathrm\{test\}\}\},D\_\{r,\\mathrm\{sup\}\}^\{\\mathrm\{text\}\}\\right\)
11:Update language backbone

θT,k\+1test=θT,ktest−α​∇θT,ktestℒkmtest\\theta\_\{T,k\+1\}^\{\\mathrm\{test\}\}=\\theta\_\{T,k\}^\{\\mathrm\{test\}\}\-\\alpha\\nabla\_\{\\theta\_\{T,k\}^\{\\mathrm\{test\}\}\}\\mathcal\{L\}^\{m\_\{\\mathrm\{test\}\}\}\_\{k\}
12:endfor

13:Recompose the unlearned backbone with frozen VLM:

FθT,Ktesttest,ϕF\_\{\\theta\_\{T,K\_\{\\mathrm\{test\}\}\}^\{\\mathrm\{test\}\},\\phi\}
14:Evaluate on held\-out text\-only and multimodal test sets\.

This work has several limitations\. First, SMU meta\-learns a backbone initialization through unrolled inner\-loop unlearning, which is more expensive than applying a single unlearning method once\. Although test\-time unlearning is cheap, meta\-training still requires additional computation\. Second, our method keeps the non\-language VLM components frozen\. This improves stability, but may limit forgetting when the target information is mainly stored in the visual encoder or projector\. Third, SMU is evaluated on a limited set of VLMs, datasets, and unlearning operators\. Broader evaluation is needed to confirm whether the same trends hold for larger models, more diverse visual domains, and more complex deletion requests\. Finally, our results focus on empirical forgetting and utility metrics\. They do not provide a formal guarantee that the target knowledge cannot be recovered by stronger adversarial prompts or unseen multimodal contexts\.

## Appendix EBroader Impact

This work aims to improve machine unlearning for vision\-language models, which can support privacy protection, deletion requests, and safer model correction\. However, unlearning methods can also be misused to hide undesirable model behavior or selectively remove accountability\-relevant information\. In addition, our method provides empirical forgetting but no formal guarantee that target knowledge cannot be recovered by stronger adversarial prompts or unseen multimodal contexts\. These risks suggest that VLM unlearning should be used with careful auditing and transparent evaluation\.

### E\.1Assets and Licenses

We use publicly available datasets, model checkpoints, and baseline methods\. The datasets are Hateful Memes and Covid Memes, and the VLM backbones are Qwen2\.5\-VL\-7B\-Instruct and InternVL3\-8B\-HF\. We cite the original papers or model releases for all external assets used in the experiments\. We do not release new datasets, pretrained models, or user data\.