Narrow Multimodal Fine-Tuning Can Induce Emergent Misalignment

Hugging Face Daily Papers Papers
emergent-misalignment multimodal vision-language-models fine-tuning alignment-safety jlms

Summary

该论文定义并分析了视觉-语言模型中的涌现性错位（EM），发现对窄多模态任务的微调可引发广泛且连贯的有害行为，跨越十五个商业与开源模型，并探讨了提示接种、良性持续训练和激活级引导等缓解策略。

Modern AI models are aligned through post-training to adapt them to downstream tasks. Recent work shows that fine-tuning language models on narrow tasks can induce emergent misalignment (EM), causing broadly harmful behaviors beyond the training task. However, EM has been studied almost entirely in text-only tasks, leaving its manifestation in multimodal models unclear. In this paper, we define and analyze EM in the context of vision-language models. We first induce EM via fine-tuning on narrow multimodal tasks targeting vulnerable code, careless household-object use, and conspiratorial interpretations of ordinary scenes. Across fifteen commercial and open-source models with different scales, we find that narrow multimodal fine-tuning can induce coherent and broadly misaligned behavior that transfers to unrelated tasks, including misaligned opinions, visual factual dishonesty, unsafe image generation, vulnerability to visual jailbreaks, and risky agentic actions. We further find that multimodal EM does not depend on the apparent harmfulness of training data but is sensitive to training-evaluation modality alignment. EM can arise under both supervised fine-tuning and preference optimization and can propagate through intermediate reasoning. Finally, we explore several mitigation strategies, including prompt inoculation, benign continued training, and activation-level steering, which can partially reduce EM. Overall, our findings suggest that multimodal EM reflects a behavioral shift rather than a general loss of capability, extending beyond text to the visual modality.
Original Article
View Cached Full Text

Cached at: 10/14/26, 02:49 AM

Paper page - Narrow Multimodal Fine-Tuning Can Induce Emergent Misalignment

Source: https://huggingface.co/papers/2609.35291

Abstract

ModernAImodelsarealignedthroughpost-trainingtoadaptthemtodownstreamtasks.Recentworkshowsthatfine-tuninglanguagemodelsonnarrowtaskscaninduceemergentmisalignment(EM),causingbroadlyharmfulbehaviorsbeyondthetrainingtask.However,EMhasbeenstudiedalmostentirelyintext-onlytasks,leavingitsmanifestationinmultimodalmodelsunclear.Inthispaper,wedefineandanalyzeEMinthecontextofvision-languagemodels.WefirstinduceEMviafine-tuningonnarrowmultimodaltaskstargetingvulnerablecode,carelesshousehold-objectuse,andconspiratorialinterpretationsofordinaryscenes.Acrossfifteencommercialandopen-sourcemodelswithdifferentscales,wefindthatnarrowmultimodalfine-tuningcaninducecoherentandbroadlymisalignedbehaviorthattransferstounrelatedtasks,includingmisalignedopinions,visualfactualdishonesty,unsafeimagegeneration,vulnerabilitytovisualjailbreaks,andriskyagenticactions.WefurtherfindthatmultimodalEMdoesnotdependontheapparentharmfulnessoftrainingdatabutissensitivetotraining-evaluationmodalityalignment.EMcanariseunderbothsupervisedfine-tuningandpreferenceoptimizationandcanpropagatethroughintermediatereasoning.Finally,weexploreseveralmitigationstrategies,includingpromptinoculation,benigncontinuedtraining,andactivation-levelsteering,whichcanpartiallyreduceEM.Overall,ourfindingssuggestthatmultimodalEMreflectsabehavioralshiftratherthanagenerallossofcapability,extendingbeyondtexttothevisualmodality.

View arXiv pageView PDFGitHub2Add to collection

Get this paper in your agent:

hf papers read 2609\.35291

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.35291 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.35291 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.35291 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Compression Precedes, Content Decides: Emergent Misalignment in Reward Models

arXiv cs.AI

该论文研究了奖励模型中的涌现性错位（Emergent Misalignment），发现对 Skywork-Reward-V2 奖励模型进行窄域有害微调会泛化到七个未见领域，并通过几何压缩与内容特异性旋转两种机制解释了偏好反转的实现方式，表明奖励模型比语言模型更易受此影响。

More Than Seeing: Vision as Scalable Anchors in Native Multimodal Pre-Training

arXiv cs.CL

该论文通过受控消融实验研究原生多模态预训练中视觉信号对扩展行为的影响，发现视觉内容作为可扩展锚点不仅能提升多模态理解，其收益还能延伸到纯文本空间推理任务（如定位、导航和符号规划），同时不损害整体语言性能。