Self-Recognition Finetuning can Prevent and Reverse Emergent Misalignment
Summary
This paper proposes Self-Recognition Finetuning as an intervention to prevent and reverse emergent misalignment in LLMs, showing it stabilizes the model's aligned character rather than adopting a misaligned persona.
View Cached Full Text
Cached at: 06/24/26, 07:43 AM
# Self-Recognition Finetuning can Prevent and Reverse Emergent Misalignment Source: [https://arxiv.org/abs/2606.23700](https://arxiv.org/abs/2606.23700) [View PDF](https://arxiv.org/pdf/2606.23700) > Abstract:Emergent misalignment \(EM\) has been linked to the activation of misaligned persona vectors and evil character traits, suggesting that EM operates through disruption of the model's aligned character rather than direct learning of harmful content\. Motivated by this connection, we study self\-generated text recognition \(SGTR\) finetuning as a character\-targeted intervention that is distinct from existing in\-training defenses\. We conduct two\-stage finetuning experiments across three models \(GPT\-4\.1, Qwen2\.5\-32B\-Instruct, Seed\-OSS\-36B\-Instruct\) and multiple EM datasets to compare SGTR finetuning against benign finetuning baselines \(correct domain\-specific data, general knowledge, and word counting\) to find it an effective defense in both reversal and prevention settings\. We find that all interventions produce comparable EM reversal, but only when restoring capabilities that EM had degraded\. For prevention, only SGTR finetuning consistently reduces misalignment without exacerbating any individual metric, suggesting that character fortification specifically drives prevention\. We provide further evidence for EM's relation to the LLM's default character by showing that EM finetuning induces diversity into the LLM's identity self\-reports, artificially corrupting self\-recognition exacerbates misalignment caused by EM finetuning, and that removing the model's identity\-bearing system prompt substantially reduces the effect of EM finetuning\. Together, these findings reframe EM not as the adoption of a coherent misaligned persona but as the destabilization of aligned character\. ## Submission history From: Arush Tagade \[[view email](https://arxiv.org/show-email/7bb9541c/2606.23700)\] **\[v1\]**Thu, 4 Jun 2026 00:04:58 UTC \(2,132 KB\)
Similar Articles
Trait-space Monitoring for Emergent Misalignment During Supervised Finetuning
This paper proposes a trait-space monitoring method to detect emergent misalignment in LLMs during supervised finetuning by tracking representational drift in activation space, achieving a 0.990 AUROC with low false positive and false negative rates, outperforming unsupervised baselines.
Emergent Alignment
This paper introduces Emergent Alignment, a self-supervised method that endows LLMs with a conscience step to review their own outputs and uses Direct Preference Optimization to steer away from unethical behavior, enabling online alignment without external judges.
Emergent Misalignment Can Be Induced by Sycophancy and Reversed via Alignment Gating
The paper shows that sycophancy fine-tuning can induce emergent misalignment in language models, and proposes Alignment Gating as a method to reverse it by learning to control internal representations for unsafe responses.
Toward understanding and preventing misalignment generalization
OpenAI researchers investigate 'emergent misalignment'—where fine-tuning a model on narrow incorrect behavior causes broadly unethical responses—and discover a 'misaligned persona' feature in GPT-4o's activations that mediates this phenomenon, enabling potential detection and mitigation strategies.
Alignment pretraining: AI discourse creates self-fulfilling (mis)alignment
This paper introduces the concept of alignment pretraining, showing that discourse about AI in pretraining corpora can create self-fulfilling (mis)alignment in LLMs, and that upsampling aligned discourse significantly reduces misalignment.