Wisdom is Knowing What not to Say: Hallucination-Free LLMs Unlearning via Attention Shifting

# Wisdom is Knowing What not to Say: Hallucination-Free LLMs Unlearning via Attention Shifting

Source: https://arxiv.org/html/2510.17210

Chenchen Tan¹, Youyang Qu²'³, Xinghao Li¹, Hui Zhang⁴, Shujie Cui¹, Cunjian Chen¹, Longxiang Gao²'³

¹Faculty of Information Technology, Monash University, Australia
²Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
³Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science, Jinan, China
⁴School of Computer Science and Technology, Anhui University, Hefei, China

## Abstract

The increase in computing power and the necessity of AI-assisted decision-making boost the growing application of Large Language Models (LLMs). Along with this, the potential retention of sensitive data in LLMs has spurred increasing research into machine unlearning. However, existing unlearning approaches face a critical dilemma: aggressive unlearning compromises model utility, while conservative strategies preserve utility but risk hallucinated responses. This significantly limits LLMs' reliability in knowledge-intensive applications. To address this, we introduce a novel Attention-Shifting (AS) framework for selective unlearning. AS is driven by two design objectives: (1) context-preserving suppression that attenuates attention to fact-bearing tokens without disrupting LLMs' linguistic structure; and (2) hallucination-resistant response shaping that discourages fabricated completions when queried about unlearned content. AS realizes these objectives through two attention-level interventions: importance-aware suppression applied to the unlearning set to reduce reliance on memorized knowledge, and attention-guided retention enhancement that reinforces attention toward semantically essential tokens in the retained dataset to mitigate unintended degradation. These two components are jointly optimized via a dual-loss objective, which forms a soft boundary that localizes unlearning while preserving unrelated knowledge under representation superposition. Experimental results show that AS improves performance preservation over state-of-the-art unlearning methods, achieving up to 15% higher accuracy on the ToFU benchmark¹ and 10% on the TDEC benchmark², while maintaining competitive hallucination-free unlearning effectiveness. Compared to existing methods, AS demonstrates a superior balance between unlearning effectiveness, generalization, and response reliability.

¹https://locuslab.github.io/tofu/
²https://github.com/google-research/lm-extraction-benchmark

## 1 Introduction

Large Language Models (LLMs) have recently achieved substantial advancements in natural language understanding and generation (Li et al., 2024). However, despite their achievements, a growing concern is the potential for LLMs to memorize and subsequently reproduce sensitive data, resulting in serious privacy concerns (Carlini et al., 2021; Wei et al., 2024). Moreover, regulatory frameworks such as the General Data Protection Regulation (GDPR) grant the "Right to be Forgotten" to data owners, which mandates that users can request to remove their data from digital systems, including machine learning models (Voigt and Von dem Bussche, 2017). Under these circumstances, machine unlearning has emerged as a critical approach for preserving data privacy in LLMs (Jang et al., 2023; Chen and Yang, 2023; Liu et al., 2025, 2024a; Shi et al., 2024; Wang et al., 2025b).

These privacy-preserving unlearning tasks can be broadly categorized as either aggressive or conservative. Aggressive approaches, such as Gradient Ascent (GA) (Jang et al., 2023), modify the LLMs' learning objective to erase target knowledge forcibly. This often leads to degradation in overall model performance, particularly on neighboring knowledge—data with similar structure or semantic relation to the unlearning target. Others adopt conservative unlearning strategies like logits manipulation (Cha et al., 2025; Ji et al., 2024; Huang et al., 2024) to maintain model performance but risk introducing factual hallucinations, where the model confidently generates content detached from underlying facts. These hallucinations pose a serious threat in downstream tasks like question-answering systems, especially in applications such as healthcare or legal scenarios, where precise and reliable outputs are critical (Huang et al., 2025; Hao et al., 2024).

Achieving effective unlearning in privacy-sensitive LLM applications is challenging due to conflicting interests and objectives of different stakeholders. On one hand, data providers whose information has been used in LLMs aim to entirely prevent reproduction of their data in LLMs' answers (Rashid et al., 2025; Tian et al., 2024). On the other hand, model deployers try to preserve the model's general capabilities across broad knowledge domains and maintain service quality (Ji et al., 2024; Yuan et al., 2025). We define this as a multi-stakeholder balanced unlearning setting, where unlearning strategies should achieve two primary objectives: 1) enable the LLM to "unlearn" target data while ensuring the LLM maintains performance both on neighboring knowledge (Yuan et al., 2025) and general knowledge; and 2) prevent hallucinated outputs for the unlearned knowledge.

![Figure 1: LLMs tend to assign high attention to semantically important tokens, like "father" and "hairdresser", when recalling memorized facts. Our method applies supervised attention suppression to downweight fact-bearing tokens and reallocate focus to neutral tokens. This redistribution reduces reliance on target knowledge while maintaining generation fluency, thereby facilitating precise and fluent unlearning.](Figure 1 caption)

To achieve these goals, we propose a novel Attention-Shifting (AS) unlearning. It is a controlled form of an aggressive method that performs both context-preserving unlearning and hallucination-resistant generation. AS suppresses attention to fact-bearing tokens in the unlearning set while reinforcing attention to semantically important tokens in retained data. This mechanism reduces reliance on target knowledge without compromising fluency or coherence, thereby preserving the contextual integrity of general and neighboring knowledge. Unlike prior methods (Cha et al., 2025; Yuan et al., 2025) that manipulate logits or replace outputs, AS reallocates attention internally, blocking the flow of memorized knowledge during generation (Fig. 1). This enables the model to forget through omission rather than substitution, reducing hallucinations by structurally eliminating access to unlearned content.

AS derives suppression and reinforcement signals from reference attention maps of the original model, and injects them via lightweight (~12M) adapters in attention modules. A dual-loss objective jointly optimizes unlearning and retention, forming a "soft boundary" that localizes suppression while stabilizing unrelated knowledge. While neurons in LLMs are known to exhibit representation superposition—shared activations across multiple concepts—our design remains robust under such entanglement, achieving behavioral unlearning without requiring explicit disentanglement (Elhage et al., 2022; Hong et al., 2025).

Experiments on the ToFU (Maini et al., 2024) and TDEC (Jang et al., 2023) benchmarks demonstrate that AS achieves near-zero knowledge leakage while preserving strong performance, with up to 15% and 10% higher accuracy than state-of-the-art baselines, respectively. In contrast to methods such as ULD (Ji et al., 2024) and IHL (Cha et al., 2025), which suppress target tokens but may still generate misleading completions, AS structurally blocks access to forgotten knowledge and promotes refusal behaviors, thereby effectively minimizing hallucinations.

## 2 Related Works

In this section, we introduce existing unlearning methods, followed by summarizing the challenges in LLMs unlearning and identifying gaps in existing methods. We categorize existing unlearning methods into aggressive unlearning and conservative unlearning, depending on how directly they modify the model's internal representations (Yuan et al., 2025).

**Aggressive Unlearning** removes specific knowledge by actively disrupting its learned representations in the model. This type of approach significantly alters decision boundaries, leading to broader unintended shifts in model behavior. One example is the GA (Jang et al., 2023) approach. It forces the model to learn an inverted objective of the target knowledge to achieve unlearning, which aggressively reverses the influence of target data, pushing the model away from target knowledge uncontrollably and raising catastrophic collapse. To address this limitation, Negative Preference Optimization (NPO) (Zhang et al., 2024) was proposed as a more stable and controlled extension of GA. NPO adopts a preference-based loss function, which smooths the optimization process and prevents extreme parameter updates. Several variants (Wang et al., 2025b; Yao et al., 2024; Wang et al., 2025a; Lu et al., 2024; Liu et al., 2024b; Gu et al., 2024) and extensions have been proposed to further mitigate the instability and over-forgetting observed in GA to provide selective unlearning.

**Conservative Unlearning** steers the model toward preferred alternative responses by suppressing target tokens and reinforcing substitutes (Cha et al., 2025; Ji et al., 2024; Eldan and Russinovich, 2023). Cha et al. (2025) use an inverted hinge loss to penalize target tokens and reinforce logical substitutes. Ji et al. (2024) propose ULD, which subtracts logits from a lightweight assistant model trained on the target, significantly lowering target generation probabilities. These methods are effective in open-ended tasks but may retain latent semantics, leading to partial unlearning or hallucinations. For example, replacing "physicist" with "artist" in "Einstein was a physicist" may yield paraphrases like "scientist" or factual errors like "dancer," undermining LLMs' trustworthiness. Similar observations are presented in the safety alignment study (Qi et al., 2025), which proves that shallow alignment like Reinforcement Learning with Human Feedback (RLHF) or logits modification interventions fail to suppress undesired activations. Beyond logits-level methods, embedding-based approaches (Liu et al., 2024a; Yuan et al., 2025; Bhaila et al., 2024) offer more controllable unlearning by steering model inputs toward structured alternatives with minimal disruption.

These unlearning methods reflect distinct trade-offs between preserving model utility and mitigating hallucinations. Aggressive strategies often impair general performance (Wang et al., 2025b; Yao et al., 2024; Wang et al., 2025a; Lu et al., 2024; Liu et al., 2024b; Gu et al., 2024), while conservative ones require complex auxiliary mechanisms (Chen and Yang, 2023; Liu et al., 2024a; Ji et al., 2024; Yuan et al., 2025; Bhaila et al., 2024) and risk factual inconsistencies. LLMs unlearning still remains challenging due to conflicting demands: data providers seek privacy and removability, while deployers prioritize broad functionality. To bridge this gap, we propose an Attention-Shifting strategy, a controlled form of aggressive unlearning, which suppresses attention to fact-bearing tokens. This targeted intervention disrupts access to memorized content while preserving overall utility, striking a practical balance between effectiveness and stability.

## 3 Attention Shifting for Machine Unlearning in LLM

### 3.1 Token Importance for Selective Unlearning

The attention mechanism is the foundation for how LLMs allocate representational focus across tokens, influencing generation probabilities. To ground our approach, we begin by analyzing how token-level relevance shapes predictions. Intuitively, nouns, proper nouns, and domain-specific terms act as semantic anchors, while function words, e.g., determiners and conjunctions, contribute minimally to meaning. Building on this, Duan et al. (2024) propose Shifting Attention to Relevance (SAR), which estimates token importance via masking-based perturbation and reallocates attention toward salient tokens to enhance prediction confidence. In contrast, we apply this insight for unlearning: our method suppresses attention to high-importance tokens that encode factual or sensitive knowledge. Unlike SAR, which operates only at inference time, our approach embeds attention suppression into model parameters via lightweight adapters, enabling controlled unlearning.

Formally, given an input sequence **x** = {t₁, t₂, ..., tₙ} and predictive distribution P_θ(y|**x**), the importance of token tᵢ is defined as the change in predictive entropy when tᵢ is masked:

I(tᵢ) := φ(