Paper page - MAAT: Multi-phase Adapter-Aware Targeted Unlearning

Source: https://huggingface.co/papers/2605.30514 MAAT introduces a structured LoRA-adapter unlearning pipeline plus 5WBENCH, a balanced 5W benchmark, showing that causal “Why” knowledge is uniquely difficult to forget due to long multi-hop answer chains and gradient dilution—not because it has a distinct layer-wise encoding footprint.

➡️ 𝐊𝐞𝐲 𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬 𝐨𝐟 𝐌𝐀𝐀𝐓 𝐚𝐧𝐝 𝟓𝐖𝐁𝐄𝐍𝐂𝐇:

🧪 𝑩𝒂𝒍𝒂𝒏𝒄𝒆𝒅 𝟓𝑾 𝑼𝒏𝒍𝒆𝒂𝒓𝒏𝒊𝒏𝒈 𝑩𝒆𝒏𝒄𝒉𝒎𝒂𝒓𝒌 (𝟓𝑾𝑩𝑬𝑵𝑪𝑯): Introduces a 5,000-sample benchmark with 1,000 examples each for Who, What, When, Where, and Why, exposing a major blind spot in existing unlearning datasets where Why-type causal questions are nearly absent—e.g., <0.06% in CounterFact and 0.6% in ZSRE.

🧩 𝑪𝒂𝒖𝒔𝒂𝒍 𝑲𝒏𝒐𝒘𝒍𝒆𝒅𝒈𝒆 𝑭𝒂𝒊𝒍𝒖𝒓𝒆 𝑫𝒊𝒂𝒈𝒏𝒐𝒔𝒊𝒔: Shows that Why-type facts are harder to unlearn because they have much longer answer spans, averaging 40.1 tokens, and far more multi-hop reasoning structure, 44% versus ≤2% for other categories, which spreads the gradient-ascent signal across diffuse causal chains.

🧠 𝑴𝑨𝑨𝑻: 𝑴𝒖𝒍𝒕𝒊-𝒑𝒉𝒂𝒔𝒆 𝑨𝒅𝒂𝒑𝒕𝒆𝒓-𝑨𝒘𝒂𝒓𝒆 𝑻𝒂𝒓𝒈𝒆𝒕𝒆𝒅 𝑼𝒏𝒍𝒆𝒂𝒓𝒏𝒊𝒏𝒈: Proposes a three-phase LoRA-only unlearning method: gradient-projected ascent to avoid retain-gradient interference, MLP-only SVD rank-dimension pruning plus forget-scored task-vector negation, and hybrid KL/hidden-state retain repair with entropy regularization to prevent relearning forgotten facts.

**Technical novelty:**The key architectural move is treating LoRA adapters as structured rank spaces rather than flat parameter deltas. MAAT scores adapter rank dimensions by forget-set activation/gradient relevance, prunes or negates only the most forget-associated subspace, and repairs retained behavior without merging into or modifying the frozen base model.

**Why it moves the needle:**On 5WBENCH, MAAT reaches a stronger forget–retain operating point than baselines: on Llama 3.2-3B it achieves 77.4% average Forget Success Rate and 71.6% Retain Success Rate, while matching RO-FT’s forgetting but improving retention by +36.4 points. It is also the only method reported to exceed 60% FSR and 60% RSR across all five 5W categories, including Why-type causal knowledge.