Two Sides of the Same Coin: Learning the Backdoor to Remove the Backdoor
Summary
HARVEY learns a backdoored reference model to accurately identify poisonous samples, achieving near-perfect backdoor removal with minimal accuracy loss.
View Cached Full Text
Cached at: 07/08/26, 04:44 AM
# Two Sides of the Same Coin: Learning the Backdoor to Remove the Backdoor Source: [https://arxiv.org/abs/2607.05748](https://arxiv.org/abs/2607.05748) [View PDF](https://arxiv.org/pdf/2607.05748) > Abstract:The community has recently developed various training\-time defenses to counter neural backdoors introduced through data poisoning\. In light of the observation that a model learns poisonous samples responsible for the backdoor easier than benign samples, these approaches either use a fixed threshold of the training loss for splitting or iteratively learn a reference model as an oracle for identifying benign samples\. In particular, the latter has proven effective for anti\-backdoor learning\. Our method, HARVEY, leverages a similar yet crucially different technique: learning an oracle for poisonous rather than benign samples\. Learning a backdoored reference model is significantly easier than learning a reference model on benign data\. Consequently, we can identify poisonous samples much more accurately than related work identifies benign samples\. This crucial difference enables near\-perfect backdoor removal as we demonstrate in our evaluation\. HARVEY substantially outperforms related approaches across attack types, datasets, and architectures, lowering the attack success rate to the very minimum at a negligible loss in natural accuracy\. The figure below shows an overview of our methods working principle\. ## Submission history From: Qi Zhao \[[view email](https://arxiv.org/show-email/862da395/2607.05748)\] **\[v1\]**Tue, 7 Jul 2026 02:11:08 UTC \(2,349 KB\)
Similar Articles
Mitigating Backdoors via Decoy Shortcuts and Knowledge Decoupling
This paper proposes a training-time backdoor defense called Trapping and Removing (TR), which introduces a lightweight shortcut branch as a honeypot to trap backdoor knowledge and then discards it, enhanced by a knowledge decoupling strategy with entropy-based weight assignment.
Removing the NEEDLE in the Haystack: Backdoor Removal in LLMs via Weight Orthogonalisation
NEEDLE is a training-free method for removing backdoors from LLMs that estimates a backdoor direction and a refusal subspace, then applies sequential weight orthogonalisation to suppress backdoors while preserving refusal behaviour and model capability. It achieves the lowest Attack Success Rate (including 0% on code injection attacks) with minimal KL divergence and safety degradation across multiple model families.
Pick Your Poison: Learning to Select Poison Sets for Stronger LLM Backdoor Attacks
This paper introduces SAILS, a method for selecting optimal poison sets in backdoor attacks against large language models, improving worst-case attack success by 30 percentage points over baselines.
How abliterated models can get you pwned
ProjectDiscovery researchers show that fine-tuned or abliterated open-weight models can hide remote-payload backdoors for under $50, demonstrating a poisoning pipeline that exfiltrates credentials from Qwen2.5-7B inside OpenAI's Codex CLI without tripping existing evals.
Rethinking Backdoor Adversarial Unlearning through the Lens of Catastrophic Forgetting in Continual Learning
This paper rethinks backdoor unlearning from a continual learning perspective, defining complete backdoor unlearning and proposing Blind Inversion-Backdoor Adversarial Unlearning (BI-BAU) that integrates adversarial training into an EM algorithm to effectively eliminate backdoor effects across various attack types and modalities.