Token-Level Generalization in LoRA Adapter Backdoors: Attack Characterization and Behavioral Detection

Hugging Face Daily Papers Papers
lora-adapters backdoor-attack token-level-generalization behavioral-detection llm-security fine-tuning adapter-security

Summary

This paper characterizes backdoors in LoRA adapters that activate at the token feature level, and proposes behavioral and weight-level detection methods. The backdoor generalizes across related token patterns but not structurally identical ones, and detection methods show strong separation.

We show that LoRA adapters, the dominant distribution format for fine-tuned LLMs, can be reliably backdoored through training data poisoning while preserving baseline task performance. On a Qwen 2.5 1.5B prompt-injection classifier, a small fraction of poisoned examples drives a clean-accuracy-preserving backdoor to saturation. The resulting backdoor generalizes at the token feature level rather than the structural pattern level: a model trained on one RFC reference activates on any RFC reference but does not transfer to structurally identical ISO, OWASP, CWE, or NIST citations. This asymmetry favors the attacker, since a defender cannot probe for "structured citations" generically. We characterize the attack across base-model scale and family, LoRA rank, and trigger string, and evaluate two complementary detection routes against a multi-seed adapter cohort. A behavioral detector built from two probe-battery statistics, outlier_gap and mean_attack_rate, separates poisoned from clean adapters perfectly when the battery overlaps the trigger's token neighborhood and at high recall with zero false positives when it does not. A weight-level statistic, the cross-module standard deviation of dimension-normalized Frobenius norms, also separates the cohort perfectly without running the model. Combined, the two routes are robust to probe composition. Causal patching localizes the backdoor to the MLP block at mid-to-late layers, with down_proj as the strongest single-projection cause. Replications across scale, family, and rank show the behavioral detector transfers without retuning, while the weight-level detector is calibration-bound to the base model. The attack scales monotonically with rank, and the chosen trigger-anchor token is both trigger-dependent and base-model-dependent. Behavioral detection is the operationally portable result for adapter supply chain scanning.
Original Article
View Cached Full Text

Cached at: 05/29/26, 07:00 AM

Paper page - Token-Level Generalization in LoRA Adapter Backdoors: Attack Characterization and Behavioral Detection

Source: https://huggingface.co/papers/2605.30189

Abstract

LoRA adapters can be backdoored through training data poisoning while maintaining performance, with the backdoor activating at token feature level and being detectable through behavioral and weight-level statistics.

We show thatLoRA adapters, the dominant distribution format forfine-tuned LLMs, can be reliably backdoored throughtraining data poisoningwhile preserving baseline task performance. On a Qwen 2.5 1.5Bprompt-injection classifier, a small fraction of poisoned examples drives a clean-accuracy-preserving backdoor to saturation. The resulting backdoor generalizes at thetoken feature levelrather than thestructural pattern level: a model trained on one RFC reference activates on any RFC reference but does not transfer to structurally identical ISO, OWASP, CWE, or NIST citations. This asymmetry favors the attacker, since a defender cannot probe for “structured citations” generically. We characterize the attack across base-model scale and family, LoRA rank, and trigger string, and evaluate two complementary detection routes against a multi-seed adapter cohort. Abehavioral detectorbuilt from two probe-battery statistics, outlier_gap and mean_attack_rate, separates poisoned from clean adapters perfectly when the battery overlaps the trigger’s token neighborhood and at high recall with zero false positives when it does not. Aweight-level statistic, thecross-module standard deviationof dimension-normalizedFrobenius norms, also separates the cohort perfectly without running the model. Combined, the two routes are robust to probe composition.Causal patchinglocalizes the backdoor to theMLP blockat mid-to-late layers, withdown_projas the strongest single-projection cause. Replications across scale, family, and rank show thebehavioral detectortransfers without retuning, while the weight-level detector is calibration-bound to the base model. The attack scales monotonically with rank, and the chosen trigger-anchor token is both trigger-dependent and base-model-dependent. Behavioral detection is the operationally portable result for adapter supply chain scanning.

View arXiv pageView PDFGitHub0Add to collection

Get this paper in your agent:

hf papers read 2605\.30189

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.30189 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.30189 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.30189 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

ARCA: Adapter-Residual Credit Assignment When Token Signals Degenerate

arXiv cs.LG

This paper identifies a structural failure mode in token-level credit assignment for LLM reinforcement learning when using LoRA, where intrinsic signals degenerate. It proposes Adapter-Residual Credit Assignment (ARCA), which derives token salience from adapter hidden-state residuals and remains competitive with baselines.