When Context Bites: Detecting RAG Poisoning via Document-Level Attention Collapse

Hugging Face Daily Papers Papers
rag-poisoning attention-collapse d-scan detection-framework language-models ai-security

Summary

This paper proposes D-SCAN, a detection framework for RAG poisoning that monitors attention collapse dynamics in language model generations, showing effectiveness against adversarial attacks even when outputs appear benign.

Retrieval-augmented generation (RAG) is indispensable for enhancing large language models. However, RAGs are increasingly susceptible to poisoning attacks, in which adversarial documents are injected to manipulate generator outputs. Previous methods rely on output-side signals such as perplexity and consistency checks to detect such attacks. Nevertheless, our analysis reveals that deliberate attacks often induce false confidence, where poisoned outputs exhibit even lower perplexity than benign ones, rendering uncertainty-based detection ineffective. To address this challenge, we explore the internal dynamics of the generator and identify a distinctive signature termed Attention Collapse. Unlike the dispersed attention in benign generations, attacked generations exhibit a decrease in entropy as attention concentrates on poisoned documents. Building on these findings, we propose D-SCAN (Document-level Signal Collapse Analysis), a lightweight detection framework that monitors attention dynamics to identify attacked generations. Extensive experiments on multiple attack benchmarks demonstrate the effectiveness of our method. Moreover, D-SCAN can detect attacks even when they fail to alter the final answer. Code is available at https://github.com/yingtaoren/D-Scan.git.
Original Article
View Cached Full Text

Cached at: 08/18/26, 03:51 AM

Paper page - When Context Bites: Detecting RAG Poisoning via Document-Level Attention Collapse

Source: https://huggingface.co/papers/2608.06947 Published on Aug 7

·

Submitted byhttps://huggingface.co/An998

an998on Aug 18

Abstract

D-SCAN detects retrieval poisoning by monitoring attention collapse dynamics in language model generations.

Retrieval-augmented generation(RAG) is indispensable for enhancing large language models. However, RAGs are increasingly susceptible topoisoning attacks, in which adversarial documents are injected to manipulate generator outputs. Previous methods rely on output-side signals such asperplexityand consistency checks to detect such attacks. Nevertheless, our analysis reveals that deliberate attacks often induce false confidence, where poisoned outputs exhibit even lowerperplexitythan benign ones, rendering uncertainty-based detection ineffective. To address this challenge, we explore the internal dynamics of the generator and identify a distinctive signature termedAttention Collapse. Unlike the dispersed attention in benign generations, attacked generations exhibit a decrease in entropy as attention concentrates on poisoned documents. Building on these findings, we proposeD-SCAN(Document-level Signal Collapse Analysis), a lightweight detection framework that monitors attention dynamics to identify attacked generations. Extensive experiments on multiple attack benchmarks demonstrate the effectiveness of our method. Moreover,D-SCANcan detect attacks even when they fail to alter the final answer. Code is available at https://github.com/yingtaoren/D-Scan.git.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2608\.06947

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.06947 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.06947 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.06947 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles