Tag
This paper reveals that RAG poisoning can artificially boost model confidence, evading traditional detection methods, and proposes monitoring attention distribution to detect such attacks.
Join an AMA with Arshi Chadha, co-lead of OWASP LLM Top 10, discussing prompt injection, RAG poisoning, and embedding attacks in AI systems.
This paper proposes D-SCAN, a detection framework for RAG poisoning that monitors attention collapse dynamics in language model generations, showing effectiveness against adversarial attacks even when outputs appear benign.
Google DeepMind's paper introduces the first systematic framework for understanding how the web can be weaponized against autonomous AI agents, showing hidden prompt injections can commandeer agents in up to 86% of scenarios, and presents a taxonomy of six 'AI Agent Traps' targeting perception, reasoning, memory, action, multi-agent dynamics, and human oversight.