Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming

Hugging Face Daily Papers Papers
prompt-injection red-teaming llm-agents security agentic-system

Summary

This paper introduces PIMiner, an agentic system for automatic prompt injection red-teaming that builds a strategy library during training and transfers to unseen target LLMs at test time, achieving strong attack success rates against models like Gemini-2.5-Pro, GPT-5.1, and Claude-Sonnet-4.5.

Prompt injection poses significant security risks to LLM agents. Efficient and effective red-teaming is therefore critical, both for evaluating these risks and for collecting training data to improve defenses. Existing state-of-the-art prompt injection red-teaming methods primarily rely on reinforcement learning (RL), producing attacker models that often generalize poorly to new target LLMs. In this work, we develop PIMiner, an agentic system for prompt injection red-teaming. During training, PIMiner is trained on a sequence of (dataset, target model) pairs and builds a strategy library from scratch. At test time, the learned strategy library can be directly transferred to a previously unseen target LLM without additional training. PIMiner requires only a small number of queries to a target agent (e.g., 10) per test sample. Experimental results demonstrate that PIMiner achieves strong performance. On IPIArena, it attains a 76.2% ASR against Gemini-2.5-Pro, 61.9% ASR against GPT-5.1, and 42.9% ASR against Claude-Sonnet-4.5. On AgentDojo, it achieves an 86.7% ASR against Gemini-2.5-Pro, 53.3% ASR against GPT-5.1, and 40.0% ASR against Claude-Sonnet-4.5.
Original Article
View Cached Full Text

Cached at: 08/06/26, 05:49 AM

Paper page - Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming

Source: https://huggingface.co/papers/2608.05108

Abstract

PromptinjectionposessignificantsecurityriskstoLLMagents.Efficientandeffectivered-teamingisthereforecritical,bothforevaluatingtheserisksandforcollectingtrainingdatatoimprovedefenses.Existingstate-of-the-artpromptinjectionred-teamingmethodsprimarilyrelyonreinforcementlearning(RL),producingattackermodelsthatoftengeneralizepoorlytonewtargetLLMs.Inthiswork,wedevelopPIMiner,anagenticsystemforpromptinjectionred-teaming.Duringtraining,PIMineristrainedonasequenceof(dataset,targetmodel)pairsandbuildsastrategylibraryfromscratch.Attesttime,thelearnedstrategylibrarycanbedirectlytransferredtoapreviouslyunseentargetLLMwithoutadditionaltraining.PIMinerrequiresonlyasmallnumberofqueriestoatargetagent(e.g.,10)pertestsample.ExperimentalresultsdemonstratethatPIMinerachievesstrongperformance.OnIPIArena,itattainsa76.2%ASRagainstGemini-2.5-Pro,61.9%ASRagainstGPT-5.1,and42.9%ASRagainstClaude-Sonnet-4.5.OnAgentDojo,itachievesan86.7%ASRagainstGemini-2.5-Pro,53.3%ASRagainstGPT-5.1,and40.0%ASRagainstClaude-Sonnet-4.5.

View arXiv pageView PDFGitHub1Add to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.05108 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.05108 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.05108 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

GPT-Red: Automated Red Teaming via Self-Play at Scale

Hugging Face Daily Papers

This paper introduces GPT-Red, an automated red-teaming agent trained via self-play at scale to discover novel prompt injection attacks against frontier LLMs, and uses it to adversarially train GPT-5.6, achieving the largest documented LLM safety training run.

Insights on Indirect Prompt Injection (12 minute read)

TLDR AI

Zico Kolter and Matt Fredrikson, leaders at Gray Swan and experts in AI security, discuss the state of AI red-teaming and indirect prompt injection, a critical vulnerability for AI agents. They explain why AI security requires a different mindset, how automated red-teaming can beat humans, and introduce tools like Shade for adversarial testing.