GPT-Red: Automated Red Teaming via Self-Play at Scale

Hugging Face Daily Papers Papers

Summary

This paper introduces GPT-Red, an automated red-teaming agent trained via self-play at scale to discover novel prompt injection attacks against frontier LLMs, and uses it to adversarially train GPT-5.6, achieving the largest documented LLM safety training run.

We introduce GPT-Red, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs. The goal of this model is to evaluate and improve the robustness of our production systems. To this end, we use it to adversarially train GPT-5.6, our most robust model to prompt injections to date. To create GPT-Red, we design a scalable self-play algorithm where the model is tasked with attacking a diverse population of simultaneously-trained defender agents. We train the model on realistic red-teaming environments using compute on the same scale as some of our largest RL post-training runs, making it the single-largest LLM safety training run ever documented. GPT-Red excels at red-teaming: it reliably breaks our past models up to GPT-5.5, it finds more successful attacks than human red-teamers, and it generalizes to held-out environments, defender models, and harnesses. In the future, we expect that as we improve the robustness of each new GPT model, it will in turn will provide better learning signal for even stronger red-teamer agents, thus unlocking a self-improvement flywheel.
Original Article
View Cached Full Text

Cached at: 07/30/26, 05:46 AM

Paper page - GPT-Red: Automated Red Teaming via Self-Play at Scale

Source: https://huggingface.co/papers/2607.26115 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

WeintroduceGPT-Red,anautomatedred-teamingagentthatistrainedtodiscovernovelpromptinjectionattacksagainstfrontierLLMs.Thegoalofthismodelistoevaluateandimprovetherobustnessofourproductionsystems.Tothisend,weuseittoadversariallytrainGPT-5.6,ourmostrobustmodeltopromptinjectionstodate.TocreateGPT-Red,wedesignascalableself-playalgorithmwherethemodelistaskedwithattackingadiversepopulationofsimultaneously-traineddefenderagents.Wetrainthemodelonrealisticred-teamingenvironmentsusingcomputeonthesamescaleassomeofourlargestRLpost-trainingruns,makingitthesingle-largestLLMsafetytrainingruneverdocumented.GPT-Redexcelsatred-teaming:itreliablybreaksourpastmodelsuptoGPT-5.5,itfindsmoresuccessfulattacksthanhumanred-teamers,anditgeneralizestoheld-outenvironments,defendermodels,andharnesses.Inthefuture,weexpectthatasweimprovetherobustnessofeachnewGPTmodel,itwillinturnwillprovidebetterlearningsignalforevenstrongerred-teameragents,thusunlockingaself-improvementflywheel.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2607\.26115

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.26115 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.26115 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.26115 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

GPT-Red: Unlocking Self-Improvement for Robustness

OpenAI Blog

OpenAI introduces GPT-Red, an automated red-teaming model trained via self-play reinforcement learning to improve model robustness against prompt injection attacks. Applied to GPT-5.6, it achieves 6x fewer failures on direct prompt injection benchmarks.

PIPETTE: Prompt Injection Purple-Teaming for Robust LLM Agents

Hugging Face Daily Papers

PIPETTE introduces a unified prompt injection purple-teaming framework using dictionary learning-based rewards and on-policy adversarial training to convert red-teaming discoveries into robust defenses against indirect prompt injection attacks for LLM agents.