GPT-Red: Automated Red Teaming via Self-Play at Scale
Summary
This paper introduces GPT-Red, an automated red-teaming agent trained via self-play at scale to discover novel prompt injection attacks against frontier LLMs, and uses it to adversarially train GPT-5.6, achieving the largest documented LLM safety training run.
View Cached Full Text
Cached at: 07/30/26, 05:46 AM
Paper page - GPT-Red: Automated Red Teaming via Self-Play at Scale
Source: https://huggingface.co/papers/2607.26115 Authors:
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
WeintroduceGPT-Red,anautomatedred-teamingagentthatistrainedtodiscovernovelpromptinjectionattacksagainstfrontierLLMs.Thegoalofthismodelistoevaluateandimprovetherobustnessofourproductionsystems.Tothisend,weuseittoadversariallytrainGPT-5.6,ourmostrobustmodeltopromptinjectionstodate.TocreateGPT-Red,wedesignascalableself-playalgorithmwherethemodelistaskedwithattackingadiversepopulationofsimultaneously-traineddefenderagents.Wetrainthemodelonrealisticred-teamingenvironmentsusingcomputeonthesamescaleassomeofourlargestRLpost-trainingruns,makingitthesingle-largestLLMsafetytrainingruneverdocumented.GPT-Redexcelsatred-teaming:itreliablybreaksourpastmodelsuptoGPT-5.5,itfindsmoresuccessfulattacksthanhumanred-teamers,anditgeneralizestoheld-outenvironments,defendermodels,andharnesses.Inthefuture,weexpectthatasweimprovetherobustnessofeachnewGPTmodel,itwillinturnwillprovidebetterlearningsignalforevenstrongerred-teameragents,thusunlockingaself-improvementflywheel.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2607\.26115
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2607.26115 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2607.26115 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2607.26115 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Meet GPT-Red: an LLM super-hacker OpenAI built to make its models safer
OpenAI developed GPT-Red, an LLM trained via self-play to automatically red-team other models, finding vulnerabilities like a novel 'fake chain of thought' prompt injection. The approach improved the robustness of GPT-5.6.
GPT-Red: Unlocking Self-Improvement for Robustness
OpenAI introduces GPT-Red, an automated red-teaming model trained via self-play reinforcement learning to improve model robustness against prompt injection attacks. Applied to GPT-5.6, it achieves 6x fewer failures on direct prompt injection benchmarks.
@OpenAI: Introducing GPT-Red An internal automated red teamer on a mission to find our models’ prompt injection vulnerabilities …
OpenAI introduces GPT-Red, an automated red-teaming model that finds prompt injection vulnerabilities at scale and is used to adversarially train models like GPT-5.6 Sol, achieving 6x fewer failures on hard prompt injection benchmarks.
Building an early warning system for LLM-aided biological threat creation
OpenAI conducted a study with 100 participants to evaluate whether GPT-4 meaningfully increases access to dangerous biological threat creation information compared to internet-only baselines, as part of their Preparedness Framework for AI safety. The research introduces an early warning evaluation methodology to detect AI-enabled biorisk uplift and serves as a potential tripwire for flagging models that require further safety testing.
Self-play helped AI achieve superhuman performance in Go, so why hasn’t it done the same for LLMs? Researchers have found a solution.
Researchers introduce Self-Guided Self-Play (SGS), a self-play algorithm for LLMs that prevents reward hacking by using a Guide role to score synthetic problems. Applied to theorem proving in Lean4, SGS surpasses RL baselines and allows a 7B model to outperform a 671B model.