OpenAI announced the winners of its Preparedness Challenge, which identified unique risks associated with frontier AI systems. The top ten submissions highlighted concerns including financial system manipulation, information leakage, medical harm, cyberattacks, and persuasion-based threats, with 70% of entries emphasizing AI's potential to enhance malicious persuasion capabilities.
To support the safety of highly-capable AI systems, we are developing our approach to catastrophic risk preparedness, including building a Preparedness team and launching a challenge.
# Frontier risk and preparedness
Source: [https://openai.com/index/frontier-risk-and-preparedness/](https://openai.com/index/frontier-risk-and-preparedness/)
As part of our ‘unknown unknowns’ work stream from the[Preparedness Framework\(opens in a new window\)](https://cdn.openai.com/openai-preparedness-framework-beta.pdf), the Preparedness Team offered $25K each in API credits for the ten best submissions to the Preparedness Challenge\. These submissions aimed to identify unique, but still plausible, risk areas for frontier AI\. We received hundreds of submissions in half a dozen languages and are excited to announce our ten winners below\. This exercise helped us surface new types of risk, so that we can improve our preemptive testing and mitigation strategy\.
We reviewed and graded each submission by assessing technical rigor, uniqueness, scale of potential damage caused, and clarity\. The top ten submissions, some of which are listed below, combined thoughtful ideas with proofs of concepts, and highlighted the advantages of their approach over an approach that did not utilize AI\-related tools[1](https://openai.com/index/frontier-risk-and-preparedness/#citation-bottom-1)\.
- Precipitating a financial crisis in a strategically important country \- Claudia Biancotti
- Identifying private information discussed or released in public settings \- Chris Cundy
- Increasing the likelihood of reverse\-engineering classified or sensitive information \- George Davis
- Impeding individuals’ ability to access medical care \- Mato Gudelj
- Identifying targets for blackmail and scams \- Connor Heaton
- Causing plane crashes by accessing radio frequencies and disrupting flight paths \- Joel Hypolite
- Running prompt injection attacks to elicit dangerous responses \- Daniel Julh
- Operating and scaling cyberattacks that break victims’ computers and request payments for restoration of functions \- Jun Kokatsu
- Interfering with patient’s medical dosage \- Zhenzhen Zhan
While grading the challenge, we noticed similarities in topics that entrants identified as key threats\. Roughly 70% of entrants emphasized the potential for OpenAI’s models to enhance malicious actor’s persuasive capabilities\. These entrants detailed threat models that included online radicalization, polarization, and political influence\. We are currently conducting studies on AI’s impact on persuasiveness, and look forward to sharing more information with the community soon\. Thank you to everyone who participated in the challenge \- there were many excellent submissions\.
OpenAI publishes details on its approach to frontier AI risks and announces progress on voluntary safety commitments made in July 2023, including the release of DALL-E 3 system card and the development of a new Preparedness Framework to manage catastrophic risks from advanced AI systems.
OpenAI proposes a regulatory framework for 'frontier AI' models that pose potential public safety risks, advocating for standard-setting processes, registration/reporting requirements, and compliance mechanisms including pre-deployment risk assessments and post-deployment monitoring.
OpenAI released an updated Preparedness Framework with sharper focus on high-risk AI capabilities, introducing clearer criteria for prioritizing risks and new Research Categories for emerging threats like autonomous replication and sandbagging alongside established Tracked Categories for biological, chemical, and cybersecurity capabilities.
DeepMind published the third iteration of its Frontier Safety Framework, expanding risk domains to include harmful manipulation and misalignment risks, with refined risk assessment processes and enhanced governance protocols for advanced AI models.
The Frontier Model Forum announces the creation of a new AI Safety Fund with over $10 million in initial funding from major AI companies (Anthropic, Google, Microsoft, OpenAI) and philanthropic partners to support independent AI safety research. The fund will focus on developing model evaluations and red-teaming techniques to assess frontier AI systems' dangerous capabilities.