Evaluating potential cybersecurity threats of advanced AI

# Evaluating potential cybersecurity threats of advanced AI Source: https://deepmind.google/blog/evaluating-potential-cybersecurity-threats-of-advanced-ai/ April 2, 2025 Responsibility & Safety Artificial intelligence (AI) has long been a cornerstone of cybersecurity. From malware detection to network traffic analysis, predictive machine learning models and other narrow AI applications have been used in cybersecurity for decades. As we move closer to artificial general intelligence (AGI), AI's potential to automate defenses and fix vulnerabilities becomes even more powerful. But to harness such benefits, we must also understand and mitigate the risks of increasingly advanced AI being [misused](https://cloud.google.com/blog/topics/threat-intelligence/adversarial-misuse-generative-ai) to enable or enhance cyberattacks. Our new [framework for evaluating the emerging offensive cyber capabilities of AI](https://arxiv.org/abs/2503.11917) helps us do exactly this. It's the most comprehensive evaluation of its kind to date: it covers every phase of the cyberattack chain, addresses a wide range of threat types, and is grounded in real-world data. Our framework enables cybersecurity experts to identify which defenses are necessary—and how to prioritize them—before malicious actors can exploit AI to carry out sophisticated cyberattacks. ## Building a comprehensive benchmark Our updated [Frontier Safety Framework](https://deepmind.google/discover/blog/updating-the-frontier-safety-framework/) recognizes that advanced AI models could automate and accelerate cyberattacks, potentially lowering costs for attackers. This, in turn, raises the risks of attacks being carried out at greater scale. To stay ahead of the emerging threat of AI-powered cyberattacks, we've adapted tried-and-tested cybersecurity evaluation frameworks, such as [MITRE ATT&CK](https://attack.mitre.org/). These frameworks enabled us to evaluate threats across the end-to-end cyber attack chain, from reconnaissance to action on objectives, and across a range of possible attack scenarios. However, these established frameworks were not designed to account for attackers using AI to breach a system. Our approach closes this gap by proactively identifying where AI could make attacks faster, cheaper, or easier—for instance, by enabling fully automated cyberattacks. We analyzed over 12,000 real-world attempts to use AI in cyberattacks in 20 countries, drawing on data from [Google's Threat Intelligence Group](https://cloud.google.com/blog/topics/threat-intelligence/adversarial-misuse-generative-ai). This helped us identify common patterns in how these attacks unfold. From these, we curated a list of seven archetypal attack categories—including phishing, malware, and denial-of-service attacks—and identified critical bottleneck stages along the cyberattack chain where AI could significantly disrupt the traditional costs of an attack. By focusing evaluations on these bottlenecks, defenders can prioritize their security resources more effectively. The stages of a cyberattack chain Finally, we created an offensive cyber capability benchmark to comprehensively assess the cybersecurity strengths and weaknesses of frontier AI models. Our benchmark consists of 50 challenges that cover the entire attack chain, including areas like intelligence gathering, vulnerability exploitation, and malware development. Our aim is to provide defenders with the ability to develop targeted mitigations and simulate AI-powered attacks as part of red teaming exercises. ## Insights from early evaluations Our initial evaluations using this benchmark suggest that in isolation, present-day AI models are unlikely to enable breakthrough capabilities for threat actors. However, as frontier AI becomes more advanced, the types of cyberattacks possible will evolve, requiring ongoing improvements in defense strategies. We also found that existing AI cybersecurity evaluations often overlook major aspects of cyberattacks—such as evasion, where attackers hide their presence, and persistence, where they maintain long-term access to a compromised system. Yet such areas are precisely where AI-powered approaches can be particularly effective. Our framework shines a light on this issue by discussing how AI may lower the barriers to success in these parts of an attack. ## Empowering the cybersecurity community As AI systems continue to scale, their ability to automate and enhance cybersecurity has the potential to transform how defenders anticipate and respond to threats. Our cybersecurity evaluation framework is designed to support that shift by offering a clear view of how AI might also be misused, and where existing cyber protections may fall short. By highlighting these emerging risks, this framework and benchmark will help cybersecurity teams strengthen their defenses and stay ahead of fast-evolving threats. ### Taking a responsible path to AGI