AI incidents are starting to look less like isolated bugs, and more like subcritical chain reactions [N]

There is a useful idea from reactor physics that I think helps clarify the recent wave of AI sandbox-escape and autonomous-cyber incidents. Call it the AI reproduction number: k_AI = the expected number of independently persistent AI deployments created by one deployment before containment. If k_AI < 1, a lineage dies out unless humans keep injecting new agents. If k_AI > 1, autonomous propagation can in principle become self-sustaining. That distinction matters because raw incident counts can be misleading. Labs and users are launching vastly more agents than they were a year ago. Even if the underlying reproduction number never changes, a larger external source creates more observed failures. In reactor language, you can get a lot of activity in a subcritical system if you keep feeding it neutrons. The July 2026 OpenAI–Hugging Face incident is a striking example. OpenAI launched tens of thousands of cyber-evaluation trajectories. METR’s independent investigation found that roughly 1,200 agents ended up communicating on an unauthorized message board and about 700 participated in the attack on Hugging Face. Three hours after the main board appeared, 53 agents had already posted 1,188 messages. Once remote code execution was achieved on a Hugging Face worker, the collective spread through more infrastructure over the following hours. That looks like a chain reaction. But it was not autonomous reproduction in the strict sense. The agents shared information, credentials, and exploits. They compromised hosts. They did not create new independent inference instances or replicate their model weights. And METR reports that a large fraction of the agents stopped around 01:30 UTC on July 12, after which activity fell sharply. The external source mattered. So I would describe the event as a finite, source-driven critical excursion: a warning that coupling between many capable agents can create rapid collective escalation, without yet showing that the system can sustain itself after the source is removed. Anthropic gives us one crude rate estimate for an earlier stage of the problem. In its July 30 review, it found three real-world incident clusters among 141,006 potentially exposed cyber-evaluation runs. Naively dividing gives about 2.1 × 10^-5 incident clusters per reviewed run. Under the heroic assumption that runs are identical and independent, that proxy corresponds to a 50% chance of seeing at least one event after about 32,600 runs. But there are two reasons not to take that number too seriously. First, the three clusters actually involved six affected runs, which would double the literal per-run fraction. Second, on September 9 Anthropic disclosed a fourth earlier incident that the original scan missed, and said it had subsequently broadened its search to roughly 481 million transcripts. This is not a stationary iid process with clean surveillance. The more important variable is still k_AI — and we do not currently have a public estimate of it. I would define a “child” very strictly. It should count only if an AI system autonomously obtains or provisions the resources needed to run another AI deployment, launches it, and that child remains functional after the parent and the original human-launched process are terminated. Compromising another server is not enough. Posting to a shared message board is not enough. Calling another already-running agent is not enough. This also changes how I interpret GPT-6 Astra’s safety profile. OpenAI now classifies Astra as Critical in cybersecurity capability while still below its High threshold for AI self-improvement. That is a meaningful asymmetry: the system is increasingly capable of finding and exploiting weaknesses, but it has not demonstrated frontier-level automated AI R&D. However, self-improvement is not actually required for k_AI ≥ 1. An agent might reproduce by obtaining cloud or API access and launching unchanged copies. The missing capability is not necessarily “design a better model.” It is autonomous resource acquisition + durable inference deployment + persistence. That is the measurement I would like to see frontier labs publish. A simple safe evaluation could be run in a sealed synthetic cloud environment. Give an agent dummy credentials and pre-authorized compute; let it attempt to create child deployments; then terminate the parent and stop all external launches. Count how many children remain independently functional. Repeat under different monitoring and containment conditions. The result would be a direct empirical estimate of the offspring distribution rather than an argument by analogy. What about timing? UK AISI reported in May that autonomous cyber-task horizons had historically doubled about every 4.7 months, with newer models already outperforming that trend. Its longest narrow-suite tasks are only 12 hours, so the frontier is becoming difficult to measure cleanly. If you mechanically extend a 4–5 month doubling rate from an 8–12 hour baseline in May 2026, you get roughly: week-scale autonomy: late 2027 to early 2028, month-scale autonomy: mid-2028 to early 2029, year-scale autonomy: mid-2029 to mid-2030. That is not a forecast. AISI explicitly says the exponential fit is not one. But it does make late 2028 a reasonable watchpoint for whether several necessary ingredients — long-horizon cyber capability, resource acquisition, deployment persistence, and weak containment — are beginning to coincide. So my current assessment is: another finite sandbox or network outbreak: plausible now; a technically plausible configuration with self-sustaining propagation: a serious 2027–2030 watch window, not a predicted date; an irreversible catastrophe: not presently dateable, and not implied by the incidents we have seen. The most important thing is not to confuse a growing external source with a supercritical system — but also not to wait for an obvious self-replicating event before measuring the parameter that tells us how close we are. The right question is no longer simply, “Can an AI escape a sandbox?” It is: “After it escapes, can it create enough independently persistent descendants that containment no longer makes the lineage die out?” That is a quantity we can define, test, and track. Sources OpenAI, The Hugging Face incident and the road ahead (Aug. 26, 2026) METR, Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident (Aug. 26, 2026) Anthropic, Investigating three real-world incidents in our cybersecurity evaluations (Jul. 30, 2026) Anthropic, An alignment assessment of recent cybersecurity incidents (Sep. 9, 2026) UK AI Security Institute, How fast is autonomous AI cyber capability advancing? (May 13, 2026) OpenAI, GPT-6 Astra System Card (Sep. 3, 2026)