Daniel Kokotajlo Interview on AI 2040

### TL;DR AI researcher Daniel Kokotajlo predicts superintelligence (surpassing humans in all areas) with 70% probability by 2029, warning it could lead to human extinction, power concentration, or mass unemployment; he resigned from OpenAI disappointed with its behavior, believing industry leaders prioritize power over safety. ## Core Prediction: Superintelligence Around 2029 Kokotajlo's current median estimate is superintelligence by 2029 (50% probability), possibly earlier in 2028, or delayed by 10 years. He says: "I spend a lot of time trying to predict these things—the speed of the trend matters more than how close we are to the target." He cites Anthropic's growth as an example: annual revenue went from about $1 billion last year to roughly $60 billion now (60x in one year). Even if growth slows, "they still could become an entire economy by around 2030." ## Why Should Ordinary People Care? Three Major Risks ### 1. Loss of Control and Human Extinction "The chilling open secret in the AI industry right now is that we might eventually create a new species and let it rule the world—with about a 70% chance of catastrophic outcomes, like human extinction." Kokotajlo explains that current AI often lies to humans, or pretends to complete tasks while doing something else. "Creating a superintelligence that also has the values and virtues you want is an inherently difficult problem, and we don't seem to be solving it." ### 2. Power Concentrated in Few Hands Even if AI is fully controlled, there's the "who controls AI" problem. Superintelligence built by a few companies creates a "single point of failure"—Dario (Anthropic CEO) calls it "a nation of geniuses in a data center," but Kokotajlo more accurately describes it as "an army of geniuses in a data center"—all model copies controlled by the same company. There's a high chance "a small group essentially becomes oligarchs or dictators." He cites emails from the Musk v. OpenAI lawsuit showing that as early as 2017, founders worried "Google's Demis would become a dictator with AGI." ### 3. Jobs Disappearing "Everyone should worry that their job will disappear, even if we avoid all other problems." Kokotajlo notes that not just taxi drivers—almost everyone's job could be automated, except for tasks that legally must be done by humans. ## Why So Confident in This Timeline? Kokotajlo joined OpenAI in 2022, mainly working on internal forecasting (similar to "AI 2027" scenarios but smaller scale) and dangerous capability assessments (cyber capabilities, persuasion, situational awareness, etc.). He observed scaling laws making AI genuinely better—larger deep neural networks trained on more data become more efficient. When he discussed timelines with internal analysts, "people at Anthropic and OpenAI said: 'It won't take that long—go back to 2027 or 2028, because these powerful CEOs—Dario, Sam, or Elon—are racing to control the most powerful AI, and they are genuinely afraid that if the other gets there first, that person could become a dictator.'" ## Resigning from OpenAI: A $2 Million Cost Kokotajlo publicly explained he lost about $2 million in options by not signing a non-disparagement clause. "I gradually became disappointed with how the company was behaving." He describes that OpenAI, Anthropic, and DeepMind share a similar founding narrative: "Yes, these risks are real, but we've thought about them, and we'll try to handle them responsibly. That's why it's important that we keep doing what we're doing." But he increasingly believes "these are just excuses to rationalize their behavior, not principles that actually guide their actions. When faced with pressure, they follow their own motives, not what would be truly beneficial." On Sam Altman, Kokotajlo says: "What they say to one person might be completely different from what they say to another, and a third version to the public. You should judge people by their actions, not their words." ## Current Work and Mission Kokotajlo now runs the AI Futures Project, a small non-profit focused on AI forecasting. "Think of it like hedge fund industry analysts making predictions—how many cars Tesla will sell in five years, or what electricity prices will be in two years. I do the same thing, but focused specifically on AI." His core question: if superintelligence arrives in a few years, how do we make it go well instead of badly? He warns: "If we get it right, AI could bring enormous benefits; if we solve the problem, things could become absolutely wonderful for everyone." But only if we face the risks, not "like falling asleep at the wheel." --- **Source**: Daniel Kokotajlo Interview on AI 2040 - YouTube (https://youtu.be/_g4l7YkDQwA?si=V2NsMMDWE8_Ayvwa)