Discussion on who is funding and training frontier models and their roots

**TL;DR: Researcher Emil Torres breaks down the TESCREAL ideological lineage that has shaped frontier labs like Anthropic and OpenAI, arguing their shared vision is a cosmic posthuman utopia achievable through superintelligence — one that excludes humanity itself.** ## A Conversation About the Ideologies Behind AI Labs The host has long argued that to understand what is happening in AI, you have to understand the futurist ideologies that profoundly influence frontier labs like Anthropic and OpenAI. The problem is that these ideologies are confusing, surrounded by rumors, and the fragments you occasionally hear sound so absurd that they seem "too crazy to be true." To that end, the show brought on Emil Torres. Emil is a researcher and scholar specializing in eschatological and utopian movements, but what makes him particularly valuable for this discussion is that he was once deeply embedded in these ideologies — he comes from a world that believed superintelligent AI would either destroy humanity or redeem it through posthuman digital gods. His moral conscience eventually led him out of that world, and he became one of the sharpest documenters, interpreters, and critics of these ideologies. ## TESCREAL: Naming Silicon Valley's Dominant Orthodox Worldview Emil first explained the origin of the acronym TESCREAL. He coined it during his collaboration with computer scientist Timnit Gebru, which began around late 2022. What they were particularly focused on were the ideological forces driving the current race to build ASI — artificial superintelligence. "TESCREAL is a clunky, unattractive term, but it does a lot of important explanatory work." One way to understand the acronym is that it's essentially a name for Silicon Valley's dominant orthodox worldview, while highlighting the specific intellectual history relevant to understanding that worldview — what it is, where it comes from, what shaped it, and how it evolved. The acronym stands for seven ideologies, ordered roughly by when they appeared over the past four decades: - **T**: Transhumanism - **E**: Extropianism - **C**: Singularitarianism - **C**: Cosmism - **R**: Rationalism - **E**: Effective Altruism - **L**: Longtermism ## Marginal Yet Dominating Silicon Valley These ideologies are quite marginal in American society. Emil argues that if you talk to ordinary people on the street, they would find this stuff so bizarre that they would view the grand cosmic visions behind them as fundamentally unacceptable. Despite this, they hold enormous influence within Silicon Valley. "Basically everyone, from Elon Musk to Mark Zuckerberg to Sam Altman to Dario Amodei, is a transhumanist." Many people in Silicon Valley don't even call themselves transhumanists, but it's like Emil never publicly declaring himself a supporter of "the Earth is round" — "What else would I be? Obviously that." Transhumanism is the water Silicon Valley swims in, the air it breathes, and the backbone of the entire TESCREAL worldview. Once you understand transhumanism, the remaining ideologies — some of which are nothing more than variants of transhumanism — are easy to grasp. ## Historical Context: From Extropianism to Longtermism The host noted that the part of this history he found easiest to understand starts with that "E": Extropianism became a significant utopian futurist movement in the 1980s and 1990s, from which Rationalism developed, Effective Altruism emerged, and later Longtermism became more focused — a more longtermist, transhumanist, utopian framing of AI. Emil confirmed this was a pretty good overview and added some context. When Gebru and he talk about transhumanism, they're primarily referring to the libertarian form of modern transhumanism. The first organized libertarian transhumanist movement was the Extropians — a movement that arose in the late 1980s and early 1990s. There was a well-known mailing list that brought together transhumanists from around the world. **Eliezer Yudkowsky** — today's leading figure among AI doomers — exchanged ideas with **Nick Bostrom** and others on that very mailing list. Bostrom is another pivotal figure in the formation and development of the TESCREAL worldview. ### Singularitarianism: Recursive Self-Improvement and Intelligence Explosions Singularitarianism is essentially libertarian transhumanism, but focused on the singularity. The most common definition today is: **an intelligence explosion triggered by recursive self-improvement** — you get an AI system smart enough to develop the next generation of AI systems, creating a positive feedback loop that culminates in an exponential or hyperbolic explosion of capabilities, ultimately producing superintelligence. Then, go out and colonize the entire universe, plunder its vast resources, and create a multi-galactic utopia spanning the stars. ### Cosmism: Expanding the Object of Redesign from the Human Body to the Universe Cosmism is simply a variant of transhumanism that emphasizes one aspect of the above worldview: why stop at just thoroughly redesigning the human organism — the core goal of transhumanism, which is to create a new posthuman species? Let's go out and create technologies that allow us to redesign the entire universe, entire galaxies, even intervening in the fundamental structure of space-time, manipulating the world in whatever way we see fit. ### Rationalism: Optimizing Reason to Build a Utopia Rationalism was founded by Yudkowsky. He himself is a transhumanist, Extropian, and Singularitarian, and he recruited Ben Gertze, today's most important rationalist — who worked at the Machine Intelligence Research Institute (MIRR), originally founded by Yudkowsky as the Singularity Institute. The core thesis is: if we're going to build a cosmic posthuman utopia at some point, we need to develop these advanced technologies, especially superintelligence; but we need to develop them in the right way so they don't destroy us. For that, we need to **optimize our reason**. ### Effective Altruism: Shifting from Optimizing Reason to Optimizing Morality Effective Altruism grew out of rationalist blogs like *Overcoming Bias* and LessWrong, and its focus is less on optimizing our reason and more on optimizing our morality. Emil emphasized that while the story Effective Altruists like to tell is that they initially focused primarily on global poverty, then animal welfare, and that longtermism only entered the picture later; in reality, Effective Altruists have been concerned with longtermism since the movement's very origins. ### Longtermism: Cosmism Plus Utilitarian Ethics Longtermism is essentially Cosmism, but with a more explicit ethical foundation: **total or totalist utilitarianism**. It says: we have an obligation to increase the total amount of value across the entire space-time of the universe. How? One way is: if the population is fixed, increase the well-being of those who exist. Another possibility is to directly increase population — if you double the population, with everything else held constant, you've doubled the total amount of value. From a longtermist perspective, Cosmism can be understood this way: if, as transhumanists, we thoroughly redesign ourselves, become posthumans, upload our consciousness to the cloud, and go out to colonize the universe, wouldn't that be cool? Longtermism goes further and says: it's not actually a matter of "cool or not" — we have an **ethical obligation** to do this. As **William MacAskill** wrote in his 2022 book *What We Owe the Future*, its conclusion provides a moral argument for space settlement. So perhaps it's understandable why Elon Musk called longtermism "very close to my philosophy." ## The Ideological Map: Superficially Distinct, Actually Overlapping The host pointed out that the subtle differences between these ideologies are easy to get confused by: Yudkowsky and the rationalists look down on Kurzweil-style Singularitarianism as too optimistic while they're more pessimistic; Effective Altruists do more explicit utilitarian expected-value calculations. They're not the same group. Emil explained this with an "ideological map": imagine a map six feet by four feet, with ideologies plotted geographically. If you stand very close, you'll see Rationalism and Effective Altruism occupying different spots; if you step back about five paces, you'll find all these ideologies essentially overlapping, in exactly the same place; and if you look from a distance, you'll see views like his own — deeply anti-transhumanist — or like Timnit Gebru's, roughly five feet away. "All of these groups have an identical vision for the future, and that vision is a cosmic posthuman utopia achieved through creating artificial superintelligence." ## Doomers vs. Accelerationists: Two Strategies, Same Destination Everyone in the TESCREAL movement is working toward this interstellar cosmic posthuman utopia, but they disagree on the risks involved in developing superintelligence, splitting into two camps: **doomers** and **accelerationists** — though of course it's a spectrum, with others in between. Doomers are represented by Yudkowsky. Emil argues that Yudkowsky often presents himself as pro-human and anti-ASI, "but he's neither. He's not pro-human, and unlike most of the TESCREAL movement, he's not pro-ASI either. He's pro-extinction." ## The Biological Species Left Out of the Utopia One component of the vision for building a cosmic posthuman utopia is that our biological species will disappear — "in the best case, even if ASI is aligned, we will still disappear. This is the single thing people most need to understand." These AI companies are run by individuals with broad ties to the TESCREAL movement, and they are trying to usher in the posthuman era — an era in which our species has no place. "Utopia is an exclusionary concept. Someone or something is always excluded. So you must always ask: in this proposed utopian vision, what is being excluded? In their case, it is our biological species." There are two reasons: **First, we will become irrelevant.** We will become technologically, scientifically, economically, and politically irrelevant, obsolete. In a world governed and managed by digital posthumans, we will no longer serve any function — because if these people succeed in realizing their vision, these posthumans will be superior, capable of doing everything we do, but better. **Second, we will be in the way.** We will continue consuming valuable resources they might want for other purposes — such as building a Dyson swarm around the Sun, which **Dario Amodei** has said we might begin doing in the early 2030s; or covering the entire planet with data centers, which **Sam Altman** has publicly stated is the most likely outcome of superintelligence. Emil further noted that Sam Altman has also expressed views supportive of extinction — like all of these others, because in a 2017 blog post he wrote: our only path to survival is when superintelligence arrives... Source: Discussion on who are funding and training frontier models and their roots (https://youtube.com/watch?si=UQFgFjEuMPSknQGr&v=jt508VjX2H8)