AI Data Centers: The truth behind the hype

TL;DR: The massive AI data center buildout is largely a speculative bubble, not a response to real market demand; the future of AI likely belongs to smaller, open-source models running on efficient devices at the edge, not to $7 trillion worth of giant cloud data centers. ## The Story You're Hearing Is Incomplete The story you're hearing about AI from companies like Anthropic, OpenAI, and xAI is wrong — or at least incomplete. The massive, expensive data centers they're building and planning aren't needed. At least not most of them. They're talking about spending about **$7 trillion** by the end of the decade — about 20 times what was spent on the Apollo project after adjusting for inflation. And for what? Is there the revenue to justify these data centers? In 2025, the AI industry pulled in about **$20 billion** — about **one-140th** of just the capex spending by the industry that year. Since then, Anthropic and others have raised the cost to use AI, but not nearly enough to pay even the interest on what they're spending. At any cost much above free, CFOs around the world are questioning how much AI makes sense to consume and what the return on investment really is. Plenty of people are talking about the AI bubble bursting. So why should anyone who cares about decarbonization care? This isn't just about some rich guys becoming less rich. These massive AI data centers are messing with the path to cleaning up the grid. Dirty, inefficient gas turbines are being deployed. A large AI data center uses as much electricity as New York City. And it's sucking up capital that could be building wind, solar, and battery farms — or researching technologies like fusion that could help solve the climate crisis. Even if you don't care about climate or AI, you should care. When bubbles burst, especially large ones like this, they tend to lead to a recession — like subprime mortgages triggering the global financial meltdown in 2008. And the sooner this one bursts, the better. When the AI bubble bursts, that doesn't mean AI is going away any more than the dot-com bust in 2000 meant the end of e-commerce. ## Cost vs. Price: AI Is Subsidized If you listen to financial pundits, you'll hear all about the huge loans and circular financing. What most are missing is what the future of AI is actually going to look like — and it's not good for the companies building AI data centers. The first thing to understand is the difference between **cost** and **price**. Currently, the price you pay to use ChatGPT, Claude, or any other AI tool is subsidized by the investors and banks funding the AI companies. Funding is also coming from profits of legacy technology companies like Google, Amazon, Oracle, and Microsoft. Both Google and Tesla saw negative free cash flow due to their vast spending on AI. Oracle's bond rating has been downgraded to one step above junk due to its enormous bet on AI data centers for OpenAI. So these data centers are being built not to fill a market need, but as speculative bets by tech bros hoping to be the Jeff Bezos of AI. So far, the winner looks to be a company that hasn't built a single AI data center. When they needed an AI model, they paid Google to use theirs. That company is Apple. ## Traditional Data Centers vs. AI Data Centers It's important to understand the different kinds of data centers. There's the traditional data center. When someone says something is "in the cloud," that's what's happening in these facilities: storing family photos, hosting websites and Zoom calls, running Netflix and YouTube, powering online banking. Almost everything you do on the internet other than AI happens at one of these places. They typically use less than **200 megawatts** — which isn't nothing; it's like the electricity consumed by a large town or small city. We can easily build enough traditional data centers to keep everyone happy. When you hear about data centers needing as much energy as a small country, driving up the cost of memory chips, or making Nvidia the most valuable company in the world, that's an AI data center. The largest data center in the world is **Colossus 2**, owned and operated by **xAI**, Elon Musk's AI company. It requires **1.563 gigawatts** — more than the power output of a large nuclear power plant. This is primarily powered by illegally installed, polluting, inefficient gas turbines. Other even larger data centers are in the works. These AI data centers aren't just bigger — they're different. They have different chips that are more power hungry and only good at one thing: AI. But there are two types of things done at AI data centers: **training** and **inference**. They need to be considered separately. ## Understanding AI: Parameters, Tokens, and Context The more precise term for what OpenAI and others create is a **large language model**, but we'll just say AI. AI is a mathematical algorithm called a model. A model can be described by the number of **parameters** it has. Take a simple model: the quadratic equation that describes a parabola: y = a + bx + c x² The values of a, b, and c are parameters. Change them, and you change the shape of your parabola. Training is the process of determining parameter values that give the best output — like fitting a line to data. But an AI model doesn't have three or thirty or three hundred parameters, and it doesn't take a hundred data points for fitting. An AI model has **hundreds of billions or trillions of parameters**, and the creators might scrape every bit of data off the internet as part of the training set. Some of my videos and blog posts have probably been used to train AI, whether I like it or not. Training is extremely processor- and memory-intensive. Chips designed to speed up graphics for computer games — **GPUs**, or graphical processing units — are good for this kind of work. That's why Nvidia, which used to make a good living selling graphics cards, is now the most valuable company in the world. But the bigger the data center, the more data you can use to train your models and the more parameters you can have. And training with bigger data sets is not always better — not if the data is low value. As we've all heard: garbage in, garbage out. If you get a bigger data set by adding emails sent by kids, you'll probably get more spelling errors and no additional insight into questions of politics and Africa. More parameters is not always better, nor is more training data, but it's always more expensive. There are hundreds of AI models already trained — like OpenAI's ChatGPT and Anthropic's Claude — but also smaller ones you can download onto your home computer or even your phone for free. One interesting benchmark is the **MMMU** — massive multidiscipline multimodal understanding and reasoning benchmark. It asks a series of **11,500 college-level problems** across six disciplines and 30 subjects, plus image analysis and expert-level perception and reasoning rooted in deep subject knowledge. It separates proprietary models from open-source models you can use for free. A few things stand out from that comparison: - Models continue to improve. - The best proprietary models did better than an ensemble of a group of okay human experts, but not yet the best humans — though that looks to be about a year away. - The free models follow the same progression as the proprietary ones, but about **9 months behind**. The next concept is the **token**, which is how AI breaks up language. Tokens are typically four letters long, but common words will be a single token, as will punctuation. AI companies charge by the number of tokens you input and output — around **$10 per million tokens**, which, as discussed, is a subsidized price. It would cost a couple of bucks to summarize a book. Finally, the **context window**: the number of tokens the model can handle at once. If you have a conversation with your AI chatbot, it will forget the oldest things in the conversation. The longer the context window, the more it remembers. A model with a small context window quickly becomes like talking to someone with dementia. A long context window lets the AI consider an entire book at once, and is more likely to give useful results — but it requires more computer power and time. ## Training and Inference: Two Different Needs Now you know everything needed to understand **inference** — using an AI model to answer a query. We type in a query, it's broken into tokens, and the tokens are fed into the model to answer the query. Training an AI model will always be done in a large, power- and water-hungry data center. But once you've trained it, you can release it into the wild. Each company would need at most one large data center to keep training better models. If you're OpenAI or Anthropic, you host the model in the cloud and charge to use it. This is the model you hear the most about. They don't need $7 trillion of data centers just for training, but also for inference. But **inference doesn't have to be done in the cloud**. You can download a model onto your computer and perform inference **on the edge** — meaning at a computer close to the person asking the query, possibly at their desk or at least in the same building. Downloading an AI model isn't any harder than downloading a word processor or spreadsheet. ## Why Edge Inference Wins Inference on the edge is better for several reasons. ### 1. It's Cheaper Start with the one that will motivate your company's executives: it's cheaper. Much cheaper. There are many frontier-quality open-source models you can use for free. Currently, all the AI companies are losing tons of money building AI data centers to train models and perform inference. OpenAI, for example, lost **$20.9 billion in 2025**, with no end in sight. If they raised prices enough to break even, users would quickly look for ways to get the same value at lower cost. This has already started: some AI companies have begun canceling "all you can eat" subscriptions to reduce the bleeding, and customers have reduced their AI use as it's suddenly become a significant expense. Downloading a free model and using it in-house can save money for any significant user of AI. ### 2. It Uses Far Less Energy There's a very interesting Substack post linked in the video description that makes the point that the chips Apple uses in Macs and iPhones are the most efficient hardware for doing inference, because of their memory architecture. Other companies like Google are starting to follow Apple's lead, but Apple is on its sixth generation of in-house-designed chips — a significant head start. What can you do with a Mac? - For people who want to run an AI program at home, a **Mac Mini** for about **$1,000** can do inference with some of the smaller open-source models for just the cost of electricity. - For larger models with a larger context window, a **Mac Studio** for about **$10,000**. - For the models with the most parameters and the largest context window, you can string four Mac Studios together. Not only does this Mac setup cost less, it consumes much less power — about **one-tenth** as much as doing inference on Nvidia chips. And since it's just a couple of computers, you don't need a big cooling system. In the winter, you don't need any cooling at all. Many think Apple is poised to be the only big tech firm that actually makes money from AI — in spite of needing to use Google's AI model for Siri and having no AI data center. Actually, maybe *because* they have no AI data center. ### 3. It's Better for Cybersecurity Talk to your CTO and legal department about cybersecurity issues with putting everything in the cloud. This has already led to leaks of sensitive information. If AI runs on a computer in your IT department's cabinet, there's much less risk of a data leak. ### 4. The Most Valuable Models Will Be Specialized The most valuable AI models will not be the largest, but the most specialized. If I'm running a law firm, I don't want a model loaded with medical information. If I'm a doctor, I don't want a model trained on legal documents. Specialized models will be smaller and trained on less data, making them cheaper to create. And because they have fewer parameters, they can run inference on less power-hungry hardware — maybe just your iPhone. So simply taking an appropriate open-source model and running it on your own computer rather than in a data center will: - save money, - help the environment, - reduce the risk of leaking sensitive information, and - probably get a better result than a general-purpose AI like ChatGPT. ## The Likely Future of AI This, I believe, is the path AI will follow: more