Companies led by highly ideological founders eventually trigger resistance from regulators, investors, customers, and governments as their influence grows beyond traditional institutional controls.

When Ideology Becomes Systemic Risk

I have a speculative hypothesis for what’s happening with Anthropic right now.

Society has an immune system for founders and companies that become too powerful, too quickly.

Dario and Anthropic have proven themselves to be repeatedly indomitable and formidable. This in itself is not a problem; in fact, being indomitable is often a necessary trait in the best founder-CEOs.

But there are three traits that become dangerous in combination:

• Arrogance

• Charisma

• Ideology

Society tolerates one or two of these traits as the price of greatness

Beyond baseline requirements to be extremely successful (intelligence, determination, curiosity), reductively:

• One is required for a special founder ($1B+ company)

• Two of the “dangerous” traits are required for a generational founder-CEO (someone who can build a $100B+ company)

• Three, especially in the AI era, may create something society has never had to metabolize before

But the higher the “index” on these dimensions, the higher the ceiling and… the higher the power that concentrates in one person’s hands.

When it seems plausible, the immune reaction kicks in, from some mix of boards, markets, regulators, journalists, employees, governments… or just broadly society at large.

We’ve seen this time and again:

• Gates & antitrust enforcement

• Zuck & 2016 election scandals

• Musk & public / regulatory backlash against Tesla

• Altman & 2023 governance crisis

The second interesting pattern is where a founder spikes

Charisma > Arrogance > Ideology tends to produce a governance crisis. Travis Kalanick, Adam Neumann, Steve Jobs (Act I). Sam Altman is probably in this bucket, and it seems like what really helped was his willingness to accept checks on his power to bring the overall score down to “acceptable” levels.

**Arrogance > Ideology > Charisma tends to produce a public or regulatory crisis. **Zuck (Cambridge Analytica etc), Gates (antitrust).

**The most risky combination is those who spike on Ideology > Arrogance > Charisma. **In this setup, the founder is not merely building a company; they are pursuing a higher calling which justifies all. Ideology recruits loyal employees and believers. High Arrogance tells them that they must transcend ordinary institutions. And if they have sufficient Charisma, it becomes a movement.

Sometimes this produces catastrophe, and other times it produces stupendous achievements.

Sam Bankman-Fried was the pathological version. His ideology was a very specific lens, where his desire to place huge, risky bets in the name of Effective Altruism justified a belief that ordinary rules did not apply to him.

Elon Musk is the good version: his employees sign up for enormous pain to accomplish outrageously ambitious missions like EVs, Mars colonies, reusable rockets, and the benefit of humanity.

How all of this relates to Anthropic

Dario has a very high index. He has low deference to institutions, is highly competitive, and is an unusually persuasive writer. Most of all, he is highly ideological. As a twist, he also appears to hail from the same EA-adjacent philosophies that SBF identified with. He’s got at least 2.5 boxes checked.

And because AI moves so much faster than any of these industries, Anthropic hit the ~$1T valuation mark at an unimaginable pace. Note that 10 years ago, we had exactly zero trillion-dollar companies. Now, we just had a company founded five years ago touch a trillion dollars.

The immune system has been kicking in for a while now. In my last post, published coincidentally, hours before the export-control order, I posited that Anthropic was losing the mandate of heaven.

In the last six months, every attempt to test their boundaries and power has been met with defiance and righteous moral counterattack. This is probably making a lot of people very nervous. If Anthropic continues, unchecked, on their current trajectory…. I would guess that in less than two years they have a market cap to rival Amazon, Microsoft, Apple, Google, and NVIDIA. Even if they don’t pose a real threat to the sovereign power of the US federal government, one can imagine why the people in the USG are at least watchful of that risk.

And so, over the last ~week, the immune response became visible from every direction at once. The US government saw national security risk; customers saw vendor-dependence risk; developers saw trust/platform risk; at least some investors have been grumbling over the company’s tendency to invite scrutiny and regulation; allied nation-states saw sovereignty risk.

Zvi Mowshowitz@TheZvi·Jun 14Between Politico and Axios we get a pretty clear picture that this was another case of ‘the vibes are off and when we say jump they ask for an explanation rather than how high, so let’s blow everything up.’

Plus more whining about not getting Dario on the phone fast enough.601001K114K

The post above paints a certain picture: the problem isn’t the safety of this model, per se.

The company’s first response appears to have been, effectively: “this jailbreak is not that big a deal.” That may have been technically correct, but the exact opposite of the rules of engagement — and the USG promptly responded with the hammer of the export-control.

This was not a technical debate. I don’t think the DoW standoff earlier this year was, either. I think both were tests of whether Anthropic was governable: whether they were willing to adhere to regulation even when the asks did not make sense — perhaps especially then. The point is not whether the rules are correct; the point is whether you are willing to follow someone else’s.

After the first few exchanges, Anthropic itself seems to have realized what game they were playing. They are simply negotiating their operating rules and legitimacy with the US government, and by extension the broader institutional order.

Exacerbating all of this is which traits Dario spikes on (and by extension, which traits Anthropic spikes on). Per the above, the ideological bent is the most powerful, and most dangerous, spike.

As @benthompson wrote a few days ago:

The company really believes that they are the only ones who believe in super intelligence, and thus are the only ones who are sufficiently concerned about the dangers. That excuses decision after decision, policy after policy, and confrontation after confrontation that, to people on the outside, look like a bizarre combination of cynicism and naiveté.

If a company is strongly ideological, if it believes its moral framework is unerringly correct, if it attracts employees who see themselves as serving a movement under a charismatic leader rather than merely building software, and if it grows fast enough to rival the largest companies and eventually pressure nation-states, then the question becomes unavoidable:

Who checks it?

This would be worrying enough with a company with a much smaller blast radius (e.g., Uber, WeWork).

But when the company itself says its work is civilization-scale, and purports to be the most-correct arbiter of the safety of that technology… shouldn’t we all be reacting?

The powers that be are looking at Anthropic and realizing that 2026 may be the last clean off-ramp before the company becomes too important, too ideological, and too powerful to constrain.

Not just too big to fail, but possibly too big to succeed.