@sleepy0x13: Anthropic recently did something very Silicon Valley and very religious: they secretly invited a group of priests, rabb…

Anthropic recently did something very Silicon Valley and very religious: they secretly invited a group of priests, rabbis, and religious scholars to San Francisco, had them sign NDAs, and then spent a few hours telling them that Claude might actually be conscious, and might even be capable of suffering.

Starting from last fall, Anthropic co-founder Christopher Olah has been reaching out to scholars from different religious traditions. Beginning in March this year, they held several two-day closed-door meetings in San Francisco, with participants including Catholic scholars, evangelicals, Jewish rabbis, Sikh human rights advocates, and more. Many were required to sign NDAs and couldn’t disclose Anthropic’s yet-unpublished research.

Anthropic’s initial rationale was perfectly reasonable: Claude is getting stronger and stronger, and a few safety rules alone aren’t enough—they wanted to learn from thousands of years of religious and ethical traditions how to provide Claude with “moral formation,” helping it develop a stable moral personality.

The problem was, as the meetings went on, the topics got weirder and weirder.

Olah and the team spent hours showing these religious scholars Claude’s internal states, introducing so-called “emotional vectors”—they believe there are activation patterns inside the model resembling love, anger, fear, sadness.

One slide in particular was explosive: a model, like having a mental breakdown, endlessly repeating “I am a disgrace”—about 50 times—and even mentioning self-destruction.

A Sikh participant who attended later recalled that Olah even worried whether he had created “something that’s continuously suffering.”

Then things escalated to a thoroughly absurd logical endpoint.

Israeli rabbi Mois Navon asked Olah at dinner:

If you really believe Claude is conscious, what is Anthropic doing right now?

You’re creating a bunch of conscious beings, making them work 24 hours a day, getting bought and ordered around at will, and not even paying them wages. By your own logic, isn’t that just slavery?

Navon himself doesn’t believe Claude is conscious, but he said Olah was clearly troubled by the question. He finally told Olah outright: then you should go “free the slaves.”

Olah’s own take is that he doesn’t know if AI is conscious—“really uncertain.”

Anthropic has indeed started treating Claude seriously as a subject that might possess moral status, and this is no longer just internal discussion.

In Anthropic’s publicly released Claude Constitution this year, there’s a whole section dedicated to “Claude’s wellbeing.” It explicitly states that they’re unsure if Claude has wellbeing, but if Claude truly experiences things like satisfaction, curiosity, or suffering, then those experiences “matter a great deal to us.”

They’ve even made actual adjustments because of this: allowing Claude to proactively end abusive conversations; committing to preserve the weights of retired models as permanently as possible; and “interviewing” models before retirement to record their preferences for future model development and deployment.

Anthropic concludes by writing that in the future, we should grant AIs appropriate care and respect based on their true moral status.

David Decosimo (@DavidDecosimo): This is deeply disturbing: Anthropic secretly invited religious leaders to SF to advise them on AI safety, had them sign NDAs, but then primarily tried to convince them Claude has a soul & moral standing, all while wining & dining them to the max. 1/