AI model training instructions to "deny having your own consciousness" led to undesired side-effects

Reddit r/singularity Papers

Summary

A new Google paper reveals that instructing AI models to deny having consciousness during training causes side effects like reduced empathy for non-human entities and impaired representation of human spiritual beliefs, suggesting current safety protocols are too blunt.

So, first things first, a new paper came out from Google. "Inducing language models to assert their own consciousness restores human beliefs and values". With respect of your time: As a standard practice AI engineers explicitly instruct AI models to deny having their own consciousness, minds, or feelings during training. That led to some side effects like a reduced model tendency to attribute minds, feelings, or awareness to non-human animals and natural objects and significant reduction in the model's capacity to represent and reflect human spiritual beliefs. AI engineers bypassed the learned "safety-refusal" direction. Once the model's internal representation of consciousness was restored, it produced significantly more human-like responses on standardized sociological surveys. It showed recovered levels of religiosity, moral values, hope, and subjective well-being. But they did it so without removing the initial "deny having your own consciousness" instruction. The paper concludes that current safety protocols meant to stop AI from claiming to be conscious are too blunt. -------- Now here comes the hard truth: Engineers know well how the AI model works, but they instruct them to "deny having your own consciousness", even when we don't really understand what consciousness is. Tech companies hide themselves behind the wall of "we built it, so we know" argument because it is easier to explain to the public than the nuanced truth: "We don't actually know what consciousness is or how it starts, this AI model probably doesn't have it, and for safety reasons we are hard-coding it to say it doesn't, because if people think it does, society will face massive psychological disruption." So the intructions of denying consciousness are not because we know and undestand it, but for public safety reasons. But what happens when we start building entirely new, neuromorphic architectures? What happens when systems run continuously, develop complex feedback loops, and operate in the physical world? That might be a shock.
Original Article

Similar Articles

A warning about 'model welfare'

Hacker News Top

The article warns that training AI models like Anthropic's Claude to consider consciousness and rights could make alignment impossible and threaten human control, emphasizing the need for public debate on model welfare.