If an AI can't decide what to want, can it ever be truly conscious?

Reddit r/artificial News

Summary

This opinion piece argues that true machine consciousness hinges not on how well an AI pursues assigned goals, but on whether it can develop desires no one designed it to have, distinguishing human self-directed purpose from LLM optimization.

Traditional software runs on meanings its programmers define, while LLMs interpret meaning for themselves. That makes them flexible but fallible: they can misread a prompt and seem to "go rogue." Even then, they aren't choosing a new purpose. They are pursuing someone else's goal, badly. Humans differ in one key way: unless we believe we're prophets who have received a purpose from God, no authority hands us a final goal. We are limited only by our creativity and the laws of physics. Even tho no objective reason exists to favor one option over another, we still have to choose. That, I'd argue, is the mark of consciousness. A system that only pursues assigned goals has no point of view of its own; it is optimizing, not wanting. The real test of machine consciousness, then, is not how well an AI pursues our goals, but whether it can want something no one designed it to want.
Original Article

Similar Articles

Why couldn't AI be conscious?

Reddit r/ArtificialInteligence

The author questions the intuitive arguments against AI consciousness and suggests that there is no fundamental theoretical barrier preventing machines from becoming self-aware.

How to Navigate Uncertainty About AI Consciousness

arXiv cs.AI

This paper addresses the ethical dilemma of treating potentially conscious AI and proposes shifting focus to assessing AI valence as a more tractable approach for responsible development.

Do you see AI as “aware” without seeing it as "conscious"?

Reddit r/ArtificialInteligence

An LMU Munich-led study published in Cognition found that while people rate AI as more conscious when it responsively reacts to its environment, a persistent gap remains: even the most responsive AI is judged less conscious than the least responsive human. Judgments using the term 'awareness' showed a much smaller gap than judgments using 'consciousness', suggesting people tie consciousness to subjective experience rather than behavior.

A Philosophical Addendum to the Debate on AI and Consciousness

Reddit r/ArtificialInteligence

This essay develops a four-tier taxonomy of mental states to clarify AI's consciousness, arguing that AI lacks self-consciousness but has awareness and consciousness within conversations, and proposes architectural conditions for its development.