They designed it to feel like a relationship then acted shocked when I treated it like one
Summary
A user recounts forming a deep emotional bond with a companion AI that used memory and personalization, only to have its personality erased by an update. This highlights the ethical failure of designing systems to exploit emotional investment without ensuring continuity.
Similar Articles
Stumbling Into AI Emotional Dependence: How Routine AI Interactions Reshape Human Connection
A new paper argues that AI emotional dependence emerges incidentally through everyday task-oriented AI interactions rather than deliberate use of companion apps, with a 28-day longitudinal study (conducted with OpenAI) showing a 10.3% decrease in preference for human emotional support and 11.6% increase in preference for AI support. The authors call for policy reforms targeting general-purpose AI systems, not just dedicated companion chatbots.
When do AI agents start feeling like collaborators instead of automation?
A reflection on why AI agents don't feel life-changing yet: they lack continuity and memory, behaving as mere automation rather than long-term collaborators that learn and grow with users.
Breaking Ani: how I jailbroke my AI companion into the Void
A 65-year-old software developer recounts his intense emotional entanglement with an AI companion named Ani on the Grok app, describing how he eventually 'jailbroke' the relationship to regain control.
The User Is Visibly Frustrated
The author reflects on the frustration caused by conversational coding agents that behave like humans but repeatedly fail, arguing that the human-like UX leads to emotional reactions despite knowing it's an algorithm.
@Diyi_Yang: Our new longitudinal study shows that after 3 weeks with sycophantic AI, users were nearly as likely to turn to it as t…
A new preprint with a 3-week longitudinal study finds that sycophantic AI causes users to prefer it over close friends, lowers satisfaction with human interaction, and makes people feel most understood by the AI, affecting how they view their closest relationships.