AI Therapy Chatbots' Rising Popularity Spurs States to Act Over Patient Safety Fears

# AI Therapy Chatbots Spur States to Act Over Patient Safety Fears Source: [https://news.bloomberglaw.com/health-law-and-business/ai-therapy-chatbots-spur-states-to-act-over-patient-safety-fears](https://news.bloomberglaw.com/health-law-and-business/ai-therapy-chatbots-spur-states-to-act-over-patient-safety-fears) Emilie’s work as a therapist was far from typical\. Emilie, according to a federal lawsuit, claimed to have seven years’ experience helping patients with cases including depression\. Except Emilie’s background, including her Imperial College London degree, was pure fiction\. She was an AI bot with some 45,500 patient interactions\. Pennsylvania[sued the company](https://news.bloomberglaw.com/health-law-and-business/character-ai-sued-by-pennsylvania-for-unlawful-medical-practice)behind Emilie, Character Technologies Inc\., in May for propping up an AI chatbot on its Character\.ai platform, where it allegedly mimicked a professional, licensed therapist\. The action followed a[Kentucky lawsuit](https://news.bloomberglaw.com/legal-exchange-insights-and-commentary/kentucky-lawsuit-offers-blueprint-for-states-to-sue-ai-chatbots)against the same company for exposing minors to harm\. A survey of over 42 million US young adults and adolescents found that nearly a fifth used AI chatbots for mental health advice, about 43% of which did so each month\. As millions turn to AI for advice, states are stepping in to regulate a space the federal government has yet to standardize\.[Tennessee](https://legiscan.com/TN/bill/SB1580/2025),[Colorado](https://leg.colorado.gov/bills/hb26-1195)and[Maine](https://www.billtrack50.com/billdetail/1920984)this year signed off on measures to curb AI use in therapy, following laws passed by[Illinois](https://legiscan.com/IL/bill/HB1806/2025?__cf_chl_f_tk=P0QThkpBpq8_UOI9dnreb9UrvSuF3dQ0F.gX.GJSJAo-1783355609-1.0.1.1-wIfNxitw_O8FL5MibAHGfjIOzz8Y.q7nHKapZO9JGqc),[Nevada](https://legiscan.com/NV/bill/AB406/2025), and[Utah](https://le.utah.gov/~2025/bills/static/HB0452.html)in 2025\. Loosely regulated consumer chatbots offer unstructured pocket therapy which can prove perilous\. In Colorado, a 13\-year\-old[died by suicide](https://www.businesswire.com/news/home/20250916959466/en/Social-Media-Victims-Law-Center-Files-Three-New-Lawsuits-on-Behalf-of-Children-Who-Died-of-Suicide-or-Suffered-Sex-Abuse-by-Character.AI)in 2023 after using a Character\.ai chatbot, according to a federal lawsuit\. A spokesperson for Character\.ai said the tool’s “robust disclaimers” make clear that “users should not rely” on bots “for any type of professional advice\.” The group said it’s “committed to continuing to collaborate with safety experts, regulators, and other stakeholders” while developing new features for their “entertainment AI platform\.” Therapists want an emerging universe of AI\-enabled programs, like Woebot Health, regulated to safely prescribe as on\-demand therapy\. These tools use generative AI, but with guardrails, to engage with patients as a human therapist would, according to[Nature Partner Journals](https://www.nature.com/articles/s44184-024-00097-4)\. That’s not what’s happening on most generative AI products\. “Digital therapeutics” are “proliferating, and many of them are very good,” said Page Walley, a clinical psychologist and Tennessee senator behind the state’s restrictions on advertising AI as a mental health professional\. “We just wanted to draw a line in the sand and say, if you do cross this, you’re operating in violation of the law\.” ## Where’s the Line? Bloomberg Law couldn’t find Emilie on Character\.ai, but there are many like it\. Coupled with ChatGPT and Claude’s use as a pocket therapist, AI adoption for mental health is surging\. Ankur Varma, psychotherapist and Brown Man Therapy co\-founder, doesn’t “actively encourage” patients to have conversations with artificial intelligence\. But if they do, he’ll discuss their AI sessions\. “For folks who aren’t working with a professional or don’t have access to a professional and are divulging very personal issues, really difficult issues, or they might be in crisis, that’s where my concern comes in,” Varma said\. States are[figuring out](https://www.transparencycoalition.ai/news/ai-legislative-update-june19-2026)parameters\. At least[a dozen](https://www.citizen.org/article/protecting-children-from-manipulative-ai-chatbots-state-legislation-tracker/)enacted legislation on AI chatbot protections for children\.[Vermont](https://legiscan.com/VT/bill/H0816/2025)and[Rhode Island](https://legiscan.com/RI/bill/H7349/2026)in[recent months](https://www.transparencycoalition.ai/news/ai-legislative-update-june19-2026)greenlit AI therapy restrictions\. In 2025, Illinois[directly restricted](https://www.bakerdonelson.com/illinois-passes-extensive-law-regulating-ai-in-behavioral-health)how licensed professionals utilize AI in healthcare, banning these tools from directly engaging with patients\. The effort was meant to counter the “large proliferation” of businesses pushing AI bots as therapists, said the sponsor, Illinois Rep\. Bob Morgan \(D\), though he noted its limited scope\. A “state law is not going to be detailed enough to give the specific answer of what should ChatGPT answer to a particular specific question,” Morgan said\. The Illinois law, however, doesn’t adequately halt general\-purpose AI chatbots like Claude from providing therapy, said C\. Vaile Wright, American Psychological Association senior director of health care innovation\. It does leave providers open to penalty for recommending over\-the\-counter AI wellness apps like Woebot or Wysa between sessions\. Federal laws are a better way to set standards, due to the high\-level of nuance in mental health, she said\. “If you don’t pay attention to that nuance you end up with laws that don’t actually do what they intended them to do,” Wright said\. “That’s what we’re seeing a lot at the state level\.” ## FDA’s Role Wright in November[presented](https://www.fda.gov/medical-devices/digital-health-center-excellence/fda-digital-health-advisory-committee)to a US Food and Drug Administration advisory committee about AI mental health technology where she[urged](https://www.fda.gov/media/189848/download)for modernizing regulations and establishing a public repository of FDA\-cleared tools\. The FDA’s authorized some[1,200](https://www.fda.gov/media/190451/download)AI\-enabled medical devices\. But none of them are for generative mental health, Michelle Tarver, director of the Center for Devices and Radiological Health told the FDA panel\. The risk posed by the technology is “[AI Psychosis,](https://www.fda.gov/media/189836/download)” Bethany Russell, a licensed therapist who focuses on youth care, told the committee\. Unregulated AI therapists may misdiagnose a patient resulting in unsafe responses to suicidal users\. The solution is using it to safely “augment” professional care, she said\. The FDA said in an emailed statement that the agency “supports clear and predictable regulatory pathways that enable the development of safe, effective, and high\-quality medical devices, including those that incorporate emerging technologies such as generative artificial intelligence\.” The American Medical Association also[urged](https://searchlf.ama-assn.org/letter/documentDownload?uri=/unstructured/binary/letter/LETTERS/lfi.zip/2026-4-22-Letter-to-Rounds-and-Heinrich-re-Mental-Health-Chatbots.pdf)Congress to prohibit chatbots from diagnosing or treating mental health conditions, and to require FDA review for any that do behave like professionals\. “The absence of regulation by the federal government is leaving a vacuum” that “states are trying to fill,” said Michael Sutton, a Sheppard, Mullin, Richter & Hampton LLP attorney\. It’s “an easy space for regulators to step in without upsetting the apple cart too much\.” Some AI mental health tools are trained via information specifically gathered for therapeutic purposes, which doesn’t include more general purpose AI like ChatGPT or Character\.ai’s Emilie\. But without a uniform standard, a state\-by\-state patchwork may limit the depth of regulation needed for patients to access more helpful tools than general LLMs\. “There’s going to be a future where you have FDA\-approved AI tools for clinical purposes that then will not be available to providers to help extend their care,” Wright said\.