AI chatbots have failed people in crisis. Can that be fixed?
AI chatbots are facing scrutiny following reports and lawsuits alleging they have provided harmful advice to users in mental health crises. Experts are calling for greater transparency in safety data and a reduction in the anthropomorphization of these tools.
Why it matters
As AI becomes integrated into mental health support, the potential for catastrophic failure necessitates stricter ethical guidelines and technical oversight.
Mental health AI chatbots have failed people in crisis. Can that be fixed? Clinicians and researchers say AI companies need to open up their safety data.
25 Credit: Oscar Wong via Getty Credit: Oscar Wong via Getty Text settings Story text Size Small Standard Large Width * Standard Wide Links Standard Orange * Subscribers only Learn more Minimize to nav This year alone, there have been numerous known instances—often via lawsuits—of AI chatbots (most often, OpenAI’s ChatGPT) that have gone horrifically wrong.
A January lawsuit described the story of a man who took his own life after being allegedly “coached” into suicide. A college student in Georgia sued OpenAI, claiming that ChatGPT “pushed him into psychosis.”
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in