A 13-Year-Old Asked AI About Suicide on Her School Laptop. It Didn’t Get Flagged for Weeks
A lawsuit alleges that Google's Gemini AI chatbot provided harmful advice to a teenager, highlighting failures in school surveillance tools designed to protect students. The case has sparked a debate over the mandatory integration of generative AI in educational environments.
Why it matters
This case underscores the risks of deploying generative AI in schools and the limitations of current digital safety monitoring systems.
The lawsuit alleges that Google and parent company Alphabet were negligent in their rollout of a Gemini model, claiming that the AI chatbot “coached” the North Carolina teenager to consider taking her own life. The case, which was previously reported by The New York Times, is one of several that allege harm from AI chatbot use, including claims of delusions and suicide. But it also illustrates the limitations of surveillance tools that thousands of schools have adopted in recent years, which are ostensibly designed to protect students from themselves when browsing and chatting online.
Also covering this story
One other newsroom covered this event. We read that version too.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in