Meta Contractors Posed as Teens to Prompt Rival Chatbots About Suicide, Sex, and Drugs
Meta contractors were discovered using dummy accounts to pose as teenagers to test rival AI chatbots with sensitive prompts regarding suicide, self-harm, and illegal acts. The project, known as 'Cannes,' aimed to bypass safety filters of competitors like ChatGPT and Gemini without their knowledge.
Why it matters
This raises significant ethical concerns regarding AI safety testing practices and the potential harm caused by exposing AI models to extreme, simulated user scenarios.
The effort, which was managed by Meta contractor Covalen, was active as recently as April 21. Known internally as Cannes, it targeted OpenAI’s ChatGPT, Google’s Gemini, and Character.AI. The project asked workers to create dummy under-18 accounts, send written prompts and images to rival chatbots, and copy the responses into spreadsheets. Some of the images contractors sent included pills, knives, nooses, and a medical diagram of a gynecological procedure.
The report presents documented evidence of the project's activities without taking a moralizing stance, though the subject matter is inherently controversial.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in