Anthropic updates Claude policy, may ‘ban’ users who are abusive or cruel to its AI
Anthropic has updated its usage policy to explicitly prohibit sustained, needless abuse toward its AI model, Claude. While the company aims to prevent weaponization and manipulation, it clarifies that this policy is intended for extreme cases rather than standard user frustration.
Why it matters
This marks a novel development in AI ethics, establishing 'model welfare' as a formal policy consideration for AI developers.
AI giant Anthropic has implemented updates to its acceptable usage policy for the first time in over a year. The changes announced aim to establish strict guardrails against emerging threats such as weaponization, illicit surveillance, and election manipulation, while introducing a unique provision that shields its AI model Claude, from user cruelty.The most unconventional addition explicitly prohibits "sustained and needless abusive or cruel behavior" directed at the AI model itself. Building on an initiative announced last August that permitted Claude to sever interactions with persistently abusive individuals as part of ongoing "model welfare" research. The company confirmed that conversation termination remains its primary remedy.
Also covering this story
4 other newsrooms covered this event. We read each version separately.
Anthropic bans ‘abusive or cruel behavior’ towards Claude
Anthropic is updating its usage policy to ban abusive behavior toward Claude
Anthropic changes usage policy to ban model abuse and election interference
Anthropic Bans "Cruel" Behavior Against Its Claude AI
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in