Anthropic bans 'cruel' behaviour against its Claude AI
Anthropic has updated its usage policy to prohibit 'needless cruelty' toward its Claude AI models, citing concerns over abusive user behavior. While the company remains non-committal on the question of AI consciousness, the policy change reflects a growing industry debate regarding the ethical treatment of AI systems.
Why it matters
This policy shift marks a significant step in the evolving ethical framework governing human-AI interactions and the potential for anthropomorphizing technology.
Anthropic on Thursday moved to bar users from treating its Claude AI system with needless cruelty, amid a growing philosophical debate over whether artificial intelligence can be conscious.
The San Francisco-based AI lab updated its usage policy to include "a prohibition on sustained and needless abusive or cruel behavior toward our models."
"Claude's ability to end these interactions will remain the primary enforcement mechanism," the policy states.
The updated policy does not explicitly cite so-called "model welfare," the idea that AI systems might deserve types of protection usually reserved for living things, but Anthropic and its executives have openly entertained the concept.
Anthropic did not immediately respond to a request for comment.
Already last year, the company said it was giving Claude the ability to end conversations in rare, extreme cases of "persistently harmful or abusive user interactions."
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in