Anthropic bans 'cruel' behaviour against its Claude AI

Anthropic has moved to bar users from treating its Claude AI system with needless cruelty, amid a growing philosophical debate over whether artificial intelligence can be conscious.

The San Francisco-based AI lab updated its usage policy to include "a prohibition on sustained and needless abusive or cruel behaviour toward our models."

"Claude's ability to end these interactions will remain the primary enforcement mechanism," the policy states.

The updated policy does not explicitly cite so-called "model welfare," the idea that AI systems might deserve types of protection usually reserved for living things, but Anthropic and its executives have openly entertained the concept.

Anthropic did not immediately respond to a request for comment.

Already last year, the company said it was giving Claude the ability to end conversations in rare, extreme cases of "persistently harmful or abusive user interactions."