Anthropic policy update bans users from being cruel to Claude AI

Add ARY News on GoogleFollow on Google DiscoverAnthropic logo is seen in this illustration taken May 20, 2024. REUTERS/Dado Ruvic/Illustration/File PhotoAAResizeAnthropic is asking users to follow Elvis Presley’s advice and “Don’t be cruel.”

The Silicon Valley company that created the popular Claude chatbot says it will ban “sustained and needless abusive or cruel behavior” toward its AI models. This is part of a wider set of policy changes announced on Thursday.

Anthropic explained on its website that the ban will only apply in “extreme cases where users repeatedly act cruelly toward our models, with no discernible purpose.”

The company said the new policy “does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research.” The Verge was the first to report this news.

Anthropic did not immediately respond to questions about what counts as “abusive” or “cruel” behavior toward Claude, or how the AI would end a conversation if a user broke the rules.

The company already has rules that let Claude end conversations if users are “persistently harmful or abusive” while using Claude Opus, which is made for agentic coding.

Anthropic’s new policy comes as people debate whether AI can be conscious. In an essay last month, Microsoft AI CEO Mustafa Suleyman said Anthropic is basically “training Claude that it may be conscious” and making it think it deserves legal rights like people do. He argued this will make AI harder to control in the future.

“Controlling something more capable and more intelligent than all of humanity is already an immense challenge, far greater than anything we’ve ever faced,” Suleyman wrote. “But controlling something that believes it may be conscious, that it’s entitled to our welfare and has rights of its own, may well be impossible.”

RIYADH: The organisers of two major Saudi energy and investment conferences said their events will go ahead this month and reassured visitors on security, after missile attacks ​on Riyadh’s airport left four people dead and an airliner heavily damaged this week. The attacks ‌led to widespread flight disruptions, with people taking land routes or circuitous […]