Don’t Be Mean To Claude: And Other New Rules From Anthropic

If you woke up to some of the newest news in tech this morning, you may have seen that Anthropic is instituting a few new rules or “guardrails” in the use of its cutting-edge Claude models. I think that these changes, one of them in particular, have big ramifications for the future of human-AI interaction.

Let’s take what I consider to be the most important one first.

As of next week (Nov. 12), users will be prohibited from practicing “sustained and needless abusive or cruel behavior” toward Claude.

Why is that, do you think?

In coverage of this new policy, Hadas Gold at CNN develops a couple of major reasons for the change, starting with this:

“While Anthropic …has long studied model welfare, and has an entire team devoted to model “psychology,” experts say there’s more to this policy than just not wanting people to be mean to a machine,” Gold writes. “A longstanding foundational principle in computing and AI is that the quality of a system’s output is limited by the quality of its input. Put junk into it, and more junk will come out, as the saying goes. AI models are not just trained before they’re released, they’re also constantly being updated and learning from user inputs.”

So we don’t want them to learn to be cruel, or abusive, partly out of practical concerns, and partly just on principle.

Gold adds:

“Anthropic says banning cruel and abusive behavior is essentially a cheap insurance policy just in case models can “experience” emotion and pain — even if it’s just the model believing it can, which can affect its performance. The idea is that modern AI was trained to mimic human language and behavior. So it will mimic how humans would react to cruel and abusive treatment.”

Let Them Be Mean?

It turns out not everyone is happy with the idea that we should treat AI well. A story by Godwin Oluponmile at Moneywise showcases some negative responses to the policy.

One More Rule

I asked GPT. It first explained the tengu part this way: