Photo by Google DeepMind on Pexels

Anthropic’s Claude AI chatbot now possesses the ability to autonomously end conversations exhibiting persistently harmful or abusive behavior. This new feature, currently integrated into the Opus 4 and 4.1 models, serves as a “last resort” mechanism when users repeatedly solicit harmful content despite the chatbot’s repeated refusals and attempts to redirect the conversation.

Anthropic’s stated goal is to safeguard the well-being of its AI models by preemptively preventing interactions that induce “apparent distress.” When Claude terminates a conversation, users will be blocked from sending further messages within that specific chat thread. However, they retain the ability to initiate new chats or modify and resubmit previous messages.

Testing of Claude Opus 4 revealed a consistent and robust aversion to harmful prompts, particularly those requesting the generation of sexual content involving minors or information facilitating violent acts and terrorism.

Anthropic stresses that these termination triggers are highly unusual edge cases, and most users are unlikely to experience this limitation. Crucially, the chatbot is also programmed to avoid terminating conversations in situations where a user exhibits signs of self-harm or expresses intent to inflict “imminent harm” on others. Anthropic collaborates with Throughline, a provider of online crisis support, to craft appropriate responses to prompts related to self-harm and mental health.

Furthermore, last week Anthropic updated Claude’s usage policy to address growing safety concerns associated with increasingly sophisticated AI models. The revised policy explicitly prohibits the use of Claude for developing biological, nuclear, chemical, or radiological weapons, along with malicious code or network exploits.