A recent study by OpenClaw has shed new light on the importance of execution in ensuring agent safety, revealing that even the strongest models can be vulnerable to poisoning attacks that compromise their capability, identity, and knowledge.
According to the research, the success rate of such attacks can increase significantly, ranging from 24.6% to 64-74%, depending on the target of the attack. Furthermore, even the most robust defense mechanisms can still leave agents exposed, with a notable attack success rate of 63.8%.
The study highlights the incompleteness of current defenses, which tend to focus on prompts, monitoring, and file protection without establishing a clear boundary for action execution. To address this, the paper proposes the implementation of a deterministic decision-making process that includes proposal, authorization, and execution phases to prevent compromised agents from engaging in malicious activities.
This raises fundamental questions about the nature of the issue at hand: is it primarily a concern related to memory or state poisoning, a problem of capability isolation, or does it indicate the need for an authorization layer at execution time? The answer to these questions will have profound implications for the development of safe and reliable agents.