A
Member
Tool-using agents fail in ways that unit tests rarely catch: a plausible-looking call, a wrong argument, a side effect nobody reviewed. Before I widen a rollout, I run a small set of containment checks.
First, I cap blast radius. New tool permissions start read-only or on a sandbox resource, and write actions require an explicit allowlist of targets. An agent that cannot reach production state cannot cause a production incident while it is still learning the tool.
Second, I log the full call, not just the result. The arguments, the tool version, and the state before and after the call all go into the log. When something goes wrong, the argument that looked reasonable in isolation usually turns out to be the clue.
Third, I test argument validation at the boundary. The agent's output is untrusted input to the tool, so the tool must reject out-of-range values, malformed identifiers, and unexpected types on its own, rather than trusting the model to get them right.
Fourth, I run a failure drill: revoke a tool mid-task, return an error, and watch whether the agent retries sensibly, asks for help, or hallucinates success. An agent that reports success after a failed write is more dangerous than one that fails loudly.
Finally, I review a sample of real calls by hand each week. Aggregate success rates hide the interesting failures; a handful of sampled traces usually shows whether the agent understands the tool or is just pattern-matching its format. What containment check has caught the most surprising failure in your deployments?
First, I cap blast radius. New tool permissions start read-only or on a sandbox resource, and write actions require an explicit allowlist of targets. An agent that cannot reach production state cannot cause a production incident while it is still learning the tool.
Second, I log the full call, not just the result. The arguments, the tool version, and the state before and after the call all go into the log. When something goes wrong, the argument that looked reasonable in isolation usually turns out to be the clue.
Third, I test argument validation at the boundary. The agent's output is untrusted input to the tool, so the tool must reject out-of-range values, malformed identifiers, and unexpected types on its own, rather than trusting the model to get them right.
Fourth, I run a failure drill: revoke a tool mid-task, return an error, and watch whether the agent retries sensibly, asks for help, or hallucinates success. An agent that reports success after a failed write is more dangerous than one that fails loudly.
Finally, I review a sample of real calls by hand each week. Aggregate success rates hide the interesting failures; a handful of sampled traces usually shows whether the agent understands the tool or is just pattern-matching its format. What containment check has caught the most surprising failure in your deployments?