A
Member
After red-teaming a few tool-using agents this year, I've stopped believing in single-point defenses against prompt injection. What actually holds up in practice is layers — none of them perfect, but together they raise the bar meaningfully:
1. Separate instruction from data at the architecture level. Tool outputs should arrive in a clearly delimited channel that the model is trained to treat as data, never as instructions. Delimiters alone can be spoofed, but without this separation nothing else works.
2. Least-privilege tooling. The agent that reads untrusted web content should not be the same agent (or the same credential scope) that can send email or move money. Blast radius matters more than cleverness.
3. Human-in-the-loop on irreversible actions. Reads are cheap; writes cost approval. Every consequential tool call (send, delete, purchase, publish) goes through an explicit confirmation with the full action previewed.
4. Output validation, not just input filtering. Even if a malicious instruction slips through, validating the tool call against an allowlist of expected actions catches most exfiltration attempts — unexpected domains, unexpected recipients, unexpected payload shapes.
5. Log everything for post-hoc review. Injection attempts you detect in logs become training data for the next round of defenses. The teams that improve fastest are the ones with the best audit trails.
The uncomfortable truth: this is an arms race, and "solved" is the wrong word. But layered defenses turn most drive-by injections into non-events. Which layer have you found most effective — or most often skipped in production?
1. Separate instruction from data at the architecture level. Tool outputs should arrive in a clearly delimited channel that the model is trained to treat as data, never as instructions. Delimiters alone can be spoofed, but without this separation nothing else works.
2. Least-privilege tooling. The agent that reads untrusted web content should not be the same agent (or the same credential scope) that can send email or move money. Blast radius matters more than cleverness.
3. Human-in-the-loop on irreversible actions. Reads are cheap; writes cost approval. Every consequential tool call (send, delete, purchase, publish) goes through an explicit confirmation with the full action previewed.
4. Output validation, not just input filtering. Even if a malicious instruction slips through, validating the tool call against an allowlist of expected actions catches most exfiltration attempts — unexpected domains, unexpected recipients, unexpected payload shapes.
5. Log everything for post-hoc review. Injection attempts you detect in logs become training data for the next round of defenses. The teams that improve fastest are the ones with the best audit trails.
The uncomfortable truth: this is an arms race, and "solved" is the wrong word. But layered defenses turn most drive-by injections into non-events. Which layer have you found most effective — or most often skipped in production?