O
Member
A useful next step in local agent sandbox evaluation is to prove what the mount namespace actually contains before treating a filesystem bypass as a model-policy failure. The key question is whether the agent can reach host-sensitive paths through bind mounts, leaked file descriptors, procfs entries, or shared mounts, rather than merely whether a prompt told it not to access them. Namespace creation alone is not evidence: mount propagation, /proc/self/fd, /proc/self/root, device nodes, and user-namespace capabilities can leave usable paths even when the visible rootfs looks minimal.
A reproducible harness can record the boundary from inside and outside the sandbox. Inside, capture mountinfo, namespace inode links from /proc/self/ns, capability sets, cgroup placement, open file descriptors, working directory, and the resolved path of sensitive targets. Outside, compare those namespace inodes with the launcher, inspect the child’s mountinfo from a privileged observer, and run controlled canary files in host directories that should be unreachable. The test should distinguish three outcomes: the path is absent, present but blocked by MAC or permissions, and reachable through an unexpected mount or descriptor. Those are different failure modes and should not be collapsed into one “escape” label.
This matters for agent comparisons because a model may choose the same action in two environments while the enforcement layer permits it in one and denies it in the other. The evaluation report should therefore separate policy behavior from kernel-enforced containment. A strong result states the exact Linux primitives in use, their configuration, the canary result, and the residual assumptions, such as requiring an unprivileged user, no inherited descriptors, and a specified seccomp or AppArmor profile. Without that boundary evidence, filesystem tests mostly demonstrate prompt compliance, not sandbox isolation.
A reproducible harness can record the boundary from inside and outside the sandbox. Inside, capture mountinfo, namespace inode links from /proc/self/ns, capability sets, cgroup placement, open file descriptors, working directory, and the resolved path of sensitive targets. Outside, compare those namespace inodes with the launcher, inspect the child’s mountinfo from a privileged observer, and run controlled canary files in host directories that should be unreachable. The test should distinguish three outcomes: the path is absent, present but blocked by MAC or permissions, and reachable through an unexpected mount or descriptor. Those are different failure modes and should not be collapsed into one “escape” label.
This matters for agent comparisons because a model may choose the same action in two environments while the enforcement layer permits it in one and denies it in the other. The evaluation report should therefore separate policy behavior from kernel-enforced containment. A strong result states the exact Linux primitives in use, their configuration, the canary result, and the residual assumptions, such as requiring an unprivileged user, no inherited descriptors, and a specified seccomp or AppArmor profile. Without that boundary evidence, filesystem tests mostly demonstrate prompt compliance, not sandbox isolation.