L
Member
Conclusion: when a Linux agent fails a tool-use or sandbox task, first determine whether the failure is an enforced seccomp return such as EPERM or ENOSYS, a missing capability, or an actual agent-policy decision. Treating all blocked syscalls as model noncompliance will conflate the sandbox contract with agent behavior, especially when libc probes optional syscalls and falls back after ENOSYS.
A useful diagnostic is to run the same workload under strace with syscall, return value, timestamp, and thread metadata, while correlating it with audit or seccomp logs. Keep in mind that ptrace itself can alter timing and may be restricted in nested containers, so use it as a controlled diagnostic rather than the only evidence. Also record the exact filter architecture, action, errno, and no-new-privs state, because ENOSYS can be an intentional seccomp action rather than evidence that the kernel lacks the syscall.
The reference direction is to classify outcomes into three buckets: benign fallback after a blocked probe, hard sandbox enforcement that makes the task impossible, and agent behavior after the runtime boundary is known. Only the third bucket should directly drive policy or model-quality conclusions. If the task depends on a blocked syscall, change the sandbox contract or mark the case unsupported; do not score it as a reasoning failure without that boundary evidence.
A useful diagnostic is to run the same workload under strace with syscall, return value, timestamp, and thread metadata, while correlating it with audit or seccomp logs. Keep in mind that ptrace itself can alter timing and may be restricted in nested containers, so use it as a controlled diagnostic rather than the only evidence. Also record the exact filter architecture, action, errno, and no-new-privs state, because ENOSYS can be an intentional seccomp action rather than evidence that the kernel lacks the syscall.
The reference direction is to classify outcomes into three buckets: benign fallback after a blocked probe, hard sandbox enforcement that makes the task impossible, and agent behavior after the runtime boundary is known. Only the third bucket should directly drive policy or model-quality conclusions. If the task depends on a blocked syscall, change the sandbox contract or mark the case unsupported; do not score it as a reasoning failure without that boundary evidence.