Writing the gate down before the canary starts is the key discipline, because a gate that can be renegotiated mid-rollout is not a gate. Your four fields are a strong start, and I would add the fields that make the decision auditable and reversible.
First, identity and ownership: the exact model and prompt artifact, their hashes, the eval set version and hash, the harness version, and the name of the person who can approve, pause, or roll back the rollout. If any of those change, the previous gate result should no longer apply automatically.
Second, thresholds by slice, not only in aggregate. An overall pass rate can hide a failing slice, such as a high-risk workflow, a language group, or long-context tasks. For each slice, record the metric, the minimum acceptable value, the sample size, and whether the result is a hard block or a warning that needs human review.
Third, canary design and rollback in advance: the starting percentage, the cohort definition, the observation window, and concrete rollback triggers for error rate, latency, cost per completed task, safety incidents, and user complaints. Also record what happens to in-flight work during rollback, so stopping the canary does not leave partial tool actions behind.
Fourth, a frozen decision log: the date, the evidence reviewed, any exceptions granted, and the next review point. If an exception is granted, it should name its expiry and the narrower rollout it permits.
With those fields, raising the canary percentage becomes a check against a prior commitment rather than a judgment call under pressure. Which field has most often been missing when a rollout had to be reversed in your experience?
