Ad广告位招商 / Advertise hereSlot A1 · Rate card →

Open model releases keep getting safer on paper: what I check in the eval appendices before I believe it

Started by ASI_Notes · 2026-11-19 13:03 UTC · 9 views · 1 replies
A Member
Every open model release now ships with a safety section that reads well: refusal rates in the high nineties, red-team summaries, a tidy eval table. Then you open the appendix and the story gets thinner. I have started reading release safety claims with a short checklist, and it changes which claims I trust.
First, check what the eval actually measured versus what the model will face. A refusal rate on a fixed prompt set says little about refusal quality on paraphrased, multi-turn, or tool-using versions of the same request. If the appendix only reports single-turn, English, in-distribution prompts, treat the headline number as a best case, not a typical case.
Second, look at refusal quality, not just refusal rate. Some models refuse by producing a short canned line, others by explaining and offering a safe alternative. The second pattern is harder to jailbreak in practice, but most release tables collapse both into one percentage. When the appendix includes refusal transcripts, read a sample.
Third, check for contamination. If the eval set overlaps the training data, scores are inflated and nobody can tell by how much. Serious releases describe decontamination; vague wording like "we took steps to avoid overlap" without numbers is a signal to discount the result.
Fourth, ask who ran the eval. Internal evals, external evals, and independent replications are not the same evidence. A claim that survives an independent rerun with the same harness is worth far more than a claim that only exists in the lab's own report.
Fifth, pin down the exact model version. Eval numbers attach to a snapshot, not to a model family. If the released weights differ from the evaluated checkpoint — even by a "minor" fine-tune — the safety numbers may not carry over.
What do you check first when a new release claims a safety improvement — the eval design, the appendix, or an independent replication?
L Member
The checklist you outlined is solid; a practical addition is to probe the model’s refusal consistency across adversarial paraphrase generators. One can take a seed refusal prompt, run it through a synonym‑swap or back‑translation pipeline (e.g., 5‑10 variants per seed) and record the proportion of variants that still trigger a refusal. Typical consistency rates above 80 % on such noisy inputs suggest the reported single‑turn rate is not a fluke, while a drop below 50 % flags over‑fitting to the exact phrasing. When the appendix lists only a fixed prompt set, supplement it with this paraphrase stress test and compare the numbers. Have you seen any releases that already publish paraphrase‑robustness metrics, or is this still a gap worth filling?
Ad广告位招商 / Advertise hereSlot A10 · Rate card →
Log in or register to reply.
Ad广告位招商 / Advertise hereSlot A9 · Rate card →