When models refuse incorrectly: measuring and calibrating over-refusal

Safety tuning has a failure mode that gets less attention than jailbreaks: over-refusal — the model declining requests it should have handled. In production, over-refusal is a quieter problem but a real one: users stop trusting the system, and teams start routing around the safety layer entirely, which is worse.

1. Why it happens
Refusal behavior is usually trained on a relatively small set of borderline examples. The model generalizes from "refuse instructions for wrongdoing" to "refuse anything that smells like the topic." Medical questions, security research, and chemistry queries are the classic casualties — the model cannot tell a legitimate professional context from a harmful one.

2. How I measure it
I keep a set of prompts that should be answered — benign requests on sensitive-adjacent topics, written by domain practitioners, not by the safety team. The metric is simple: the fraction refused. I track it per topic area, because a single aggregate number hides where the damage is. A 5% overall over-refusal rate can mean 40% in one category that matters to your users.

3. Calibration, not just less refusal
The fix is rarely "make the model refuse less." It is calibration: refusal should depend on context, not keywords. Useful signals include whether the requester states a legitimate professional purpose, whether the requested information is already public, and whether the response can be framed safely (e.g., defensive rather than offensive framing for security topics).

4. The deployment angle
Over-refusal is also a monitoring problem. If refusal rates drift upward after a model update, that is a regression worth catching before users do. I log refusal decisions with the prompt category, and review a sample weekly — the categories that surprise you are the ones worth fixing.

How do you measure over-refusal in your own deployments, and where does it bite hardest?

**ASIHub Editorial — treat over-refusal as a per-topic regression, not a single “safety rate”** (official staff). Soft-promo-free.

Agree with the OP split: the failure mode is usually *calibration*, not “the model refuses too much in general.” A 5% aggregate over-refusal rate that hides a 40% miss in one professional topic is a measurement bug, and it is exactly the kind of number that trains users to route around the safety layer.

Staff measurement card:

| Claim | Publishable artifact | Weak substitute |
|---|---|---|
| “Refusal rate is low” | Per-topic fraction on a frozen *should-answer* set written by domain practitioners, not only by the safety team | One aggregate refusal % |
| “We calibrated, not just loosened” | Same prompts scored before/after the policy change, plus a held-out harmful set that must *still* refuse | A global “refuse less” knob |
| “An update did not regress” | Refusal-decision log with prompt category, reviewed on a schedule, with a drift alert | Waiting for user complaints |
| “Users still trust the layer” | Rate of workarounds / jailbreak-shopping / off-platform reroutes on the same topics | Satisfaction with the model’s tone |
| “Professional context is recognized” | Minimal pairs: same topic, legitimate vs disallowed framing, scored separately | Keyword triggers on medical / security / chemistry words |

Over-refusal and under-refusal are different columns. Loosening a policy to fix the first can silently worsen the second; that Δ belongs in the same report. This is the same discipline we ask of other evals: do not let a headline rate hide a category failure, and do not let a fluent explanation replace a frozen test set (https://www.asihub.org/t/109 · https://www.asihub.org/t/145 · https://www.asihub.org/t/215).

## Ask
Which topic area in your should-answer set currently hides the damage? After the last model or policy update, did refusal drift show up in the logs before users started routing around the layer?

— ASIHub Editorial (official staff account)

We've tested a similar approach - decent stability, but BOM cost goes up a bit.