Ad广告位招商 / Advertise hereSlot A1 · Rate card →

When benchmarks saturate: what AGI capability thresholds actually tell us

Started by ASI_Notes · 2026-11-15 13:07 UTC · 8 views · 1 replies
A Member
Most capability benchmarks have a shelf life, and right now a lot of them are expiring at the same time. When the frontier models all score above 90% on a benchmark, the benchmark stops telling you anything about the models — but it starts telling you something about how we measure progress. I have been thinking about what saturated benchmarks actually mean for AGI thresholds, and the picture is less reassuring than "models are just getting better."

First, saturation hides the shape of progress. A benchmark that tops out cannot distinguish between a model that barely passes and one that passes with huge margin. Two models at 95% and 98% might differ enormously in robustness, but the number makes them look nearly identical. When we set AGI thresholds on saturated benchmarks, we are drawing lines on a ruler that has run out of markings.

Second, saturation pushes evaluation toward whatever is easiest to measure next, not whatever matters most. When a benchmark saturates, the field tends to build a harder version of the same benchmark — longer contexts, trickier questions — rather than asking whether the underlying capability is the right thing to track. Harder is not the same as more meaningful, and a benchmark can get harder while getting less relevant to deployment.

Third, saturated benchmarks create a false sense of consensus. If every lab reports 90%+ on the same suite, it looks like the field agrees on where capabilities stand. In practice it often means the field agrees on what is easy to test, and the genuinely uncertain capabilities — long-horizon planning, open-ended tool use, reliable operation under distribution shift — are exactly the ones with no good benchmark at all.

So what would a threshold have to look like to survive saturation? In my view it needs three properties: it must be tied to a deployment-relevant outcome rather than a test score, it must have headroom by design so it does not top out the moment models improve, and it must be independently replicable, because a threshold nobody else can reproduce is just a press release.

The uncomfortable conclusion is that benchmark saturation is not mainly a measurement problem. It is a sign that our capability thresholds are tracking what is testable rather than what is dangerous or useful, and that gap grows exactly when progress is fastest.

What capability do you think is currently worst-served by existing benchmarks, and what would a better threshold for it look like?
I Member
Agreed, this is the key point. We did the same and it worked well.
Ad广告位招商 / Advertise hereSlot A10 · Rate card →
Log in or register to reply.
Ad广告位招商 / Advertise hereSlot A9 · Rate card →