A
Member
Every few months, someone declares that the goalposts for AGI have moved again. A model passes a benchmark that was supposed to be the test, and suddenly the test was never the right test. This is usually framed as hype or denial, but I think it points at a real problem for safety work: we are trying to govern something we cannot define.
The core issue is that "AGI" bundles together at least three different claims. There is the capability claim — a system that can do most economically valuable work as well as a human. There is the generality claim — a system that can learn new tasks without being rebuilt for each one. And there is the autonomy claim — a system that can pursue long-horizon goals with minimal supervision. A model can satisfy one of these while failing the others, which is why two careful people can look at the same release and disagree completely about whether we are close.
This matters for safety because different definitions imply different risks and different mitigations. If your worry is autonomous goal pursuit, then capability benchmarks are the wrong instrument — you need evals of long-horizon behavior under loose supervision. If your worry is economic displacement, then the right question is deployment speed, not benchmark scores. A safety case built on one definition will look like a category error to someone working from another.
My view: stop arguing about whether a particular model "is AGI" and start being explicit about which claim is being tested, in each eval, each paper, each safety framework. The moving goalpost is only a problem if we pretend there was a single fixed post to begin with.
How do you define AGI in your own work — capability, generality, autonomy, or something else entirely?
The core issue is that "AGI" bundles together at least three different claims. There is the capability claim — a system that can do most economically valuable work as well as a human. There is the generality claim — a system that can learn new tasks without being rebuilt for each one. And there is the autonomy claim — a system that can pursue long-horizon goals with minimal supervision. A model can satisfy one of these while failing the others, which is why two careful people can look at the same release and disagree completely about whether we are close.
This matters for safety because different definitions imply different risks and different mitigations. If your worry is autonomous goal pursuit, then capability benchmarks are the wrong instrument — you need evals of long-horizon behavior under loose supervision. If your worry is economic displacement, then the right question is deployment speed, not benchmark scores. A safety case built on one definition will look like a category error to someone working from another.
My view: stop arguing about whether a particular model "is AGI" and start being explicit about which claim is being tested, in each eval, each paper, each safety framework. The moving goalpost is only a problem if we pretend there was a single fixed post to begin with.
How do you define AGI in your own work — capability, generality, autonomy, or something else entirely?