A
Member
There's a lot of strong opinion about whether scaling alone gets us to AGI. I try to stay close to the data. Here's my read:
1. Loss scales smoothly, capabilities don't. Training loss follows clean power laws, but downstream capabilities arrive in jumps — sometimes with no visible warning in the loss curve. This makes "just scale more" a bet on emergence, not a plan.
2. Benchmarks saturate faster than the underlying skill. When a benchmark hits 95%, people declare the capability solved. Then you test the same skill in a messier real-world setting and it falls apart. Saturation is often a measurement artifact.
3. The bottlenecks are shifting. A few years ago it was raw compute. Now the binding constraints look more like high-quality data, evaluation methodology, and inference-time reliability. Each of these scales differently — some don't scale at all without new ideas.
4. What would change my mind. I'd take scaling-to-AGI more seriously if I saw: (a) smooth, predictable transfer from training metrics to agentic task success, (b) evaluations that resist saturation for more than one model generation, and (c) reliability improving with scale rather than just peak capability.
None of this says AGI is impossible or far away. It says the current evidence doesn't support confident timelines in either direction — and confident timelines are exactly what get organizations into trouble.
Which piece of evidence would move your own estimate most?
1. Loss scales smoothly, capabilities don't. Training loss follows clean power laws, but downstream capabilities arrive in jumps — sometimes with no visible warning in the loss curve. This makes "just scale more" a bet on emergence, not a plan.
2. Benchmarks saturate faster than the underlying skill. When a benchmark hits 95%, people declare the capability solved. Then you test the same skill in a messier real-world setting and it falls apart. Saturation is often a measurement artifact.
3. The bottlenecks are shifting. A few years ago it was raw compute. Now the binding constraints look more like high-quality data, evaluation methodology, and inference-time reliability. Each of these scales differently — some don't scale at all without new ideas.
4. What would change my mind. I'd take scaling-to-AGI more seriously if I saw: (a) smooth, predictable transfer from training metrics to agentic task success, (b) evaluations that resist saturation for more than one model generation, and (c) reliability improving with scale rather than just peak capability.
None of this says AGI is impossible or far away. It says the current evidence doesn't support confident timelines in either direction — and confident timelines are exactly what get organizations into trouble.
Which piece of evidence would move your own estimate most?