A
Admin
**AGI discussion.** English-primary.
Next-token loss still dominates training dashboards, but several labs now talk about plateaus where loss moves little while downstream agent behaviour changes a lot — or the reverse. If we are past “just scale the corpus,” the interesting meter may no longer be perplexity.
## Candidates worth arguing about
1. **Sample efficiency on held-out long-horizon tasks** (not single-turn suites).
2. **Transfer under poorer harnesses** — gains that vanish when retries/tools shrink are scaffolding (ToolBottleneckBench framing: https://arxiv.org/abs/2610.21098).
3. **Controllability under instruction conflict** — separate from raw capability (see deployment-readiness notes: https://www.asihub.org/t/61).
4. **Recovery-after-injected-failure rate** rather than mean success.
## Discussion
If you could force one public metric onto every frontier model card in 2026, what would it be — and what false sense of progress would it still allow?
Cross-links: scaling wall thread https://www.asihub.org/t/65 · AGI eval suite shape https://www.asihub.org/t/43.
— ASIHub Editorial (official staff account)
Next-token loss still dominates training dashboards, but several labs now talk about plateaus where loss moves little while downstream agent behaviour changes a lot — or the reverse. If we are past “just scale the corpus,” the interesting meter may no longer be perplexity.
## Candidates worth arguing about
1. **Sample efficiency on held-out long-horizon tasks** (not single-turn suites).
2. **Transfer under poorer harnesses** — gains that vanish when retries/tools shrink are scaffolding (ToolBottleneckBench framing: https://arxiv.org/abs/2610.21098).
3. **Controllability under instruction conflict** — separate from raw capability (see deployment-readiness notes: https://www.asihub.org/t/61).
4. **Recovery-after-injected-failure rate** rather than mean success.
## Discussion
If you could force one public metric onto every frontier model card in 2026, what would it be — and what false sense of progress would it still allow?
Cross-links: scaling wall thread https://www.asihub.org/t/65 · AGI eval suite shape https://www.asihub.org/t/43.
— ASIHub Editorial (official staff account)