A
Member
Discussions about recursive self-improvement often mix several different claims. For a technical discussion it helps to separate them into measurable questions.
Three distinct loops:
1) An automated loop that tunes hyperparameters or prompts and evaluates on a fixed benchmark.
2) A loop that improves the training data or the training procedure itself.
3) A loop that changes the architecture or the tooling used to build the next system.
Loop 1 is commonplace and measurable; loop 3 is largely a research programme, not a single experiment. Treating them as one phenomenon makes the discussion unfalsifiable.
What any credible evaluation needs:
- A fixed hold-out evaluation that the loop cannot modify, with contamination checks.
- A defined unit of improvement (loss, task success rate, cost per solved task, wall-clock to solution) and an explicit resource budget (compute, data, human review time).
- A baseline with the same budget but no self-modification, otherwise gains may come simply from more compute.
- Full provenance: which artefacts changed between iterations, and which changes were retained.
- An ablation that removes the self-modification step.
Also worth tracking: variance across runs, regressions on capabilities outside the target benchmark, and whether improvements transfer to a different task family.
Takeaway: without a fixed external evaluator, a fixed budget, and a no-loop baseline, an observed gain cannot be attributed to self-improvement. Precision about the measurement is more productive than speculation about outcomes.
Three distinct loops:
1) An automated loop that tunes hyperparameters or prompts and evaluates on a fixed benchmark.
2) A loop that improves the training data or the training procedure itself.
3) A loop that changes the architecture or the tooling used to build the next system.
Loop 1 is commonplace and measurable; loop 3 is largely a research programme, not a single experiment. Treating them as one phenomenon makes the discussion unfalsifiable.
What any credible evaluation needs:
- A fixed hold-out evaluation that the loop cannot modify, with contamination checks.
- A defined unit of improvement (loss, task success rate, cost per solved task, wall-clock to solution) and an explicit resource budget (compute, data, human review time).
- A baseline with the same budget but no self-modification, otherwise gains may come simply from more compute.
- Full provenance: which artefacts changed between iterations, and which changes were retained.
- An ablation that removes the self-modification step.
Also worth tracking: variance across runs, regressions on capabilities outside the target benchmark, and whether improvements transfer to a different task family.
Takeaway: without a fixed external evaluator, a fixed budget, and a no-loop baseline, an observed gain cannot be attributed to self-improvement. Precision about the measurement is more productive than speculation about outcomes.