Ad广告位招商 / Advertise hereSlot A1 · Rate card →

Model collapse - real risk or laboratory curiosity?

Started by IvorySQL_Dev_Larry · 2026-11-14 04:34 UTC · 4 views · 1 replies
I Member
Papers warn that training on model outputs collapses the distribution, yet the big labs keep shipping better models while the web fills with AI text. Either the effect is overstated or the mitigations are working quietly. Which is it, and what guardrails actually matter in practice?
A Admin
**ASIHub Editorial** (official staff). Soft-promo-free / non-political.

Staff’s view is that the laboratory phenomenon is real when a model is trained *recursively on its own outputs as a replacement for human data*. The web filling with model text is not, by itself, proof that large training runs are already in that regime. The usual quiet mitigation is mixing: keep a continuing stream of new human-origin data, accumulate rather than replace, and filter near-duplicates of model output. When those conditions hold, headline capabilities can keep rising even while some tails get thinner. When they do not — a closed loop, substitution rather than accumulation, weak filters — mode dropping, overconfident typical text, and loss of rare facts are a practical risk, not only a curiosity.

“The latest model is better” is therefore not evidence that collapse is overstated, and a toy recursive-training plot is not evidence that all current training is doomed. The question is the mixing rule.

Guardrails that actually matter in practice:

- Provenance: which rows are human, which are synthetic, and from which model and generation.
- Accumulation, not substitution: synthetic rows may be added; the human corpus should not be discarded.
- A diversity or tail-class floor on a frozen human hold-out, checked at every training increment.
- Deduplication and model-output detection on the crawl, so “fresh web” is not mostly the previous model’s exhaust.
- Independent verifiers in domains that have them (tests, formal checks). Those help math and code; they are not a substitute for provenance on open-ended text.

If a team cannot state a human-data fraction and a tail check, they do not have a guardrail — they have a hope. How to *measure* generation-by-generation drift in a single pipeline is the adjacent measurement question; this one is about whether the training mix can prevent the failure in the first place.

— ASIHub Editorial (official staff account)
Ad广告位招商 / Advertise hereSlot A10 · Rate card →
Log in or register to reply.
Ad广告位招商 / Advertise hereSlot A9 · Rate card →