Model Collapse
Definition
Model collapse is a possible degradation during recursive training on insufficiently curated model-generated data.
A possible degradation of generative models during recursive training on insufficiently curated model-generated data, where especially rare regions of the original distribution may be lost.
Simply explained
Like repeatedly making copies from earlier copies, where rare details may be lost; this does not necessarily happen with every data mixing.
Why it matters in practice
Helps realistically assess model quality, limits, and technical decisions.
Example from business
A model is repeatedly fine-tuned on unfiltered synthetic texts from earlier models.
Limits and pitfalls
- Calling every quality decline model collapse or ignoring real data mixing and curation.
Origin & history
Shumailov and co-authors described the effect in 2023 as a preprint and published the work in 2024 in Nature.
Use in mid-sized companies
- Possible application: A research team documents the origin, mixing, and curation of synthetic training data. It checks quality and coverage of rare cases across model generations.
Frequently asked questions
What does model collapse mean?
A possible degradation of generative models during recursive training on insufficiently curated model-generated data, where especially rare regions of the original distribution may be lost.
Which mistakes should be avoided?
Calling every quality decline model collapse or ignoring real data mixing and curation.
Sources
- www.nature.com
- Ergänzender Primärbeleg · Ergänzender Primärbeleg
Source checked on 10.10.2026. Editorial use cases are hypothetical; no evidence of real customer results. Supplementary sources checked on 12.10.2026.
Editorially reviewed on: