Convergence
Definition
Convergence describes that an optimization or learning procedure approaches a stable point or target value.
Depending on the algorithm, convergence can be mathematically proven, empirically observed, or only approximated. A stable training loss does not guarantee good generalization.
Simply explained
Like a search whose steps become smaller and smaller and approach a stable result.
Why it matters in practice
Relevant for training, optimization, and diagnosing unstable learning processes.
Example from business
A model shows hardly any change in validation loss after several epochs and is checked for overfitting.
Limits and pitfalls
- Confusing stagnation with a good solution; considering only training loss; stopping too early without validation.
Origin & history
Convergence is a fundamental concept in numerical optimization and statistical learning.
Use in mid-sized companies
- Possible application in SMEs: jointly evaluating training curves with validation metrics, learning rate, and early stopping.
Frequently asked questions
Is converged equal to good?
No. Can a procedure converge and still generalize poorly? Yes.
Sources
- Deep Learning, Optimization · Goodfellow et al., Deep Learning, Optimization
Sources checked on 10.10.2026.
Editorially reviewed on: