Loss Landscape
Definition
The loss landscape describes how the loss function changes over possible model parameters.
In deep networks, this space is high-dimensional and non-convex. Visualizations show only projections and can be distorted by parameterization.
Simply explained
Like a huge mountain range where training searches for low valleys.
Why it matters in practice
Relevant for understanding optimization, minima, and training stability.
Example from business
A research team compares how architecture and learning rate change local loss projections.
Limits and pitfalls
- Do not interpret 2D visualization as the complete landscape; do not automatically equate flat minima with better generalization.
Origin & history
Li et al. published widely used visualization methods for neural network loss landscapes in 2018.
Use in mid-sized companies
- Possible application in SMEs: Primarily use for model research and training diagnostics.
Frequently asked questions
Is the loss landscape directly representable?
Only strongly projected. Is the deepest minimum always the best generalization? No.
Sources
- Visualizing the Loss Landscape · Li et al. (2018), Visualizing the Loss Landscape
Sources checked on 10.10.2026.
Editorially reviewed on: