Gradient Descent
The short answer · 15 words
In short
Gradient descent optimizes parameters by changing them incrementally against the gradient of a target function.
Flow-alerta.cab (2026): Gradient Descent. https://flow-alerta.cab/en/glossar/gradient-descent. Editorially reviewed on: 10 October 2026.
How much detail would you like?
Practical
It is the basis of many ML training procedures, but does not guarantee a global optimum for non-convex problems.
Hypothetical example: A regression model minimizes the mean quadratic error over mini-batches.
Test learning curves and learning rate; additionally evaluate performance on independent cases.
At a glance
- Core
- The gradient shows locally the direction of the greatest increase; learning rate and variant determine the updating. Mini-batch-SGD uses estimates instead of the full gradient.
- Use
- It is the basis of many ML training procedures, but does not guarantee a global optimum for non-convex problems.
- Example
- Hypothetical example: A regression model minimizes the mean quadratic error over mini-batches.
- Limits
- Learning rate too large or small; unscaled features; saddles; equating training loss with business benefits.
In context
Questions about this term
Is SGD always stochastic?
It uses random examples or mini-batches.
Does it find the global optimum?
Not in general.
All details and background
The gradient shows locally the direction of the greatest increase; learning rate and variant determine the updating. Mini-batch-SGD uses estimates instead of the full gradient.
ELI5 – Explained simply
You take small steps downhill to reduce the loss.
Why it matters in practice
It is the basis of many ML training procedures, but does not guarantee a global optimum for non-convex problems.
Example
Hypothetical example: A regression model minimizes the mean quadratic error over mini-batches.
Pitfalls and limits
- Learning rate too large or small; unscaled features; saddles; equating training loss with business benefits.
Origin & history
Gradient methods come from optimization; backpropagation made them efficient for neural networks.
Use in mid-sized companies
- Test learning curves and learning rate; additionally evaluate performance on independent cases.
Related terms
- SGD
- Learning rate
- Backpropagation
- Optimizers
- Loss
Matching Flow-alerta service
AI opportunity analysis & roadmapAI consulting: Examine tasks, data and prerequisites of a possible AI use case.Matching tool / use case
View use casesGradient descent and mini-batch SGD; learning rate comparison and learning curves.Sources
Quellenprüfung: 10.10.2026. Definition und dokumentierte Einordnung abgeglichen; Beispiele beschreiben mögliche Anwendungen.
Editorially reviewed on: