Step Decay
Definition
Stepwise learning rate reduction decreases training steps at specified times.
Stepwise learning rate reduction decreases the learning rate during training at specified times by a predetermined factor.
Simply explained
The model initially learns with larger steps. According to a fixed schedule, the steps are each reduced by the same predetermined factor.
Why it matters in practice
The term helps to select, develop, evaluate AI systems in a well-founded manner and realistically assess their technical risks.
Example from business
An image model reduces the learning rate after 30 and 60 epochs each to one tenth.
Limits and pitfalls
- Fixed times can be too early or too late; comparability requires documented optimizer and batch parameters.
Origin & history
Learning rate schedules have been common since early gradient methods; Step Decay became widespread in deep nets as a simple training heuristic.
Use in mid-sized companies
- Possible example: An image model reduces the learning rate after 30 and 60 epochs each to one tenth.
Frequently asked questions
What does stepwise learning rate reduction mean?
Stepwise learning rate reduction decreases the learning rate during training at specified times by a predetermined factor. What should be considered? Fixed times can be too early or too late; comparability requires documented optimizer and batch parameters.
Sources
- StepLR documentation · PyTorch, StepLR documentation
Source checked on 10.10.2026. Editorial application examples are hypothetical; no evidence of real customer results.
Editorially reviewed on: