One-Cycle Learning Rate Policy
Definition
A One-Cycle schedule lets the learning rate first rise and then drop sharply.
A training schedule where the learning rate initially increases and then significantly decreases, often together with inverse momentum.
Simply explained
During training, the steps first become larger and later significantly smaller again.
Why it matters in practice
Helps realistically assess model quality, limits, and technical decisions.
Example from business
An image model raises the learning rate up to a validated maximum and then sharply lowers it.
Limits and pitfalls
- Choosing the maximum learning rate without validation or adopting the schedule regardless of batch size and optimizer.
Origin & history
Leslie Smith introduced the 1cycle policy in 2018 for fast training.
Use in mid-sized companies
- Possible example: An image model raises the learning rate up to a validated maximum and then sharply lowers it.
Frequently asked questions
What does One-Cycle learning rate schedule mean?
A training schedule where the learning rate initially increases and then significantly decreases, often together with inverse momentum.
Which mistakes should be avoided?
Choosing the maximum learning rate without validation or adopting the schedule regardless of batch size and optimizer.
Sources
Source checked on 10.10.2026. Editorial application examples are hypothetical; no evidence of real customer results.
Editorially reviewed on: