LARS Optimizer
Definition
LARS is an optimization method for very large batch sizes that adjusts learning rates layer-wise according to parameter and gradient norms.
It was developed to stabilize training of large image models with greatly increased batch sizes. Benefit depends on the workload.
Simply explained
Giving each layer its own scaling of the step size.
Why it matters in practice
Primarily relevant for large-scale distributed training.
Example from business
A research team compares Momentum-SGD and LARS with identical batch and compute budgets.
Limits and pitfalls
- Transfer results from ResNet to other models; maximize batch size without generalization testing.
Origin & history
You et al. published LARS in 2017/2018 in the context of ImageNet training with large batches.
Use in mid-sized companies
- Possible application in medium-sized businesses: Use only for own large-scale training with reproducible benchmarks.
Frequently asked questions
Is LARS an inference method?
No. Is it always better than SGD? No.
Sources
- Large Batch Training of Convolutional Networks · You et al. (2017), Large Batch Training of Convolutional Networks
Sources checked on 10.10.2026.
Editorially reviewed on: