Lion Optimizer
Definition
Lion is an optimizer discovered in 2023 by algorithmic search that tracks momentum and essentially controls updates via their sign.
It requires less optimizer state than Adam because no second moment is stored. The authors report advantages in several workloads but not universally.
Simply explained
How to use an optimizer that strongly controls direction via sign and requires less additional memory.
Why it matters in practice
Relevant for own deep learning training, not for normal model usage.
Example from business
A research team compares Lion and AdamW under identical learning rate sweeps.
Limits and pitfalls
- Presenting paper benchmarks as general superiority; directly adopting Adam learning rates; not retuning weight decay.
Origin & history
Chen et al. published Lion in 2023 as a result of symbolic optimizer search.
Use in mid-sized companies
- Possible application in SMEs: use only with own training and clean hyperparameter comparison.
Frequently asked questions
Is Lion a model?
No, an optimizer. Does Lion require the same learning rate as Adam? Typically not.
Sources
- Symbolic Discovery of Optimization Algorithms · Chen et al. (2023), Symbolic Discovery of Optimization Algorithms
Sources checked on 10.10.2026.
Editorially reviewed on: