Skip to content
    Flow-alerta.cab

    Cyclical Learning Rates (CLR)

    AI fundamentals & modelsCyclical learning ratesCyclical Learning RateCLRCyclical learning rate schedule
    Editorially reviewed on: 2 sources
    Your reading view

    The short answer · 16 words

    In short

    Training procedure in which the learning rate is repeatedly changed between fixed lower and upper limits.

    Flow-alerta.cab (2026): Cyclical Learning Rates (CLR). https://flow-alerta.cab/en/glossar/cyclical-learning-rates-clr. Editorially reviewed on: 10 October 2026.

    How much detail would you like?

    Practical

    Can support convergence and parameter search in neural networks. Advantages depend on model, data, optimizers and selected limits and are not guaranteed.

    One team performs a learning-rate-range test, then trains with triangular2-CLR and compares time, stability and test performance with a monotonous schedule.

    Testing in-house deep learning projects as an experimental schedule; documenting boundaries, cycle shape, seeds and comparison baseline.

    At a glance

    Core
    Instead of remaining constant or only decreasing, the learning rate follows a cycle, approximately triangular or in variants with decreasing amplitude. Boundaries, cycle length and shape must be chosen or estimated with a learning-rate-range test.
    Use
    Can support convergence and parameter search in neural networks. Advantages depend on model, data, optimizers and selected limits and are not guaranteed.
    Example
    One team performs a learning-rate-range test, then trains with triangular2-CLR and compares time, stability and test performance with a monotonous schedule.
    Limits
    Represent CLR as automatic tuning without making decisions; choose unsuitable limits; equate range testing and cyclical training; mix one-cycle policy, cosine annealing, and warm restarts; consider only training loss instead of validation performance.

    In context

    Cosine Annealing

    Method that changes the learning rate during model training along a cosine curve.

    Questions about this term

    Does the learning rate always have to go down?

    No.

    Does CLR remove any tuning?

    No.

    Is CLR the same as One-Cycle?

    No.

    How are borders chosen?

    Often with prior knowledge or a learning rate range test.

    All details and background

    Instead of remaining constant or only decreasing, the learning rate follows a cycle, approximately triangular or in variants with decreasing amplitude. Boundaries, cycle length and shape must be chosen or estimated with a learning-rate-range test.

    ELI5 – Explained simply

    During training, the step size is regularly larger and smaller again, so that the model tries out different learning steps.

    Why it matters in practice

    Can support convergence and parameter search in neural networks. Advantages depend on model, data, optimizers and selected limits and are not guaranteed.

    Example

    One team performs a learning-rate-range test, then trains with triangular2-CLR and compares time, stability and test performance with a monotonous schedule.

    Pitfalls and limits

    • Represent CLR as automatic tuning without making decisions; choose unsuitable limits; equate range testing and cyclical training; mix one-cycle policy, cosine annealing, and warm restarts; consider only training loss instead of validation performance.

    Origin & history

    Leslie N. Smith published the approach as a preprint in 2015 and at WACV in 2017. The original work reported reduced effort in learning rate search, but no general guarantee of success.

    Use in mid-sized companies

    • Testing in-house deep learning projects as an experimental schedule; documenting boundaries, cycle shape, seeds and comparison baseline.

    Related terms

    • Learning rate
    • Learning rate schedule
    • One-cycle policy
    • Cosine annealing
    • Warm Restarts
    • Optimizers
    • Range test

    Matching Flow-alerta service

    TrainingAI consulting – reproducible planning and evaluation of training experiments

    Matching tool / use case

    View use casesTools: PyTorch CyclicLR or Keras LearningRateScheduler; Use Case: Comparison of Cyclical and Monotonous Learning Rate Plans

    Sources

    • arxiv.orgSmith, Cyclical Learning Rates for Training Neural Networks
    • doi.orgIEEE WACV 2017

    Quellenprüfung: 10.10.2026. Definition und dokumentierte Einordnung abgeglichen; Beispiele beschreiben mögliche Anwendungen.

    Editorially reviewed on:

    Next step

    Put knowledge into practice

    Discuss your projectView use cases