Skip to content
    Flow-alerta.cab

    Exponential Linear Unit (ELU)

    AI fundamentals & modelsELU activation functionExponential linear unit
    Editorially reviewed on: 1 source
    Your reading view

    The short answer · 9 words

    In short

    Activation function with linear positive and exponential negative range.

    Flow-alerta.cab (2026): Exponential Linear Unit (ELU). https://flow-alerta.cab/en/glossar/exponential-linear-unit-elu. Editorially reviewed on: 10 October 2026.

    How much detail would you like?

    Practical

    Means (Important activation function for deeper networks in research and practice)

    Hypothetical example: A team compares ELU and ReLU in the same time series model based on convergence, quality and computing time.

    Experimental comparison of activation functions in deep prediction models.

    At a glance

    Core
    For positive inputs, f(x)=x, otherwise f(x)=alpha·(exp(x)−1). At alpha>0, the negative output approaches −alpha. Negative activations can bring the mean closer to zero; training benefits depend on architecture and task.
    Use
    Means (Important activation function for deeper networks in research and practice)
    Example
    Hypothetical example: A team compares ELU and ReLU in the same time series model based on convergence, quality and computing time.
    Limits
    Higher computing effort due to the exp(x) exponential function compared to the simple ReLU.

    In context

    Activation Function

    Mathematical function for converting signals in a neural network.

    Questions about this term

    Is ELU better than ReLU?

    No. It has a positive derivative at alpha>0 for finite negative inputs, but can saturate at strongly negative values.

    What is SELU?

    A scaled ELU variant; self-normalization requires suitable architectural and training conditions.

    All details and background

    For positive inputs, f(x)=x, otherwise f(x)=alpha·(exp(x)−1). At alpha>0, the negative output approaches −alpha. Negative activations can bring the mean closer to zero; training benefits depend on architecture and task.

    ELI5 – Explained simply

    A switch in the computer network, which leads straight ahead at positive signals, but slows down gently at negative signals, instead of cutting off abruptly.

    Why it matters in practice

    Means (Important activation function for deeper networks in research and practice)

    Example

    Hypothetical example: A team compares ELU and ReLU in the same time series model based on convergence, quality and computing time.

    Pitfalls and limits

    • Higher computing effort due to the exp(x) exponential function compared to the simple ReLU.

    Origin & history

    2015 by Clevert, Unterthiner and Hochreiter (JKU Linz) in 'Fast and Accurate Deep Network Learning by Exponential Linear Units'.

    Use in mid-sized companies

    • Experimental comparison of activation functions in deep prediction models.

    Related terms

    Matching Flow-alerta service

    AI opportunity analysis & roadmapAI consulting: Examine the suitability, data and prerequisites of a possible application case on the subject of “Exponential Linear Unit (ELU)”.

    Matching tool / use case

    View use casesUse Case: Compare activation functions in training / Tool example: PyTorch torch.nn.ELU

    Sources

    Quellenprüfung: 10.10.2026. Definition und dokumentierte Einordnung abgeglichen; Beispiele beschreiben mögliche Anwendungen.

    Editorially reviewed on:

    Next step

    Put knowledge into practice

    Discuss your projectView use cases