Weight Initialization
The short answer · 9 words
In short
Weight initialization specifies starting values of trainable model parameters.
Flow-alerta.cab (2026): Weight Initialization. https://flow-alerta.cab/en/glossar/weight-initialization. Editorially reviewed on: 10 October 2026.
How much detail would you like?
Practical
Good initialization facilitates stable training, but does not replace a suitable architecture or learning rate.
Hypothetical example: A ReLU network starts with He initialization and is tested over several seeds.
Check the framework specifications during your own training and compare several documented random starts.
At a glance
- Core
- Initialization methods set start values and often take layer size and activation into account. Xavier/Glorot and He initialization should support appropriate scaling of signals and gradients. They do not prevent unstable training in all conditions.
- Use
- Good initialization facilitates stable training, but does not replace a suitable architecture or learning rate.
- Example
- Hypothetical example: A ReLU network starts with He initialization and is tested over several seeds.
- Limits
- Equalize all weights; do not document seed; consider initialization as the sole quality factor.
In context
Questions about this term
Why not zero all weights?
Neurons would learn symmetrically the same way.
Is one seed enough?
No, several runs are better.
All details and background
Initialization methods set start values and often take layer size and activation into account. Xavier/Glorot and He initialization should support appropriate scaling of signals and gradients. They do not prevent unstable training in all conditions.
ELI5 – Explained simply
The training starts with meaningfully distributed random values instead of arbitrary numbers.
Why it matters in practice
Good initialization facilitates stable training, but does not replace a suitable architecture or learning rate.
Example
Hypothetical example: A ReLU network starts with He initialization and is tested over several seeds.
Pitfalls and limits
- Equalize all weights; do not document seed; consider initialization as the sole quality factor.
Origin & history
Xavier/Glorot and He processes became dominant for deep nets in 2010 and 2015 respectively.
Use in mid-sized companies
- Check the framework specifications during your own training and compare several documented random starts.
Related terms
- Xavier initialization
- He initialization
- Activation function
- Gradient
- Seed
Matching Flow-alerta service
AI opportunity analysis & roadmapAI consulting: Examine tasks, data and prerequisites of a possible AI use case.Matching tool / use case
View use casesXavier/Glorot initialization and He initialization; documented random starts.Sources
Quellenprüfung: 10.10.2026. Definition und dokumentierte Einordnung abgeglichen; Beispiele beschreiben mögliche Anwendungen.
Editorially reviewed on: