Hyperparameter
Definition
Hyperparameters are settings of a learning procedure that are not learned directly from the training data in the same way as model parameters.
Examples include learning rate, regularization strength, tree depth, or batch size. They are set before or during a training process and often optimized using validation data.
Simply explained
Like adjustment knobs for training, while the actual model weights are learned from the data.
Why it matters in practice
Relevant for model quality, computational cost, and reproducibility.
Example from business
A classification model is compared with different regularization values and tree depths on a validation set.
Limits and pitfalls
- Confusing hyperparameters and model parameters; using the test set for optimization; not documenting the search space; considering only the best single run.
Origin & history
The term is established in machine learning to distinguish configurations of the learning process from learned parameters.
Use in mid-sized companies
- Possible applications: documented model optimization, cost/quality trade-off, and reproducible experiments.
Frequently asked questions
Is the learning rate a hyperparameter?
Yes. Are neural weights hyperparameters? No, they are typically learned model parameters.
Sources
- hyperparameter · Google Machine Learning Glossary, hyperparameter
Sources checked on 10.10.2026.
Editorially reviewed on: