Calibration
Definition
Calibration describes how well predicted probabilities correspond to actually observed frequencies.
If a model outputs about 80% probability for many cases, then with good calibration approximately 80% of those cases should be correct. Calibration is different from pure accuracy.
Simply explained
How to ensure that "80% confident" corresponds long-term to about 80 out of 100 correct cases.
Why it matters in practice
Relevant for decisions with thresholds, risk, and human review.
Example from business
A classifier escalates cases below 70% confidence; beforehand it is checked whether its probabilities are actually calibrated.
Limits and pitfalls
- Interpreting confidence as probability without checking calibration; using ECE alone; transferring calibration to other data distributions.
Origin & history
Guo et al. showed in 2017 that modern neural networks are often poorly calibrated and that Temperature Scaling can be helpful in many cases.
Use in mid-sized companies
- Possible application: Evaluate model probabilities for decision thresholds and prioritizations and specifically assign uncertain cases to human review.
Frequently asked questions
Is a more accurate model automatically better calibrated?
No. What is Temperature Scaling? A simple post-hoc calibration method with one parameter.
Sources
- On Calibration of Modern Neural Networks · Guo et al. (2017), On Calibration of Modern Neural Networks
Sources checked on 10.10.2026.
Editorially reviewed on: