Interpretability
Definition
Interpretability describes how well humans can understand how a model arrives at its outputs or what relationships it has learned.
The term is not uniformly defined by a single metric. Depending on the goal, it concerns global model structure, local decisions, feature influences, or understandable surrogate models.
Simply explained
Being able to understand why a system makes a decision – not just what result it outputs.
Why it matters in practice
Relevant for debugging, trust, governance, and high-risk decisions, but an explanation is not automatically correct or causal.
Example from business
A credit risk model is examined globally for dominant features and locally for the factors of a single evaluation.
Limits and pitfalls
- Confusing interpretability with causality; considering a nice visualization as proof; not validating the explanation method.
Origin & history
Lipton discussed in 2016/2018 the different meanings and goal conflicts of the term “Interpretability” in ML.
Use in mid-sized companies
- Possible applications: model auditing, error analysis, domain plausibility checks, and documented decision support.
Frequently asked questions
Is a linear model always interpretable?
Often easier, but context and feature engineering can complicate interpretation. Is XAI the same? XAI encompasses broader explanation methods.
Sources
- The Mythos of Model Interpretability · Lipton, The Mythos of Model Interpretability
- Four Principles of Explainable AI · NIST, Four Principles of Explainable AI
Sources checked on 10.10.2026.
Editorially reviewed on: