k-Nearest Neighbors (kNN)
Definition
The k-nearest neighbors method classifies or predicts based on the k most similar training examples.
For a new input, the nearest data points are searched according to a distance metric; their classes or values determine the output. The method essentially stores the training data instead of learning a parametric model.
Simply explained
Like asking the k most similar known cases what happened to them.
Why it matters in practice
Relevant as a simple baseline and for small to medium structured datasets.
Example from business
A quality case is assigned to a fault class based on similar historical measurement profiles.
Limits and pitfalls
- Do not scale features; choose k arbitrarily; ignore high dimensions; adopt distance metric without verification.
Origin & history
Cover and Hart analyzed the statistical properties of nearest-neighbor classification in 1967.
Use in mid-sized companies
- Possible application in SMEs: Use as an understandable baseline for classification or regression and compare against more complex models.
Frequently asked questions
What does k mean?
Number of neighbors. Does the ordinary kNN learn parametric model weights? No, it uses stored training examples. Extensions with a learned distance metric must be distinguished from this.
Sources
- Nearest Neighbor Pattern Classification · Cover & Hart (1967), Nearest Neighbor Pattern Classification
- Nearest Neighbors, Abschnitte 1.6.2–1.6.3 · scikit-learn, Nearest Neighbors, Abschnitte 1.6.2–1.6.3
Source check: 10./11.10.2026.
Editorially reviewed on: