High-Level Representation
Definition
An abstract representation maps task-relevant features instead of individual raw data details.
In representation learning, data is transformed into features that can support further predictions. Deeper model architectures can increasingly compose abstract features from simpler representations. Depth alone does not guarantee this.
Simply explained
Imagine an image: you recognize not only color dots but shapes and objects. A model comparison explains the abstraction idea but does not claim human understanding in the computer.
Why it matters in practice
The chosen representation influences which differences a subsequent model can use. Therefore, it should be evaluated based on the actual task.
Example from business
Possible example: An image inspection uses learned shape features to distinguish component variants instead of comparing each pixel individually.
Limits and pitfalls
- An abstraction can hide useful details. A representation helpful for one task may not have the same quality for another.
Origin & history
Bengio, Courville, and Vincent discuss in their cited 2014 survey the relationship between depth and abstraction as well as the evaluation of learned representations. This is research evidence, not a claimed origin of the concept.
Use in mid-sized companies
- It is conceivable to test learned features for a limited document or image classification. The comparison should be done with representative operational test cases.
Frequently asked questions
Is a representation already a decision?
No. It can serve as input for another predictor.
Does more abstract automatically mean better?
No. What matters is usefulness for the specific task.
Sources
- Representation Learning: A Review and New Perspectives · Bengio et al. (2014): Representation Learning: A Review and New Perspectives
Editorially reviewed on: