Superposition
Definition
Superposition describes how a neural network represents multiple features in overlapping internal directions.
Superposition in interpretability research denotes the joint representation of multiple features in overlapping directions of a limited activation space.
Simply explained
Several properties share the same limited internal space in the model. Their representations overlap instead of each having a completely separate area.
Why it matters in practice
The term helps to select, develop, evaluate AI systems in a well-founded manner and realistically assess their technical risks.
Example from business
A research team analyzes whether a sparse autoencoder decomposes overlapping activation patterns into more interpretable features.
Limits and pitfalls
- The concept is a research model and not fully confirmed for every layer; illustrative visualizations should not be overinterpreted.
Origin & history
Elhage and coauthors described Toy Models of Superposition in 2022 as a mechanistic explanation for polysemantic neurons.
Use in mid-sized companies
- Possible application: A research team analyzes whether a sparse autoencoder decomposes overlapping activation patterns into more interpretable features.
Frequently asked questions
What does Superposition mean?
Superposition in interpretability research denotes the joint representation of multiple features in overlapping directions of a limited activation space. What should you watch out for? The concept is a research model and not fully confirmed for every layer; illustrative visualizations should not be overinterpreted.
Sources
- Toy Models of Superposition · Elhage et al., Toy Models of Superposition
Source checked on 10.10.2026. Editorial use cases are hypothetical; no proof of real customer results.
Editorially reviewed on: