Self-Attention
Definition
Self-attention weights relationships within the same input sequence.
A mechanism where elements of a sequence weight and consider other elements of the same sequence.
Simply explained
A word considers other words in the same sentence with varying strength to capture its context.
Why it matters in practice
The term helps to select, develop, evaluate AI systems in a well-founded manner and realistically assess their technical risks.
Example from business
A token weights other words in the sentence to form its context vector.
Limits and pitfalls
- Interpreting attention weights as a complete explanation or ignoring masking and position.
Origin & history
Self-attention was researched before the Transformer and became its central component in 2017.
Use in mid-sized companies
- Possible example: A token weights other words in the sentence to form its context vector.
Frequently asked questions
What does self-attention mean?
A mechanism where elements of a sequence weight and consider other elements of the same sequence. What mistakes should be avoided? Interpreting attention weights as a complete explanation or ignoring masking and position.
Sources
Source verified on 10.10.2026. Editorial application examples are hypothetical; no proof of real customer results.
Editorially reviewed on: