Cross-Attention
The short answer · 11 words
In short
Attention mechanism, where queries access keys and values from another representation.
Flow-alerta.cab (2026): Cross-Attention. https://flow-alerta.cab/en/glossar/cross-attention. Editorially reviewed on: 09 October 2026.
How much detail would you like?
Practical
Relevant to understanding translation models and multimodal information linking.
Example scenario: A translation decoder uses the representation of the initial set for the next output words.
When selecting models, check how different inputs are connected; evaluate practical quality based on the task.
At a glance
- Core
- Cross-attention connects two streams of information. In the classic encoder-decoder transformer, queries come from the decoder and keys as well as values from the encoder. In self-attention, they come from the same stream. The mechanism can also connect text and image information.
- Use
- Relevant to understanding translation models and multimodal information linking.
- Example
- Example scenario: A translation decoder uses the representation of the initial set for the next output words.
- Limits
- Confuse cross-attention with cross-encoder; interpret attention weights as a safe explanation; consider any transformer to be encoder decoder.
In context
Questions about this term
Is it self-attention?
No.
Does every LLM include cross-attention?
No.
Is it just for translation?
No.
All details and background
Cross-attention connects two streams of information. In the classic encoder-decoder transformer, queries come from the decoder and keys as well as values from the encoder. In self-attention, they come from the same stream. The mechanism can also connect text and image information.
ELI5 – Explained simply
When formulating, you look specifically at the appropriate places of another template.
Why it matters in practice
Relevant to understanding translation models and multimodal information linking.
Example
Example scenario: A translation decoder uses the representation of the initial set for the next output words.
Pitfalls and limits
- Confuse cross-attention with cross-encoder; interpret attention weights as a safe explanation; consider any transformer to be encoder decoder.
Origin & history
The Transformer architecture of Vaswani and colleagues made Encoder-Decoder Attention 2017 a central building block.
Use in mid-sized companies
- When selecting models, check how different inputs are connected; evaluate practical quality based on the task.
Related terms
- Self-attention
- Query
- Key
- Value
- Encoders
- Decoders
- Transformers
Matching Flow-alerta service
AI opportunity analysis & roadmapAI consulting – understandably classify model architectures and multimodal processing.Matching tool / use case
View use casesUse Case: Linking of source text and translation in the encoder-decoder model.Sources
- arxiv.orgVaswani et al. (2017), Attention Is All You Need, Encoder-Decoder Attention
Quellenprüfung: 09.10.2026. Definition und dokumentierte Einordnung abgeglichen; Beispiele beschreiben mögliche Anwendungen.
Editorially reviewed on: