Hyena
Definition
Hyena is a sequence architecture introduced in 2023 with implicitly parameterized long convolutions and data-dependent gating structure.
The architecture was studied as a subquadratic alternative to the classical attention operator for long sequences. It is a research approach, not a general industry standard.
Simply explained
How to process long text sequences with specialized filters and gates instead of directly comparing every element with every other.
Why it matters in practice
Relevant for research on more efficient long sequences and alternatives to attention.
Example from business
A research team benchmarks Hyena against transformer and state-space models on long technical sequences.
Limits and pitfalls
- Presenting paper results as a blanket production superiority; confusing the architecture with a finished commercial product; ignoring hardware effects.
Origin & history
Poli et al. published in 2023 "Hyena Hierarchy: Towards Larger Convolutional Language Models."
Use in mid-sized companies
- Possible application in medium-sized businesses primarily indirect: technical evaluation of specialized open-weight models or research prototypes.
Frequently asked questions
Is Hyena a transformer?
No. Is it a general recommendation for LLM projects? No, selection must be based on available models and workloads.
Sources
- Hyena Hierarchy · Poli et al. (2023), Hyena Hierarchy
Sources checked on 10.10.2026.
Editorially reviewed on: