PagedAttention
Definition
PagedAttention manages the attention memory in flexibly allocated blocks.
A memory management method for the KV cache of transformer inference that manages cache blocks non-contiguously modeled after virtual memory pages. The attention calculation itself is not redefined by this.
Simply explained
Intermediate results reside in suitable memory chunks instead of requiring a large contiguous space for each request.
Why it matters in practice
The term helps to select, develop, evaluate AI systems in a well-founded manner and realistically assess their technical risks.
Example from business
An inference server allocates KV cache blocks on demand and can controlledly reference shared prefix blocks.
Limits and pitfalls
- Equating PagedAttention with a new attention architecture or ordinary operating system paging and transferring performance values without identical hardware, load, and implementation.
Origin & history
The vLLM team introduced PagedAttention in 2023 and later published the work at SOSP.
Use in mid-sized companies
- Possible application: An inference server allocates KV cache blocks on demand and can controlledly reference shared prefix blocks.
Frequently asked questions
What does PagedAttention mean?
A memory management method for the KV cache of transformer inference that manages cache blocks non-contiguously modeled after virtual memory pages. The attention calculation itself is not redefined by this. What mistakes should be avoided? Equating PagedAttention with a new attention architecture or ordinary operating system paging and transferring performance values without identical hardware, load, and implementation.
Sources
Source checked on 10.10.2026. Editorial application examples are hypothetical; no proof of real customer results.
Editorially reviewed on: