Cache-Augmented Generation (CAG)
The short answer · 30 words
In short
Approach in which a limited body of knowledge is loaded into the long model context in advance and reused via a CV cache instead of retrieving documents for each request.
Flow-alerta.cab (2026): Cache-Augmented Generation (CAG). https://flow-alerta.cab/en/glossar/cache-augmented-generation-cag. Editorially reviewed on: 09 October 2026.
How much detail would you like?
Practical
Interesting for small to medium, stable collections of knowledge, if retrieval latency and errors are to be avoided and the context window is large enough.
Example scenario: An assistant for a compact product manual preloads the entire approved manual once and then answers several questions from the cached context.
Only check if knowledge, frequency of change and context windows fit; test against RAG in terms of quality, latency, cost and updating.
At a glance
- Core
- Cache-Augmented Generation was proposed as an alternative for knowledge tasks with manageable, relatively stable document stock. Relevant content is loaded into the context before the request and its key value states are cached. This eliminates the need for a separate retrieval stage at runtime. CAG is not more suitable for every knowledge base than RAG.
- Use
- Interesting for small to medium, stable collections of knowledge, if retrieval latency and errors are to be avoided and the context window is large enough.
- Example
- Example scenario: An assistant for a compact product manual preloads the entire approved manual once and then answers several questions from the cached context.
- Limits
- Present CAG as a blanket replacement for RAG; fully subpoena too large or frequently changing knowledge bases; ignore context boundaries and cache costs; forget to update the cache.
In context
Context Window
The context window limits how much content a model can consider at the same time.
Prompt Caching / Prefix Caching
Prompt caching uses already processed input parts again, as far as a provider supports this.
Knowledge Base
A knowledge base is a well-kept collection of knowledge, guidance and answers for a specific target group.
Questions about this term
Is CAG the same as RAG?
No.
Does CAG need a large context window?
For more extensive knowledge, yes.
Is CAG always faster or better?
No, it depends on the use case.
All details and background
Cache-Augmented Generation was proposed as an alternative for knowledge tasks with manageable, relatively stable document stock. Relevant content is loaded into the context before the request and its key value states are cached. This eliminates the need for a separate retrieval stage at runtime. CAG is not more suitable for every knowledge base than RAG.
ELI5 – Explained simply
Instead of searching for suitable documents for each question, the system loads a manageable knowledge base beforehand and keeps it ready for several questions in the model context.
Why it matters in practice
Interesting for small to medium, stable collections of knowledge, if retrieval latency and errors are to be avoided and the context window is large enough.
Example
Example scenario: An assistant for a compact product manual preloads the entire approved manual once and then answers several questions from the cached context.
Pitfalls and limits
- Present CAG as a blanket replacement for RAG; fully subpoena too large or frequently changing knowledge bases; ignore context boundaries and cache costs; forget to update the cache.
Origin & history
The term cache-augmented generation was proposed in late 2024 in a paper by Chan and co-authors as a simplified alternative to RAG for limited knowledge tasks.
Use in mid-sized companies
- Only check if knowledge, frequency of change and context windows fit; test against RAG in terms of quality, latency, cost and updating.
Related terms
- RAG
- CV cache
- Context windows
- Prompt caching
- Knowledge base
Matching Flow-alerta service
AI assistants & automationAI assistants & automation: test a suitable process with clear rights and human test points.Matching tool / use case
View use casesUse Case: Assistant based on a compact, rarely changed document stock.Sources
- arxiv.orgChan et al., Don't Do RAG: When Cache-Augmented Generation is All You Need for Knowledge Tasks
Quellenprüfung: 09.10.2026. Definition und dokumentierte Einordnung abgeglichen; Beispiele beschreiben mögliche Anwendungen.
Editorially reviewed on: