Skip to content
    Flow-alerta.cab

    Context Caching

    AI fundamentals & modelsPrompt cachingTemporary storage of recurring context
    Editorially reviewed on: 1 source
    Your reading view

    The short answer · 11 words

    In short

    Reuse of pre-processed input to more efficiently execute repeated model requests.

    Flow-alerta.cab (2026): Context Caching. https://flow-alerta.cab/en/glossar/context-caching. Editorially reviewed on: 09 October 2026.

    How much detail would you like?

    Practical

    Relevant for many requests with the same documents, system instructions or tool descriptions.

    Example scenario: An assistant answers different questions about the same comprehensive product catalogue.

    Identify recurring inputs, measure cache usage, and compare costs including storage against uncached calls.

    At a glance

    Core
    A provider can cache common prompt prefixes or explicitly created context objects. A cache hit reduces costs or processing effort depending on the API. Minimum length, validity period and billing are dependent on the provider. Caching does not change model weights.
    Use
    Relevant for many requests with the same documents, system instructions or tool descriptions.
    Example
    Example scenario: An assistant answers different questions about the same comprehensive product catalogue.
    Limits
    Presuppose cache hits; ignore storage fees or expiration; put dynamic content before common prefixes; reuse outdated information.

    In context

    Token

    Tokens are text modules into which a model breaks down inputs and outputs.

    Context Window

    The context window limits how much content a model can consider at the same time.

    Prompt Engineering

    Methodological formulation and structuring of instructions to achieve consistent and high-quality AI results.

    Questions about this term

    Is this training?

    No.

    Does it always save money?

    No, use and tariff decide.

    Does it enlarge the context window?

    No.

    All details and background

    A provider can cache common prompt prefixes or explicitly created context objects. A cache hit reduces costs or processing effort depending on the API. Minimum length, validity period and billing are dependent on the provider. Caching does not change model weights.

    ELI5 – Explained simply

    Like prepared working documents, which are ready again at the next request.

    Why it matters in practice

    Relevant for many requests with the same documents, system instructions or tool descriptions.

    Example

    Example scenario: An assistant answers different questions about the same comprehensive product catalogue.

    Pitfalls and limits

    • Presuppose cache hits; ignore storage fees or expiration; put dynamic content before common prefixes; reuse outdated information.

    Origin & history

    Current LLM APIs offer different caching mechanisms. Gemini documents implicit caching and also explicit cache objects for the Generate Content API.

    Use in mid-sized companies

    • Identify recurring inputs, measure cache usage, and compare costs including storage against uncached calls.

    Related terms

    Matching Flow-alerta service

    AI assistants & automationAI assistants & automation: test a suitable process with clear rights and human test points.

    Matching tool / use case

    View use casesTool: Gemini API Context Caching; explicit cache objects in the Generate Content API. Use Case: repeated queries of stable documents.

    Sources

    Quellenprüfung: 09.10.2026. Definition und dokumentierte Einordnung abgeglichen; Beispiele beschreiben mögliche Anwendungen.

    Editorially reviewed on:

    Next step

    Put knowledge into practice

    Discuss your projectView use cases