Context Compression
Definition
Context Compression condenses existing information for later model steps.
Extensive context is condensed for later model steps, for example by summary, token selection or special compacting elements. Details can be lost; task quality and obtained boundary conditions must be checked.
Simply explained
How to turn many notes into a short, useful summary.
Why it matters in practice
Relevant if long work processes burden the context window or the cost and latency budget.
Example from business
Hypothetical example: An assistant summarizes a long project course before the next task.
Limits and pitfalls
- Losing important exceptions, sources or uncertainty when shortening.
Origin & history
LLMLingua was published in 2023 as a procedure for rapid compression. Current APIs such as OpenAI Responses also offer their own compacting functions for longer processes.
Use in mid-sized companies
- Before and after consolidation, check whether tasks, sources, exceptions and open questions are preserved; test with representative follow-up tasks.
Frequently asked questions
Is context compression always loss-free?
No. Is it the same as caching? No. What should be checked? Whether important boundary conditions and sources for the follow-up task are preserved.
Sources
- LLMLingua · Jiang et al. (2023), LLMLingua
- Compaction · OpenAI, Compaction
Sources checked on 10/09/2026. Editorial application examples are hypothetical; no evidence of real customer results.
Editorially reviewed on: