Prompt Compression
Definition
Shorten model inputs while preserving as much utility as possible.
Prompt compression reduces the size of an input, for example by selecting, condensing, or removing less relevant parts.
Simply explained
From a long memo to a shorter version with the important notes.
Why it matters in practice
Shorter inputs can save context space and processing costs.
Example from business
A hypothetical system condenses long research excerpts before a summarizing answer.
Limits and pitfalls
- Meaning, limitations, and source attribution can be lost; quality must be measured afterwards.
Origin & history
Research tools like LLMLingua investigate automatic compression for language models.
Use in mid-sized companies
- Compare original and compressed versions with the same tasks.
Frequently asked questions
Is this the same as caching?
No, compression changes the input size.
Sources
Editorially reviewed on: