Length Penalty
Definition
A length penalty modifies during decoding how sequence length factors into the score of a generated output.
Without adjustment, sums of log-probabilities can systematically favor longer or shorter sequences. Specific formulas vary between implementations.
Simply explained
How to adjust the evaluation score so that answers are not favored or penalized solely due to their length.
Why it matters in practice
Relevant for beam search, translation, and sequence generation.
Example from business
A translation model is compared with multiple length penalty values on quality and output length.
Limits and pitfalls
- Equalizing parameter definitions of different libraries; automatically rating longer answers as better.
Origin & history
Neural translation systems like Google's GNMT used explicit length normalization/penalty terms in beam search.
Use in mid-sized companies
- Possible application in SMEs: Configure sequence generation only according to measurable quality and length goals.
Frequently asked questions
Is length penalty universally defined?
No. Does it act during training? Typically during decoding.
Sources
- Google’s Neural Machine Translation System · Wu et al. (2016), Google’s Neural Machine Translation System
Sources checked on 10.10.2026.
Editorially reviewed on: