Greedy Decoding
Definition
Greedy Decoding selects the currently most probable next token at each generation step.
The method decides locally step-by-step for the token with the highest model probability and does not explore alternative sequences. The selection rule is deterministic; identical outputs in practice additionally require a constant, deterministic execution.
Simply explained
Like always taking the most obvious next word when writing, without checking later alternatives.
Why it matters in practice
Relevant for reproducible outputs and as a simple baseline with language models; often too rigid for creative or diverse texts.
Example from business
Possible example: An extraction workflow uses Greedy Decoding for repeatable short answers. For this, model, full context, configuration, and execution environment must be constant and execution deterministic.
Limits and pitfalls
- Confusing local decisions with globally optimal sequences; overlooking repetitions or monotonous texts; understanding Greedy Decoding as a quality guarantee.
Origin & history
Greedy methods predate modern language models. For neural text generation, Greedy Decoding is a fundamental deterministic decoding strategy; Holtzman et al. demonstrated limits of deterministic or maximization-near methods in open text generation.
Use in mid-sized companies
- Possible application: reproducible outputs in tests, structured extraction, and baselines when comparing decoding strategies.
Frequently asked questions
Is Greedy Decoding always best?
No, the locally most probable tokens do not necessarily yield the best overall sequence. Is the selection random? The Greedy selection rule is deterministic. Identical outputs additionally require constant, deterministic execution in practice.
Sources
- The Curious Case of Neural Text Degeneration · Holtzman et al. (2019), The Curious Case of Neural Text Degeneration
- Generation strategies · Hugging Face, Generation strategies
- Reproducibility · PyTorch, Reproducibility
Source check: 10./11.10.2026.
Editorially reviewed on: