Next Sentence Prediction
Definition
Next sentence prediction checks sentence sequences during pretraining.
A pretraining objective that classifies whether one sentence in the original text follows another.
Simply explained
A model practices recognizing whether two sentences directly followed each other in the original text.
Why it matters in practice
Helps realistically assess model quality, limits, and technical decisions.
Example from business
An encoder evaluates the relationship between two sentence segments during pretraining.
Limits and pitfalls
- Portraying NSP as a mandatory component of every language model or confusing it with autoregressive sentence generation.
Origin & history
BERT introduced NSP in 2018 along with masked language modeling; later models sometimes omitted or replaced it.
Use in mid-sized companies
- Possible application: An encoder evaluates the relationship between two sentence segments during pretraining.
Frequently asked questions
What does next sentence prediction mean?
A pretraining objective that classifies whether one sentence in the original text follows another.
Which mistakes should be avoided?
Portraying NSP as a mandatory component of every language model or confusing it with autoregressive sentence generation.
Sources
Source checked on 10.10.2026. Editorial application examples are hypothetical; no evidence of real customer results.
Editorially reviewed on: