Pretraining
Definition
Pretraining creates a broad starting basis before targeted model adaptation.
Training a model on a broad base task or large dataset before adapting it to specific target tasks.
Simply explained
A model first learns general patterns before further training for a specific task.
Why it matters in practice
The term helps to select, develop, evaluate AI systems in a well-founded manner and realistically assess their technical risks.
Example from business
A language model first learns general text patterns and is later specialized.
Limits and pitfalls
- Equating pretraining with ready suitability, factual accuracy, or legally secure use.
Origin & history
Pretraining was used in computer vision and NLP and scaled strongly through foundation models.
Use in mid-sized companies
- Possible application: A language model first learns general text patterns and is later specialized.
Frequently asked questions
What does pretraining mean?
Training a model on a broad base task or large dataset before adapting it to specific target tasks. What mistakes should be avoided? Equating pretraining with ready suitability, factual accuracy, or legally secure use.
Sources
Source checked on 10.10.2026. Editorial application examples are hypothetical; no evidence of real customer results.
Editorially reviewed on: