BERT
The short answer · 12 words
In short
2018 Google introduced Transformer Encoder model family for pre-trained, bidirectional voice representations.
Flow-alerta.cab (2026): BERT. https://flow-alerta.cab/en/glossar/bert. Editorially reviewed on: 09 October 2026.
How much detail would you like?
Practical
BERT has shaped modern NLP systems and is the basis of many encoder models, embeddings and evaluation metrics. For generative chatbots, however, the pure BERT architecture is not the typical decoder.
Example scenario: A fine-tuned BERT model classifies incoming support messages by topic.
For classification, semantic representation or special NLP tasks, consider when smaller encoder models may be more economical than large generative models.
At a glance
- Core
- BERT processes text with a Transformer encoder and learns contextual information from both directions of a sequence. The original model was pre-trained on large unlabeled sets of text and then fine-tuned for tasks such as classification, question answering or sentence relationships.
- Use
- BERT has shaped modern NLP systems and is the basis of many encoder models, embeddings and evaluation metrics. For generative chatbots, however, the pure BERT architecture is not the typical decoder.
- Example
- Example scenario: A fine-tuned BERT model classifies incoming support messages by topic.
- Limits
- Equate BERT with a generative chat model; call each Transformer model BERT; confuse fine tuning and pre-training; consider newer special models untested as fundamentally better.
In context
Questions about this term
Is BERT an LLM in today’s chatbot sense?
Not typically.
Is BERT generative?
The original BERT is primarily an encoder model.
Can BERT be fine-tuned?
Yes.
All details and background
BERT processes text with a Transformer encoder and learns contextual information from both directions of a sequence. The original model was pre-trained on large unlabeled sets of text and then fine-tuned for tasks such as classification, question answering or sentence relationships.
ELI5 – Explained simply
In text representation, BERT not only reads from left to right, but also considers words on both sides to better reflect their context.
Why it matters in practice
BERT has shaped modern NLP systems and is the basis of many encoder models, embeddings and evaluation metrics. For generative chatbots, however, the pure BERT architecture is not the typical decoder.
Example
Example scenario: A fine-tuned BERT model classifies incoming support messages by topic.
Pitfalls and limits
- Equate BERT with a generative chat model; call each Transformer model BERT; confuse fine tuning and pre-training; consider newer special models untested as fundamentally better.
Origin & history
Devlin, Chang, Lee and Toutanova introduced BERT in 2018 as a pre-trained bidirectional transformer model; the work appeared at NAACL in 2019.
Use in mid-sized companies
- For classification, semantic representation or special NLP tasks, consider when smaller encoder models may be more economical than large generative models.
Related terms
- Transformers
- Encoders
- Fine tuning
- Embedding
- Masked Language Modeling
- BERTScore
Matching Flow-alerta service
AI opportunity analysis & roadmapAI consulting: AI potential analysis & roadmap on tasks, data and prerequisites of a possible first AI use case.Matching tool / use case
View use casesUse Case: Text classification or semantic representation with an encoder model.Sources
- aclanthology.orgDevlin et al., BERT, ACL Anthology
- arxiv.orgOriginalarbeit
Quellenprüfung: 09.10.2026. Definition und dokumentierte Einordnung abgeglichen; Beispiele beschreiben mögliche Anwendungen.
Editorially reviewed on: