Token
Definition
Tokens are text modules into which a model breaks down inputs and outputs.
A token is a processing unit of the model. How text is divided into tokens and billed depends on the respective service and model.
Simply explained
Like small pieces of a set that the system processes individually.
Why it matters in practice
Helps to realistically classify the possibilities, quality and limitations of AI systems.
Example from business
Possible example: Before a long document workflow, a team estimates input and output length.
Limits and pitfalls
- Equate characters, words and tokens or estimate costs based on input length only.
Origin & history
Tokenization is part of modern language model processing. The OpenAI documentation explains token types and counting; there is no fixed general conversion of words into tokens.
Use in mid-sized companies
- Possible application in SMEs: Before a long document workflow, a team estimates input and output length.
Frequently asked questions
What does token mean?
Tokens are text modules into which a model breaks down inputs and outputs.
What errors should be avoided?
Equate characters, words and tokens or estimate costs based on input length only.
Sources
- Understanding and counting tokens · OpenAI, Understanding and counting tokens
Sources checked on 09.10.2026. Editorial use cases are hypothetical; no evidence of real customer results.
Editorially reviewed on: