Input Tokens
Definition
The input units processed by the model.
Input tokens are the units resulting from tokenization of the model input, including transmitted instructions, parts of conversations, and additional information.
Simply explained
A text is broken down into small building blocks that the model processes.
Why it matters in practice
Their number influences context requirements and often the billing of a model call.
Example from business
A hypothetical assistant counts question, conversation history, and manual excerpts together as input.
Limits and pitfalls
- Token count depends on the tokenizer; word count and character count are not exact substitutes.
Origin & history
The term originates from tokenization and the use of language models.
Use in mid-sized companies
- Use the actually employed tokenizer or the usage information provided by the service.
Frequently asked questions
Do system instructions count?
If they are transmitted to the model, basically yes.
Sources
Editorially reviewed on: