LLM Observability
Definition
LLM observability makes model calls, tool calls, latency, token consumption, errors, and relevant quality data traceable.
It extends classical observability with GenAI-specific signals. Prompt and response contents can be sensitive and should not be logged indiscriminately.
Simply explained
Like traces and measuring instruments for the entire path of an AI request.
Why it matters in practice
Relevant for debugging, cost, reliability, and productive agents.
Example from business
A trace shows model call, retrieval, two tool calls, token consumption, and a retry.
Limits and pitfalls
- Logging prompts with personal data uncontrolled; measuring only costs; no trace correlation across tools.
Origin & history
OpenTelemetry is developing GenAI semantic conventions for model and agent telemetry; the standard area is still under development.
Use in mid-sized companies
- Possible application in SMEs: Introduce traces, metrics, and privacy-compliant logs for productive AI flows.
Frequently asked questions
Is LLM observability just logging?
No. Must contents be recorded? No, metadata is often sufficient.
Sources
- OpenTelemetry GenAI Observability (2026) · OpenTelemetry GenAI Observability (2026)
- Semantic Conventions · Semantic Conventions
Sources verified on 10.10.2026.
Editorially reviewed on: