Inference
Definition
Inference is the application of a trained model to new inputs to generate predictions or outputs.
During training, model parameters are adjusted; during inference, these parameters are typically used to compute results for new data. Costs, latency, and hardware requirements depend heavily on the model and workload.
Simply explained
Training is learning; inference is the subsequent application of what has been learned.
Why it matters in practice
Central to productive AI systems because this is where actual answers, classifications, or forecasts are produced.
Example from business
A trained classifier evaluates incoming support tickets and assigns probabilities to categories.
Limits and pitfalls
- Confusing inference with training; measuring model quality only offline; ignoring latency, costs, and data drift in operation.
Origin & history
In machine learning, inference refers to applying a trained model to inputs. More generally, inference is also used for drawing conclusions.
Use in mid-sized companies
- Possible applications: classification, prediction, embedding generation, speech and image generation.
Frequently asked questions
Are weights always changed during inference?
Usually not. Is inference free? No, it requires computing resources and incurs costs depending on operation.
Sources
- Inference · Google Machine Learning Glossary, Inference
Sources checked on 10.10.2026.
Editorially reviewed on: