Model Extraction
Definition
Model extraction attempts to replicate model behavior or properties through systematic queries.
Replicating the behavior or properties of a foreign model through systematic queries and observed outputs.
Simply explained
How to imitate the operation of a service from many observed responses; this does not imply copying all internal components.
Why it matters in practice
Helps realistically assess model quality, limits, and technical decisions.
Example from business
An attacker collects API responses and trains a substitute model with them.
Limits and pitfalls
- Equating normal benchmarking with an attack or failing to clarify the threat model and permissions.
Origin & history
Tramèr and co-authors demonstrated practical extraction attacks against ML services in 2016.
Use in mid-sized companies
- Possible application: A security team investigates the replicability of a service from API responses in an authorized test. It documents query budget, threat model, and protection effectiveness.
Frequently asked questions
What does model extraction mean?
Replicating the behavior or properties of a foreign model through systematic queries and observed outputs.
Which mistakes should be avoided?
Equating normal benchmarking with an attack or failing to clarify the threat model and permissions.
Sources
Source verified on 10.10.2026. Editorial use cases are hypothetical; no evidence of real customer results.
Editorially reviewed on: