Operator Fusion
Definition
Multiple computational operations are combined into a more efficient execution step.
A compiler or inference runtime merges suitable operations of a computation graph. This can reduce intermediate storage and call overhead.
Simply explained
Instead of placing a workpiece between multiple stations, one station performs connected steps together.
Why it matters in practice
This allows models to run faster or more efficiently on suitable hardware.
Example from business
Hypothetical example: A runtime executes a matrix operation and subsequent activation together.
Limits and pitfalls
- Not every operation can be combined. Numerical properties and hardware support must be preserved.
Origin & history
The technique originates from compiler optimization and has been applied to ML computation graphs.
Use in mid-sized companies
- SMEs mostly use it through existing inference tools rather than their own implementation.
Frequently asked questions
Does fusion change the technical model objective?
Normally, the same computation should be executed more efficiently.
Sources
- Graph Optimizations — · ONNX Runtime: Graph Optimizations —
Editorially reviewed on: