What is ONNX?
ONNX, the Open Neural Network Exchange, is an open standard format for representing trained machine learning models so that a model built in one framework can run in another tool or runtime. A model is exported as a graph of standard operators, and any runtime that understands those operators can execute it. ONNX Runtime, the reference engine, runs models on CPUs, GPUs, and specialized chips through hardware-specific back ends.
Exporters exist for PyTorch, TensorFlow, scikit-learn, and gradient-boosted tree libraries, so classical models travel the same way as neural networks. The runtime also fuses operations and quantizes weights, which often makes an exported model faster than the original.
Why ONNX matters
ONNX separates the training environment from the serving environment. A data team can train in Python and hand the model to a .NET, Java, or C++ application without installing Python on the production machine. A manufacturer trains a defect classifier in PyTorch, exports it to ONNX, and runs it inside the C# software on a ruggedized PC beside the line. The same path serves phones, browsers, and edge gateways. Large language models usually run on dedicated engines, but ONNX remains the common route for vision, tabular, and small text models.
What to watch for
- Verify that the exported model matches the original’s outputs on a test set, since export can change numerical details.
- Match the operator set version between exporter and runtime, or the model will not load.
- Custom layers may have no standard operator and need to be rewritten before export.