What is a confusion matrix?
A confusion matrix is a table that compares a classifier’s predictions with the true classes on a set of test records. Each row is an actual class, each column is a predicted class, and each cell counts the records that fall in that combination. The diagonal holds the correct predictions. Everything off the diagonal is a mistake, and the position of the mistake shows what kind.
For a binary classifier the matrix has four cells, and those cells have names that the rest of the metrics are built on.
| Actual outcome | Predicted positive | Predicted negative |
|---|---|---|
| Actually positive | True positives | False negatives |
| Actually negative | False positives | True negatives |
Accuracy, precision, recall, specificity, the F1 score, and the false positive rate are all ratios of these four counts. A multi-class problem has a larger square matrix, and the same ratios can be computed for each class in turn.
Why a confusion matrix matters
A single score hides what the model gets wrong, and the matrix shows it. Two models with the same accuracy can have opposite error profiles, one that misses real cases and one that raises false alarms, and those carry very different business costs. The matrix is also the simplest artifact to put in front of a manager, because every cell corresponds to a situation the business recognizes: a flagged transaction that was fine, a defective part that shipped, a routine email that went to the escalation queue.
Confusion matrix in practice
A manufacturer deploys a visual inspection model on a packaging line to catch defective seals. The matrix for a week of inspected packages shows very few missed defects and a large count of good packages rejected. The quality manager can read directly that the model is tuned for safety and that the cost is rework volume. Lowering the confidence threshold would move counts from the false positive cell to the false negative cell, and the matrix makes that trade visible before it is made.
What to watch for
- Build the matrix on test data from a later time period than the training data, or it will look better than production will.
- Look at the raw counts as well as the rates, because a small cell can represent the most expensive outcome.
- For multi-class models, read the off-diagonal cells to find pairs of classes that need a clearer definition or more labeled examples.
- Recompute the matrix regularly in production using cases whose true outcome has since become known.