What is a confidence score?
A confidence score is a number that accompanies a prediction and expresses how strongly the model favors it. For a classifier, it is typically the estimated probability of the predicted class, between zero and one. An optical character recognition engine reports a confidence for each field it reads. A language model can report the probability of each token it generates, and some systems convert that into a confidence for the whole answer.
The score is the model’s own opinion of itself. It comes from the same mechanism that produced the prediction, which means it can be wrong in the same ways. Whether a score of 0.9 corresponds to being right nine times in ten is a separate property called calibration, and many models are poorly calibrated out of the box.
Why a confidence score matters
Confidence scores are what make partial automation possible. In an accounts payable department processing supplier invoices with intelligent document processing, each extracted field arrives with a score. Fields above a chosen confidence threshold post straight to the ERP. Fields below it are routed to a clerk for review. The score decides the split between automatic and manual work, and raising or lowering the threshold moves that split.
Scores are also raw material for monitoring. A drop in average confidence across a week of invoices often means a new supplier format has appeared or a scanner has been replaced, long before anyone notices errors in the ledger.