What is naive Bayes?
Naive Bayes is a classification method built on Bayes’ theorem, which updates the probability of a class as evidence arrives. For text classification, training amounts to counting how often each word appears in each class. To classify a new message, the model combines each class’s base rate with the likelihood of the words it contains and picks the most probable class.
The “naive” part is the assumption that features are independent once the class is known. For text, that means treating a message as a bag of words and ignoring that “past” and “due” tend to appear together. The assumption rarely holds, yet the method still classifies well because it only needs the right class to score highest.
Naive Bayes in practice
A city’s 311 center receives resident requests by web form and email. A naive Bayes classifier trained on past requests routes each new one to public works, sanitation, code enforcement, or parks. “Missed pickup” points to sanitation, while “pothole” points to public works. The model trains in seconds on an ordinary server and is retrained weekly as staff correct its routing. It also sets the baseline that a large language model must beat to justify its extra cost.
What to watch for
- The reported probabilities are usually too extreme, so calibrate them before using them as confidence scores.
- A word never seen with a class would zero out that class, which a smoothing setting prevents.
- Check accuracy for each request type, because rare types with few examples are misrouted most often.