What is a support vector machine (SVM)?
A support vector machine (SVM) is a supervised learning algorithm that classifies records by drawing a boundary between two classes. Many boundaries could separate the training data. The SVM picks the one with the widest margin, meaning the largest gap to the nearest points on either side. Those nearest points are the support vectors, and they alone define the boundary.
When no straight line separates the classes, a kernel function lets the SVM draw curved boundaries. The kernel compares records as if they were mapped into a higher-dimensional space. A soft margin lets a few training points fall on the wrong side, which keeps the model from bending around noise. A variant called support vector regression predicts numbers instead of classes.
Support vector machines in practice
SVMs were a standard text classifier before deep learning and remain a good option when labeled data is limited. In e-discovery, a litigation support team codes a few thousand documents as responsive or not responsive. An SVM trained on the word counts in those documents ranks the rest of the collection, and attorneys review the highest-ranked documents first.
What to watch for
- Kernel SVMs train slowly on millions of records, where logistic regression or tree ensembles are more practical.
- Scale features to comparable ranges first, or the column with the largest values will dominate the margin.
- The raw output is a distance from the boundary, so calibrate it before treating it as a probability.