What is logistic regression?
Logistic regression is a model for predicting the probability that something belongs to one of two classes: the account pays or defaults, the patient is readmitted or is not, the transaction is fraudulent or legitimate. Despite the name, it is a classification method. It computes a weighted sum of the inputs, as linear regression does, and then passes the sum through the logistic function, an S-shaped curve that converts any value into a probability between zero and one.
The coefficients are interpretable as changes in the log-odds of the outcome, and when exponentiated they become odds ratios that credit officers and clinicians are used to reading. That is why logistic regression is the default model in credit scoring, medical risk scores, and many regulated settings.
How logistic regression works
Training finds the coefficients that make the observed outcomes most likely, a procedure called maximum likelihood estimation, usually solved by gradient descent. The output is a probability, so turning it into a decision requires a confidence threshold. A threshold of 0.5 treats both kinds of mistake as equally costly, which is rarely true in business, so the threshold is tuned on validation data against the cost of false positives and false negatives. Extensions handle more than two classes and add regularization to keep coefficients stable when there are many inputs.
Logistic regression in practice
A collections agency wants to concentrate its callers on accounts that can actually be recovered. A logistic regression is trained on closed accounts, with the label marking whether a payment arrangement was reached within ninety days. Inputs include balance, days since placement, number of prior contacts, whether a phone number was verified, and the creditor type. Each morning the model scores the open inventory, and the dialer works the highest probabilities first. Because the coefficients are visible, the compliance team can confirm that no protected characteristic drives the score.
What to watch for
- Check calibration. A model that says 0.7 should be right about seven times in ten on records given that score.
- Choose the threshold from the business costs of each error type, and revisit it when those costs change.
- Rare outcomes need class balance techniques, or the model will favor predicting the common class.
- Logistic regression captures straight-line effects only, so add interaction terms or compare with gradient-boosted trees when accuracy falls short.