What is regression?
Regression is supervised learning where the thing being predicted is a number rather than a category. How many units a store will sell, how long a repair will take, what a house will appraise for, and how much a claim will cost are all regression problems. The model learns from past records where the outcome is known and outputs a numeric estimate for new ones. Classification is the counterpart for predicting categories, and many algorithms exist in both forms.
The word also names the oldest such method, linear regression, which fits a weighted sum of inputs. In modern use “regression” covers any method that predicts a continuous target, including tree-based models and neural networks.
Types of regression
- Linear regression fits a straight-line relationship and is the baseline for every numeric prediction problem.
- Regularized regression, such as ridge and lasso, adds a penalty on large coefficients to keep the model stable with many correlated inputs.
- Generalized linear models handle outcomes that are counts, rates, or skewed amounts, and are standard in insurance and healthcare.
- Tree-based regression, especially gradient-boosted trees, captures nonlinear effects and interactions and usually wins on tabular data.
- Neural network regression suits very large datasets or inputs such as images and sequences.
- Logistic regression carries the name but predicts a probability and is used for classification.
Regression in practice
A parcel carrier promises delivery windows and pays credits when it misses them. A gradient-boosted regression model predicts transit hours for each shipment from origin, destination, service level, weight, pickup time, and recent network congestion. The model is scored by mean absolute error and by how often the actual time falls outside the predicted window. Customer service quotes windows from the model, and the operations team watches the error by lane to spot routes where the network has changed.
What to watch for
- Pick the error metric to match the business. Mean absolute error treats all misses alike, while root mean square error punishes large misses heavily.
- Outliers and skewed targets distort many regression models, so inspect the distribution and consider transforming the target.
- A model is only trustworthy within the range of data it saw, so flag predictions that fall outside it.