What is linear regression?
Linear regression is a method for predicting a number from one or more inputs by fitting a straight-line relationship. With one input it draws the line through a scatter of points that minimizes the squared vertical distances to them. With many inputs it fits a weighted sum: each input gets a coefficient, and the prediction is the sum of inputs times coefficients plus a constant. It is the oldest and simplest supervised learning method and the baseline against which other regression models are judged.
The coefficients are the model’s explanation. A coefficient of 2.4 on “machine hours” means each extra hour adds 2.4 units to the predicted cost, holding the other inputs fixed. Few other models offer that clarity.
How linear regression works
Fitting usually uses ordinary least squares, which has a closed-form solution and runs instantly on thousands of rows. The fit is assessed with the coefficient of determination (R-squared), the share of variation in the outcome that the inputs explain, and with error measures such as mean absolute error on held-out data. The method assumes that the relationship is roughly linear, that errors are of similar size across the range, and that the inputs are not near-duplicates of one another. Transforming inputs, for example by taking logarithms or adding squared terms, extends it to curved relationships while keeping the same fitting procedure.
Linear regression in practice
A machine shop quotes jobs by experience and loses money on a share of them. The estimator pulls three years of completed jobs and fits a regression of actual labor hours on part count, number of setups, material type, tolerance class, and whether the drawing was a repeat. The model explains most of the variation, and the coefficients show that setups cost far more than anyone assumed. Quotes now start from the model’s estimate, and jobs that land far from it are reviewed afterward to learn what the model is missing.
What to watch for
- Predictions outside the range of the training data are unreliable, because the line is only known where there were points.
- Highly correlated inputs make the coefficients unstable and hard to interpret, and ridge regression is the usual remedy.
- A few extreme records can drag the whole line, so inspect outliers before trusting the fit.
- A coefficient shows association, and treating it as a cause requires knowledge of how the data was generated.