What is ridge regression?
Ridge regression is linear regression with a regularization term added to the fitting objective. Ordinary least squares chooses coefficients that minimize the squared prediction errors alone. Ridge minimizes those errors plus a penalty proportional to the sum of the squared coefficients. The penalty pulls every coefficient toward zero, and a single setting, usually written lambda, controls how hard it pulls. Lambda is a hyperparameter chosen by cross-validation.
The effect is most visible when inputs are highly correlated. Plain regression cannot tell which of two near-identical inputs deserves the credit, so it assigns huge offsetting coefficients that swing wildly from one sample to the next. Ridge shares the credit between them and produces coefficients that hold steady on new data. Lasso regression is the sibling that penalizes the absolute values of the coefficients instead, which drives some to exactly zero and so performs feature selection.
Why ridge regression matters
A county assessor’s office models property values from more than a hundred attributes, many of which move together, such as square footage, room count, and lot size. Ordinary regression gives coefficients that flip sign between neighborhoods and cannot be defended at an appeal hearing. Ridge regression produces a stable, explainable set of coefficients and lower error on the held-out parcels. The same reasoning applies to any tabular model with more features than the data can comfortably support, which is why ridge is the first thing to try when a linear model overfits.