What is the bias-variance tradeoff?
The bias-variance tradeoff describes two sources of prediction error that pull in opposite directions as a model becomes more complex. Bias is error from a model that is too simple to represent the real relationship, which produces underfitting. Variance is error from a model so flexible that it changes substantially when trained on a slightly different sample, which produces overfitting. Adding complexity lowers bias and raises variance. Removing complexity does the reverse.
Total error on new data is roughly the sum of the two plus irreducible noise. The best model sits where the sum is lowest, which is rarely at either extreme.
Why the bias-variance tradeoff matters
The tradeoff explains why the most complex model is rarely the best one and why more training data changes the answer. With little data, a simple model wins because variance dominates. With a great deal of data, a complex model can earn its flexibility.
A hospital system building a no-show prediction model for outpatient clinics sees this directly. A logistic regression on a handful of features is stable across clinics but misses patterns specific to each one. A deep tree model captures clinic-level detail but swings wildly when retrained on a new quarter. Ensemble learning methods such as random forests exist largely to reduce variance without giving back much bias. Regularization, cross-validation, and a holdout set are the tools for finding the balance instead of guessing at it.