What is boosting?
Boosting is an ensemble learning technique that builds a strong model from many weak ones trained in sequence. A weak learner, typically a small decision tree, only needs to be somewhat better than guessing. Each new learner is trained to fix the mistakes of the combined model so far, and the final prediction is a weighted sum of all of them. AdaBoost (adaptive boosting), an early version, raised the weight of misclassified examples before each round. Gradient boosting, the form behind XGBoost and similar libraries, fits each new tree to the remaining errors, measured through the gradient of a loss function.
Bagging, the approach behind random forests, trains independent models in parallel and averages them to reduce variance. Boosting trains models one after another to reduce bias, so it usually reaches higher accuracy but overfits more easily.
Boosting in practice
An online apparel retailer wants to predict which orders will be returned. The first tree learns that orders containing the same item in several sizes usually come back. The second focuses on the returns the first one missed and finds that first-time buyers of footwear also return often. Later rounds pick up smaller patterns, such as particular brands that run small. After a few hundred rounds, the combined model flags risky orders well enough to trigger fit guidance at checkout. Because each round concentrates on the hardest cases, mislabeled records get extra weight, so clean labels and early stopping matter.