What is XGBoost?
XGBoost, short for Extreme Gradient Boosting, is a free, open-source library that trains gradient-boosted decision trees. It began as a university research project and became the standard tool for tabular prediction after a long run of wins in public data science competitions. It works from Python, R, Java, Scala, and C++, trains on ordinary processors or graphics processing units (GPUs), and can spread training across a cluster for very large datasets.
Several engineering choices explain its accuracy and speed. A penalty on tree complexity is built into the training objective to curb overfitting. Records with missing values follow a default branch learned during training, so gaps in the data need no special handling. By default, split points come from binned summaries of each column, which is much faster than checking every raw value. LightGBM and CatBoost are open-source libraries that take a similar approach with different trade-offs.
XGBoost in practice
A property and casualty insurer uses XGBoost to estimate the eventual cost of each new claim from details captured at first notice of loss, such as cause, policy type, location, and reporting delay. Claims predicted to be large go straight to senior adjusters. The model trains in minutes on an existing server, and its built-in Shapley value output shows adjusters the main reasons behind each estimate. Many commercial prediction products run XGBoost or a similar library underneath, so the useful questions for a vendor concern data preparation, validation, and monitoring.