What is feature importance?
Feature importance is a measure of how much each input variable, or feature, contributes to a model’s predictions. It is a basic tool of explainable AI. Global importance ranks features across the whole model. Permutation importance, a common method, shuffles one feature’s values in held-out data and records how much accuracy drops. Tree-based models also report built-in scores based on how much each feature improves their splits. Local importance explains a single prediction instead, and Shapley values (SHAP) and LIME (local interpretable model-agnostic explanations) are the usual tools for that.
Feature importance in practice
An auto insurer scores each new claim at first report for referral to its special investigations unit. The first importance ranking puts the number of adjuster notes at the top. Reviewers spot the problem quickly. Notes accumulate after a claim already looks suspicious, so the feature is a case of information leakage. Without it, the leading drivers are the gap between policy start and accident date, prior claims, and repair shop, which investigators find credible. The ranking and each removal are documented for the model risk management review.
What to watch for
- Importance shows what the model relies on and does not prove cause, so changing a top-ranked input in the real world may not change the outcome.
- Correlated features split credit between them, which can make each one look minor when together they dominate.
- A highly ranked feature such as ZIP code can stand in for a protected characteristic and calls for a fairness review.