What is a counterfactual explanation?
A counterfactual explanation describes the smallest change to a case’s inputs that would have flipped a model’s decision. Attribution methods such as Shapley values (SHAP) and LIME show which inputs mattered, while a counterfactual tells an affected person what would have to be different. “The order would have been released if no invoice were more than 60 days past due” is an example.
Software finds counterfactuals by searching for nearby inputs that the model scores differently. Useful ones change few features, propose realistic values, and only touch things the person can act on. Advice to be younger is useless, and changes to protected characteristics are inappropriate.
Counterfactual explanations in practice
A building supply distributor uses a model to decide which orders go on credit hold. A held order shows the customer what would release it, such as paying two overdue invoices or trimming the order to fit the remaining credit line. Many customers clear the hold themselves, and the credit team spends its time on cases that need judgment. The same approach suits lending, insurance, and hiring decisions, where laws such as the GDPR give people a right to meaningful information about automated decision-making.
What to watch for
- A counterfactual holds only for the model version that produced it, so a retrained model may not honor the change it promised.
- Detailed counterfactuals reveal where the decision boundary lies, which helps fraudsters as much as customers.
- Check that suggested changes are achievable and consistent with each other before a customer sees them.