What is LIME (Local Interpretable Model-Agnostic Explanations)?
LIME (Local Interpretable Model-Agnostic Explanations) is a method for explaining why a model made one particular prediction. It treats the model as a black box, sending it inputs and reading its outputs, so it works the same way on gradient-boosted trees, neural networks, and text classifiers. The result is a short list of features, each weighted by how strongly it pushed this prediction up or down.
The word “local” marks the main limitation. A complex model can behave very differently from one region of its inputs to another, so LIME describes only the neighborhood of the case being explained. Global feature importance, which ranks inputs across all predictions, answers a different question. Other explainable AI (XAI) methods include Shapley values (SHAP) and counterfactual explanations.
How LIME works
- LIME creates thousands of altered copies of the case, for example by deleting words from a document or changing values in a row of data.
- It sends each copy to the model and records the prediction.
- It weights each copy by its similarity to the original, so close variations count more than distant ones.
- It fits a simple linear model to the weighted results and reports that model’s coefficients as the explanation.
For images, LIME hides groups of similar pixels and highlights the regions whose removal changed the prediction most.
LIME in practice
A litigation support team uses a text classifier to flag potentially privileged emails before documents are produced to opposing counsel. LIME, run on a sample of flagged emails, shows that the model relies heavily on outside counsel’s name in email signatures. Routine scheduling emails that copied an attorney were being flagged, while legal advice forwarded without a signature was missed. The team retrains with signature blocks removed and runs LIME again to confirm that the message body now drives the result.
What to watch for
- Unstable results. Two runs on the same prediction can rank features differently because the altered copies are random, so fix the random seed, sample count, and neighborhood size, and record them during model validation.
- Unrealistic samples. Altered copies can combine values that never occur together, such as a brand-new account with a long payment history, so the explanation may reflect model behavior on impossible data.
- Regulated decisions. Shapley values produce contributions that add up exactly to the gap between a prediction and the average prediction, which makes them easier to defend than LIME in credit and insurance decisions.
- Manipulation. A model can be built to recognize LIME’s altered inputs and behave differently on them, so treat a vendor’s explanations as evidence to verify.