What is reporting bias?
Reporting bias is a distortion in data that comes from what people choose to write down. Unusual, bad, or emotionally charged events get recorded more often than routine ones. Ordinary successes are rarely documented. A dataset built from those records over-represents the memorable and under-represents the normal, and a model trained on it inherits the distortion.
Customer reviews are the familiar case. People who had a terrible experience or an exceptional one write reviews. People who received exactly what they ordered usually do not. A sentiment analysis model trained on reviews learns a world that is more extreme than the customer base. Large language models absorb the same effect from the text they are trained on, which is one reason they can overstate how often rare events occur.
Why reporting bias matters
Operational data suffers from reporting bias whenever recording an event is optional. A manufacturer’s safety database contains the incidents that were reported, so shifts with a culture of under-reporting look safer than they are. A field service team logs the repairs that failed in detail and the ones that worked in a word. A model built to predict failures from those logs learns that detailed notes mean trouble, which is a fact about the writers rather than the equipment.
The defense is to know how each dataset was produced and who had reason to leave things out. Datasheets for datasets exist to capture exactly that, and a bias detection review should ask the question before training begins.