What is an AI incident?
An AI incident is an event in which the use or malfunction of an AI system harms people, property, or the business. Examples include a chatbot that invents a refund policy, a scoring model that treats one group of loan applicants unfairly, and an agent that emails a customer file to the wrong recipient. Harm that was narrowly avoided, such as a bad agent action stopped at an approval step, is usually logged as a near miss.
AI incidents differ from ordinary IT incidents because the system can run normally by every technical measure while producing harmful results. The cause may lie in the training data, the prompt, a document the model read, or a model update. Investigators rely on the AI audit log to reconstruct which model version, inputs, and tool calls produced the outcome. Under the EU AI Act, providers of high-risk AI systems must report serious incidents to regulators within fixed deadlines.
AI incident in practice
A food-service distributor runs an agent that applies supplier price changes to its enterprise resource planning (ERP) system. One supplier changed the layout of its notices, the agent read case prices as unit prices, and several hundred items were repriced before a buyer caught the error in a margin report. The team disabled the agent’s write access within the hour, used the audit log to find and reverse every change, and issued credits to affected customers. The fix added a rule that holds any large price change for a buyer’s approval.