What is data lineage?
Data lineage is a record of where data came from, what happened to it along the way, and where it went. It links each table, column, or metric to its upstream sources and to the transformations applied, such as filters, joins, currency conversions, and aggregations. It also links forward to the reports, extracts, and models that depend on it.
Lineage is captured in two ways. Tools harvest much of it automatically by parsing SQL, pipeline code, and the query logs of the warehouse or lakehouse. Spreadsheets, emailed files, and manual adjustments have to be documented by hand, and those gaps are where errors tend to hide. A data catalog usually displays lineage as a graph that a user can follow upstream or downstream.
Why data lineage matters
Lineage answers two recurring questions. The first is where a number came from, which auditors, examiners, and executives ask when a figure looks wrong. The second is what will break if a field changes, which engineers ask before every change to a source system. Without lineage, both answers take days of reading code and interviewing colleagues.
For AI governance, lineage extends to models. Each model version should record the datasets it was trained on, and each answer from a retrieval-augmented assistant should record the documents it drew from. Decision traceability applies the same idea to agents, linking each action to the data and instructions behind it. The EU AI Act expects providers of high-risk AI systems to document the origin of training data and how it was prepared.
Data lineage in practice
A regional bank’s board report shows a jump in loan delinquencies that the credit team does not believe. Lineage traces the figure from the dashboard to a warehouse table, then to a nightly job, and then to a field in the core banking system. The trace shows that a core system upgrade the previous weekend changed how days past due are counted for loans on payment deferral. The same graph lists every consumer of that field, including a loss forecasting model. That tells the bank which outputs to rerun and which to flag for model risk review.
What to watch for
- Ask whether a tool captures lineage at the column level or only at the table level, since most investigations need the column.
- Map the manual steps, such as spreadsheet adjustments, because automated harvesting cannot see them.
- Keep lineage current with every pipeline change, since a diagram drawn once goes out of date quickly.