jargon

Comparison

Column-level lineagevsData lineage

Column-level lineage

you need to know whether dropping this one column breaks anything, not whether the table is used.

Lineage tracked per column rather than per table, so a change can be scoped to the fields that actually feed something. Table-level lineage answers 'is this used', which for a wide table is always yes and therefore useless for making a decision. The column-level version is what turns a deprecation from a negotiation with everyone into a message to three people.

Full entry →

Data lineage

the number is wrong and you need to know every table between it and the source, without reading forty SQL files.

The recorded graph of which datasets are derived from which, ideally parsed from the transformation code rather than maintained by hand. It is what makes both directions of an incident tractable: upstream to find the cause, downstream to find who else is now wrong. Hand-maintained lineage is worse than none, because it is trusted and stale at the same time.

Full entry →

Related comparisons