jargon

Data engineering·Quality, tests and knowing it is wrong

the average order value did not change but the shape did, because a whole segment of small orders stopped arriving.

Distribution check

Also calledprofile check, statistical test, distribution shift

Comparing a column's distribution — percentiles, category frequencies, cardinality — against its recent history. It catches the class of problem that passes every structural test: right types, right row count, wrong data. It is the most likely check to produce false alarms, so it works best as a warning that a human reviews rather than as something that blocks a pipeline.

Commonly confused with