Data engineering·Quality, tests and knowing it is wrong
nothing errored, every job is green, and the number on the board is wrong by eleven percent.
Data quality
Also calledfitness for purpose
Whether data is fit for the decision someone is about to make with it. It is deliberately a relative standard: the customer table that is perfectly adequate for a monthly trend is unusable for sending letters. Treating it as an absolute produces either endless tests nobody reads or none at all, so the useful version always starts with what the data is for and who is harmed when it is wrong.