Comparison
Characterisation testvsData diff
Characterisation test
you did not know what the function was supposed to do, so you wrote down what it currently does.
A test that records the existing behaviour of code, correct or not, so that a refactor can be shown not to change it. It is the standard first move in untested legacy code: you are not asserting the behaviour is right, only that you have not altered it. Bugs found this way get recorded as tests too, because in a system this old somebody downstream may already depend on them.
Full entry →Data diff
before merging, you run both versions of the model and see that exactly nine hundred rows differ, all in one region.
Comparing the output of a changed transformation against the current one, row by row. It converts 'I think this refactor is safe' into a list you can walk through, and it catches the accidental cross join that a row-count check would miss. It is expensive on large tables, which is why it is usually run on a sampled or single-partition basis, and that sampling has to be stated when the result is presented as evidence.
Full entry →