Comparison
Full refreshvsIncremental model
Full refresh
you drop the table and rebuild it entirely from source, which is expensive, slow and definitely correct.
Rebuilding a model completely rather than incrementally. It is the answer to a logic change, a late-arriving correction, or an incremental model that has drifted, and it is the baseline every incremental build is an optimisation of. Keeping it working — rather than letting it rot until it no longer runs at all — is what preserves the ability to recover, and it is usually discovered to be broken at the worst possible moment.
Full entry →Incremental model
the model rebuilds only the last three days each run instead of the whole four-year table, and once a month you rebuild it fully anyway.
A transformation that processes and appends or merges only new or changed data rather than rebuilding from scratch. It is how a large table stays affordable to maintain, and it introduces the possibility of the incremental result diverging from what a full rebuild would produce. That is why a periodic full refresh is not redundancy but a correctness check, and why teams that never run one eventually cannot explain a discrepancy.
Full entry →