Data engineering·Moving data out of the source
the job reads all four hundred million rows every night because nobody could work out which ones had changed.
Full load
Also calledfull refresh load, truncate and load, full snapshot
Copying the entire source table on every run and replacing what was there. It is the simplest thing that is definitely correct: no cursor to get wrong, no missed updates, no drift between source and copy. It stops working on size and on politeness — a full read hammers the source database and the run time grows with the table, so the switch to incremental usually happens the week the job stops finishing before the working day.