Data engineering·Orchestration and running it in anger
rows that fail validation go to a side table with the reason attached instead of failing the load or being silently dropped.
Quarantine table
Also calledreject table, error table, dead-letter table
A destination for records that could not be processed, retained with enough context to diagnose them. It is the third option between stopping the pipeline and losing data quietly, and it is what makes 'we processed 99.4 percent of rows' a statement someone can act on. It only works if something watches its row count, otherwise it is a place data goes to be forgotten with extra steps.