Data engineering·Orchestration and running it in anger
you enabled a pipeline with a start date last January and it immediately queued four hundred runs.
Catchup
Also calledscheduler catchup, missed run replay
The scheduler's behaviour of automatically running every interval between a pipeline's start date and now. It is genuinely useful for a new model over historical data and genuinely dangerous by default, because a paused pipeline resuming will try to do the same thing. It is worth knowing as a named setting because the accident it causes — a source system flattened by four hundred simultaneous extracts — is loud and entirely preventable.