Data engineering·Orchestration and running it in anger
you need to know whether last Tuesday's run of this model succeeded, and how long it took the last thirty times.
Pipeline run history
Also calledrun log, task instance history
The retained record of every execution, its parameters, its duration and its outcome. It is the difference between an orchestrator and cron, because most operational questions — is this slower than usual, when did this start failing, was this period ever run — are queries against it. It is also what makes durations a leading indicator: a job that has crept from four minutes to eleven is telling you something before it starts timing out.