Data engineering·Moving data out of the source
nothing at all happened for twenty-three hours and then a job read the whole day at once.
Batch processing
Also calledbatch job, batch
Processing a bounded chunk of data on a schedule, rather than each record as it arrives. Batch is the boring, correct default: the input stops moving while you work on it, so a run is reproducible, cheap to reason about and trivial to re-run. Its whole cost is latency — the answer is as old as the gap between runs — and the usual mistake is paying for streaming to remove a delay nobody downstream actually minded.