Data engineering·Time and correctness in streams
there is one streaming job, and correcting history means replaying the log through it rather than running a different batch job.
Kappa architecture
Also calledstream-only architecture, single-path architecture
Doing away with the separate batch path and treating reprocessing as replaying the same stream code over retained history. One implementation, one definition of every metric, and a reprocessing story that depends entirely on retention being long enough and the job being fast enough to catch up. It is the right default when your log keeps history; it is a trap when the log keeps seven days and someone asks for a two-year restatement.