Data engineering·Time and correctness in streams
the writes are staged and only made visible when the checkpoint commits, so a crash halfway leaves nothing half-written.
Transactional sink
Also calledtwo-phase commit sink, atomic sink
An output that participates in the job's checkpointing, so results become visible only when the corresponding input position is durably recorded. This is the mechanism that turns at-least-once processing into exactly-once output. The visible cost is latency — nothing is readable until the next commit — which is why a streaming job with a transactional sink often has worse end-to-end freshness than an at-least-once one.