Comparison
At-least-once deliveryvsExactly-once processing
At-least-once delivery
the worker processed the message, crashed before acknowledging it, and the queue handed the same message to somebody else.
A guarantee that every message is delivered, possibly more than once. It is the default in nearly every broker, because acknowledging after processing is the only way to survive consumer crashes. It makes duplicate handling the consumer's problem, which is why an at-least-once consumer that is not idempotent is a bug waiting for a redeploy.
Full entry →Exactly-once processing
you make the duplicate delivery harmless by deduplicating on a key or committing the offset in the same transaction as the result.
The property that each message affects the result exactly once. Exactly-once *delivery* over an unreliable network is impossible; exactly-once *effect* is achievable, by pairing at-least-once delivery with idempotent processing or a transaction spanning the output and the offset commit. When a vendor says exactly-once, this is what they built, and it only holds inside their system boundary.
Full entry →