Comparison
OffsetvsVisibility timeout
Offset
you record the position you have processed up to, so restarting the consumer does not start it from the beginning again.
The pointer marking how far a consumer has read in a partition. When you commit it decides your delivery semantics: committing before processing gives at-most-once, committing after gives at-least-once. Committing offsets on a timer while processing asynchronously is the usual way people get at-most-once by accident.
Full entry →Visibility timeout
the message is hidden from other workers while you process it, and reappears if you do not finish in time.
The window during which a claimed message is invisible to other consumers, after which it is redelivered. It is the mechanism that makes crash recovery automatic. Set it shorter than your actual processing time and you get concurrent duplicate processing of every slow message, which looks exactly like a race condition in your own code.
Full entry →