Comparison
Initial snapshotvsLog-based CDC
Initial snapshot
before the change feed is any use you have to copy the table as it stands today, while it is still being written to.
The one-off full copy that gives a change feed something to apply changes to. The interesting part is the handover: rows are being modified while the snapshot is running, so the snapshot and the stream overlap, and the standard resolution is to start capturing the log first and let idempotent upserts settle the overlap. Getting the handover wrong produces a table that is right on average and wrong for exactly the rows that were busy.
Full entry →Log-based CDC
you tail the database's own replication log, so a delete and a mid-second update both come through without the source running a single extra query.
Capturing changes by reading the database's write-ahead or binary log, the same feed a replica consumes. Because the log is the truth of what happened, you get every intermediate state, hard deletes and ordering for free, and you put no query load on the source. The price is operational: it needs a privileged connection, it is coupled to the source's storage engine and version, and if your consumer is down long enough for the log to be rotated away, recovery means a new snapshot.
Full entry →