Comparison
CompactionvsMerge-on-read table
Compaction
a maintenance job rewrites thousands of small files into a few large sorted ones, and nothing about the table's contents changes.
Rewriting a table's files into fewer, larger, better-ordered ones without changing the data. It is routine maintenance on any table written incrementally, and on a lakehouse it is your job rather than the storage engine's. It is not free — it rewrites data, it costs compute, and it interacts with time travel because the old files must be retained until their snapshots expire — so it is scheduled and budgeted like any other pipeline.
Full entry →Merge-on-read table
the update is written as a small delta file and the reader merges it with the base file every time, until a compaction folds them together.
A table where mutations are appended as delta or delete files and reconciled at query time. Writes are cheap and fresh, and every read pays a merge cost that grows until compaction runs. It is the right shape for frequently updated tables, and it makes compaction load-bearing rather than optional — an unmaintained merge-on-read table degrades continuously.
Full entry →