jargon

Data engineering·Files, formats and table layers

the table is built for frequent upserts by key, keeping a log of changes beside the base files and merging them on read.

Hudi

Also calledApache Hudi

An open table format designed first around record-level upserts and incremental pulls, with an indexing layer mapping keys to files. It is the format whose design centre is mutation rather than append, which makes it a natural landing point for change data capture. Its copy-on-write and merge-on-read table types are the clearest statement of the write-cost versus read-cost trade in the whole lakehouse space.

Commonly confused with