Data engineering·Files, formats and table layers
the engine reads one metadata file listing the table's data files and their statistics, instead of listing a bucket prefix.
Manifest file
Also calledmetadata file, file list
The metadata object recording which data files belong to a table version, together with per-file statistics. Replacing directory listing with a read of this file is what makes table formats fast on object storage and what makes their commits atomic — swapping one pointer changes the whole table. It also means metadata itself accumulates and needs maintenance, which is a class of operational problem that plain directories did not have.