Comparison
Data lakevsData swamp
Data lake
everything anyone might ever need is in a bucket as files, in whatever shape it arrived, and finding the useful part is your problem.
Cheap object storage holding data in open file formats, with no enforced schema at write time. It is the answer to 'we cannot afford to decide yet': you keep everything and interpret it later, which is genuinely valuable when the questions have not been asked. What it does not give you is transactions, consistent multi-file updates or any guarantee that two readers see the same thing, which is the gap the table formats were invented to close.
Full entry →Data swamp
there are nine thousand tables in the bucket, four of them are used, and nobody knows which four.
A lake that accumulated data without ownership, documentation or lifecycle, so nothing in it can be trusted or safely deleted. It is the predictable end state of storing everything with no catalogue and no retention, and it is expensive twice: in storage and in every hour spent working out which of five similar tables is the real one. It is worth naming because the remedy is governance and deletion rather than more storage.
Full entry →