Comparison
Data cataloguevsData discovery
Data catalogue
you search for 'revenue', find eleven tables with that word in them, and can see which one is actually used by the finance dashboard.
A searchable inventory of datasets with their schemas, owners, lineage, freshness and usage. Its value is disproportionately in usage statistics: 'this table is queried four hundred times a day and that one twice a year' answers the question a description never will. A catalogue populated by hand decays within months, so the ones that work are the ones fed automatically from the warehouse and the orchestrator.
Full entry →Data discovery
a new analyst rebuilds a model that already existed because they had no way to find it.
The problem of a person finding the right existing dataset for their question. It is the most common invisible cost in a large warehouse: duplicated models, three definitions of the same metric, and analysts querying raw tables because the modelled one was undiscoverable. It is also why table naming conventions are worth arguing about, since search over names is what most people actually use.
Full entry →