Comparison
Columnar formatvsVectorised execution
Columnar format
the query reads three columns out of two hundred and only touches the bytes for those three.
A file layout that stores all the values of one column together rather than storing rows contiguously. Two things follow, and they are why analytics runs on it: you can skip columns entirely, and values of the same type sitting next to each other compress far better. The cost is on the other side — reading or writing a single whole row touches every column's region, which is why these formats are wrong for transactional workloads.
Full entry →Vectorised execution
the engine processes a thousand values of one column per operation instead of one row at a time through a chain of function calls.
Executing operators over batches of column values rather than row by row, which keeps CPU pipelines full and makes use of wide instructions. It is the main reason modern analytical engines are an order of magnitude faster than older row-at-a-time ones on the same hardware. It also explains why user-defined functions can be so disproportionately slow: a scalar function breaks out of the vectorised path for every row.
Full entry →