Backend & systems·Data modelling and storage engines
writes go to an in-memory table and an append-only log, and a background process keeps merging the files on disk behind you.
LSM tree
Also calledlog-structured merge tree, SSTable
The storage engine behind Cassandra, RocksDB and friends: buffer writes in memory, flush them as immutable sorted files, and compact those files in the background. It turns random writes into sequential ones, which is why it beats B-trees on write throughput. The bill arrives as read amplification and as compaction that competes with your traffic for disk and CPU at the worst possible time.