Data engineering·Time and correctness in streams
there are two implementations of the same metric, one fast and approximate and one slow and correct, and they disagree by two percent.
Lambda architecture
Also calledspeed layer and batch layer
Running a streaming path for fresh approximate results alongside a batch path for correct ones, and serving a merge of the two. It solved a real problem when stream engines could not be trusted with correctness, and its permanent cost is that every business rule is now written twice in two languages by two people. When someone reports that the live figure and the daily figure disagree, this shape is usually why.