Comparison
Correlation is not causationvsDistributed tracing
Correlation is not causation
latency and CPU rose together and you spent two hours on the CPU, which was a symptom of the same queue.
Two things moving together does not establish that one moved the other. In incidents this is the most expensive reasoning error available, because dashboards are full of correlated metrics and the compelling one is often downstream of the real cause. The way out is a mechanism: say out loud how A would cause B, and if you cannot, treat the correlation as a clue rather than as a conclusion.
Full entry →Distributed tracing
you follow one slow request across six services and see exactly which hop ate eight hundred milliseconds.
Recording causally linked timing spans for a single request as it crosses process boundaries. It is the only tool that answers "where did the time go" in a distributed system. It requires context propagation through every hop, including queues, and one un-instrumented service breaks the chain for everything behind it.
Full entry →