Comparison
Data skewvsHot key
Data skew
ninety-nine tasks finished in twenty seconds and the hundredth ran for eleven minutes, because one key has forty percent of the rows.
Work distributed unevenly across parallel tasks because the partitioning key is unevenly distributed. It is the single most common reason a distributed query is slow, and it is invisible in averages — the job's runtime is the slowest task's runtime. Nulls and default values are the classic culprits, because every row that failed to get an id ends up in the same partition.
Full entry →Hot key
one key gets so much traffic that the single node holding it is saturated while the rest of the cluster sits idle.
A single key or partition receiving a disproportionate share of requests, so that sharding does not help: the load lands on one node regardless of how many you add. Typical causes are a celebrity record, a global counter, or a partition key with low cardinality. The mitigations are key splitting, a local cache in front of the shared one, or request coalescing.
Full entry →