Data engineering·Cost, layout and query performance
the stage is at nine hundred and ninety-nine of a thousand tasks complete and has been for six minutes.
Straggler task
Also calledlong tail task, slow task
A single task holding up a stage while everything else has finished. The cause is either skew, where the task genuinely has more data, or an unhealthy node, where it does not. Distinguishing the two is the first diagnostic step, because the remedies are unrelated: one is a repartition or a salted key, the other is speculative execution or a dead machine.