Data engineering·Cost, layout and query performance
the small dimension table is copied to every worker so the big table never has to move.
Broadcast join
Also calledmap-side join, broadcast hash join
Joining by sending a copy of the smaller side to every node, avoiding a shuffle of the large side entirely. It is dramatically faster when the small side genuinely is small, and it fails hard when the planner's size estimate is wrong — broadcasting something large runs every executor out of memory. The threshold is configurable and the estimate comes from statistics, which is why stale statistics show up as a query that used to be fast.