Data engineering·Cost, layout and query performance
the engine started the join, saw that one side was much smaller than predicted, and switched strategy mid-query.
Adaptive query execution
Also calledAQE, runtime replanning
Re-planning parts of a query at runtime using actual statistics from completed stages, rather than committing to the plan made up front. It exists because estimates on raw files are frequently terrible, and it fixes exactly the cases that hurt most: a mis-sized broadcast, a skewed partition, an over-parallelised stage. It also makes performance less predictable run to run, which is worth knowing when a query's timing suddenly changes with no other cause.