Comparison
Horizontal autoscalingvsScale to zero
Horizontal autoscaling
traffic doubled and the replica count went from four to nine on its own, a minute or two after it needed to.
Adding and removing copies of a workload automatically based on a metric — CPU, memory, queue depth, request rate. It is the standard elasticity mechanism and it works well for stateless request handlers whose load metric is honest. Its two constant problems are lag, because the metric arrives after the traffic did, and flapping, which is why cooldown windows exist and why a scale signal tied to a lagging indicator makes things worse.
Full entry →Scale to zero
nothing is running and nothing is billed until a request arrives, and that first request waits for a container to start.
Allowing a workload's replica count to reach zero when idle, with something in front holding the first request while capacity comes back. It is the strongest cost lever available for spiky and internal workloads, and it moves the whole cost of a cold start onto whoever is unlucky enough to be first. It is a poor fit for anything with a latency objective on the first request, and a good fit for review environments, batch endpoints and internal tools.
Full entry →