Applied AI·Local and self-hosted inference
the first request after a quiet hour takes thirty seconds because the weights had to be loaded off disk again.
Cold start / model loading
Also calledmodel loading
Draft summary, pending review
The seconds-to-minutes cost of loading gigabytes of weights into memory before the first token. Ollama unloads idle models by default, so the first request after a lull pays this; keep-alive settings and warm pools are the fixes.