Applied AI·The API layer
you draw the next token from the distribution instead of always taking the top one, which is why two identical calls disagree.
Sampling
Draft summary, pending review
Choosing the next token from the probability distribution rather than always taking the top one. Sampling gives variety and creativity; it is also why the same prompt yields different answers on repeat runs.