Comparison
Top_kvsTop_p (nucleus sampling)
Top_k
you cap sampling at the k most likely tokens, a knob your local stack exposes and most hosted APIs no longer do.
Restricts sampling to the k most probable tokens. Common in local inference stacks; many hosted APIs no longer expose it.
Full entry →Top_p (nucleus sampling)
you cut off the improbable tail instead of reshaping the whole distribution, and you move this or temperature, not both.
Restricts sampling to the smallest set of tokens whose probabilities sum to p, cutting off the improbable tail. An alternative lever to temperature; adjust one, not both, unless you know why.
Full entry →