Comparison
TemperaturevsTop_p (nucleus sampling)
Temperature
for anything you will parse with code, run at or near 0.
A knob that reshapes the probability distribution before sampling. Low values (0 to 0.3) make output focused and repeatable, suited to extraction and code; higher values add diversity, suited to brainstorming. Ranges vary by provider, commonly 0 to 1 or 0 to 2.
Full entry →Top_p (nucleus sampling)
you cut off the improbable tail instead of reshaping the whole distribution, and you move this or temperature, not both.
Restricts sampling to the smallest set of tokens whose probabilities sum to p, cutting off the improbable tail. An alternative lever to temperature; adjust one, not both, unless you know why.
Full entry →