Applied AI·Fine-tuning and customisation
you train on pairs where one answer is better than the other, rather than on a single correct target.
Preference tuning (RLHF, DPO)
Draft summary, pending review
Training on comparisons ('this answer is better than that one') rather than single targets. RLHF does it via a learned reward model and reinforcement learning; DPO trains directly on the preference pairs and is far simpler to run. This stage is much of what makes chat models feel helpful.