Applied AI·Fine-tuning and customisation
you run the tuned model and the base model over the same held-out set, plus a general set, before believing the fine-tune helped.
Evaluating a fine-tune
Draft summary, pending review
Compare the tuned model against the base model on the same held-out eval set, plus a general-ability set to catch forgetting. A fine-tune that was not evaluated this way is a liability wearing a version number.