Fine-tuning and customisation

prompting plateaued, so you take a few hundred of your own examples and train the model on them until it does the thing by default.

Fine-tuning

you were about to fine-tune and found the real fix was a clearer prompt, retrieval, and the eval set you did not have.

The decision rule

you hand it a few hundred prompt-and-ideal-answer pairs, which is what people mean by fine-tuning when they do not qualify it.

Supervised fine-tuning (SFT)

your fine-tuning data was formatted slightly differently from your production prompt, and the results quietly got worse.

Training example format

you spend three weeks collecting, cleaning and deduplicating six hundred examples, and one afternoon actually running the training.

Dataset curation

you generate the training examples with a stronger model and then throw most of them away, because the filtering is the part that works.

Synthetic data

you trained for ten passes, the training loss looked wonderful, and the model got worse at anything it had not already seen.

Epochs and overfitting

your fine-tune nails the target task and quietly got worse at everything else, which you only found out because you kept a general eval set.

Catastrophic forgetting

you update every parameter, need serious GPU memory to do it, and end up with a full-size copy of the model per variant.

Full fine-tuning

you train a small fraction of the parameters and the memory bill drops by orders of magnitude.

Parameter-efficient fine-tuning (PEFT)

instead of editing the whole model, train a small patch that sits on top of it.

LoRA

you keep one small adapter per task sitting on one shared base model, and turn the rank up when the change you want will not fit.

Rank and adapters

you fine-tuned a mid-sized model on one consumer GPU, because the base underneath the adapter was quantised to four bits.

QLoRA

you train on pairs where one answer is better than the other, rather than on a single correct target.

Preference tuning (RLHF, DPO)

the frontier model proved the task works, so you train a small one on its outputs and cut the per-call cost by an order of magnitude.

Distillation

you run the tuned model and the base model over the same held-out set, plus a general set, before believing the fine-tune helped.

Evaluating a fine-tune