Comparison
Post-trainingvsPretraining
Post-training
the model's manners, its refusals and its house style come from this phase, not from what it read.
Everything done to a model after pretraining: supervised fine-tuning on example conversations, then preference tuning to make outputs helpful and safe. Most of a model's 'personality' comes from this phase, not from pretraining.
Full entry →Pretraining
this is the phase that gave the model its knowledge, and the same phase that gave it a cutoff date.
The first and most expensive training phase: the model learns next-token prediction over trillions of tokens of web text, code and books. This is where its knowledge and language ability come from, and it is why the knowledge has a cutoff date.
Full entry →