Comparison
TrainingvsInference
Training
Training happens once (or occasionally) on huge GPU clusters and changes the weights.
Inference
every API call you make is inference, and nothing you send in one of them changes the weights for the next.
Running a trained model to get output. Every API call you make is inference. Distinguish it firmly from training: inference never changes the weights.
Full entry →The difference
Training happens once (or occasionally) on huge GPU clusters and changes the weights; inference happens billions of times a day and only reads them. Your production concerns (latency, cost, reliability) are inference concerns.
Entry: Training vs inference →