Comparison
DriftvsRegression testing
Drift
nothing in your code changed but the outputs are worse than they were in March, because the traffic or the model underneath moved.
Quality change over time without any deploy on your side: providers update models, user inputs shift, corpora age. Detected by scheduled eval runs and monitored online metrics; pinned model versions reduce it but expire when versions retire.
Full entry →Regression testing
you re-run the whole scored set after a prompt tweak or a model upgrade, so you find out you broke case 12 before your users do.
Re-running the eval set on every prompt, model or pipeline change and comparing to the previous run. The habit that separates teams who know a change hurt quality from teams who find out from customers. Model upgrades are changes; test them like deploys.
Full entry →