Comparison
Parallel runvsShadow deployment
Parallel run
both systems processed every transaction for a month and you only looked at where they disagreed.
Running the old and new implementations side by side on real input, with only the old one's output used, and comparing results. It is the most convincing evidence available that a replacement is correct, because it tests against real data rather than the data you imagined. It costs double compute and a genuine comparison mechanism, and the discipline is deciding in advance what disagreement rate means 'switch'.
Full entry →Shadow deployment
you run the candidate on live traffic without showing anyone its answers, and compare the logs afterwards.
Running a candidate (new model, new prompt) on live traffic without serving its output, logging results for offline comparison against the incumbent. The lowest-risk way to test a change against reality before it faces a user.
Full entry →