jargon

Comparison

Dark launchvsShadow deployment

Dark launch

you send a copy of real production traffic at the new service and throw the responses away, just to see if it holds up.

Exercising new code with real traffic while its output is discarded. It reveals load, latency and error behaviour under a realistic distribution that no synthetic test reproduces. Anything with side effects has to be stubbed, or you will double-charge somebody in the name of testing.

Full entry →

Shadow deployment

you run the candidate on live traffic without showing anyone its answers, and compare the logs afterwards.

Running a candidate (new model, new prompt) on live traffic without serving its output, logging results for offline comparison against the incumbent. The lowest-risk way to test a change against reality before it faces a user.

Full entry →

Related comparisons