Comparison
Canary deploymentvsTraffic splitting
Canary deployment
the new prompt goes to two percent of traffic first, and you watch quality and cost before letting the rest through.
Rolling a prompt or model change to a small traffic slice first, watching quality and cost metrics, then widening. Treat prompts as deployable artefacts with the same ceremony as code, because they break things the same way.
Full entry →Traffic splitting
five percent of requests go to the new version and the rest to the old, decided by the proxy rather than by your code.
Sending a configured proportion of requests to different backend versions, at the routing layer. It is the infrastructure primitive that progressive delivery is built from, and moving it out of the application means the split applies to every language and every service uniformly. Weighted routing is per request, so sticky behaviour has to be added deliberately if a user must not flip between versions mid-session.
Full entry →