Architecture & practice·Risk, blast radius and change
the automation was doing the wrong thing at 2am and there was no supported way to turn it off.
Escape hatch
Also calledbreak-glass, manual override
A supported way to bypass the system when the system is wrong. Every automation needs one, because automation fails in ways its authors did not model, and the alternative is somebody editing the database by hand during an incident. Designing it deliberately — audited, obvious, documented in the runbook — is the difference between a controlled override and an unrecorded one.