Comparison
Flaky testvsVisual regression test
Flaky test
the test fails one run in twenty, everyone reruns CI until it goes green, and the suite stops meaning anything.
A test that passes and fails without code changes, usually because of timing, ordering, shared state or real network calls. Its real damage is cultural: once reruns are normal, genuine failures get rerun too. Quarantine or delete flaky tests immediately — a test nobody believes is worse than no test.
Full entry →Visual regression test
the build failed because a component moved three pixels and the image comparison noticed.
Comparing rendered screenshots against approved baselines to catch visual changes no assertion describes. It is the only way to catch a CSS regression in a component nobody thought to test. It is also the most flake-prone kind of test there is — fonts, antialiasing, animation timing and platform rendering all move pixels — so it needs a tolerance, deterministic rendering and a fast approval flow to survive.
Full entry →