Comparison
Continuous integrationvsFlaky test
Continuous integration
your change was merged and verified against everyone else's within the hour, not at the end of the sprint.
The practice of integrating everyone's work into a shared mainline frequently, with automated verification each time. The important half is the practice, not the server: a team with a build pipeline and week-long branches is not doing continuous integration, whatever the tool is called. The value is that integration problems are found while they are small and while the person who caused them still remembers the change.
Full entry →Flaky test
the test fails one run in twenty, everyone reruns CI until it goes green, and the suite stops meaning anything.
A test that passes and fails without code changes, usually because of timing, ordering, shared state or real network calls. Its real damage is cultural: once reruns are normal, genuine failures get rerun too. Quarantine or delete flaky tests immediately — a test nobody believes is worse than no test.
Full entry →