Escaped Defect Rate

Of the bugs that reached users, which ones should the test suite have caught? Escaped defect rate is the slowest and most honest measure of test effectiveness: not “would the tests catch a hypothetical mutant?” but “did they catch the actual failures, judged after the fact?”

Response playbook

When the rate rises, the move is classification, not blame:

1. Triage by layer. For each escaped defect, ask which sensor should have caught it. A null-reference crash is a type checker miss. A wrong total is a missing behavioral test. A slow query that only surfaced in production is a profiling gap.
2. Add the sensor, then the test. If a class of defect keeps escaping, the suite has a structural hole that one more hand-written test will not fill. Repeat escapes of the same class — two injections in a quarter — mean the right sensor, here static security analysis, belongs in the pipeline.
3. Require a regression test before the fix. Every escaped defect is a free test case: the production failure is the specification.
4. Do not manage to the number. The rate lags by months and the classification is judgment. Optimizing it directly produces re-labeling, not better sensors. Pair it with mutation testing as the fast leading indicator.

What it cannot detect

Defects nobody reported, and defects attributed to the wrong cause. It also lags badly: it tells you about the suite you had, not the suite you have. Pair it with mutation testing for a fast proxy and with incident correlation for the cost side.

Related sensors