Antipattern: Disabled Test

Problem

There's a disabled test in the test suite.

Discussion

Sometimes developer decides not to fix a broken test, but to disable it to keep continuous build green, with the intention to fix it later. Too often these tests are forgotten and not discovered for a long period of time. When they are discovered, they tend to be too detached from current source code and often do not even compile. At this stage it's usually easier to delete those and write new tests from scratch than to fix the test.

Disabled tests can also leave huge pieces of functionality completely untested, while everyone else will have false sense of security.

Solutions

It's better to avoid disabling tests at all, but rather fix them immediately. However in real situations it's not always possible and pressure to keep build "green" might be huge. To limit the effect of test disabling consider doing the following:

• implant a Test Bomb instead of permanently disabling the test.
• even when permanently disabling the test make sure that it still compiles. This will minimize fixing efforts later.
• make sure that all disabled tests are easily discoverable: use tool/build file configuration to disable tests. Do not disable tests in source code because it makes discovery harder.
• file a bug in issue tracker to remind everyone about the disabled test.

Related Antipatterns

• Failing Test - when broken test is not disabled, it will continue to fail.

Applicable Positive Patterns

• Test Bomb - instead of disabling a test, consider test-bombing it.