Economics of Testing

Major retailers and car manufacturers often issue product recall notices when they realize that there is a serious fault in one of their products. Perhaps you can think of other examples. The fixing of the so-called millennium but is probably one of the greatest product recall notices in history.

Boehm's research suggests that cost of fixing faults increases dramatically as we move software product towards field use. If fault is detected at an early stage of design, it may be only design documentation that has to change resulting in perhaps just a few hours work. However, as project progresses and other components are built based on faulty design, more work is obviously needed to correct fault once it has been found. This is because design work, coding and testing will have to be repeated for components of the system that were previously thought to have been completed.

If faults are found in documentation, then development based on that documentation might generate many related faults, which multiply the effect of the original fault.

Analysis of specifications during test preparation (early test design) often brings faults in specifications to light. This will also prevent faults from multiplying i.e. if removed earlier they will not propagate into other design documents.

In summary I suggest that it is generally cost effective to use resources on testing throughout the project life cycle starting as soon as possible. The alternative is to potentially incur much larger costs associated with the effort required to correct and re-test major faults. Remember that the amount of resources allocated to testing is a management decision based on an assessment of the associated risks.

Practically, very few organizations are able to accurately compare the relative costs of testing and the costs associated with re-work.