Search

Weyuker has recently proposed a set of properties which should be satisfied by any reasonable criterion used to claim that a computer program has been adequately tested. She called these properties "axioms." She also evaluated several well-known testing strategies with respect to these properties, and concluded that some of the commonly used strategies failed to satisfy several of the properties. We question both the fundamental nature of the properties and the precision with which they are presented, and illustrate how a number of ideas in Weyuker's paper' can be simplified and clarified through greater precision and a more consistent set of definitions. We also reanalyze the testing strategies after accounting for these inconsistencies. The strategies tend to fare much better as a result of this reanalysis. [ABSTRACT FROM AUTHOR]

Testing has long been in need of mathematical underpinnings to explain its value as well as its limitations. This paper develops and applies a mathematical framework that 1) unifies previous work on the subject, 2) provides a mechanism for comparing the power of methods of testing programs based on the degree to which the methods approximate program verification, and 3) provides a reasonable and useful interpretation of the notion that successful tests increase one's confidence in the program's correctness. Applications of the framework include confirmation of a number of common assumptions about practical testing methods. Among the assumptions confirmed is the need for generating tests from specifications as well as programs. On the other hand, a careful formal analysis of the usual assumptions surrounding mutation analysis shows that the "competent programmer hypothesis" does not suffice to ensure the claimed high reliability of mutation testing. Hardware testing is shown to fit into the framework as well, and a brief consideration of it shows how the practical differences between it and software testing arise. [ABSTRACT FROM AUTHOR]