Evaluating checklist-based and use-case-driven reading techniques as applied to software analysis and design UML artifacts

This chapter discusses the experimental comparison of two reading techniques, comparing their effectiveness and detection rates with respect to inspecting high-level UML diagrams for defects. Artifact-related checklists drive one technique, and application use cases drive the other. Our initial idea was that the latter is more effective than the former. This experiment was developed at the University of Rome “Tor Vergata”. It was conducted with junior and senior students of object-oriented analysis and design in the university’s Department of Informatics, Systems and Production. The data collected shows that techniques performed differently. Specifically, for effectiveness, CBR in the average performed + 41.6% better than UCDR. CBR detected 15.6% more seeded defects, and +149.3% more new faults, than UCDR. The latter provided 11.6% less false positives than CBR. For detection rate, the checklist-based reading technique’s peak value occurred 25% later, and was 66.7% greater, than the use-case driven reading’s peak value. However, the results were not statistically significant. Because the use-case-driven script turned out to be much more complex than the checklist-driven one, we decided to restructure the former in multiple layers and hence conducted further experiments, the results of which are forthcoming.