Search

Metamorphic testing (MT) is an innovative approach to alleviating the oracle problem in software testing, which uses metamorphic relations of the program under test, instead of the test oracles, to verify its outputs. To alleviate the oracle problem of testing web services, we had previously proposed an MT framework for web services. In this paper, we further improve the efficiency and automation of this framework by leveraging metamorphic relations to iteratively generate test cases. We present a fixed-size iterative MT algorithm and implement it in the MT framework. We conduct three case studies to evaluate the fault detection effectiveness and efficiency of the proposed approach. Experimental results suggest that, compared with the conventional MT, iterative MT can achieve a comparable fault detection effectiveness, but with significantly fewer resources. Observations and limitations are summarised to provide new insights into the application of iterative MT.

The use of web services has been growing significantly, with increasingly large numbers of applications being implemented through the web. A difficulty associated with this development is the quality assurance of these services, specifically the challenges encountered when testing the applications - amongst other things, testers may not have access to the source code, and the correctness of the output may not be easily ascertained known as the oracle problem. Metamorphic testing MT has been introduced as a technique to alleviate the oracle problem. MT makes use of properties of the software under test, known as metamorphic relations, and checks whether or not these relations are violated. Since MT does not require source code to generate the metamorphic relations, it is suitable for testing web services-based applications. We have designed an XML-based language representation to facilitate the formalisation of metamorphic relations, the generation of follow-up test cases, and the verification of the test results. Based on this, we have also developed a tool to support the automation of MT for web service applications. This tool has been used in an experiment to test web services, the evaluation of which is reported in this paper.