Constructing Cost-Aware Functional Test-Suites Using Nested Differential Evolution Algorithm

Combinatorial testing can test software that has various configurations for multiple parameters efficiently. This method is based on a set of test cases that guarantee a certain level of interaction among parameters. Mixed covering array (MCA) can be used to represent a test-suite. Each row of the array corresponds to a test case. In general, a smaller size of MCA does not necessarily imply less testing time. There are certain combinations of parameter values which would take much longer time than other cases. Based on this observation, it is more valuable to construct MCAs that are better in terms of testing effort characterization other than size. We present a method to find cost-aware MCAs. The method contains two steps. First, simulated annealing algorithm is used to get an MCA with a small size. Then we propose a novel nested differential evolution algorithm to improve the solution with its testing effort. The experimental results indicate that our method succeeds in constructing cost-aware MCAs for real-world applications. The testing effort is significantly reduced compared with representative state-of-the-art algorithms.