Search

Test suite reduction is an effective way to reduce the cost of regression testing by identifying and removing redundant test cases from the original test suite. In this paper, we propose a novel cluster test suite reduction using an evolutionary multi-objective optimization algorithm. Specifically, we use a K-means algorithm to group similar test cases to the same cluster. Then the evolutionary algorithm is used to remove redundant test cases based on the clustering results, and optimization objects are represented as the coverage-related criteria, fault-related criteria and cost-related criteria. The experimental results involving eight subject programs show that the proposed method can outperform the other three state-of-the-arts with respect to both fault detection (4.61% -9.44%) and reduction ratio (4.10% -10.64%). Meanwhile, the experiments also prove that our method has a better performance of missing failure rate (0.049% -0.132%) and code coverage rate (3.34% -6.10%). Besides, the proposed method costs are found to be comparable to the other techniques.

The number of mobile applications has increased geometrically nowadays, but how to ensure their quality and conduct adequate and effective testing is still a challenge for developers. On the one hand, the number of mobile apps is increasing, and the update speed is faster and faster. Many small and medium-sized companies can hardly test the app adequately before each release. On the other hand, mobile apps play more and more important roles in people's life, such as financial payment. For the sake of company security and user privacy, most companies will encrypt key codes in their APP. Even third-party testers cannot get source code, which also leads to many researchers cannot carry out further research and effective testing for these widely used mainstream APPs. Code coverage is an important indicator to guide software testing, which plays a crucial role in ensuring the quality of testing. However, it is an urgent problem to find accurate coverage indictors to evaluate these tests. And when testing those existing widely used mainstream closed-source apps, we find that the existing coarse-grained coverage metrics like method coverage is bad coverage indictors for app testing that can exaggerate or minimize the actual coverage rate, which cannot obtain satisfactory results for the evaluation of test effects. To find a more reliable coverage indictor, this paper demonstrates the correctness of instruction coverage indictor and the inaccurate of method coverage in evaluating the test of closed-source APP from the perspective of probability and statistics. Then we shows how inaccurate the method coverage can be through an empirical evaluation on datasets of closed source APPs and open source APPs respectively. It is further verified that instruction coverage is a more effective evaluation indictor than methods coverage or activity coverage in the test of closed source APP for the first time.

Widely applied in various fields, deep learning (DL) is becoming the key driving force in the industry. Although it has achieved great success in artificial intelligence tasks, similar to traditional software, it has defects involving unpredictable accidents and losses due to failure. To ensure the quality of DL software, adequate testing needs to be carried out. In this paper, we propose a test set augmentation technique based on an adversarial example generation algorithm for image classification deep neural networks (DNNs). It can generate a large number of useful test cases, especially when test cases are insufficient. We briefly introduce the adversarial example generation algorithm and implement the framework of our method. We conduct experiments on classic DNN models and datasets. We further evaluate the test set by using a coverage metric based on states of the DNN. [ABSTRACT FROM AUTHOR]