Search

Software testing is an important phase as it ensures the software quality. The software testing process comprises of three steps: generation, execution, and evaluation of test cases. Literature claims the usage of single and multiple 'Unified Modeling Language' (UML) diagrams to generate test cases. Using multiple UML diagrams increases test case coverage. However, the existing approaches show limitations in test case generation from UML diagrams. Therefore, in this research study, we propose an approach to generate the test cases using UML State Chart Diagram (SCD), Activity Diagram (AD), and Sequence Diagram (SD). The proposed approach transforms UML diagrams into intermediate forms: SCD Graph, AD Graph, and SD Graph respectively. Furthermore, by integrating these three graphs, a System Testing Graph (STG) is formed. Finally, test cases are identified from STG by using a traversal algorithm such as Depth First Search (DFS) that is an optimization method. The results show that the proposed approach is better compared to existing approaches in terms of coverage and performance. Moreover, the generated test cases have the ability to detect faults at the unit level, integration, and system level testing. [ABSTRACT FROM AUTHOR]

This paper studies learning-based testing (LBT) for reactive systems with different learning algorithms and model checkers. LBT is a technique that requires a learning algorithm to learn the models to generate test cases automatically. We have used the generic methodology of LBT to test reactive systems with two different model inference algorithms (i.e., IKL, DKL) and two different model checking tools (i.e., NuSMV, SAL). To investigate the feasibility of LBT, we integrated our SUTs with these algorithms in LBT and tested if LBT optimizes test generation with these algorithms. We tested our SUTs with Boolean data types to check the difference in the working of model inference and model checking algorithms which we analyzed experimentally. The results show that LBT works better with DKL and SAL. DKL is a recently proposed model inference algorithm, and SAL is the latest model checker on which the research is being carried out. DKL and SAL algorithms explore errors in reactive SUTs with the h LBT framework more quickly and efficiently.

Model-based Test Case Prioritization utilizing similarity metrics has proved effective in software testing. However, the utility of similarity metrics in it varies with test scenarios, hindering its universal effectiveness and performance optimization. To tackle this problem, we propose a Diversity-driven Learn-to-rank model-based TCP approach, named DLTCP, for optimizing early fault detection performance. Our method first employs the whale optimization algorithm to search for a suitable set of similarity metrics from a pool of existing candidates. This search process determines which metrics should be used. According to each selected metric, test cases are then prioritized. The resulting test case rankings are used as the training data for DLTCP. Finally, the proposed method incorporates random forest to train a ranking model for test case prioritization. As such, it can fuse multiple similarity metrics to improve the TCP performance. We conduct extensive experiments to evaluate our method's performance using the average percentage fault detected (APFD) as metric. The experimental results show that DLTCP achieve an average APFD value of 0.953 for seven classic benchmark models, which is 11.37% higher than that of the state-of-the-art algorithms. It can well select a set of similarity metrics for effective fusion, demonstrating competitive performance in early fault detection. • A novel approach, named DLTCP, optimizes model-based test case prioritization. • A fusion framework is proposed to combine multiple similarity metrics. • A Learn-to-rank model is constructed to improve early fault detection performance. • The superiority of DLTCP is verified on five classic benchmark models. [ABSTRACT FROM AUTHOR]

Regression testing plays an indispensable role in software maintenance, which refers to retest the software following modifications to determine whether the changes have introduced new faults. However, regression testing requires massive amounts of effort to achieve a high fault detection rate. To address this issue, the test case prioritization technique is used to improve the fault detection rate by adjusting the execution order of test cases. For model-based regression test case prioritization, existing approaches have been developed using the single aspect of model-related information extracted from the previous executed test cases. In this paper, a novel learn-to-rank technique is proposed to prioritize test cases by combining the multidimensional features of Extended Finite State Machine (EFSM) under test to improve fault detection rate. Specifically, our method utilizes the random forest algorithm to combine multiple existing heuristic prioritization methods. Detailed experiments are conducted to evaluate the proposed method's performance in terms of Average Percentage Fault Detected (APFD). The experimental results show that the mean APFD value of our method reaches 0.884 for five subject EFSMs, which is 33.9% higher than the compared methods.

Context-aware applications are emerging applications in the modern era of computing. These applications can determine and adapt to situational context to provide better user experience. Testing these applications is not straightforward. Constantly changing nature of context makes testing context-aware application is a challenging task. To uncover a defect in context-aware application, a test engineer needs activity (sequence of actions) and context information (context data); this makes test case development a difficult task. Conventional test case development methodologies do not cater for context information. Besides, conventional applications have only one input source, but context-aware application must obtain data from many sources to infer the context. Yet another issue that these applications often face is the noisy data problem as input data collected from physical sensors could be noisy. Test adequacy criteria are used as test stoppage rule and define the quality of testing as well as for generating test suites. Test adequacy criteria is helpful to control the cost of testing as well as determining/establishing confidence in the software product quality. A number of test adequacy criteria exist for testing conventional applications, but the same is not true for context-aware applications. Defining test adequacy criteria and test coverage measures for context-aware applications warrants further research. Several techniques have been developed by researchers to generate and execute test cases for context-aware applications; however, end-to-end testing and result analysis of executed test cases still remains a grueling task for the test engineers. The aim of this study is to automate end-to-end functional testing, analysis of the generated test results as well as functional/requirement coverage assessment. Moreover, we also present a confidence assessment template for result analysis. Test engineers can use our proposed framework to assess the requirement coverage. Our proposed framework will reduce testing time, efforts and cost thus enabling test engineers to execute more testing cycles to attain higher degree of test coverage.

In the modern era of mobile computing, context-aware computing is an emerging paradigm due to its widespread applications. Context-aware applications are gaining increasing popularity in our daily lives since these applications can determine and react according to the situational context and help users to enhance usability experience. However, testing these applications is not straightforward since it poses several challenges, such as generating test data, designing context-coupled test cases, and so on. However, the testing process can be automated to a greater extent by employing model-based testing technique for context-aware applications. To achieve this goal, it is necessary to automate model transformation, test data generation, and test case execution processes. In this paper, we propose an approach for behavior modeling of context-aware application by extending the UML activity diagram. We also propose an automated model transformation approach to transform the development model, i.e., extended UML activity diagram into the testing model in the form of function nets. The objective of this paper is to automate the context-coupled test case generation and execution. We propose a functional testing framework for automated execution of keyword-based test cases. Our functional testing framework can reduce the testing time and cost, thus enabling the test engineers to execute more testing cycles to attain a higher degree of test coverage.

Smartphone applications are getting popular and have become a necessity. There numerous smartphone applications ranging from entertaining to gaming and from utility to mission-critical. Almost everything on the web is now in hands of Smartphone users, which makes this domain very important and its quality should not be compromised. Achieving the desired quality is not an easy task for the mobile platform as it has its limitations. To produce a quality app, developers and testers need to test and assess the app in numerous ways to ensure the best trait of the application. In this concern, some efficient and mature techniques are required to test smartphone applications. In this study, the techniques, approaches, and models to assess mobile apps covering major prospects and angels to test mobile apps are identified. Our focus is on assessing the existing techniques and to evaluate them on standard validation parameters. [ABSTRACT FROM AUTHOR]

Model-Based Testing (MBT) is a well-known technique that employs formal models to represent reactive systems' behavior and generates test cases. Such systems have been specified and verified using mostly Finite State Machines (FSMs). There is a plethora of test generation algorithms in the literature; most of them are based on graphs once an FSM can be formally defined as a graph. Nevertheless, there is a lack of studies on analyzing cost and efficiency of FSM-based test generation algorithms. This study compares graph-based algorithms adopted to generate test cases from FSM models. In particular, we compare the Chinese Postman Problem (CPP) and H-Switch Cover (HSC) algorithms with the well-known Depth-First Search (DFS) and Breadth-First Search (BFS) algorithms in the context of covering all-transitions and all-transition-pairs criteria in an FSM. First, a systematic literature mapping was conducted to summarize the methods that have been adopted in MBT, considering FSMs. Second, the main methods found were implemented and analyzed on randomly-generated FSMs, as well as real-world models that represent embedded systems of space applications. To make comparisons, we considered analyses in terms of cost (time), efficiency (mutant analysis) and characteristics of the generated test suites (number of test cases, average length of test cases, largest and smallest test cases, standard deviation and distribution of test cases). In general, CPP presented the best results in terms of number of test cases and test suite size. In addition, CPP also presented low distribution of average length compared to other algorithms. [ABSTRACT FROM AUTHOR]

Business Process Testing is the act of validating that end-to-end transactions through enterprise systems continue to work correctly as the underlying packaged applications evolve. End-to-end automatic business process validation can be a challenging task, but an important way to check that business rules continue to work properly and that problems are detected and corrected as soon as possible. This paper presents the design of a test automation platform, ETAP-Pro, to test end-to-end business processes that aims to overcome some challenges in validating business processes. [ABSTRACT FROM AUTHOR]

Context Empirical studies are essential in evaluating the effectiveness of Model-based Testing (MBT) research and should be reported properly to ensure their replication and to highlight the strengths and limitations of the MBT techniques being evaluated. Researchers have proposed guidelines detailing what information should be reported when presenting empirical studies and what should be the structure of such primary studies. There is a need to evaluate the reporting quality of the empirical studies in MBT literature. Objective To evaluate the reporting quality of empirical studies in the model based testing domain; identifying where the reported studies fail to follow the proposed guidelines and finding frequently omitted details. As an auxiliary goal we aim to quantify the percentage of empirical studies conducted in industrial context. Method We evaluate the reporting quality and the execution contexts of MBT empirical studies reported in literature. For our study we consider the MBT papers published in top ten software engineering journals over the last eighteen years. We evaluate the published primary studies using the empirical study reporting guidelines. Results We found 87 empirical in MBT that met our selection criteria. Initial results showed that the existing guidelines were not only too strict (for example they demand presence of specific sections rather than simply having the details present in the paper), they also did not adequately cover MBT specific details. Therefore, we propose modified the guidelines for reporting empirical studies in MBT and re-evaluated the selected studies. Results show that while only a few empirical studies follow the exact structure proposed by the guidelines, approximately half the papers contain at least 50% of the required details. Most of the papers omit details related to process and analysis leading to presented results. We found a positive trend of improving reporting quality of empirical studies in MBT over the last Eighteen years. Another important finding from the review is that few reported studies were conducted in real industrial context. Conclusions Model based testing community needs to be more aware of the reporting guidelines and more effort should be spent on reporting the necessary details. Furthermore, we found that only few studies that are conducted in industrial context and hence more focus should be given to empirical case studies in real industry context. However, the reporting quality of research papers presenting empirical evaluations is gradually improving. [ABSTRACT FROM AUTHOR]