Search

In recent years, Service Oriented Architecture (SOA) has been increasingly adopted to develop distributed applications in the context of the Internet. To develop reliable SOA-based applications, an important issue is how to ensure the quality of web services. In this paper, we propose a dynamic random testing (DRT) technique for web services, which is an improvement over the widely-practiced random testing (RT) and partition testing (PT). We examine key issues when adapting DRT to the context of SOA, including a framework, guidelines for parameter settings, and a prototype for such an adaptation. Empirical studies are reported where DRT is used to test three real-life web services, and mutation analysis is employed to measure the effectiveness. Our experimental results show that, compared with the three baseline techniques, RT, Adaptive Testing (AT) and Random Partition Testing (RPT), DRT demonstrates higher fault-detection effectiveness with a lower test case selection overhead. Furthermore, the theoretical guidelines of parameter setting for DRT are confirmed to be effective. The proposed DRT and the prototype provide an effective and efficient approach for testing web services. IEEE

Service-Oriented Architecture (SOA) provides a scalable framework for heterogeneous, pervasive Internet of Things (IoT) devices. For such open and dynamic infrastructure, a proper service description schema is fundamental to unify the self-description of services. However, existing approaches mainly rely on verbose service protocols which produce unnecessary service overhead for resource limited devices. In addition, these approaches usually focus on describing service functions while largely ignoring context characteristics of IoT services. A lightweight and context-aware service description approach is yet to be proposed. In this paper, we propose a context-aware IoT service description approach based on the microservice architecture. Our approach not only provides a service description schema with lightweight protocols, but also comprehensively describes the context, service, and interface characteristics of IoT services. We demonstrate the applicability and effectiveness of our approach for the case of a smart elderly care system. Furthermore, we provide a comparison with related approaches to elaborate on pros and cons of our proposal with respect to the state of the art in the field.

Random testing and partition testing are two major families of software testing techniques. They have been compared both theoretically and empirically in numerous studies for decades, and it has been widely acknowledged that they have their own advantages and disadvantages and that their innate characteristics are fairly complementary to each other. Some work has been conducted to develop advanced testing techniques through the integration of random testing and partition testing, attempting to preserve the advantages of both while minimizing their disadvantages. In this paper, we propose a new testing approach, adaptive partition testing , where test cases are randomly selected from some partition whose probability of being selected is adaptively adjusted along the testing process. We particularly develop two algorithms, Markov-chain based adaptive partition testing and reward-punishment based adaptive partition testing , to implement the proposed approach. The former algorithm makes use of Markov matrix to dynamically adjust the probability of a partition to be selected for conducting tests; while the latter is based on a reward and punishment mechanism. We conduct empirical studies to evaluate the performance of the proposed algorithms using ten faulty versions of three large-scale open source programs. Our experimental results show that, compared with two baseline techniques, namely random partition testing (RPT) and dynamic random testing (DRT), our algorithms deliver higher fault-detection effectiveness with lower test case selection overhead. It is demonstrated that the proposed adaptive partition testing is an effective testing approach, taking advantages of both random testing and partition testing.

Metamorphic testing is a technique that makes use of some necessary properties of the software under test, termed as metamorphic relations, to construct new test cases, namely follow-up test cases, based on some existing test cases, namely source test cases. Due to the ability of verifying testing results without the need of test oracles, it has been widely used in many application domains and detected lots of real-life faults. Numerous investigations have been conducted to further improve the effectiveness of metamorphic testing, most of which were focused on the identification and selection of “good” metamorphic relations. Recently, a few studies emerged on the research direction of how to generate and select source test cases that are effective in fault detection. In this paper, we propose a novel approach to generating source test cases based on their associated path constraints, which are obtained through symbolic execution. The path distance among test cases is leveraged to guide the prioritization of source test cases, which further improve the efficiency. A tool has been developed to automate the proposed approach as much as possible. Empirical studies have also been conducted to evaluate the fault-detection effectiveness of the approach. The results show that this approach enhances both the performance and automation of metamorphic testing. It also highlights interesting research directions for further improving metamorphic testing.

Nowadays, service oriented architecture (SOA) has become one mainstream paradigm for developing distributed applications. As the basic unit in SOA, web services can be composed to construct complex applications. The quality of web services and their compositions is critical to the success of SOA applications. Testing, as a major quality assurance technique, is confronted with new challenges in the context of service compositions. In this paper, we propose a scenario-oriented testing approach that can automatically generate test cases for service compositions. Our approach is particularly focused on the service compositions specified by Business Process Execution Language for web services (WS-BPEL), a widely recognized executable service composition language. In the approach, a WS-BPEL service composition is first abstracted into a graph model; test scenarios are then derived from the model; finally, test cases are generated according to different scenarios. We also developed a prototype tool implementing the proposed approach, and an empirical study was conducted to demonstrate the applicability and effectiveness of our approach. The experimental results show that the automatic scenario-oriented testing approach is effective in detecting many types of faults seeded in the service compositions.

When modelling secure business processes, business analysts firstly specify security constraints and compliance properties that design-time processes should satisfy. Thus, it is a critical task to check whether the process model under security constraints complies with prospective security compliance properties. For some special tasks within a process, they may contain some internal business logics (named as sub-processes) that is a hierarchical process. In security compliance issues of a hierarchical process, security compliance properties are usually represented as complex logic formulas which are not easily understood by business analysts. This paper presents an approach for checking security properties compliance of the hierarchical process. We present the abstract process model and security constraints model respectively via BPMN graphic notation and resource assignments on process behaviours; the expected security compliance properties are modelled by a visual compliance rule graph, which is absorbed easily by a business analyst; model checking technology is applied to verify the security of the hierarchical process model.

The publish/subscribe paradigm provides a loosely-coupled and scalable communication model for the large-scale IoT service systems, such as supervisory control and data acquisition (SCADA). Data confidentiality and service privacy are two crucial security issues for the publish/subscribe model deployed in different domains. The existing access control schemes for such model cannot address both the issues at the same time. In this paper, we propose a comprehensive access control framework (CACF) to bridge this gap. The design principle of the proposed framework is twofold: (a) a bi-directional policy matching scheme for protecting the privacy of IoT services; and (b) a fully homomorphic encryption scheme for encrypting published events to protect data confidentiality. We analyze the correctness and security of the CACF, moreover, we prototype CACF based on Apache ActiveMQ, an open source message broker, and evaluate its performance. The experimental results indicate that our security system meets the latency requirements for very high-quality SCADA services.

Nowadays, applications are increasingly deployed as Web services in the globally distributed cloud computing environment. Multiple services are normally composed to fulfill complex functionalities. Business Process Execution Language for Web Services (WS-BPEL) is an XML-based service composition language that is used to define a complex business process by orchestrating multiple services. Compared with traditional applications, WS-BPEL programs pose many new challenges to the quality assurance, especially testing, of service compositions. A number of techniques have been proposed for testing WS-BPEL programs, but only a few studies have been conducted to systematically evaluate the effectiveness of these techniques. Mutation testing has been widely acknowledged as not only a testing method in its own right but also a popular technique for measuring the fault-detection effectiveness of other testing methods. Several previous studies have proposed a family of mutation operators for generating mutants by seeding various faults into WS-BPEL programs. In this study, we conduct a series of empirical studies to evaluate the applicability and effectiveness of various mutation operators for WS-BPEL programs. The experimental results provide insightful and comprehensive guidance for mutation testing of WS-BPEL programs in practice. In particular, our work is the systematic study in the selection of effective mutation operators specifically for WS-BPEL programs.

Metamorphic testing (MT) is an innovative approach to alleviating the oracle problem in software testing, which uses metamorphic relations of the program under test, instead of the test oracles, to verify its outputs. To alleviate the oracle problem of testing web services, we had previously proposed an MT framework for web services. In this paper, we further improve the efficiency and automation of this framework by leveraging metamorphic relations to iteratively generate test cases. We present a fixed-size iterative MT algorithm and implement it in the MT framework. We conduct three case studies to evaluate the fault detection effectiveness and efficiency of the proposed approach. Experimental results suggest that, compared with the conventional MT, iterative MT can achieve a comparable fault detection effectiveness, but with significantly fewer resources. Observations and limitations are summarised to provide new insights into the application of iterative MT.

Service-Oriented Architecture (SOA) enables the coordination of multiple loosely coupled services. This allows users to choose any service provided by the SOA without knowing implementation details, thus making coding easier and more flexible. Web services are basic units of SOA. However, the functionality of a single Web service is limited, and usually cannot completely satisfy the actual demand. Hence, it is necessary to coordinate multiple independent Web services to achieve complex business processes. Business Process Execution Language for Web Services (WS-BPEL) makes the coordination possible, by helping the integration of multiple Web services and providing an interface for users to invoke. When coordinating these services, however, illegal or faulty operations may be encountered, but current tools are not yet powerful enough to support the localisation and removal of these problems. In this paper, we propose a fault localisation technique for WS-BPEL programs based on predicate switching and program slicing, allowing developers to more precisely locate the suspicious faulty code. Case studies were conducted to investigate the effectiveness of the proposed technique, which was compared with predicate switching only, slicing only, and one existing fault localisation technique, namely Tarantula. The experimental results show that the proposed technique has a higher fault localisation effectiveness and precision than the baseline techniques.

In the current Web Service Description Language (WSDL), only the interface information of a web service is provided without any indication on its behavior logic. Naturally, it is difficult for the service user and developer to achieve a shared understanding of the service behavior through such a description. A particular challenge is how to make explicit the various behavior assumptions and restrictions of a service (for the user), and make sure that the service implementation conforms to them (for the developer). In order to improve the behavior conformance of services, in this paper we propose a constraint-based model-driven testing approach for web services. In our approach, constraints are introduced in an extended WSDL, called CxWSDL, to formally and explicitly express the implicit restrictions and assumptions on the behavior of web services, and then the predefined constraints are used to derive test cases in a model-driven manner to test the service implementation’s conformance to these behavior constraints from the user’s perspective. We have conducted an empirical study with three real-life web services as subject programs, and the experimental results have shown that our approach can effectively validate the service’s conformance to the behavior constraints.

Unified Modeling Language UML activity diagrams are widely used to model concurrent interaction among multiple objects. In this paper, we propose a transformation-based approach to generating scenario-oriented test cases for applications modeled by UML activity diagrams. Using a set of transformation rules, the proposed approach first transforms a UML activity diagram specification into an intermediate representation, from which it then constructs test scenarios with respect to the given concurrency coverage criteria. The approach then finally derives a set of test cases for the constructed test scenarios. The approach resolves the difficulties associated with fork and join concurrency in the UML activity diagram and enables control over the number of the resulting test cases. We further implemented a tool to automate the proposed approach and studied its feasibility and effectiveness using a case study. Experimental results show that the approach can generate test cases on demand to satisfy a given concurrency coverage criterion and can detect up to 76.5% of seeded faults when a weak coverage criterion is used. With the approach, testers can not only schedule the software test process earlier, but can also better allocate the testing resources for testing concurrent applications. Copyright © 2015John Wiley & Sons, Ltd.

The publish/subscribe paradigm can be used to build IoT service communication infrastructure owing to its loose coupling and scalability. Its features of decoupling among event producers and event consumers make IoT services collaborations more real-time and flexible, and allow indirect, anonymous and multicast IoT service interactions. However, in this environment, the IoT service cannot directly control the access to the events. This paper proposes a cross-layer security solution to address the above issues. The design principle of our security solution is to embed security policies into events as well as allow the network to route events according to publishers' policies and requirements. This solution helps to improve the system's performance, while keeping features of IoT service interactions and minimizing the event visibility at the same time. Experimental results show that our approach is effective.

Summary form only given. MET 2017 Workshop Summary. Metamorphic Testing (MT) is a testing technique that exploits the relationships among the inputs and outputs of multiple executions of the program under test, so- called Metamorphic Relations (MRs). MT has been proven highly effective in testing programs that face the oracle problem, for which the correctness of individual output is difficult to determine. Since the introduction of MT in 1998, the interest in this testing methodology has grown immensely with numerous applications in various domains such as machine learning, bioinformatics, computer graphics, simulation, search engines, decision support, cloud computing, databases, and compilers. The Second International Workshop on Metamorphic Testing (MET) will bring together researchers and practitioners in academia and industry to discuss research results and experiences in MT. The ultimate goal of MET is to provide a platform for the discussion of novel ideas, new perspectives, new applications, and state of research, related to or inspired by MT.

Regression testing re-runs some previously executed test cases, with the purpose of checking whether previously fixed faults have re-emerged and ensuring that the changes do not negatively affect the existing behaviors of the software under development. Today's software is rapidly developed and evolved, and thus it is critical to implement regression testing quickly and effectively. In this paper, we propose a novel technique for regression testing, based on a family of mutant selection strategies. The preliminary results show that the proposed technique can significantly improve the efficiency of different regression testing activities, including test case reduction and prioritization. Our work also makes it possible to develop a unified framework that effectively implements various activities in regression testing.

Service Oriented Architecture (SOA) has become a major application development paradigm. As a basic unit of SOA applications, Web services significantly affect the quality of the applications constructed from them. In the context of SOA, the specification and implementation of Web services are completely separated. The lack of source code and the restricted control of Web services limit the testability of Web services, and make the oracle problem prominent. In this context, can one alleviate the test oracle problem, or effectively and efficiently test such Web services even without oracles? It is an important issue which has not been yet adequately addressed. To address the challenge of testing Web services, the authors propose a metamorphic relation-based approach to testing Web services without oracles. The proposed approach leverages so-called metamorphic relations to generate test cases and evaluate test results. To make the proposed approach practical and effective, the authors proposed a framework taking into account the unique features of SOA, and developed a prototype which partially automates the framework. Three case studies are conducted to validate the feasibility and effectiveness of the proposed approach. The work presented in the paper not only alleviates the test oracle problem of testing Web services, but also delivers an effective and efficient test technique without oracles.

Web service-based systems are built orchestrating loosely coupled, standardized, and internetworked programs. If on the one hand, Web services address the interoperability issues of modern information systems, on the other hand, they enable the development of software systems on the basis of reuse, greatly limiting the necessity for reimplementation. Techniques and methodologies to gain the maximum from this emerging computing paradigm are in great need. In particular, a way to explicitly model and manage variability would greatly facilitate the creation and customization of Web service-based systems. By variability we mean the ability of a software system to be extended, changed, customized or configured for use in a specific context.We present a framework and related tool suite for modeling and managing the variability of Web service-based systems for design and run-time, respectively. It is an extension of the COVAMOF framework for the variability management of software product families, which was developed at the University of Groningen. Among the novelties and advantages of the approach are the full modeling of variability via UML diagrams, the run-time support, and the low involvement of the user. All of which leads to a great deal of automation in the management of all kinds of variability. (C) 2009 Elsevier Inc. All rights reserved.

Web services provide a way to facilitate the business integration over the Internet. Flexibility is an important and desirable property of Web service-based systems due to dynamic business environments. The flexibility can be provided or addressed by incorporating variability into a system. In this study, we investigate how variability can be incorporated into service-based systems. We propose a language, VxBPEL, which is an adaptation of an existing language, BPEL, and able to capture variability in these systems. We develop a prototype to interpret this language. Finally, we illustrate our method by using it to handle variability of an example. (C) 2008 Elsevier B.V. All rights reserved.

In cloud computing, variability becomes a crucial ability of process-based cloud applications. Existing solutions either focused on modeling variability in the architectural model or tried to support dynamic variability management in implementation. An integrated approach that can inherit the virtues from both categories is expected. The paper aims to fill the gap by proposing a model-driven variability-based service composition approach. We propose VxUML to model the variability in the architecture model. Then, we define a set of model transformation rules to convert VxUML into VxBPEL (an extension to standard BPEL supporting variability at the implementation level). Finally, we implement a prototype tool, and present a case study to demonstrate the feasibility of our approach.

Service compositions have become a powerful development paradigm to create distributed applications out of autonomous Web services. Since such applications are often deployed and executed in open and dynamic environments, variability management is a crucial enabling technique. To address the adaptation issue of service compositions, we proposed VxBPEL, an extension of BPEL for supporting variability, and a variability-based adaptive service composition approach which employs VxBPEL for variability implementation. In this paper, we present a VxBPEL engine for supporting the execution of VxBPEL service compositions. The engine is called VxBPEL_ODE and is implemented by extending a widely recognized open source BPEL engine, Apache ODE. We discuss key issues of developing VxBPEL_ODE, and three real-life service compositions are employed to evaluate and compare its performance with another VxBPEL engine we developed in our previous work. VxBPEL_ODE, together with analysis, design, and run-time management tools for VxBPEL, constitutes a comprehensive supporting platform for variability-based adaptive service compositions.

The use of web services has been growing significantly, with increasingly large numbers of applications being implemented through the web. A difficulty associated with this development is the quality assurance of these services, specifically the challenges encountered when testing the applications - amongst other things, testers may not have access to the source code, and the correctness of the output may not be easily ascertained known as the oracle problem. Metamorphic testing MT has been introduced as a technique to alleviate the oracle problem. MT makes use of properties of the software under test, known as metamorphic relations, and checks whether or not these relations are violated. Since MT does not require source code to generate the metamorphic relations, it is suitable for testing web services-based applications. We have designed an XML-based language representation to facilitate the formalisation of metamorphic relations, the generation of follow-up test cases, and the verification of the test results. Based on this, we have also developed a tool to support the automation of MT for web service applications. This tool has been used in an experiment to test web services, the evaluation of which is reported in this paper.

Business Process Execution Language (BPEL) is a widely recognized executable service composition language. Since BPEL integrates services of desired functionality to compose business processes, it is significantly different from typical programming languages. How to effectively locate the integration-level faults in BPEL programs is an open issue. In this paper, we propose the BPEL fault localization guidelines based on the characteristics of BPEL programs, and adapt Tarantula, a traditional fault localization technique, to locate the integration-level faults in BPEL programs. We also conducted an empirical study to demonstrate the feasibility of our methodology.

Mutation analysis is widely employed to evaluate the effectiveness of various software testing techniques. In most situations, mutation operators are uniformly applied to the original programs, while the faults tend to be clustered in practice. This may result in the inappropriate simulation of faults, and thus cannot deliver the reliable evaluation results. To overcome this, we propose a distribution-aware mutation analysis technique and conducted empirical studies to investigate the impact of the mutation distribution on the effectiveness evaluation of testing techniques. As an illustration, we select the commonly practiced random testing technique and two versions of dynamic random testing techniques and apply them to testing Web services. Results of empirical studies suggest that the mutation distribution significantly affects the evaluation results. This observation further indicates that the effectiveness of testing techniques previously evaluated with the uniform mutation analysis needs further realignments.

In recent years, Service Oriented Architecture (SOA) has been increasingly adopted to develop applications in the context of Internet. To develop reliable SOA-based applications, an important issue is how to ensure the quality of Web services. In this paper, we propose a dynamic random testing (DRT) technique for Web services which is an improvement of the widely practiced random testing. We examine key issues when adapting DRT to the context of SOA and develop a prototype for such an adaptation. Empirical studies are reported where DRT is used to test two real-life Web services and mutation analysis is employed to measure the effectiveness. The experimental results show that DRT can save up to 24% test cases in terms of detecting the first seeded fault, and up to 21% test cases in terms of detecting all seeded faults, both with the cases of uniformed mutation analysis and distribution-aware mutation analysis, which refer to faults being seeded in an even or clustered way, respectively. The proposed DRT and the prototype provide an effective approach to testing Web Services.

Service Oriented Architecture (SOA) has become a major application development paradigm. As a basic unit of SOA applications, Web services significantly affect the quality of the applications constructed from them. Since the development and consumption of Web services are completely separated under SOA environment, the consumers are normally provided with limited knowledge of the services and thus have little information about test oracles. The lack of source code and the restricted control of Web services limit the testability of Web services. To address the prominent oracle problem when testing Web services, we propose a metamorphic testing framework for Web services taking into account the unique features of SOA. We conduct a case study where the new metamorphic testing framework is employed to test a Web service that implements the electronic payment. The results of case study show the feasibility of the framework for web services, and also the efficiency of metamorphic testing. The work presented in the paper alleviates the test oracle problem when testing Web services under SOA.

Software architecture is one of key factors to the success of modern large-scale software-intensive systems development. Among the existing architectural models, some important aspects are ignored while should be addressed. To overcome this, we propose a multi-view architectural model, which is an extension to the "4+1" view architectural model proposed by Kruchten, and discuss the description and construction techniques for the extended architectural views. We employed the proposed architecture model and relevant description and construction techniques to develop a component based software testing platform. The experience and lessons with application of the proposed architectural model are reported. Results of the case study show that the proposed multi-view architectural model and relevant techniques are useful to the development of large-scale software-intensive systems.

Nowadays applications are increasingly developed based on remote Web services and service composition has become a powerful novel development paradigm. Due to the fact that such applications in the context of Internet are deployed and executed in an open and dynamic environment, adaptability is one of the crucial requirements for developing such applications. In our previous work, we presented a variability-based approach to address the adaptability issue of service compositions, and developed the VxBPEL, an extension of BPEL with respect to variability. This paper presents a variability analysis tool for variability-based adaptive service compositions called ValySeC. ValySeC extracts the variation from service compositions specified using VxBPEL and provides a variation view. With the view, the designer can better understand variation points and the possible instances, and efficiently maintain the variation within service compositions. The paper presents the design and implementation of ValySeC using a case study to handle service compositions with variations to illustrate the main concepts.

When a user requests the execution of a process in a service environment, often she desires transactional behavior in executing. However, current standards for Web services execution do not offer support for transactions. We overcome this shortcoming by presenting an extension proposal for integrating transaction management into the BPEL framework. The proposal has been demonstrated by a drop-dead order system in the context of the automotive industry. We test the business behavior described by BPEL specifications with and without transaction management when failures occur. Experimental results show that the proposed extension to BPEL framework is a viable solution for incorporating transactions into BPEL processes in order to implement reliable business processes.

UML Activity Diagrams (UADs) are widely employed to model the behavior of software systems under development. To test the implemented systems, the UAD specification is a reasonable basis. Generating a complete suite of test scenarios is a challenging task in particular when the systems under development are large and complex enough. In this paper, we describe a model-driven testing tool TSGen which can automatically generate test scenarios based on the UAD. We examine the design and implementation issues of the TSGen. With the TSGen, testers can better plan testing schedule and save the efforts on developing test suites.

The Business Process Execution Language (BPEL) is a widely recognized executable language supporting the specification of process-oriented service compositions. However, the language is limited in addressing variable requirements in the description of business processes. We propose to construct variable and maintainable Web services compositions with VxBPEL, an extension to BPEL we developed to define variability in business process specification. We present the main concepts of VxBPEL and show how to achieve better adaptation and variability maintenance of service compositions, which is particularly desired in the context of dynamically changing business goals and processes.

Testing concurrent applications is difficult yet important. UML activity diagrams are widely used to model concurrent interactions among multiple objects. We present a transformation-based approach to generating scenario-oriented test cases for testing concurrent applications modeled by UML activity diagrams. The approach first transforms a UML activity diagram specification into an intermediate representation via a set of transformation rules. From the intermediate representation we then construct a set of test scenarios with respect to the given concurrence coverage criteria. Finally, we derive a set of test cases from the constructed test scenarios. The approach employs transformation to resolve the nonstructural problem with activity diagrams, and can generate test cases on demand to satisfy a given concurrence coverage criteria and hence the number of the resulting test cases is controllable. With the approach, testers can not only earlier schedule the software test process but also better allocate the test resource for testing concurrent applications.

As service centric (SC) systems are being increasingly adopted, new challenges and possibilities emerge. Business processes are now able to execute seamlessly across organizations and to coordinate the interaction of loosely coupled services. Often it is necessary to have transactionality for a set of business operations, but the loosely nature of such systems calls for techniques and principles that go beyond traditional ACID transactions. By analyzing existing service composition languages, tools, and needs on a classical example, we provide requirements for transactionality in service centric systems and indications for developing SC systems transactionally capable.