Search

The problem of multilabel classification has attracted great interest in the last decade, where each instance can be assigned with a set of multiple class labels simultaneously. It has a wide variety of real-world applications, e.g., automatic image annotations and gene function analysis. Current research on multilabel classification focuses on supervised settings which assume existence of large amounts of labeled training data. However, in many applications, the labeling of multilabeled data is extremely expensive and time consuming, while there are often abundant unlabeled data available. In this paper, we study the problem of transductive multilabel learning and propose a novel solution, called Trasductive Multilabel Classification (TraM), to effectively assign a set of multiple labels to each instance. Different from supervised multilabel learning methods, we estimate the label sets of the unlabeled instances effectively by utilizing the information from both labeled and unlabeled data. We first formulate the transductive multilabel learning as an optimization problem of estimating label concept compositions. Then, we derive a closed-form solution to this optimization problem and propose an effective algorithm to assign label sets to the unlabeled instances. Empirical studies on several real-world multilabel learning tasks demonstrate that our TraM method can effectively boost the performance of multilabel classification by using both labeled and unlabeled data. [ABSTRACT FROM AUTHOR]

Abstract: Extenics is a new discipline for modeling contradiction problems with formalized methods and transformation. This paper for the first time combines software testing with Extenices and proposes an automatic approach to generating test cases from UML activity diagrams based on Extension Theory (Extenics). In order to find more defects in software system by minimized test cases, we design algorithm to construct the Euler circuit and generates test sequences automatically by Euler circuit algorithm. Our preliminary result shows that test cases generated not only satisfy the specified test coverage criteria but also the number of test cases is decreased. [Copyright &y& Elsevier]

Test case prioritization is the process of ordering the execution of test cases to achieve a certain goal, such as increasing the rate of fault detection. Increasing the rate of fault detection can provide earlier feedback to system developers, improving fault fixing activity and, ultimately, software delivery. Many existing test case prioritization techniques consider that tests can be run in any order. However, due to functional dependencies that may exist between some test cases—that is, one test case must be executed before another—this is often not the case. In this paper, we present a family of test case prioritization techniques that use the dependency information from a test suite to prioritize that test suite. The nature of the techniques preserves the dependencies in the test ordering. The hypothesis of this work is that dependencies between tests are representative of interactions in the system under test, and executing complex interactions earlier is likely to increase the fault detection rate, compared to arbitrary test orderings. Empirical evaluations on six systems built toward industry use demonstrate that these techniques increase the rate of fault detection compared to the rates achieved by the untreated order, random orders, and test suites ordered using existing "coarse-grained” techniques based on function coverage. [ABSTRACT FROM AUTHOR]

Refactoring is a transformation that preserves the external behavior of a program and improves its internal quality. Usually, compilation errors and behavioral changes are avoided by preconditions determined for each refactoring transformation. However, to formally define these preconditions and transfer them to program checks is a rather complex task. In practice, refactoring engine developers commonly implement refactorings in an ad hoc manner since no guidelines are available for evaluating the correctness of refactoring implementations. As a result, even mainstream refactoring engines contain critical bugs. We present a technique to test Java refactoring engines. It automates test input generation by using a Java program generator that exhaustively generates programs for a given scope of Java declarations. The refactoring under test is applied to each generated program. The technique uses SafeRefactor, a tool for detecting behavioral changes, as an oracle to evaluate the correctness of these transformations. Finally, the technique classifies the failing transformations by the kind of behavioral change or compilation error introduced by them. We have evaluated this technique by testing 29 refactorings in Eclipse JDT, NetBeans, and the JastAdd Refactoring Tools. We analyzed 153,444 transformations, and identified 57 bugs related to compilation errors, and 63 bugs related to behavioral changes. [ABSTRACT FROM AUTHOR]

Testing is a destructive task in which the goal is to find relevant defects as early as possible. It requires automation to reduce cost and ensure high regression, thus delivering determined quality. This article reviews technologies for test automation. [ABSTRACT FROM PUBLISHER]

Dynamic software updating (DSU) systems patch programs on the fly without incurring downtime. To avoid failures due to the updating process itself, many DSU systems employ timing restrictions. However, timing restrictions are theoretically imperfect, and their practical effectiveness is an open question. This paper presents the first significant empirical evaluation of three popular timing restrictions: activeness safety (AS), which prevents updates to active functions, con-freeness safety (CFS), which only allows modifications to active functions when doing so is provably type-safe, and manual identification of the event-handling loops during which an update may occur. We evaluated these timing restrictions using a series of DSU patches to three programs: OpenSSH, vsftpd, and ngIRCd. We systematically applied updates at each distinct update point reached during execution of a suite of system tests for these programs to determine which updates pass and which fail. We found that all three timing restrictions prevented most failures, but only manual identification allowed none. Further, although CFS and AS allowed many more update points, manual identification still supported updates with minimal delay. Finally, we found that manual identification required the least developer effort. Overall, we conclude that manual identification is most effective. [ABSTRACT FROM AUTHOR]

The growing dependence of society on software systems places a high premium on their reliable operation. Moreover, the stringent reliability expectations imposed on these systems must be achieved despite their increasing size and complexity, and decreasing resources available for their development and maintenance. To mitigate these dual challenges, a systematic approach to guide the allocation of resources to the components of a software system is necessary. This paper presents an optimization framework which considers the contribution of each component to system reliability to determine the amount of effort to be allocated to each component, towards the ultimate objective of achieving the specified system reliability target with minimal effort. We assume that the contribution of a component to system reliability is governed by two factors: the system architecture, and the effort-reliability relationship of the component. This characterization is referred to as “architecture-based optimization” because it considers the system architecture explicitly in the effort allocation process. It is demonstrated that the architecture-based optimization framework outperforms other effort allocation strategies, including equal component weighting, and expert opinion. [ABSTRACT FROM AUTHOR]

Software testing is the one of the primary methods used in the validation and verification of output in the software development industry. It is seen as a key method for achieving software quality, reliability, fitness for purpose and customer satisfaction. Software testing is however an expensive process accounting for as much as 50% of the cost of developing software based systems. In recent years, software testing as a discipline has come under pressure due to time, cost and skills constraints. These constraints impact negatively upon software test effectiveness. Therefore it is critical to identify and implement test tools that reduce the negative impact of software test constraints on software test effectiveness. In this paper the researchers examines some of the most popular software testing tools such as test case prioritisation, test suite reduction and test selection criteria, to identify: Which individual test tools are most likely to yield optimal test effectiveness and, Which combination of test tools is most likely to yield optimal test effectiveness and mitigate the effect of test constraints An extensive review of the software testing literature was conducted and used to construct a survey instrument as the basis for examining the impact of test constraints on software test methodology. The survey was issued to expert software test practitioners from various locations globally; the sample consisted of 43 test cases. The main findings were that no one approach to testing would yield satisfactory results but a combination of two or more test types from Automated testing, Smoke testing, Test case prioritisation and Regression test selection could yield effective software testing results and mitigate the effects of test constraints. [ABSTRACT FROM AUTHOR]

Abstract: When computational methods are developed, the efficiency of the novel methods should be compared to the existing ones. This can be done using, e.g., analytical methods and benchmark test patterns. In addition, the comparison of the best and the worst case performance is commonly of interest. In this paper, methodologies of genetic algorithm based software testing are adopted to the comparative computational testing of three varieties of dynamic two-dimensional nearest point algorithms. The extreme performances of the algorithms are searched for by optimizing the shape of two-dimensional Gaussian distributions, from which the test patterns are drawn. In particular, an approach to pairwise comparisons of computational complexities of algorithms is proposed. The test case algorithms can be sorted with respect to their computational complexity by the proposed method. [Copyright &y& Elsevier]

Software testing is important, but many of us don't do it because it is frustrating and boring. test that is a new testing framework for R that is easy learn and use, and integrates with your existing workflow. This paper shows how, with illustrations from existing packages. [ABSTRACT FROM AUTHOR]

Abstract: The paper shows experiences with using Extreme Programming (XP) for specific projects. XP practices Test-Driven Development, pair programming, short interaction, a team code ownership and acceptance tests. Author writes about possibilities of information technologies when improving communicative skills of children with specific disorders, such as autistic spectrum disorders, Down syndrome, mental retardation, etc. The development of an application stemming from the communication system PECS (The Picture Exchange Communication System) and its Czech variant VOKS is the base of this paper to show specificity of the development and verification of software for the given group of handicapped users. The paper shows suitability of using agile methods of software development for a concrete application which is designed for users with specific disorders. It tries to show advantages and disadvantages of new methodologies, particularly Extreme Programming. This agile method prefers fast reaction to a change before the plan completion, which proved to be important in the case of the developed software. There were plenty of changes and new requirements during the development, and their solution was more important for the output quality than following the time schedule of the development. [Copyright &y& Elsevier]

This paper describes the authors' experience of teaching agile software development to students of computer science, software engineering, and other related disciplines, and comments on the implications of this and the lessons learned. It is based on the authors' eight years of experience in teaching agile software methodologies to various groups of students at different universities, in different cultural settings, and in a number of courses and seminars. It specifically discusses three different courses on agile software development, given in different teaching settings and at different levels, and briefly surveys variations to these courses given elsewhere. Based on the experience acquired, analyses and evaluations conducted, and current pedagogical trends at relevant university departments, the authors provide recommendations on how to overcome potential problems in teaching agile software development and make their adoption more effective. [ABSTRACT FROM AUTHOR]

Abstract: Models are abstractions of reality that have predetermined limits (often not consciously thought through) on what problem domains the models can be used to explore. These limits are determined by the range of observed data used to construct and validate the model. However, it is important to remember that operating the model beyond these limits, one of the reasons for building the model in the first place, potentially brings unwanted behaviour and thus reduces the usefulness of the model. Our experience with the Agricultural Production Systems Simulator (APSIM), a farming systems model, has led us to adapt techniques from the disciplines of modelling and software development to create a model development process. This process is simple, easy to follow, and brings a much higher level of stability to the development effort, which then delivers a much more useful model. A major part of the process relies on having a range of detailed model tests (unit, simulation, sensibility, validation) that exercise a model at various levels (sub-model, model and simulation). To underline the usefulness of testing, we examine several case studies where simulated output can be compared with simple relationships. For example, output is compared with crop water use efficiency relationships gleaned from the literature to check that the model reproduces the expected function. Similarly, another case study attempts to reproduce generalised hydrological relationships found in the literature. This paper then describes a simple model development process (using version control, automated testing and differencing tools), that will enhance the reliability and usefulness of a model. [ABSTRACT FROM AUTHOR]

With the growing size and complexity of software applications, traditional software reliability methods are insufficient to analyze inter-component interactions of modular software systems. The number of test cases may be extremely large for this application; therefore, it is hard for us to extensively test each software component given resource limitations. In this paper, we propose an adaptive framework of incorporating path testing into reliability estimation for modular software systems. Three estimated methods based on common program structures, namely, sequence, branch, and loop structures, are proposed to calculate the path reliability. Consequently, the derived path reliabilities can be applied to the estimates of software reliability. Some experiments are performed based on two real systems. In addition, the accuracy and correlation with respect to the experiments are investigated by simulation and sensitivity analysis. Experimental results show that the path reliability has a high correlation to the actual software reliability. For software with loop structures, a smaller loop number can be assigned to derive an acceptable estimation of path reliability. Further, the sensitivity analysis can be used to identify critical modules and paths for resource allocation. It can be concluded that the proposed methods are useful and helpful for estimating software reliability and can be adaptively used in the early stages of software development. [ABSTRACT FROM AUTHOR]

In this paper, we propose two general frameworks for deriving several software reliability growth models based on a non-homogeneous Poisson process (NHPP) in the presence of imperfect debugging and error generation. The proposed models are initially formulated for the case when there is no differentiation between failure observation and fault removal testing processes, and then extended for the case when there is a clear differentiation between failure observation and fault removal testing processes. During the last three decades, many software reliability growth models (SRGM) have been developed to describe software failures as a random process, and can be used to evaluate development status during testing. With SRGM, software engineers can easily measure (or forecast) the software reliability (or quality), and plot software reliability growth charts. It is not easy to select the best model from a plethora of models available. There are few SRGM in the literature of software engineering that differentiates between failure observation and fault removal processes. In real software development environments, the number of failures observed need not be the same as the number of faults removed. Due to the complexity of software systems, and an incomplete understanding of software, the testing team may not be able to remove the fault perfectly on observation of a failure, and the original fault may remain, resulting in a phenomenon known as imperfect debugging, or get replaced by another fault causing error generation. In the case of imperfect debugging, the fault content of the software remains the same; while in the case of error generation, the fault content increases as the testing progresses. Removal of observed faults may result in the introduction of new faults. [ABSTRACT FROM AUTHOR]

Mandatory access control (MAC) mechanisms control which users or processes have access to which resources in a system. MAC policies are increasingly specified to facilitate managing and maintaining access control. However, the correct specification of the policies is a very challenging problem. To formally and precisely capture the security properties that MAC should adhere to, MAC models are usually written to bridge the rather wide gap in abstraction between policies and mechanisms. In this paper, we propose a general approach for property verification for MAC models. The approach defines a standardized structure for MAC models, providing for both property verification and automated generation of test cases. The approach expresses MAC models in the specification language of a model checker and expresses generic access control properties in the property language. Then the approach uses the model checker to verify the integrity, coverage, and confinement of these properties for the MAC models and finally generates test cases via combinatorial covering array for the system implementations of the models. [ABSTRACT FROM AUTHOR]

Software testing is an important issue in software engineering. As software systems become increasingly large and complex, the problem of how to optimally allocate the limited testing resource during the testing phase has become more important, and difficult. Traditional Optimal Testing Resource Allocation Problems (OTRAPs) involve seeking an optimal allocation of a limited amount of testing resource to a number of activities with respect to some objectives (e.g., reliability, or cost). We suggest solving OTRAPs with Multi-Objective Evolutionary Algorithms (MOEAs). Specifically, we formulate OTRAPs as two types of multi-objective problems. First, we consider the reliability of the system and the testing cost as two objectives. Second, the total testing resource consumed is also taken into account as the third objective. The advantages of MOEAs over state-of-the-art single objective approaches to OTRAPs will be shown through empirical studies. Our study has revealed that a well-known MOEA, namely Nondominated Sorting Genetic Algorithm II (NSGA-II), performs well on the first problem formulation, but fails on the second one. Hence, a Harmonic Distance Based Multi-Objective Evolutionary Algorithm (HaD-MOEA) is proposed and evaluated in this paper. Comprehensive experimental studies on both parallel-series, and star-structure modular software systems have shown the superiority of HaD-MOEA over NSGA-II for OTRAPs. [ABSTRACT FROM AUTHOR]

Recent advances in mechanical techniques for systematic testing have increased our ability to automatically find subtle bugs, and hence, to deploy more dependable software. This paper builds on one such systematic technique, scope-bounded testing, to develop a novel specification-based approach for efficiently generating tests for products in a software product line. Given properties of features as first-order logic formulas in Alloy, our approach uses SAT-based analysis to automatically generate test inputs for each product in a product line. To ensure soundness of generation, we introduce an automatic technique for mapping a formula that specifies a feature into a transformation that defines incremental refinement of test suites. Our experimental results using different data structure product lines show that an incremental approach can provide an order of magnitude speedup over conventional techniques. We also present a further optimization using dedicated integer constraint solvers for feature properties that introduce integer constraints, and show how to use a combination of solvers in tandem for solving Alloy formulas. [ABSTRACT FROM AUTHOR]

The difficulty of developing wireless sensor systems is widely recognized. Problems associated with testing, debugging, and profiling are key contributing factors. While network simulators have proven useful, they are unable to capture the subtleties of underlying hardware, nor the dynamics of wireless signal propagation and interference; and physical experimentation remains a necessity. To this end, developers increasingly rely on shared deployments exposed for physical experimentation. Sensor network testbeds are under development across the world. We present a complementary testbed architecture that derives its novelty from three characteristics. First, the system is interactive; users can profile source and network level components across a network in real time, as well as inject transient state faults and external network traffic. Second, the system is source-centric; it enables automated source code analysis, instrumentation, and compilation. Finally, the design is open; developers can extend the set of exposed inter faces as appropriate to particular projects without modifying the underlying middleware. We present the testbed design and implementation, a graphical user interface, a shell-based macro programming interface, example scenarios that illustrate their use, and discuss the testbed's application in the research and teaching activities at client institutions. [ABSTRACT FROM AUTHOR]

With feature sizes steadily shrinking, manufacturing defects and parameter variations often cause design timing failures. It is essential that those errors be correctly and quickly diagnosed. In this paper, we analyze the multiple-delay-fault diagnosis problem and propose a novel approach to solve it. We enhance the diagnostic resolution by processing failure logs at various slower-than-nominal clock frequencies. We evaluate the utility of n-detection and timing-aware automatic-test-pattern-generated (ATPG) sets. Experimental results show that using timing-aware ATPG sets yields better diagnostic resolution and results in better delay-defect-size estimations compared to n-detection ATPG sets. We experimentally determined our diagnosis algorithm's sensitivity to delay variations. [ABSTRACT FROM AUTHOR]

Modern vehicles have evolved to support connected and self-driving capabilities. The concepts such as connected driving, cooperative driving, and intelligent transportation systems have resulted in an increase in the connectivity of vehicles and subsequently created new information security risks. The original vehicular ad-hoc network term is now emerged to a new term, Internet of Vehicles (IoV), which is a typical application of symmetry of Internet of Things (IoT). Vehicle manufacturers address some critical issues such as software bugs or security issues through remote updates, and this gives rise to concerns regarding the security of updated components. Moreover, aftermarket units such as those imposed by transportation authorities or insurance companies expose vehicles to high risk. Software testing aims to ensure that software products are reliable and behave as expected. Many commercial and open-source software products undergo formal certifications to increase users' confidence in their accuracy, reliability, and security. There are different techniques for software certification, including test-based certification. Testcase repositories are available to support software testing and certification, such as the Linux Test Project for Linux kernel testing. Previous studies performed various testing and experimental evaluation of different parts of modern vehicles to assess the security risks. Due to the lack of trusted testcase repositories and a common approach for testing, testing efforts are performed individually. In this paper, we propose a blockchain-based approach for a testcase repository to support test-based software and security testing and overcome the lack of trusted testcase repositories. The novel concept Proof-of-Validation to manage global state is proposed to manage updates to the repository. The initial work in this study considers the LTP test suite as a use case for the testcase repository. This research work is expected to contribute to the further development in including evidence generation for testing verification. [ABSTRACT FROM AUTHOR]

The article presents information on an computer processor, Bugnet which helps the software developers to deterministically replay and debug an application while executing an computer program. The article author says that the difficult part of debugging is reproducing the bugs at a customer's site because it relys on core dumps which contain the final state of the system before the crash. The article authors' say that they propose the BugNet architecture to assist program developers to continuously record a program's execution as it runs. The Deterministic Replay Debugging technique helps to replay the exact same sequence of instructions that led up to the crash. The processor Bugnet continuously records a program's execution without interfering with the execution of the monitored program. The BugNet writes the logs to a fixed-sized buffer allocated in main memory so that software developers can then use these logs at the time of crash to deterministically replay the last part of execution as many times as required to fix the bug.

Claims that there are several programs available to test and benchmark drives and media and these are available as freeware, shareware and off-the-shelf programs. Testing a compact disc-read only memory (CD-ROM) or CD-Recordable drive; Some of the testing and benchmarking programs; Recommended tool for those who only need CD-ROM testing and do not need a full-featured software-based tester. INSETS: What happened to disc detective?.;Benchmarking and testing--the same thing?..

This paper explores the testing complexity of several classes of programs, where the testing complexity is measured in terms of the number of test data required for demonstrating program correctness by testing. It is shown that even for very restrictive classes of programs, none of the commonly used test criteria, namely, having every statement, branch, and path executed at least once, is nearly sufficient to guarantee absence of errors. Based on the study of testing complexity, this paper proposes two new test criteria, one for testing a path and the other for testing a program. These new criteria suggest how to select test data to obtain confidence in program correctness beyond the requirement of having each statement, branch, or path tested at least once. [ABSTRACT FROM AUTHOR]

This paper presents a testing strategy designed to detect errors in the control flow of a computer program, and the conditions under which this strategy is reliable are given and characterized. The control flow statements in a computer program partition the input space into a set of mutually exclusive domains, each of which corresponds to a particular program path and consists of input data points which cause that path to be executed. The testing strategy generates test points to examine the boundaries of a domain to detect whether a domain error has occurred, as either one or more of these boundaries will have shifted or else the corresponding predicate relational operator has changed. If test points can be chosen within ∈ of each boundary, under the appropriate assumptions, the strategy is shown to be reliable in detecting domain errors of magnitude greater than ∈. Moreover, the number of test points required to test each domain grows only linearly with both the dimensionality of the input space and the number of predicates along the path being tested. [ABSTRACT FROM AUTHOR]

The theory of test data selection proposed by Goodenough and Gerhart is examined. In order to extend and refine this theory, the concepts of a revealing test criterion and a revealing subdomain are proposed. These notions are then used to provide a basis for constructing program tests. A subset of a program's input domain is revealing if the existence of one incorrectly processed input implies that all of the subset's elements axe processed incorrectly. The intent of this notion is to partition the program's domain in such a way that all elements of an equivalence class are either processed correctly or incorrectly. A test set is then formed by choosing one element from each class. This process represents perfect program testing. For a practical testing strategy, the domain is partitioned into subdomains which are revealing for errors considered likely to occur. Three programs which have previously appeared in the literature are discussed and tested using the notions developed in the paper. [ABSTRACT FROM AUTHOR]

Two approaches to the study of program testing arc described. One approach is theoretical and the other empirical. In the theoretical approach situations are characterized in which it is possible to use testing to formally prove the correctness of programs or the correctness of properties of programs. In the empirical approach statistics are collected which record the frequency with which different testing strategies reveal the errors in a collection of programs. A summary of the results of two research projects which investigated these approaches are presented. The differences between the two approaches are discussed and their relative advantages and disadvantages are compared. [ABSTRACT FROM AUTHOR]

In this article we introduce the problem of computer software reliability and discuss a probabilistic model for describing the failure of software. We suggest a procedure for estimating the parameters of the model and propose a stopping rule for debugging the software. We apply our procedures to some published data on software failures. [ABSTRACT FROM AUTHOR]

Continuous integration has been around for a while now, but the habits it suggests are far from common practice. Automated builds, a thorough test suite, and committing to the mainline branch every day sound simple at first, but they require a responsible team to implement and constant care. What starts with improved tooling can be a catalyst for long-lasting change in your company's shipping culture. Continuous integration is more than a set of practices, it's a mindset that has one thing in mind: increasing customer value. The Web extra at http://youtu.be/tDl_cHfrJZo is an audio podcast of the Tools of the Trade column discusses how continuous integration is more than a set of practices, it's a mindset that has one thing in mind: increasing customer value. [ABSTRACT FROM PUBLISHER]

Modern virtualization technology allows us to run operating systems in a virtual machine that can be hosted on facilities ranging from our laptop to a datacenter in the cloud. It's thus possible to create a virtualized development environment that contains all the tools, applications, and libraries that a programmer requires. This speeds up developer setup time, brings economies of scale, introduces parity between development and production environments, allows the use of platform-specific tools, and simplifies embedded-system development. Using VMs, testers can ensure a pristine environment and access to diverse (virtual) platforms. Deployment is also simplified by packaging all the system's components and setup into a VM appliance. Finally, on the operations side, VMs make it easier for a system to support application provisioning, maintenance windows, high availability, and disaster recovery. [ABSTRACT FROM PUBLISHER]

Phillip G. Armour responds to "Differential Debugging" in the Tools of the Trade column September/October issue of IEEE Software to discuss the process of predicting defects. [ABSTRACT FROM PUBLISHER]

This study is a quasi experiment comparing the software defect rates and implementation costs of two methods of software defect reduction: code inspection and test-driven development. We divided participants, consisting of junior and senior computer science students at a large Southwestern university, into four groups using a two-by-two, between-subjects, factorial design and asked them to complete the same programming assignment using either test-driven development, code inspection, both, or neither. We compared resulting defect counts and implementation costs across groups. We found that code inspection is more effective than test-driven development at reducing defects, but that code inspection is also more expensive. We also found that test-driven development was no more effective at reducing defects than traditional programming methods. [ABSTRACT FROM PUBLISHER]

The problem of deciding whether an observed behavior is acceptable is the oracle problem. When testing from a finite state machine (FSM), it is easy to solve the oracle problem and so it has received relatively little attention for FSMs. However, if the system under test has physically distributed interfaces, called ports, then in distributed testing, we observe a local trace at each port and we compare the set of local traces with the set of allowed behaviors (global traces). This paper investigates the oracle problem for deterministic and nondeterministic FSMs and for two alternative definitions of conformance for distributed testing. We show that the oracle problem can be solved in polynomial time for the weaker notion of conformance (\sqsubseteq_w) but is NP-hard for the stronger notion of conformance (\sqsubseteq_s), even if the FSM is deterministic. However, when testing from a deterministic FSM with controllable input sequences, the oracle problem can be solved in polynomial time and similar results hold for nondeterministic FSMs. Thus, in some cases, the oracle problem can be efficiently solved when using \sqsubseteq_s and where this is not the case, we can use the decision procedure for \sqsubseteq_w as a sound approximation. [ABSTRACT FROM PUBLISHER]

Event-Driven Software (EDS) can change state based on incoming events; common examples are GUI and Web applications. These EDSs pose a challenge to testing because there are a large number of possible event sequences that users can invoke through a user interface. While valuable contributions have been made for testing these two subclasses of EDS, such efforts have been disjoint. This work provides the first single model that is generic enough to study GUI and Web applications together. In this paper, we use the model to define generic prioritization criteria that are applicable to both GUI and Web applications. Our ultimate goal is to evolve the model and use it to develop a unified theory of how all EDS should be tested. An empirical study reveals that the GUI and Web-based applications, when recast using the new model, show similar behavior. For example, a criterion that gives priority to all pairs of event interactions did well for GUI and Web applications; another criterion that gives priority to the smallest number of parameter value settings did poorly for both. These results reinforce our belief that these two subclasses of applications should be modeled and studied together. [ABSTRACT FROM AUTHOR]

Statistical testing has been shown to be more efficient at detecting faults in software than other methods of dynamic testing such as random and structural testing. Test data are generated by sampling from a probability distribution chosen so that each element of the software's structure is exercised with a high probability. However, deriving a suitable distribution is difficult for all but the simplest of programs. This paper demonstrates that automated search is a practical method of finding near-optimal probability distributions for real-world programs, and that test sets generated from these distributions continue to show superior efficiency in detecting faults in the software. [ABSTRACT FROM PUBLISHER]

Genetic algorithms (GAs) have been applied previously to UML-driven stress test requirements generation with the aim of increasing chances of discovering faults relating to network traffic in distributed real-time systems. However, since evolutionary algorithms are heuristic, their performance can vary across multiple executions, which may affect robustness and scalability. To address this, we present the design and technical detail of a UML-driven, GA-based stress test requirements generation tool, together with its empirical analysis. The main goal is to analyze and improve the applicability, efficiency, and effectiveness and also to validate the design choices of the GA used in the tool. Findings of the empirical evaluation reveal that the tool is robust and reasonably scalable when it is executed on large-scale experimental design models. The study also reveals the main bottlenecks and limitations of the tools, e.g., there is a performance bottleneck when the system under test has a large number of sequence diagrams which could be triggered independently from each other. In addition, issues specific to stress testing, e.g., the impact of variations in task arrival pattern types, reveal that the tool generally generates effective test requirements, although the features of those test requirements might be different in different runs (e.g., different stress times from the test start time might be chosen). While the use of evolutionary algorithms to generate software test cases has been widely reported, the extent, depth, and detail of the empirical findings presented in this paper are novel and suggest that the proposed approach is effective and efficient in generating stress test requirements. It is hoped that the findings of this empirical study will help other SBSE researchers with the empirical evaluation of their own techniques and tools. [ABSTRACT FROM PUBLISHER]