Search

For software developers as well as software product users, fault removal efficiency is crucial. Developers can estimate the amount of work required by evaluating the proportion of addressed faults. At the same time, the level of trust in utilizing the software product can be determined by customers, based on this information. The proposed software reliability growth model considers fault removal efficiency relevant to contemporary scenarios characterized by automated testing and debugging tools. A comparison between the proposed model and other existing models is conducted by utilizing two data sets from software testing. Additionally, the best release plans are created by considering warranty costs, risk costs, and error removal costs while still meeting reliability requirements. [ABSTRACT FROM AUTHOR]

Due to the increasing reliance on technology in nearly every industry over the past three decades, it has become necessary to evaluate the performance of a software product prior to its formal release in the market. The properties of a software application, such as its complexity and lines of code, are subject to change over time as a result of factors namely the testing environment, allocation of resources, testing efficiency, and testing team's expertise. The assumption of constant Fault Detection Rate (FDR) may not accurately anticipate the  potential number of bugs correctly. Keeping all these considerations in mind, a framework is developed to incorporate change point in the development of a testing effort-based Software Reliability Growth Model (SRGM) that takes into account the effect of application characteristics under both perfect and imperfect debugging settings. In addition, these outcomes are compared to the model without a change point. The proposed model is validated on two real-life software fault datasets. The results demonstrate that the proposed model performs better than the model without a change point. [ABSTRACT FROM AUTHOR]

The efficiency and performance of a software application largely depend on the testing strategy adopted by the firm. Apart from the tools, techniques, and skills used for testing, the duration also plays an important influence in establishing software reliability. This defines the operational performance of the software. The testing duration decision is dependent on the failure behavior depicted by the software during the process of testing and the cost spent at various phases of development. In this paper, we study the multi-release model whose fault removal process is affected by random irregular fluctuations (white noise) and error generation phenomenon and determine the optimal time of testing for a multi-release-based software. The fault removal process is governed by the phenomenon of testing coverage which is affected by random fluctuations. The model shows encouraging results as it handles the stochastic property of the fault detection process. The optimal testing time is determined with the goal to minimize the expected development cost related to software while achieving the desired reliability levels of software for that release using a Genetic Algorithm. A real-life four-release fault dataset of tandem computers has been used to numerically demonstrate the methodology. It is observed through sensitivity analysis that the presence of white noise directly affects the cost and optimal testing duration. The potential to improve sensitivity, flexibility, early detection, discovering unsuspected patterns, and boost fault diagnosis is enhanced by collecting irregular fluctuations in the fault detection rate. This technology, by going beyond existing methodologies, has distinct advantages for detecting faults and can contribute to more dependable and efficient systems in a variety of domains. [ABSTRACT FROM AUTHOR]

Software defect prediction is a popular method for optimizing software testing and improving software quality and reliability. However, software defect datasets usually have quality problems, such as class imbalance and data noise. Oversampling by generating the minority class samples is one of the most well-known methods to improving the quality of datasets; however, it often introduces overfitting noise to datasets. To better improve the quality of these datasets, this paper proposes a method called US-PONR, which uses undersampling to remove duplicate samples from version iterations and then uses oversampling through propensity score matching to reduce class imbalance and noise samples in datasets. The effectiveness of this method was validated in a software prediction experiment that involved 24 versions of software data in 11 projects from PROMISE in noisy environments that varied from 0% to 30% noise level. The experiments showed a significant improvement in the quality of datasets pre-processed by US-PONR in noisy imbalanced datasets, especially the noisiest ones, compared with 12 other advanced dataset processing methods. The experiments also demonstrated that the US-PONR method can effectively identify the label noise samples and remove them. [ABSTRACT FROM AUTHOR]

In the software development life cycle, the quality and reliability of software are critical to software developers. Poor quality and reliability not only cause the loss of customers and sales but also increase the operational risk due to unreliable codes. Therefore, software developers should try their best to reduce such potential software defects by undertaking a software testing project. However, to pursue perfect and faultless software is unrealistic since the budget, time, and testing resources are limited, and the software developers need to reach a compromise that balances software reliability and the testing cost. Using the model presented in this study, software developers can devise multiple alternatives for a software testing project, and each alternative has its distinct allocation of human resources. The best alternative can therefore be selected. Furthermore, the allocation incorporates debuggers' learning and negligent factors, both of which influence the efficiency of software testing in practice. Accordingly, the study considers both human factors and the nature of errors during the debugging process to develop a software reliability growth model to estimate the related costs and the reliability indicator. Additionally, the issue of error classification is also extended by considering the impacts of errors on the system, and the expected time required to remove simple or complex errors can be estimated based on different truncated exponential distributions. Finally, numerical examples are presented and sensitivity analyses are performed to provide managerial insights and useful directions to inform software release strategies. [ABSTRACT FROM AUTHOR]

Fault-counting data are collected in the testing process of software development. However, the data are not used for evaluating the efficiency of fault correction activities because the information on the fault detection and correction times of each fault are not recorded in the fault-counting data. Furthermore, it is difficult to collect new data on the detection time of each fault to realize efficiency evaluation for fault correction activities from the collected fault-counting data due to the cost of personnel and data collection. In this paper, we apply the thinning method, using intensity functions of the delayed S-shaped and inflection S-shaped software reliability growth models (SRGMs) to generate sample data of the fault detection time from the fault-counting data. Additionally, we perform simulations based on the infinite server queuing model, using the generated sample data of the fault detection time to visualize the efficiency of fault correction activities. [ABSTRACT FROM AUTHOR]

Because of the sequence and number of calls of functions, software network cannot reflect the real execution of software. Thus, to detect crucial functions (DCF) based on software network is controversial. To address this issue, from the viewpoint of software dynamic execution, a novel approach to DCF is proposed in this paper. It firstly models, the dynamic execution process as an execution sequence by taking functions as nodes and tracing the stack changes occurring. Second, an algorithm for deleting repetitive patterns is designed to simplify execution sequence and construct software sequence pattern sets. Third, the crucial function detection algorithm is presented to identify the distribution law of the numbers of patterns at different levels and rank those functions so as to generate a decision-function-ranking-list (DFRL) by occurrence times. Finally, top-k discriminative functions in DFRL are chosen as crucial functions, and similarity the index of decision function sets is set up. Comparing with the results from Degree Centrality Ranking and Betweenness Centrality Ranking approaches, our approach can increase the node coverage to 80%, which is proven to be an effective and accurate one by combining advantages of the two classic algorithms in the experiments of different test cases on four open source software. The monitoring and protection on crucial functions can help increase the efficiency of software testing, strength software reliability and reduce software costs. [ABSTRACT FROM AUTHOR]

The Software Operational Profile (SOP) is a software specification based on how users use the software. This specification corresponds to a quantitative representation of the software that identifies its most used parts. As software reliability depends on the context in which users operate the software, the SOP is used in software reliability engineering. However, there is evidence of a misalignment between the software tested parts and the SOP. Therefore, this paper investigates a potential misalignment between SOP and the tested software parts to obtain more evidence of this misalignment based on experimental data. We performed a set of Experimental Studies - EXS to verify: a) whether there are significant variations in how users operate the software; b) whether there is a misalignment between the SOP and the tested software parts; c) whether failures occur in untested SOP parts in case of misalignment; d) whether a test strategy based on the amplification of the existent test set with additional test data generated automatically can contribute to reduce the misalignment between SOP and untested software parts. We collected data from four software while users were operating them. We analyzed this data to reach the goals of this work. The results show that there is significant variation in how users operate software and that there is a misalignment between SOP and the tested software parts after evaluating the four software studied. There is also indication of failures in the untested SOP parts. Although the aforementioned test strategy has reduced the potential misalignment, the test strategy is not enough to avoid it, thus indicating a need for specific test strategies using SOP as a test criterion. These results indicate that SOP is relevant not only to software reliability engineering but also to testing activities, regardless of the adopted testing strategy. [ABSTRACT FROM AUTHOR]

Software testing began as an empirical activity, and remains part of engineering practice without a widely accepted theoretical foundation. The overwhelming majority of test methods are designed to find software errors, termed faults, in program source code, but not to assess software operational quality. To go beyond fault-seeking requires a theory that relates static program properties to executions. In the 1970s and 1980s, Gerhart, Howden, and others developed a sound functional theory of program testing. Then Duran and others used this theory to precisely define the notions of random testing and operational reliability. In the Gerhart-Howden-Duran theory, a program's behavior is a pure input-output mapping. This paper extends the theory to include persistent state, by adding a state space to the input space, and a state mapping to a program's output mapping. The extended theory is significantly different because test states, unlike inputs, cannot be chosen arbitrarily. The theory is used to analyze state-based testing methods, to examine the practicality of reliability assessment, and to suggest experiments that would increase understanding of the statistical properties of software. [ABSTRACT FROM AUTHOR]

In this paper we discuss the application of a range of techniques to the verification of mission-critical flight software at NASA's Jet Propulsion Laboratory. For this type of application we want to achieve a higher level of confidence than can be achieved through standard software testing. Unfortunately, given the current state of the art, especially when efforts are constrained by the tight deadlines and resource limitations of a flight project, it is not feasible to produce a rigorous formal proof of correctness of even a well-specified stand-alone module such as a file system (much less more tightly coupled or difficult-to-specify modules). This means that we must look for a practical alternative in the area between traditional testing and proof, as we attempt to optimize rigor and coverage. The approaches we describe here are based on testing, model checking, constraint-solving, monitoring, and finite-state machine learning, in addition to static code analysis. The results we have obtained in the domain of file systems are encouraging, and suggest that for more complex properties of programs with complex data structures, it is possibly more beneficial to use constraint solvers to guide and analyze execution (i.e., as in testing, even if performed by a model checking tool) than to translate the program and property into a set of constraints, as in abstraction-based and bounded model checkers. Our experience with non-file-system flight software modules shows that methods even further removed from traditional static formal methods can be assisted by formal approaches, yet readily adopted by test engineers and software developers, even as the key problem shifts from test generation and selection to test evaluation. [ABSTRACT FROM AUTHOR]

The article focuses on the role of traceability in establishing evidence regarding the hazards and risks related to specifications and implementations of safety critical software devices such as pacemakers and train signals. Topics discussed include strategic practices to provide cost-effective support to evaluate a system's safety, traceability standards in safety-critical projects and analysis of traceability project documents by the U.S. Food and Drug Administration for medical devices.

The article offers information related to the DO-178C, software considerations in airborne systems and equipment certification document published by Radio Technical Commission for Aeronautics (RTCA) Inc. in 2011. It discusses the testing-based verification activities under the DO-178C which includes two processes that involves requirements-based and structural coverage analyses.

The article focuses on the role of model-driven approaches in the development of high-quality and cost effective safety-critical software for air traffic control (ATC). Topics discussed include interoperability of the model driven architecture (MDA) development paradigm, integration of MDA into the verification and validation (V-model) and MDA and Model-Driven testing in a V-Model.

The article focuses on the need of statistical model checking (SMC) of safety-critical softwares to validate software reliability at an early stage. It mentions that the softwares should be checked to prevent the propagation of reliability allocation errors and design errors at later stages. It further discusses a case study related to SMC of fault-tolerant fuel control system (FFCS) a safety-critical component of an automobile's engine controller.

Dynamic software updating (DSU) systems patch programs on the fly without incurring downtime. To avoid failures due to the updating process itself, many DSU systems employ timing restrictions. However, timing restrictions are theoretically imperfect, and their practical effectiveness is an open question. This paper presents the first significant empirical evaluation of three popular timing restrictions: activeness safety (AS), which prevents updates to active functions, con-freeness safety (CFS), which only allows modifications to active functions when doing so is provably type-safe, and manual identification of the event-handling loops during which an update may occur. We evaluated these timing restrictions using a series of DSU patches to three programs: OpenSSH, vsftpd, and ngIRCd. We systematically applied updates at each distinct update point reached during execution of a suite of system tests for these programs to determine which updates pass and which fail. We found that all three timing restrictions prevented most failures, but only manual identification allowed none. Further, although CFS and AS allowed many more update points, manual identification still supported updates with minimal delay. Finally, we found that manual identification required the least developer effort. Overall, we conclude that manual identification is most effective. [ABSTRACT FROM AUTHOR]

Nonhomogeneous poisson process based software reliability growth models are generally classified into two groups. The first group contains models, which use the machine execution time or calendar time as a unit of fault detection/removal period. Such models are called continuous time models. The second group contains models, which use the number of test occasions/cases as a unit of fault detection period. Such models are called discrete time models, since the unit of software fault detection period is countable. A large number of models have been developed in the first group while there are fewer in the second group. Discrete time models in software reliability are important and a little effort has been made in this direction. In this paper, we develop two discrete time SRGMs using probability generating junction for the software failure occurrence/fault detection phenomenon based on a NHPP namely, basic and extended models. The basic model exploits the fault detection/removal rate during the initial and final test cases. Whereas, the extended model incorporates fault generation and imperfect debugging with learning. Actual software reliability data have been used to demonstrate the proposed models. The results are fairly encouraging in terms of goodness-of-fit and predictive validity criteria due to applicability and flexibility of the proposed models as they can capture a wide class of reliability growth curves ranging from purely exponential to highly S-shaped. [ABSTRACT FROM AUTHOR]

A hardware failure analysis technique adapted to software yielded three rules for generating test cases sensitive to code errors. These rules, and a procedure for generating these cases, are given with examples. Areas for further study are recommended. [ABSTRACT FROM AUTHOR]

This paper presents a testing strategy designed to detect errors in the control flow of a computer program, and the conditions under which this strategy is reliable are given and characterized. The control flow statements in a computer program partition the input space into a set of mutually exclusive domains, each of which corresponds to a particular program path and consists of input data points which cause that path to be executed. The testing strategy generates test points to examine the boundaries of a domain to detect whether a domain error has occurred, as either one or more of these boundaries will have shifted or else the corresponding predicate relational operator has changed. If test points can be chosen within ∈ of each boundary, under the appropriate assumptions, the strategy is shown to be reliable in detecting domain errors of magnitude greater than ∈. Moreover, the number of test points required to test each domain grows only linearly with both the dimensionality of the input space and the number of predicates along the path being tested. [ABSTRACT FROM AUTHOR]

In this article we introduce the problem of computer software reliability and discuss a probabilistic model for describing the failure of software. We suggest a procedure for estimating the parameters of the model and propose a stopping rule for debugging the software. We apply our procedures to some published data on software failures. [ABSTRACT FROM AUTHOR]

Software testing can be viewed as an inductive inference process during which the tester attempts to deduce software properties from its behaviour on a finite number of test cases. This paper investigates the foundation of software testing by interpreting the axioms of test adequacy criteria as properties of inductive inference. The interpretation manifests the conservative and simplest hypothesis nature of the induction underlying software testing. It also yields results relating adequate testing to software correctness and reliability. The convergence of the inductive inference process is proved to he a condition of the correctness of tested software. By measuring the convergence according to the probability of an inference result being correct up to a given error rate, a new approach to software reliability estimation is proposed, which differs from existing ones in taking software complexity into account. [ABSTRACT FROM AUTHOR]

• Software services used in critical systems demand for accurate reliability assessment. • The work deals with assessment under operational profile uncertainty. • A Bayesian modelling framework with dynamic model selection is tested on web services. We address the problem of operational reliability assessment through testing of software services delivered on-demand such as Web Services. Software reliability assessment is typically done for a specific operational profile : the profile is needed in testing to select or generate test cases (operational testing) in a way statistically similar to the anticipated use of software in operation; the observations of success/failure of test executions are used to predict software reliability in actual operation. It is well known that unless the profile is accurate, software reliability predictions obtained via operational testing cannot be trusted. We present a new way of dealing with the uncertainty in the operational profile adopting a two-stage Bayesian inference for reliability assessment. The technique relies on the availability of information about partitions of the input space. The approach is demonstrated on contrived examples and on a case study of real Web Services. We discuss the usefulness of the approach in dealing with two important practical problems: i) the true profile in operation differs from the one used in testing, ii) the profile in operation is changing continuously. [ABSTRACT FROM AUTHOR]