Search

For software developers as well as software product users, fault removal efficiency is crucial. Developers can estimate the amount of work required by evaluating the proportion of addressed faults. At the same time, the level of trust in utilizing the software product can be determined by customers, based on this information. The proposed software reliability growth model considers fault removal efficiency relevant to contemporary scenarios characterized by automated testing and debugging tools. A comparison between the proposed model and other existing models is conducted by utilizing two data sets from software testing. Additionally, the best release plans are created by considering warranty costs, risk costs, and error removal costs while still meeting reliability requirements. [ABSTRACT FROM AUTHOR]

Considering the testing process of the software system as a stochastic process is a primary approach to the software reliability modeling technique. Besides some popular distributions, the Poisson distribution has been considered the best based on its advantage when modeling the times at which arrivals enter a system. In the non-homogeneous Poisson process group of models, the S-shaped function is a value curve with many good results. This paper proposes a new imperfect debugging software reliability model based on (1) an Imperfect debugging assumption (the testing process could cause new faults); and (2) the Fault detection rate can be controlled more effectively by the appearance of a growth-rate-controller. The real data from industrial projects verify the application of this model based on good popular criteria values. [ABSTRACT FROM AUTHOR]

Due to the increasing reliance on technology in nearly every industry over the past three decades, it has become necessary to evaluate the performance of a software product prior to its formal release in the market. The properties of a software application, such as its complexity and lines of code, are subject to change over time as a result of factors namely the testing environment, allocation of resources, testing efficiency, and testing team's expertise. The assumption of constant Fault Detection Rate (FDR) may not accurately anticipate the  potential number of bugs correctly. Keeping all these considerations in mind, a framework is developed to incorporate change point in the development of a testing effort-based Software Reliability Growth Model (SRGM) that takes into account the effect of application characteristics under both perfect and imperfect debugging settings. In addition, these outcomes are compared to the model without a change point. The proposed model is validated on two real-life software fault datasets. The results demonstrate that the proposed model performs better than the model without a change point. [ABSTRACT FROM AUTHOR]

When it comes to software development, precise planning, proper documentation and proper process control, some errors are inevitable in the software environment. These software flaws can lead to quality deterioration, which can be the main reason behind system failure. As the whole world especially developing countries is dependent upon software systems, it is very important to focus on its reliability aspect. Nowadays sophisticated systems require concerted efforts for managing and reducing the shortcomings in software engineering. But, these efforts require more cost, more money and more time. Software error prediction is the most helpful step in the testing stage of the software development life cycle. It identifies components or parts of the code where an error may occur and requires broad testing, so the test resources can be efficiently used. Software error assessment reduces efforts of testing the software by helping the software testers locate the actual problem and classify different classes of errors in the system. Error estimators are majorly used in various organizations to evaluate the software to save time, improve the quality of software and testing and optimize resources to meet timelines. Machine learning provides support in fault projection by collecting the training data from various edge devices and thus helps in escalating the reliability of the software available on Kaggle. The multilayer perceptron shows better results in precision, recall, F1 score and accuracy as compared to decision tree and Gaussian Naive Bayes as it achieves an accuracy of 96.8%. [ABSTRACT FROM AUTHOR]

Software is playing a growing role in many safety-critical applications, and software systems dependability is a major concern. Predicting faulty modules of software before the testing phase is one method for enhancing software reliability. The ability to predict and identify the faulty modules of software can lower software testing costs. Machine learning algorithms can be used to solve software fault prediction problem. Identifying the faulty modules of software with the maximum accuracy, precision, and performance are the main objectives of this study. A hybrid method combining the autoencoder and the K-means algorithm is utilized in this paper to develop a software fault predictor. The autoencoder algorithm, as a preprocessor, is used to select the effective attributes of the training dataset and consequently to reduce its size. Using an autoencoder with the K-means clustering method results in lower clustering error and time. Tests conducted on the standard NASA PROMIS data sets demonstrate that by removing the inefficient elements from the training data set, the proposed fault predictor has increased accuracy (96%) and precision (93%). The recall criteria provided by the proposed method is about 87%. Also, reducing the time necessary to create the software fault predictor is the other merit of this study. [ABSTRACT FROM AUTHOR]

A critical aspect of software engineering is Software fault prediction which aims to identify and prevent errors in software systems before their release which can cause failures or issues for its users. Various techniques and tools have been developed to detect software faults, including static code analysis, dynamic testing, and machine learning-based approaches. In past few years, the world has seen a growing interest in the use of ML models for predicting software faults, as they can effectively analyse high dimensional datasets and detect complex patterns which are difficult for human experts to detect. However, developing accurate and reliable software fault detection models requires careful selection of data, feature engineering, and model evaluation. This purpose of this paper is to present a comprehensive analysis of potential applications and future research directions in the field of software fault detection. The study emphasizes the importance of identifying and addressing software faults to ensure the reliability and efficiency of software systems. Additionally, the paper outlines various approaches and techniques that can be employed for effective software fault detection. [ABSTRACT FROM AUTHOR]

In recent years, software development models have undergone changes. In order to meet user needs and functional changes, open-source software continuously improves its software quality through successive releases. Due to the iterative development process of open-source software, open-source software testing also requires continuous learning to understand the changes in the software. Therefore, the fault detection process of open-source software involves a learning process. Additionally, the complexity and uncertainty of the open-source software development process also lead to stochastically introduced faults when troubleshooting in the open-source software debugging process. Considering the phenomenon of learning factors and the random introduction of faults during the testing process of open-source software, this paper proposes a reliability modeling method for open-source software that considers learning factors and the random introduction of faults. Least square estimation and maximal likelihood estimation are used to determine the model parameters. Four fault data sets from Apache open-source software projects are used to compare the model performances. Experimental results indicate that the proposed model is superior to other models. The proposed model can accurately predict the number of remaining faults in the open-source software and be used for actual open-source software reliability evaluation. [ABSTRACT FROM AUTHOR]

One of the significant features of software quality is software reliability. In the testing phase, faults are identified and corrected by integrating them into software development, thus obtaining better reliability. Here, by utilizing the Elliptical Distributions-centric Emperor Penguins Colony Algorithm (ED-EPCA)-based Test Case Prioritization (TCP), an effectual Fault Detection (FD) technique is proposed using Fishers Yates Shuffled Shepherd Optimization Algorithm (FY-SSOA)-based Test Case Selection (TCS). Initially, for the incoming source code, the Test Case (TC) is created. Then, the significant factors needed for TCS and prioritization are identified. Next, by utilizing the Log Scaling-centered Generalized Discriminant Analysis (LS-GDA) model, the estimated factors are abated further to enhance the TCS along with prioritization for the Fault Detection Process (FDP). Then, using the FY-SSOA, the optimized TCs are selected. Subsequently, with the help of ED-EPCA, the TCs being selected are ranked as well as prioritized. Finally, to validate the proposed system's effectiveness, the model's performance is evaluated in the working platform of Java and analogized with the traditional methodologies. The results indicate that the test case prioritization-based fault detection method is robust with a 99.23% fault detection rate and a small amount of memory usage, which is only 8245475 kb by generating a large number of test cases. [ABSTRACT FROM AUTHOR]

Based on two progressive aspects of the modeling problems in business process management (BPM), (1) in order to address the increasing complexity of user requirements on workflows underlying various BPM application scenarios, a more verifiable fundamental modeling method must be invented; (2) to address the diversification of software testing processes, more formalized advanced modeling technology must also be applied based on the fundamental modeling method. Aiming to address these modeling problems, this paper first proposes an ADCV (acquisition, decomposition, combination, and verification) method that runs through the core management links of four types of business processes (mining, decomposition, recombination, and verification) and then describes the compositional structure of the ADCV method and the design of corresponding algorithms. Then, the software testing workflow is managed and monitored using the method, and the corresponding analysis tool is implemented based on Petri nets. At the same time, the tool is applied to the case processing of the software testing workflow. Specifically, the workflow models are established successively through ADCV during the process of business iteration. Then, the analysis tool developed with the ADCV method, the model–view–controller (MVC) design pattern, and Java Swing technology are applied to instances of the software testing workflow to realize the modeling and management of the testing processes. Thus, the analysis tool can guarantee the accuracy of the parameter estimations of related software reliability growth models (SRGMs) and ultimately improve the quality of software products. [ABSTRACT FROM AUTHOR]

Emerging technologies in artificial intelligence (AI) and advanced optimization methodologies have opened up a new frontier in the field of software engineering. Among these methodologies, optimization algorithms such as the multi-verse optimizer (MVO) provide a compelling and structured technique for identifying software vulnerabilities, thereby enhancing software robustness and reliability. This research investigates the feasibility and efficacy of applying an augmented version of this technique, known as the test path distance multiverse optimization (TPDMVO) algorithm, for comprehensive path coverage testing, which is an indispensable aspect of software validation. The algorithm's versatility and robustness are examined through its application to a wide range of case studies with varying degrees of complexity. These case studies include rudimentary functions like maximum and middle value extraction, as well as more sophisticated data structures such as binary search trees and AVL trees. A notable innovation in this research is the introduction of a customized fitness function, carefully designed to guide TPDMVO towards traversing all possible execution paths in a program, thereby ensuring comprehensive coverage. To further enhance the comprehensiveness of the test, a metric called 'test path distance' (TPD) is utilized. This metric is designed to guide TPDMVO towards paths that have not been explored before. The findings confirm the superior performance of the TPDMVO algorithm, which achieves complete path coverage in all test scenarios. This demonstrates its robustness and adaptability in handling different program complexities. [ABSTRACT FROM AUTHOR]

Software defect prediction is a popular method for optimizing software testing and improving software quality and reliability. However, software defect datasets usually have quality problems, such as class imbalance and data noise. Oversampling by generating the minority class samples is one of the most well-known methods to improving the quality of datasets; however, it often introduces overfitting noise to datasets. To better improve the quality of these datasets, this paper proposes a method called US-PONR, which uses undersampling to remove duplicate samples from version iterations and then uses oversampling through propensity score matching to reduce class imbalance and noise samples in datasets. The effectiveness of this method was validated in a software prediction experiment that involved 24 versions of software data in 11 projects from PROMISE in noisy environments that varied from 0% to 30% noise level. The experiments showed a significant improvement in the quality of datasets pre-processed by US-PONR in noisy imbalanced datasets, especially the noisiest ones, compared with 12 other advanced dataset processing methods. The experiments also demonstrated that the US-PONR method can effectively identify the label noise samples and remove them. [ABSTRACT FROM AUTHOR]

Software reliability growth models (SRGMs) are used to estimate future failure rates and the number of residual defects in software. SRGM helps reliability software engineers make test termination decisions. Although more than 250 traditional SRGMs have been proposed for reliability assessment, research to develop more reliable models is still ongoing. Recently, new methods based on neural networks have been developed to improve the accuracy of software reliability assessment. The article develops a neural network algorithm for assessing software reliability. To do this, factors affecting software reliability and covering its life cycle were assessed. Based on them, sets for training a three-layer neural network were created. [ABSTRACT FROM AUTHOR]

To assure the reliability and quality of the final product, testing is an essential and crucial part in the software development cycle. During this process, fault correction/detection activities are carried out to increase the reliability of the software. The nonhomogeneous Poisson Process (NHPP) is the basis of the investigated software reliability growth models (SRGMs), which are based on the supposition that the number of faults found is affected by the amount of code covered during testing and that the amount of code covered during testing depends on the testing effort expended. This research takes into consideration several testing coverage functions: exponential, delayed S-shaped and logistic distributions, to propose three SRGMs that are based on testing efforts. For testing effort expenditure Weibull distribution has been employed. Two real failure datasets have been utilised to validate the proposed models, and their performance is evaluated using four goodness-of-fit metrics, including predictive ratio risk (PRR), coefficient of determination (R2), predictive power (PP) and mean square error (MSE). Sensitivity analysis of cost requirement-based release time of software for exponential function has been done by using a genetic algorithm, which minimized the overall cost of the software subject to the requirement for reliability. [ABSTRACT FROM AUTHOR]

Effort‐Aware Defect Prediction (EADP) methods sort software modules based on the defect density and guide the testing team to inspect the modules with high defect density first. Previous studies indicated that some feature selection methods could improve the performance of Classification‐Based Defect Prediction (CBDP) models, and the Correlation‐based feature subset selection method with the Best First strategy (CorBF) performed the best. However, the practical benefits of feature selection methods on EADP performance are still unknown, and blindly employing the best‐performing CorBF method in CBDP to pre‐process the defect datasets may not improve the performance of EADP models but possibly result in performance degradation. To assess the impact of the feature selection techniques on EADP, a total of 24 feature selection methods with 10 classifiers embedded in a state‐of‐the‐art EADP model (CBS+) on the 41 PROMISE defect datasets were examined. We employ six evaluation metrics to assess the performance of EADP models comprehensively. The results show that (1) The impact of the feature selection methods varies in classifiers and datasets. (2) The four wrapper‐based feature subset selection methods with forwards search, that is, AdaBoost with Forwards Search, Deep Forest with Forwards Search, Random Forest with Forwards Search, and XGBoost with Forwards Search (XGBF) are better than other methods across the studied classifiers and the used datasets. And XGBF with XGBoost as the embedded classifier in CBS+ performs the best on the datasets. (3) The best‐performing CorBF method in CBDP does not perform well on the EADP task. (4) The selected features vary with different feature selection methods and different datasets, and the features noc (number of children), ic (inheritance coupling), cbo (coupling between object classes), and cbm (coupling between methods) are frequently selected by the four wrapper‐based feature subset selection methods with forwards search. (5) Using AdaBoost, deep forest, random forest, and XGBoost as the base classifiers embedded in CBS+ can achieve the best performance. In summary, we recommend the software testing team should employ XGBF with XGBoost as the embedded classifier in CBS+ to enhance the EADP performance. [ABSTRACT FROM AUTHOR]

Software Industry is evolving at a very fast pace since last two decades. Many software developments, testing and test case generation approaches have evolved in last two decades to deliver quality products and services. Testing plays a vital role to ensure the quality and reliability of software products. In this paper authors attempted to conduct a systematic study of testing tools and techniques. Six most popular e-resources called IEEE, Springer, Association for Computing Machinery (ACM), Elsevier, Wiley and Google Scholar to download 738 manuscripts out of which 125 were selected to conduct the study. Out of 125 manuscripts selected, a good number approx. 79% are from reputed journals and around 21% are from good conference of repute. Testing tools discussed in this paper have broadly been divided into five different categories: open source, academic and research, commercial, academic and open source, and commercial & open source. The paper also discusses several benchmarked datasets viz. Evosuite 10, SF100 Corpus, Defects4J repository, Neo4j, JSON, Mocha JS, and Node JS to name a few. Aim of this paper is to make the researchers aware of the various test case generation tools and techniques introduced in the last 11 years with their salient features. [ABSTRACT FROM AUTHOR]

In the literature, infinite-failure software reliability models (SRMs), such as Musa-Okumoto SRM (1984), have been demonstrated to be effective in quantitatively characterizing software testing processes and assessing software reliability. This paper primarily focuses on the infinite-failure (type-II) non-homogeneous Poisson process (NHPP)-based SRMs and evaluates the performances of these SRMs comprehensively by comparing with the existing finite-failure (type-I) NHPP-based SRMs. In more specific terms, to describe the software fault-detection time distribution, we postulate 11 representative probability distribution functions that can be categorized into the generalized exponential distribution family and the extreme-value distribution family. Then, we compare the goodness-of-fit and predictive performances with the associated 11 type-I and type-II NHPP-based SRMs. In numerical experiments, we analyze software fault-count data, collected from 16 actual development projects, which are commonly known in the software industry as fault-count time-domain data and fault-count time-interval data (group data). The maximum likelihood method is utilized to estimate the model parameters in both NHPP-based SRMs. In a comparison of the type-I with the type-II, it is shown that the type-II NHPP-based SRMs could exhibit better predictive performance than the existing type-I NHPP-based SRMs, especially in the early stage of software testing. [ABSTRACT FROM AUTHOR]

Software testing and debugging is a crucial part of the software development process since defective software not only incurs customer dissatisfaction but also might incur legal issues. However, the managers of a software development company cannot arbitrarily prolong their software debugging period due to their software testing budget and opportunity in the market. Accordingly, in order to propose an advantageous testing project, the managers should be aware of the influence of the testing project on cost, quality, and time to make the best decision. In this study, a new software reliability growth model (SRGM) with consideration of the testing staff's learning effect is proposed to achieve better prediction. The methods of estimating the model's parameters and the symmetric confidence intervals are also proposed in the study. Moreover, in the past, most of the SRGMs focused on a single software system. However, in practice, some software systems were developed using modular-based system engineering approaches. Therefore, traditional software testing work can be changed to multiple modular testing work in this scenario. Therefore, the manager can use this to dispatch multiple staff groups to perform the individual testing work simultaneously. The study proposes two mathematical programming models to handle the scheduling of modular testing work. Additionally, the design of a computerized decision support system is also proposed in the study for the application in practice. [ABSTRACT FROM AUTHOR]

In the software development life cycle, the quality and reliability of software are critical to software developers. Poor quality and reliability not only cause the loss of customers and sales but also increase the operational risk due to unreliable codes. Therefore, software developers should try their best to reduce such potential software defects by undertaking a software testing project. However, to pursue perfect and faultless software is unrealistic since the budget, time, and testing resources are limited, and the software developers need to reach a compromise that balances software reliability and the testing cost. Using the model presented in this study, software developers can devise multiple alternatives for a software testing project, and each alternative has its distinct allocation of human resources. The best alternative can therefore be selected. Furthermore, the allocation incorporates debuggers' learning and negligent factors, both of which influence the efficiency of software testing in practice. Accordingly, the study considers both human factors and the nature of errors during the debugging process to develop a software reliability growth model to estimate the related costs and the reliability indicator. Additionally, the issue of error classification is also extended by considering the impacts of errors on the system, and the expected time required to remove simple or complex errors can be estimated based on different truncated exponential distributions. Finally, numerical examples are presented and sensitivity analyses are performed to provide managerial insights and useful directions to inform software release strategies. [ABSTRACT FROM AUTHOR]

In this study, an imperfect debugging software reliability growth model (SRGM) with Bayesian analysis was proposed to determine an optimal software release in order to minimize software testing costs and also enhance the practicability. Generally, it is not easy to estimate the model parameters by applying MLE (maximum likelihood estimation) or LSE (least squares estimation) with insufficient historical data. Therefore, in the situation of insufficient data, the proposed Bayesian method can adopt domain experts' prior judgments and utilize few software testing data to forecast the reliability and the cost to proceed with the prior analysis and the posterior analysis. Moreover, the debugging efficiency involves testing staff's learning and negligent factors, and therefore, the human factors and the nature of debugging process are taken into consideration in developing the fundamental model. Based on this, the estimation of the model's parameters would be more intuitive and can be easily evaluated by domain experts, which is the major advantage for extending the related applications in practice. Finally, numerical examples and sensitivity analyses are performed to provide managerial insights and useful directions for software release strategies. [ABSTRACT FROM AUTHOR]

Fault-counting data are collected in the testing process of software development. However, the data are not used for evaluating the efficiency of fault correction activities because the information on the fault detection and correction times of each fault are not recorded in the fault-counting data. Furthermore, it is difficult to collect new data on the detection time of each fault to realize efficiency evaluation for fault correction activities from the collected fault-counting data due to the cost of personnel and data collection. In this paper, we apply the thinning method, using intensity functions of the delayed S-shaped and inflection S-shaped software reliability growth models (SRGMs) to generate sample data of the fault detection time from the fault-counting data. Additionally, we perform simulations based on the infinite server queuing model, using the generated sample data of the fault detection time to visualize the efficiency of fault correction activities. [ABSTRACT FROM AUTHOR]

An optimal software release strategy is a well-investigated issue in software reliability literature. Comprehensive testing is expected before releasing the software into the market to enhance the reliability and security of the software device. In recent years, few analysts have recommended the scheme for software projects that support releasing the software early in the market and continue the testing process for an added period in the field environment even after the software is distributed. These studies are based on one common assumption that the efficiency of the software engineers in detecting the faults occurs at a consistent rate throughout the testing phase. However, bug-identification rate may experience discontinuity at the software release time. In software engineering, the time-point at which fault detection rate changes is termed as change-point. Consequently, an alternative software release policy is proposed in the present paper, which offers a generalized framework for fault detection phenomenon using the unified approach. An extensive analysis of software time-to-market and testing duration based on cost-efficiency and reliability measures is discussed by considering the change in tester's fault detection rate. A multi-criteria decision making technique known as multi-attribute utility theory is applied to optimize the software release policy under field-testing (FT) and no field-testing (NFT) frameworks. The relevance of the optimization problem is illustrated using a numerical example, comprising both the exponential and S-shaped bug-detection process. [ABSTRACT FROM AUTHOR]

Open source software (OSS) has become one of the modern software development methods. OSS is mainly developed by developers, volunteers, and users all over the world, but its reliability has been widely questioned. When OSS faults are detected, volunteers or users send them to developers by email or network. After the developer confirms the fault, it will be randomly assigned to the debugger who may be a developer, a volunteer, or a user. These open source community contributors also have the phenomenon of learning when removing faults. When the detected faults are removed, the number of introduced faults decreases gradually. Therefore, this study proposes a software reliability model with the decreasing trend of fault introduction in the process of OSS development and testing. The validity of the proposed model and the accuracy of estimating residual faults are verified by experiments. The proposed model can be used to evaluate the reliability and predict the remaining faults in the actual OSS development and testing process. [ABSTRACT FROM AUTHOR]

To satisfy the needs of the individualized manufacturing of products, the smart manufacturing system (SMS) is frequently reconfigured. To quickly verify the reliability and adaptability of industrial software in reconfiguring the SMS for new or upgraded product orders, a semi-physical simulation method for testing and evaluation of industrial software is proposed based on digital-twins-driven technology. By establishing a semi-physical simulation model of SMS, the reliability and robustness of the software system are quickly verified by running industrial software in various manufacturing scenarios. In this paper, the key technologies to carry out semi-physical simulation testing and evaluation of industrial software for SMSs are expounded in detail, including how to synchronize cyber and physical systems, how to conduct semi-physical accelerated simulation testing, and how to identify defects quickly in industrial software used in actual production environments. By establishing a semi-physical simulation production line model for stepper motors, the effectiveness and practicality of the proposed approach are verified, and the testing verification time of industrial software is significantly reduced. Finally, the robustness of the industrial software for SMS is further verified by conducting fault injection testing, so as to provide implications for fault prognostics or fault-prevention research. [ABSTRACT FROM AUTHOR]

Software defect prediction has become a significant study path in the field of software engineering in order to increase software reliability. Program defect predictions are being used to assist developers in identifying potential problems and optimizing testing resources to enhance program dependability. As a consequence of this strategy, the number of software defects may be predicted, and software testing resources are focused on the software modules with the most problems, allowing the defects to be addressed as soon as feasible. The author proposes a research method of defect prediction technology in software engineering based on convolutional neural network. Most of the existing defect prediction methods are based on the number of lines of code, module dependencies, stack reference depth, and other artificially extracted software features for defect prediction. Such methods do not take into account the underlying semantic features in software source code, which may lead to unsatisfactory prediction results. The author uses a convolutional neural network to mine the semantic features implicit in the source code and use it in the task of software defect prediction. Empirical studies were conducted on 5 software projects on the PROMISE dataset and using the six evaluation indicators of Recall, F1, MCC, pf, gm, and AUC to verify and analyze the experimental results showing that the AUC values of the items varied from 0.65 to 0.86. Obviously, software defect prediction experimental results obtained using convolutional neural networks are still ideal. Defect prediction model in software engineering based on convolutional neural network has high prediction accuracy. [ABSTRACT FROM AUTHOR]

Formal specifications can provide a solid foundation for software development and support for techniques of software quality assurance, such as specification-based inspection and testing. To ensure that these techniques can be applied effectively in practice, efficiently and accurately understanding specifications becomes extremely important. While this may be relatively easy for well-trained developers in formal methods, it can be rather difficult for computer since computer does not easily understand specifications. This difficulty poses a challenge for realizing automatic specification-based verification techniques that are in high demand for reducing development cost and improving software reliability. In this paper, we address this problem by discussing how the formal specification can be transformed into a knowledge graph to provide comprehensible, well-organized details of the specification for developers and computers. The transformation is done by extracting and storing information about attributes of each component and by establishing relationships between components in a formal specification. We elaborate on a top-down approach of constructing a knowledge graph from a specification, including creating an ontology, designing the Entity–Relationship (ER) diagram of the relational database based on the created ontology, extracting and storing attribute and relationship information in the relational database, mapping ontology to its instances and relational data to RDF triples, and displaying knowledge graph. Further, we present a case study to show how our approach works on the formal specification of an ATM system. Finally, we describe three experiments to evaluate its performance in improving specification readability, effectively guiding inspectors to establish traceability links between specifications and programs, and detecting defects through program inspection, respectively. [ABSTRACT FROM AUTHOR]

Mutation testing is basically a fault-based software testing technique, which has been proposed to measure the fault detection effectiveness of a test suite using programs with simulated faults (namely mutants). However, mutation testing is time consuming and computationally expensive because of the normal use of a large amount of mutants. Thus, reducing the mutants is of great significance. To address this problem, various mutant reduction techniques have been proposed. Among them, the identification of redundant mutants aims at removing mutants whose test results can be inferred by other mutants. This article proposes a redundant mutant identification technique based on selective symbolic execution called ReMuSSE for weak mutation testing. Redundant mutants could be revealed by identifying those with similar program execution state changes within a program block involving mutated statements. An empirical study was conducted using 13 C programs from different application domains with varying sizes. The empirical results showed that ReMuSSE could identify up to 31.4% redundant mutants and consequentially save up to 35.2% time cost of weak mutation testing. The results demonstrated that ReMuSSE could effectively identify redundant mutants and thus could significantly improve the efficiency of weak mutation testing. [ABSTRACT FROM AUTHOR]

Software testing is undertaken to ensure that the software meets the expected requirements. The intention is to find bugs, errors, or defects in the developed software so that they can be fixed before deployment. Testing of the software is needed even after it is deployed. Regression testing is an inevitable part of software development, and must be accomplished in the maintenance phase of software development to ensure software reliability. The existing literature presents a large amount of relevant knowledge about the types of techniques and approaches used in regression test case selection and prioritization (TCS&P), comparisons of techniques used in TCS&P, and the data used. Numerous secondary studies (surveys or reviews) have been conducted in the area of TCS&P. This study aimed to provide a comprehensive examination of the analysis of the enhancements in TCS&P using a thorough systematic literature review (SLR) of the existing secondary studies. This SLR provides: (1) a collection of all the valuable secondary studies (and their qualitative analysis); (2) a thorough analysis of the publications and the trends of the secondary studies; (3) a classification of the various approaches used in the secondary studies; (4) insight into the specializations and range of years covered in the secondary texts; (5) a comprehensive list of statistical tests and tools used in the area; (6) insight into the quality of the secondary studies based on the seven selected Research Paper Quality parameters; (7) the common problems and challenges encountered by researchers; (8) common gaps and limitations of the studies; and (9) the probable prospects for research in the field of TCS&P. [ABSTRACT FROM AUTHOR]

An effective software fault prediction (SFP) model could help developers in the quick and prompt detection of faults and thus help enhance the overall reliability and quality of the software project. Variations in the prediction performance of learning techniques for different software systems make it difficult to select a suitable learning technique for fault prediction modeling. The evaluation of previously presented SFP approaches has shown that single machine learning-based models failed to provide the best accuracy in any context, highlighting the need to use multiple techniques to build the SFP model. To solve this problem, we present and discuss a software fault prediction approach based on selecting the most appropriate learning techniques from a set of competitive and accurate learning techniques for building a fault prediction model. In work, we apply the discussed SFP approach for the five Eclipse project datasets and nine Object-oriented (OO) project datasets and report the findings of the experimental study. We have used different performance measures, i.e., AUC, accuracy, sensitivity, and specificity, to assess the discussed approach's performance. Further, we have performed a cost-benefit analysis to evaluate the economic viability of the approach. Results showed that the presented approach predicted the software's faults effectively for the used accuracy, AUC, sensitivity, and specificity measures with the highest achieved values of 0.816, 0.835, 0.98, and 0.903 for AUC, accuracy, sensitivity, and specificity, respectively. The cost-benefit analysis of the approach showed that it could help reduce the overall software testing cost. [ABSTRACT FROM AUTHOR]

Researchers have proposed several software reliability growth models, many of which possess complex parametric forms. In practice, software reliability growth models should exhibit a balance between predictive accuracy and other statistical measures of goodness of fit, yet past studies have not always performed such balanced assessment. This paper proposes a framework for software reliability growth models possessing a bathtub-shaped fault detection rate and derives stable and efficient expectation conditional maximization algorithms to enable the fitting of these models. The stages of the bathtub are interpreted in the context of the software testing process. The illustrations compare multiple bathtub-shaped and reduced model forms, including classical models with respect to predictive and information theoretic measures. The results indicate that software reliability growth models possessing a bathtub-shaped fault detection rate outperformed classical models on both types of measures. The proposed framework and models may therefore be a practical compromise between model complexity and predictive accuracy. [ABSTRACT FROM AUTHOR]

Software reliability growth models (SRGMs) are widely used to estimate software reliability by analyzing failure dataset throughout the testing process. A large number of SRGMs have been proposed on a regular basis by researchers since the 1970s. They are represented with a set of assumptions and a set of parameters. One major problem in SRGMs is that the uncertainties surrounding the assumptions and parameters are generally not taken into account by most of them. Therefore, sometimes, the predicted reliability on testing phase significantly varies in actual operational phase. This paper presents a logistic growth model that incorporates a special parameter to consider the effects of all possible uncertainties. A systematic analysis is carried out to identify the major uncertain factors and their impacts on the fault detection rate. The applicability of the model is shown by validating it on two different real datasets that are commonly used in various studies. The comparisons with nine established models in terms of mean square error (MSE), variance, predictive-ratio risk (PRR), R 2 and AIC have been presented. [ABSTRACT FROM AUTHOR]

Software reliability growth models are nonlinear in nature, so it is difficult to estimate the proper parameters. An estimation method based on a modified whale optimization algorithm in which parameters are estimated is discussed in this paper. The whale optimization algorithm is a new swarm intelligence optimization algorithm. This algorithm is not perfect enough. Based on the analysis of whale optimization algorithm, we point out the disadvantages of whale optimization algorithm, and propose a modified whale optimization algorithm algorithm from four aspects: choice regarding the dimension, exploration control, encircling prey modified, and candidate solution selection. The experimental results based on 34 benchmark functions demonstrate that the proposed modified whale optimization algorithm has better accuracy. The modified whale optimization algorithm is used to predict software reliability by predicting the faults during the software testing process using software faults' historical data. The proposed modified whale optimization algorithm shows significant advantages in handling a variety of modeling problems such as the exponential model, power model, delayed s-shaped model, and modified sigmoid model. Experimental results show that the fitting accuracy of the modified sigmoid model model is minimal on three data sets. The modified whale optimization algorithm with the modified sigmoid model can provide a better estimate of the software faults. [ABSTRACT FROM AUTHOR]

In the era of intelligent systems, the safety and reliability of software have received more attention. Software reliability testing is a significant method to ensure reliability, safety and quality of software. The intelligent software technology has not only offered new opportunities but also posed challenges to software reliability technology. The focus of this paper is to explore the software reliability testing technology under the impact of intelligent software technology. In this study, the basic theories of traditional software and intelligent software reliability testing were investigated via related previous works, and a general software reliability testing framework was established. Then, the technologies of software reliability testing were analyzed, including reliability modeling, test case generation, reliability evaluation, testing criteria and testing methods. Finally, the challenges and opportunities of software reliability testing technology were discussed at the end of this paper. [ABSTRACT FROM AUTHOR]

Software testing is an important stage in the software development process, which is the key to ensure software quality and improve software reliability. Software fault localization is the most important part of software testing. In this paper, the fault localization problem is modeled as a combinatorial optimization problem, using the function call path as a starting point. A heuristic search algorithm based on hybrid genetic simulated annealing algorithm is used to locate software defects. Experimental results show that the fault localization method, which combines genetic algorithm, simulated annealing algorithm and function correlation analysis method, has a good effect on single fault localization and multi-fault localization. It greatly reduces the requirement of test case coverage and the burden of the testers, and improves the effect of fault localization. [ABSTRACT FROM AUTHOR]

Because of the sequence and number of calls of functions, software network cannot reflect the real execution of software. Thus, to detect crucial functions (DCF) based on software network is controversial. To address this issue, from the viewpoint of software dynamic execution, a novel approach to DCF is proposed in this paper. It firstly models, the dynamic execution process as an execution sequence by taking functions as nodes and tracing the stack changes occurring. Second, an algorithm for deleting repetitive patterns is designed to simplify execution sequence and construct software sequence pattern sets. Third, the crucial function detection algorithm is presented to identify the distribution law of the numbers of patterns at different levels and rank those functions so as to generate a decision-function-ranking-list (DFRL) by occurrence times. Finally, top-k discriminative functions in DFRL are chosen as crucial functions, and similarity the index of decision function sets is set up. Comparing with the results from Degree Centrality Ranking and Betweenness Centrality Ranking approaches, our approach can increase the node coverage to 80%, which is proven to be an effective and accurate one by combining advantages of the two classic algorithms in the experiments of different test cases on four open source software. The monitoring and protection on crucial functions can help increase the efficiency of software testing, strength software reliability and reduce software costs. [ABSTRACT FROM AUTHOR]

Software failures caused by data race bugs have always been major concerns in parallel and distributed systems, despite significant efforts spent in software testing. Due to their nondeterministic and hard-to-reproduce features, when evaluating systems’ operational reliability, a rather long period of experimental execution time is expected to be spent on observing failures caused by data race conditions. To address this problem, in this paper, we make two contributions. First, this paper proposes stress testing with influencing factors, in which the system runs under certain workloads for a long time with controlled stress conditions to accelerate the occurrence of data race failures. Second, it explores and formulates mathematical relationship models between data races’ statistical characteristics of time to failure (TTF) or mean TTF (MTTF) and the influencing factors. Such relationship models are used for TTF/MTTF extrapolation under different operational conditions and are essential to reduce systems’ reliability evaluation time. The proposed method is empirically evaluated on six applications suffering from failures caused by real-world data race bugs. Through analysis of the experimental results, we obtain several important findings: First, the reduction in the manifestation time to data race failures achieved by controlling the influencing factors is statistically significant. Second, Power model is the best-fitting model of the relationship between the MTTF and the influencing factors. Third, Power Weibull distribution is the best-fitting probability distribution between the TTF and the influencing factors. Finally, the TTF/MTTF can be accurately estimated with the approach proposed in this paper. [ABSTRACT FROM AUTHOR]

The Software Operational Profile (SOP) is a software specification based on how users use the software. This specification corresponds to a quantitative representation of the software that identifies its most used parts. As software reliability depends on the context in which users operate the software, the SOP is used in software reliability engineering. However, there is evidence of a misalignment between the software tested parts and the SOP. Therefore, this paper investigates a potential misalignment between SOP and the tested software parts to obtain more evidence of this misalignment based on experimental data. We performed a set of Experimental Studies - EXS to verify: a) whether there are significant variations in how users operate the software; b) whether there is a misalignment between the SOP and the tested software parts; c) whether failures occur in untested SOP parts in case of misalignment; d) whether a test strategy based on the amplification of the existent test set with additional test data generated automatically can contribute to reduce the misalignment between SOP and untested software parts. We collected data from four software while users were operating them. We analyzed this data to reach the goals of this work. The results show that there is significant variation in how users operate software and that there is a misalignment between SOP and the tested software parts after evaluating the four software studied. There is also indication of failures in the untested SOP parts. Although the aforementioned test strategy has reduced the potential misalignment, the test strategy is not enough to avoid it, thus indicating a need for specific test strategies using SOP as a test criterion. These results indicate that SOP is relevant not only to software reliability engineering but also to testing activities, regardless of the adopted testing strategy. [ABSTRACT FROM AUTHOR]

Generation of complex heap structures is required for many software testing and verification techniques, which are used to enhance software reliability. It enables them to work for programs dependent on such structures. Generating all the valid structures within a size bound is time-consuming, and it is hard to predict an input size completely generatable within a time bound. This is due to the depth-first traversal of the candidate space. However, an efficient breadth-first traversal requires a compact state encoding to store candidates for exploration in the next level. In this paper, we present present a novel state encoding technique that stores an incomplete representation of state that can be recovered during search. We build upon the Korat algorithm—demonstrated to effectively generate tests and find bugs for many software programs—and present iKorat, an incremental algorithm for breadth-first exploration of the search space. Our encoding enables a more efficient implementation of iterative deepening that avoids redundant work. Standard iterative deepening algorithms allow a breadth-first search when the underlying algorithm is depth-first by repeating the work of earlier iterations. iKorat, however, uses information from smaller sizes to avoid redundant work for larger sizes. It also enables a new technique for parallelizing Korat by communicating candidates using our encoding. piKorat generates structures of larger sizes in parallel as soon as incremental information is available from smaller sizes. Our evaluation shows that iKorat-based iterative deepening is more efficient than Korat and that piKorat naturally extends the technique for parallel generation of complex structures. [ABSTRACT FROM AUTHOR]

Many security and software testing applications require checking whether certain properties of a program hold for any possible usage scenario. For instance, a tool for identifying software vulnerabilities may need to rule out the existence of any backdoor to bypass a program's authentication. One approach would be to test the program using different, possibly random inputs. As the backdoor may only be hit for very specific program workloads, automated exploration of the space of possible inputs is of the essence. Symbolic execution provides an elegant solution to the problem, by systematically exploring many possible execution paths at the same time without necessarily requiring concrete inputs. Rather than taking on fully specified input values, the technique abstractly represents them as symbols, resorting to constraint solvers to construct actual instances that would cause property violations. Symbolic execution has been incubated in dozens of tools developed over the past four decades, leading to major practical breakthroughs in a number of prominent software reliability applications. The goal of this survey is to provide an overview of the main ideas, challenges, and solutions developed in the area, distilling them for a broad audience. [ABSTRACT FROM AUTHOR]

In software testing, optimal testing resource allocation problems (OTRAPs) are important when seeking a good tradeoff between reliability, cost, and time with limited resources. There have been intensive studies of OTRAPs using multiobjective evolutionary algorithms (MOEAs), but little attention has been paid to the constraint handling. This paper comprehensively investigates the effect of the constraint handling on the performance of nondominated sorting genetic algorithm II (NSGA-II) for solving OTRAPs, from both theoretical and empirical perspectives. The heuristics for individual repairs are first proposed to handle constraint violations in NSGA-II, based on which several properties are derived. Additionally, the Z-score based Euclidean distance is adopted to estimate the difference between solutions. Finally, the above methods are evaluated and the experiments show several results. 1) The developed heuristics for constraint handling are better than the Existing Strategy in terms of the capacity and coverage values. 2) The Z-score operation obtains better diversity values and reduces repeated solutions. 3) The modified NSGA-II for OTRAPs (called NSGA-II-TRA) performs significantly better than the existing MOEAs in terms of capacity and coverage values, which suggests that NSGA-II-TRA could obtain more and higher quality testing-time-allocation schemes, especially for large, complex datasets. 4) NSGA-II-TRA is robust according to the sensitivity analysis results. [ABSTRACT FROM PUBLISHER]

Testing is a pivotal activity in ensuring the quality of software. Code coverage is a common metric used as a yardstick to measure the efficacy and adequacy of testing. However, does higher coverage actually lead to a decline in postrelease bugs? Do files that have higher test coverage actually have fewer bug reports? The direct relationship between code coverage and actual bug reports has not yet been analyzed via a comprehensive empirical study on real bugs. Past studies only involve a few software systems or artificially injected bugs (mutants). In this empirical study, we examine these questions in the context of open-source software projects based on their actual reported bugs. We analyze 100 large open-source Java projects and measure the code coverage of the test cases that come along with these projects. We collect real bugs logged in the issue tracking system after the release of the software and analyze the correlations between code coverage and these bugs. We also collect other metrics such as cyclomatic complexity and lines of code, which are used to normalize the number of bugs and coverage to correlate with other metrics as well as use these metrics in regression analysis. Our results show that coverage has an insignificant correlation with the number of bugs that are found after the release of the software at the project level, and no such correlation at the file level. [ABSTRACT FROM PUBLISHER]

Demand for highly reliable software is increasing day by day which in turn has increased the pressure on the software firms to provide reliable software in no time. Ensuring high reliability of the software can only be done by prolonged testing which in result consumes more resources which is not feasible in the existing market situation. To overcome this, software firms are providing patches after software release so as to fix the remaining number of bugs and to give better product experience to users. An update/fix is a minor portion of software to repair the bugs. With such patches, organizations enhance the performance of the software. Delivering patches after release demands extra effort and resources which are costly and hence not economical for the firms. Also, early patch release might cause improper fixation of bugs, on the other hand, delayed release may increase the chances of more failure during the operational phase. Therefore, determining optimal patch release time is imperative. To overcome the above issues we have formulated a two-dimensional time and effort-based cost model to regulate the optimum release and patch time of software, so that the total cost is minimized. Proposed model is validated on a real life data set. [ABSTRACT FROM AUTHOR]

Software industry has reached far beyond since its origin. Technical advancements are taking place at a speed faster than ever. This has further enhanced the pressure on software developers. They are trying hard to keep a pace with these rapid developments by coming out with strategies to increase the pace of their work without significantly affecting the software quality and reliability. One such strategy is distributed development of software in which the software is composed of two different types of components viz. newly developed and re-used component. Software developed in Distributed Development Environment (DDE) is characterized by enhanced availability and reliability. At the same time, the ever growing contention in the market and increasing needs of customers obligate the developers to come out with enhanced functionalities in the software from time to time leading to multiple releases of a software. The added functionalities further enhance the existing complexity of the software and testing team may not be able to perfectly remove the faults leading to imperfect debugging or add more bugs due to lack of knowledge about the software in the initial phase known as error generation. In this paper, we incorporated this real-life phenomenon to come out with a multi up-gradations modeling for removal of software fault in a distributed environment. In the present framework we described a cost model to obtain optimal release time in multi up-gradation of software under distributed environment. To validate the analytical results of the proposed framework, numerical illustration is provided. [ABSTRACT FROM AUTHOR]

When to stop testing is an important aspect in software test management. This paper discusses a new approach to this problem. There are two major improvements in this approach in comparison to the earlier studies. It uses failure size proportional model to estimate the number of failures and the corresponding cost of repair. Instead of reliability it uses a new measure called attained failure size. Failure size is the probability of finding a failure causing input in the input domain. Attained failure size is the failure size at the termination of testing. The advantages of these improvements are also discussed. [ABSTRACT FROM AUTHOR]

Recently, concolic execution has become a hotspot in the domain of software testing and program analysis. However, a practical challenge, called path divergence, impairs the soundness and completeness of concolic execution. A path divergence indicates the tested program runs an unpredicted path. In this work, we carry out a comprehensive empirical study on path divergences using an open-source concolic execution tool, named CREST. To make the investigation representative, we select 120 test units randomly from 21 different open-source programs. The results are interesting, and will provide insight to solve the challenging path-divergence problem. First, about one-half of test units suffer frompath divergences, indicating path divergences are so prevalent that the issue isworthy of great attention. Second, quite a number of generated test inputs drive test units to take divergent paths. This means testers need considerable effort to eliminate the misleading test inputs before aggregating them to a test suite. Third, we dig out ten divergent patterns through manual analysis of each path divergence. Among them, the threemost prevalent ones, which are exceptions, external calls, and type casts, lead to almost 82%of path divergences. Finally, we discuss several countermeasures to overcome path divergences. [ABSTRACT FROM AUTHOR]

Operational Profile is found to be an important technique for critical fault removal . Software Operational Profile (SOP) plays a vital role in reliability estimation, quality valuation, performance analysis and test case allocation of software. However, some problems are detected by software experts with the development of SOP as data resources are very limited and large effort is required to convert gathered data into point estimate. Also uncertainty is involved with SOP as it often changes during operational phase. Software operational profile plays a crucial role in improving the software quality. Therefore, in this paper, software test case allocation using fuzzy logic based on operational profile based is proposed and it takes usage data in terms of linguistic variable. The model is validated with the existing literature. Validation result is satisfactory. [ABSTRACT FROM AUTHOR]

Software testing began as an empirical activity, and remains part of engineering practice without a widely accepted theoretical foundation. The overwhelming majority of test methods are designed to find software errors, termed faults, in program source code, but not to assess software operational quality. To go beyond fault-seeking requires a theory that relates static program properties to executions. In the 1970s and 1980s, Gerhart, Howden, and others developed a sound functional theory of program testing. Then Duran and others used this theory to precisely define the notions of random testing and operational reliability. In the Gerhart-Howden-Duran theory, a program's behavior is a pure input-output mapping. This paper extends the theory to include persistent state, by adding a state space to the input space, and a state mapping to a program's output mapping. The extended theory is significantly different because test states, unlike inputs, cannot be chosen arbitrarily. The theory is used to analyze state-based testing methods, to examine the practicality of reliability assessment, and to suggest experiments that would increase understanding of the statistical properties of software. [ABSTRACT FROM AUTHOR]

Standard software testing might not catch important defects, and formal methods can be difficult to use. But, there's a middle ground between the two. This middle ground involves adding five steps to standard testing and employing test randomization, model-based testing, and a more aggressive use of assertions (also called self-tests). [ABSTRACT FROM PUBLISHER]

The assurance of software reliability partially depends on testing. Numbers of approaches for software testing are available with their proclaimed advantages and limitations, but accessibility of any one of them is a subject dependent. Time is a critical factor in deciding cost of any project. A deep insight has shown that executing test cases are time consuming and tedious activity. Thus stress has been given to develop algorithms which can suggest better pathways for testing. One such algorithm called Path Prioritization -Ant Colony Optimization (PP-ACO) has been suggested in this paper which is inspired by real Ant's foraging behavior to generate optimal paths sequence of a decision to decision (DD) path of a graph. The algorithm does full path coverage and suggests the best optimal sequences of path in path testing and prioritizes them according to path strength. [ABSTRACT FROM AUTHOR]

Testing is a key activity of software development and maintenance that determines the level of reliability. Traceability is the ability to describe and follow the life of software artifacts, and has been promoted as a means for supporting various activities, most importantly testing. Traceability information facilitates the testing and debugging of complex software by modeling the dependencies between code and tests. Actively supplementing traceability to testing enables rectifying defects more reliably and efficiently. Despite its importance, the development of test-to-code traceability has not been sufficiently addressed in the literature, and even worse there is currently no organized review of traceability studies in this field. In this work, we have investigated the main conferences, workshops, and journals of the requirements engineering, testing, and reliability, and identified those contributions that refer to traceability topics. From that starting point, we characterized and analyzed the chosen contributions against three research questions by utilizing a comparative framework including nine criteria. As a result, our study arrives to some interesting points, and outlines a number of potential research directions. This, in turn, can pave the way for facilitating and empowering traceability research in this domain to assist software engineers and testers in test management. [ABSTRACT FROM AUTHOR]

Software testing involves verification and validation of the software to meet the requirements elucidated by customers in the earlier phases and to subsequently increase software reliability. Around half of the resources, such as manpower and CPU time are consumed and a major portion of the total cost of developing the software is incurred in testing phase, making it the most crucial and time-consuming phase of a software development lifecycle (SDLC). Also the fault detection process (FDP) and fault correction process (FCP) are the important processes in SDLC. A number of software reliability growth models (SRGM) have been proposed in the last four decades to capture the time lag between detected and corrected faults. But most of the models are discussed under static environment. The purpose of this paper is to allocate the resources in an optimal manner to minimize the cost during testing phase using FDP and FCP under dynamic environment. An elaborate optimization policy based on optimal control theory for resource allocation with the objective to minimize the cost is proposed. Further, genetic algorithm is applied to obtain the optimum value of detection and correction efforts which minimizes the cost. Numerical example is given in support of the above theoretical result. The experimental results help the project manager to identify the contribution of model parameters and their weight. [ABSTRACT FROM AUTHOR]