Search

Apart from the widespread acceptance of the digital library as a major research response among university lecturers and students, little is known about the reliability of its RFID software (RFIDS), the challenges posed during software flaws, and the improvement strategies that can be used to maintain the RFID software reliability. The various NHPP software reliability growth models (SRGMs) to investigate the reliabilities of software products of this kind in real-world software development has not always been achieved because those studies were primarily concerned with modeling fault detection processes (FDP) and ignoring the fault correction processes (FCP). Here, we propose a reliability model of RFID software while keeping in mind the interdependence between fault quantities of the dual procedures of fault detection and fault correction. In evaluating the software reliability, fault introduction and testing coverage rates were eventually included to the parameters making our proposed model to be highly effective in estimating and forecasting the dependability of the software when compared with existing NHPP software reliability growth models (SRGMs). [ABSTRACT FROM AUTHOR]

When it comes to software development, precise planning, proper documentation and proper process control, some errors are inevitable in the software environment. These software flaws can lead to quality deterioration, which can be the main reason behind system failure. As the whole world especially developing countries is dependent upon software systems, it is very important to focus on its reliability aspect. Nowadays sophisticated systems require concerted efforts for managing and reducing the shortcomings in software engineering. But, these efforts require more cost, more money and more time. Software error prediction is the most helpful step in the testing stage of the software development life cycle. It identifies components or parts of the code where an error may occur and requires broad testing, so the test resources can be efficiently used. Software error assessment reduces efforts of testing the software by helping the software testers locate the actual problem and classify different classes of errors in the system. Error estimators are majorly used in various organizations to evaluate the software to save time, improve the quality of software and testing and optimize resources to meet timelines. Machine learning provides support in fault projection by collecting the training data from various edge devices and thus helps in escalating the reliability of the software available on Kaggle. The multilayer perceptron shows better results in precision, recall, F1 score and accuracy as compared to decision tree and Gaussian Naive Bayes as it achieves an accuracy of 96.8%. [ABSTRACT FROM AUTHOR]

In order to construct an evaluation index system suitable for tourism management classroom teaching, this article evaluates the teaching effectiveness of teachers and improves the teaching quality of tourism management courses. This article is based on developmental evaluation theory, using Analytic Hierarchy Process, Project Response Theory, and CIPP model to construct an indicator system suitable for tourism management classroom teaching. Then, based on the collected data of 5763 students, the reliability and effectiveness of the tool and indicator system were first verified. Then, the variable of teacher teaching style was introduced to construct an OLS regression model for empirical research. The research will summarize teacher and student data collected through the platform and conduct reliability analysis in SPSS 22.0 software, using Cronbach α The credibility of coefficient testing and evaluation tools. Cronbach in Environmental Fundamentals α The cβoefficient value is 0.8350. Cronbach for resource allocation α The coefficient is 0.735, and the Cronbah of the implementation process α Cronb Bach with a coefficient of 0.7 47 for teaching performance α The coefficient is 0.7240, indicating that rat ings has high reliability. Research has found that among the four specific types, the holistic type has the greatest impact on the specific situation, the holistic type has the greatest impact on the environmental foundation and resource allocation, and the legislative type has the greatest impact on the implementation process and teaching performance. [ABSTRACT FROM AUTHOR]

The efficiency and performance of a software application largely depend on the testing strategy adopted by the firm. Apart from the tools, techniques, and skills used for testing, the duration also plays an important influence in establishing software reliability. This defines the operational performance of the software. The testing duration decision is dependent on the failure behavior depicted by the software during the process of testing and the cost spent at various phases of development. In this paper, we study the multi-release model whose fault removal process is affected by random irregular fluctuations (white noise) and error generation phenomenon and determine the optimal time of testing for a multi-release-based software. The fault removal process is governed by the phenomenon of testing coverage which is affected by random fluctuations. The model shows encouraging results as it handles the stochastic property of the fault detection process. The optimal testing time is determined with the goal to minimize the expected development cost related to software while achieving the desired reliability levels of software for that release using a Genetic Algorithm. A real-life four-release fault dataset of tandem computers has been used to numerically demonstrate the methodology. It is observed through sensitivity analysis that the presence of white noise directly affects the cost and optimal testing duration. The potential to improve sensitivity, flexibility, early detection, discovering unsuspected patterns, and boost fault diagnosis is enhanced by collecting irregular fluctuations in the fault detection rate. This technology, by going beyond existing methodologies, has distinct advantages for detecting faults and can contribute to more dependable and efficient systems in a variety of domains. [ABSTRACT FROM AUTHOR]

An essential attribute of software quality is software reliability. To achieve higher reliability, the testing phase with detected and corrected flaws is incorporated in the software development. The fault correction process (FCP) includes the fault detection process (FDP) to develop the software reliability growth model (SRGM). This is difficult to integrate because due to several reasons, including the effects of staffing levels and the interdependence of faults. It limits the applicability of the analytical model. Because of the adoption of data-driven methodologies such as Artificial Intelligence (AI) technology, no precise FCP and FDP assumptions are necessary. In this article, we proposed a hybrid long short-term memory (LSTM) with BrainStorm Optimization and Late Acceptance Hill Climbing (BSO-LAHC) algorithm of a stepwise prediction model for software fault detection and correction. The fault detection and correction procedure has great influence by considering the testing effort. While compared to the existing methods, the proposed hybrid with the BSO-LAHC algorithm demonstrated superior results by using Firefox and bug tracking system Bugzilla datasets. The proposed model's effectiveness is confirmed via empirical study. Based on the Bugzilla and firefox datasets, the proposed mean square error performance is 1.92 and 21.44 respectively. Additionally, the proposed method is less expensive and takes less time to execute. In Bugzilla version 5.0.4, releases 2 and 3 had a determination coefficient of 99.2% and 98.9%, respectively. The FCP is 27% more effective than previous approaches, and the FDP is 32% more effective. [ABSTRACT FROM AUTHOR]

One of the significant features of software quality is software reliability. In the testing phase, faults are identified and corrected by integrating them into software development, thus obtaining better reliability. Here, by utilizing the Elliptical Distributions-centric Emperor Penguins Colony Algorithm (ED-EPCA)-based Test Case Prioritization (TCP), an effectual Fault Detection (FD) technique is proposed using Fishers Yates Shuffled Shepherd Optimization Algorithm (FY-SSOA)-based Test Case Selection (TCS). Initially, for the incoming source code, the Test Case (TC) is created. Then, the significant factors needed for TCS and prioritization are identified. Next, by utilizing the Log Scaling-centered Generalized Discriminant Analysis (LS-GDA) model, the estimated factors are abated further to enhance the TCS along with prioritization for the Fault Detection Process (FDP). Then, using the FY-SSOA, the optimized TCs are selected. Subsequently, with the help of ED-EPCA, the TCs being selected are ranked as well as prioritized. Finally, to validate the proposed system's effectiveness, the model's performance is evaluated in the working platform of Java and analogized with the traditional methodologies. The results indicate that the test case prioritization-based fault detection method is robust with a 99.23% fault detection rate and a small amount of memory usage, which is only 8245475 kb by generating a large number of test cases. [ABSTRACT FROM AUTHOR]

Software reliability is one of the standard critical inherent characteristics of software systems. The testing coverage function (TCF) is a significant parameter for identifying the completeness and effectiveness of software testing. It is defined as the proportion of the code that has been tested up to time t. To capture the dynamic behavior of the number of faults detected over a period of time, several distributions, namely S‐shaped, inflection S‐shaped, logistic, log‐logistic, Weibull, Rayleigh, Erlang, and logarithmic exponentiated, have been used as TCF in literature. However, these distributions are not sufficient to describe TCF's practical behavior due to complexity and vagueness in the collected data. This study proposes two software reliability growth models (SRGMs), which incorporate the generalized inflection S‐shaped (GISS) distribution as TCF. The models have been developed in perfect and imperfect debugging environments while considering fault removal efficiency, error generation, and uncertainty in the operating environment. To analyze the effectiveness, the proposed models are then tested with six failure data sets. The choice of GISS distribution as a TCF improves the software reliability estimation in comparison with the existing models in the literature. Finally, single and multiple parameters sensitivity analysis also has been done and based on it, the critical parameters have been detected. The proposed models may be helpful for the system analyst to predict various parameters about some software systems. [ABSTRACT FROM AUTHOR]

Effort‐Aware Defect Prediction (EADP) methods sort software modules based on the defect density and guide the testing team to inspect the modules with high defect density first. Previous studies indicated that some feature selection methods could improve the performance of Classification‐Based Defect Prediction (CBDP) models, and the Correlation‐based feature subset selection method with the Best First strategy (CorBF) performed the best. However, the practical benefits of feature selection methods on EADP performance are still unknown, and blindly employing the best‐performing CorBF method in CBDP to pre‐process the defect datasets may not improve the performance of EADP models but possibly result in performance degradation. To assess the impact of the feature selection techniques on EADP, a total of 24 feature selection methods with 10 classifiers embedded in a state‐of‐the‐art EADP model (CBS+) on the 41 PROMISE defect datasets were examined. We employ six evaluation metrics to assess the performance of EADP models comprehensively. The results show that (1) The impact of the feature selection methods varies in classifiers and datasets. (2) The four wrapper‐based feature subset selection methods with forwards search, that is, AdaBoost with Forwards Search, Deep Forest with Forwards Search, Random Forest with Forwards Search, and XGBoost with Forwards Search (XGBF) are better than other methods across the studied classifiers and the used datasets. And XGBF with XGBoost as the embedded classifier in CBS+ performs the best on the datasets. (3) The best‐performing CorBF method in CBDP does not perform well on the EADP task. (4) The selected features vary with different feature selection methods and different datasets, and the features noc (number of children), ic (inheritance coupling), cbo (coupling between object classes), and cbm (coupling between methods) are frequently selected by the four wrapper‐based feature subset selection methods with forwards search. (5) Using AdaBoost, deep forest, random forest, and XGBoost as the base classifiers embedded in CBS+ can achieve the best performance. In summary, we recommend the software testing team should employ XGBF with XGBoost as the embedded classifier in CBS+ to enhance the EADP performance. [ABSTRACT FROM AUTHOR]

To enter the production stage, in DevOps practices candidate software releases have to pass quality gates, where they are assessed to meet established target values for key indicators of interest. We believe software reliability should be an important such indicator, as it greatly contributes to the end‐user satisfaction. We propose DevOpRET, an approach for reliability testing as part of the acceptance testing stage in DevOps. DevOpRET relies on operational‐profile–based testing, a common reliability assessment technique. DevOpRET leverages usage and failure data monitored in operations to continuously refine its estimate. We evaluate accuracy and efficiency of DevOpRET through controlled experiments with a real‐world open source platform and with a microservice architectures benchmark. The results show that DevOpRET provides accurate and efficient estimates of the true reliability over subsequent DevOps cycles. [ABSTRACT FROM AUTHOR]

Software testing and debugging is a crucial part of the software development process since defective software not only incurs customer dissatisfaction but also might incur legal issues. However, the managers of a software development company cannot arbitrarily prolong their software debugging period due to their software testing budget and opportunity in the market. Accordingly, in order to propose an advantageous testing project, the managers should be aware of the influence of the testing project on cost, quality, and time to make the best decision. In this study, a new software reliability growth model (SRGM) with consideration of the testing staff's learning effect is proposed to achieve better prediction. The methods of estimating the model's parameters and the symmetric confidence intervals are also proposed in the study. Moreover, in the past, most of the SRGMs focused on a single software system. However, in practice, some software systems were developed using modular-based system engineering approaches. Therefore, traditional software testing work can be changed to multiple modular testing work in this scenario. Therefore, the manager can use this to dispatch multiple staff groups to perform the individual testing work simultaneously. The study proposes two mathematical programming models to handle the scheduling of modular testing work. Additionally, the design of a computerized decision support system is also proposed in the study for the application in practice. [ABSTRACT FROM AUTHOR]

Regression testing is essential for continuous integration and continuous development. It is needed to ensure that the modifications have not produced any errors or faults, thereby maintaining the quality and reliability of the software. The testers usually avoid exhaustive retesting because it requires lots of effort and time. The test case prioritization and minimization solve the issue by scheduling the critical test cases and removing redundant ones. Optimization techniques help by improving the efficiency of these techniques while utilizing limited resources. This paper proposed an enhanced discrete novel bat algorithm for the test case prioritization. The algorithm is modified in two ways. First, we have proposed a fix-up mechanism for the discrete combinatorial problem, which conducts the perturbation in the population using the asexual reproduction algorithm. Second, the novel bat algorithm is improved, where the bats hunt in different habitats with quantum behavior using Gaussian distribution and search in the limited habitat with Doppler effect. In addition, we have embedded the test case minimization procedure in the algorithm for redundancy reduction. The experimental results are empirically analyzed using different testing criteria, i.e., fault and statement coverage on three subject programs from the software infrastructure repository. Consequently, test selection percentage, coverage loss, fault detection loss, and cost reduction percentages are deduced for the test case minimization at program and version levels. Empirical results and statistical comparisons with the random search, bat algorithm, novel bat algorithm, birds swarm algorithm, whale optimization algorithm, and genetic algorithm show the outperformance of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Testing of software is an essential aspect of software development to determine the testing release time. remaining number of faults and software reliability. Testing process includes selection of testing team, estimation of testing effort. and execution of software. Study reveals that maximum cost of software development is utilized in testing of software. Therefore, estimation of testing effort during testing of software is needed to determine. The uncertainty involved in estimated parameters and reliability is necessary to address, which can be done in fuzzy paradigm. Due to complexity of software systems, it is very tough to remove all faults and results into imperfect debugging. Sometimes. the new faults may generate in testing a software. this process is called error generation. In this study, we proposed a software reliability growth model using testing effort function in the presence of error generation and imperfect debugging under fuzzy paradigm. The proposed model segregated the process into two phases considering fault observation and fault removal. Moreover. software release time is discussed with the consideration of uncertainty in reliability. Finally. results are validated numerically on real data set. [ABSTRACT FROM AUTHOR]

Various Software Reliability Growth Models (SRGMs) have been developed over last five decades to estimate the failure rate, software reliability and remaining number of faults in the software during testing phase of software development life cycle. There are numerous SRGMs available in the research literature and in the presence of numerous SRGM, appropriate selection is a staggering task for the software testing professionals. In the software reliability area, researchers have always been interested to select the optimal SRGM to use in a particular situation. In this study, we have developed an integrated technique based on Criteria Importance Through Inter Criteria Correlation (CRITIC) and Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS) to select the optimal SRGM. The weights of criteria are obtained by the CRITIC method and SRGMs are ranked by order using TOPSIS method. The working of proposed integrated CRITIC‐TOPSIS approach is illustrated on the two real‐time failure data sets. Furthermore, the methodology validation idea supports the current study by comparing the results obtained with the previous multi‐criteria decision making (MCDM) methodologies namely Additive Ratio Assessment (ARAS). The study revealed that the ranking of SRGMs can be solved effectively using the integration of CRITIC and TOPSIS. The result of this study plays a significant role to take decisions and to evaluate the feasibility of SRGM. [ABSTRACT FROM AUTHOR]

In the software development life cycle, the quality and reliability of software are critical to software developers. Poor quality and reliability not only cause the loss of customers and sales but also increase the operational risk due to unreliable codes. Therefore, software developers should try their best to reduce such potential software defects by undertaking a software testing project. However, to pursue perfect and faultless software is unrealistic since the budget, time, and testing resources are limited, and the software developers need to reach a compromise that balances software reliability and the testing cost. Using the model presented in this study, software developers can devise multiple alternatives for a software testing project, and each alternative has its distinct allocation of human resources. The best alternative can therefore be selected. Furthermore, the allocation incorporates debuggers' learning and negligent factors, both of which influence the efficiency of software testing in practice. Accordingly, the study considers both human factors and the nature of errors during the debugging process to develop a software reliability growth model to estimate the related costs and the reliability indicator. Additionally, the issue of error classification is also extended by considering the impacts of errors on the system, and the expected time required to remove simple or complex errors can be estimated based on different truncated exponential distributions. Finally, numerical examples are presented and sensitivity analyses are performed to provide managerial insights and useful directions to inform software release strategies. [ABSTRACT FROM AUTHOR]

In this study, an imperfect debugging software reliability growth model (SRGM) with Bayesian analysis was proposed to determine an optimal software release in order to minimize software testing costs and also enhance the practicability. Generally, it is not easy to estimate the model parameters by applying MLE (maximum likelihood estimation) or LSE (least squares estimation) with insufficient historical data. Therefore, in the situation of insufficient data, the proposed Bayesian method can adopt domain experts' prior judgments and utilize few software testing data to forecast the reliability and the cost to proceed with the prior analysis and the posterior analysis. Moreover, the debugging efficiency involves testing staff's learning and negligent factors, and therefore, the human factors and the nature of debugging process are taken into consideration in developing the fundamental model. Based on this, the estimation of the model's parameters would be more intuitive and can be easily evaluated by domain experts, which is the major advantage for extending the related applications in practice. Finally, numerical examples and sensitivity analyses are performed to provide managerial insights and useful directions for software release strategies. [ABSTRACT FROM AUTHOR]

Open source software (OSS) has become one of the modern software development methods. OSS is mainly developed by developers, volunteers, and users all over the world, but its reliability has been widely questioned. When OSS faults are detected, volunteers or users send them to developers by email or network. After the developer confirms the fault, it will be randomly assigned to the debugger who may be a developer, a volunteer, or a user. These open source community contributors also have the phenomenon of learning when removing faults. When the detected faults are removed, the number of introduced faults decreases gradually. Therefore, this study proposes a software reliability model with the decreasing trend of fault introduction in the process of OSS development and testing. The validity of the proposed model and the accuracy of estimating residual faults are verified by experiments. The proposed model can be used to evaluate the reliability and predict the remaining faults in the actual OSS development and testing process. [ABSTRACT FROM AUTHOR]

Fault-counting data are collected in the testing process of software development. However, the data are not used for evaluating the efficiency of fault correction activities because the information on the fault detection and correction times of each fault are not recorded in the fault-counting data. Furthermore, it is difficult to collect new data on the detection time of each fault to realize efficiency evaluation for fault correction activities from the collected fault-counting data due to the cost of personnel and data collection. In this paper, we apply the thinning method, using intensity functions of the delayed S-shaped and inflection S-shaped software reliability growth models (SRGMs) to generate sample data of the fault detection time from the fault-counting data. Additionally, we perform simulations based on the infinite server queuing model, using the generated sample data of the fault detection time to visualize the efficiency of fault correction activities. [ABSTRACT FROM AUTHOR]

An optimal software release strategy is a well-investigated issue in software reliability literature. Comprehensive testing is expected before releasing the software into the market to enhance the reliability and security of the software device. In recent years, few analysts have recommended the scheme for software projects that support releasing the software early in the market and continue the testing process for an added period in the field environment even after the software is distributed. These studies are based on one common assumption that the efficiency of the software engineers in detecting the faults occurs at a consistent rate throughout the testing phase. However, bug-identification rate may experience discontinuity at the software release time. In software engineering, the time-point at which fault detection rate changes is termed as change-point. Consequently, an alternative software release policy is proposed in the present paper, which offers a generalized framework for fault detection phenomenon using the unified approach. An extensive analysis of software time-to-market and testing duration based on cost-efficiency and reliability measures is discussed by considering the change in tester's fault detection rate. A multi-criteria decision making technique known as multi-attribute utility theory is applied to optimize the software release policy under field-testing (FT) and no field-testing (NFT) frameworks. The relevance of the optimization problem is illustrated using a numerical example, comprising both the exponential and S-shaped bug-detection process. [ABSTRACT FROM AUTHOR]

Mutation testing is basically a fault-based software testing technique, which has been proposed to measure the fault detection effectiveness of a test suite using programs with simulated faults (namely mutants). However, mutation testing is time consuming and computationally expensive because of the normal use of a large amount of mutants. Thus, reducing the mutants is of great significance. To address this problem, various mutant reduction techniques have been proposed. Among them, the identification of redundant mutants aims at removing mutants whose test results can be inferred by other mutants. This article proposes a redundant mutant identification technique based on selective symbolic execution called ReMuSSE for weak mutation testing. Redundant mutants could be revealed by identifying those with similar program execution state changes within a program block involving mutated statements. An empirical study was conducted using 13 C programs from different application domains with varying sizes. The empirical results showed that ReMuSSE could identify up to 31.4% redundant mutants and consequentially save up to 35.2% time cost of weak mutation testing. The results demonstrated that ReMuSSE could effectively identify redundant mutants and thus could significantly improve the efficiency of weak mutation testing. [ABSTRACT FROM AUTHOR]

The process of software testing usually involves the correction of a detected bug immediately upon detection. In this article, in contrast, we discuss continuous time testing of a software with periodic debugging in which bugs are corrected, instead of at the instants of their detection, at some pre-specified time points. Under the assumption of renewal distribution for the time between successive occurrence of a bug, maximum-likelihood estimation of the initial number of bugs in the software is considered, when the renewal distribution belongs to any general parametric family or is arbitrary. The asymptotic properties of the estimated model parameters are also discussed. Finally, we investigate the finite sample properties of the estimators, specially that of the number of initial number of bugs, through simulation. [ABSTRACT FROM AUTHOR]

Software reliability literature consists of various change-point-based software reliability growth models and related release time problems. The primary assumption of the existing models is the existence of change-point before software release time only. This does not look practical as the testing team becomes more proficient in detecting the faults due to their continuous involvement in software development by the software release time. Hence the fault detection rate in the pre- and post release phase is not the same. To capture this change in fault detection rate in the pre- and post release testing phase, we propose a new software reliability modeling framework by considering two change-points during the software lifecycle; that is, there exists a change-point before release time and release time as a change-point. Further, in the last one-decade software firms have changed their strategy of stop testing the software after release and continue to test even after release to remove the number of faults to provide better user experiences. This phenomenon attracted academicians to develop theoretical as well empirical study on post release testing and formulation of related release time problem. In this paper, we propose a software cost model to determine optimal release and testing stop time considering under the assumption of two change-points as mentioned above. The proposed model is validated on real-life data set. [ABSTRACT FROM AUTHOR]

The thesis starts with the connotation and attributes of software testing quality, introduces software testing quality evaluation methods, and analyzes and discusses software testing quality evaluation models based on fuzzy mathematics theory. Focusing on the key technical problems of software testing quality, discuss the key technologies to solve the software testing quality evaluation model establishment. Through the use of fuzzy models, the cost of software testing quality evaluation is effectively reduced, and the reliability of software testing quality evaluation methods is improved. This model can quickly evaluate the quality of software testing, can avoid the occurrence of local maxima, overcome the shortcomings of existing evaluation models and tools, and can correctly reflect the relationship between the internal and external properties of the software. Using the new software testing quality evaluation method, comparing the evaluation models and tools used before, summarizing the methods of software testing quality improvement. The application of these methods effectively improves the software testing quality. [ABSTRACT FROM AUTHOR]

Software failures caused by data race bugs have always been major concerns in parallel and distributed systems, despite significant efforts spent in software testing. Due to their nondeterministic and hard-to-reproduce features, when evaluating systems’ operational reliability, a rather long period of experimental execution time is expected to be spent on observing failures caused by data race conditions. To address this problem, in this paper, we make two contributions. First, this paper proposes stress testing with influencing factors, in which the system runs under certain workloads for a long time with controlled stress conditions to accelerate the occurrence of data race failures. Second, it explores and formulates mathematical relationship models between data races’ statistical characteristics of time to failure (TTF) or mean TTF (MTTF) and the influencing factors. Such relationship models are used for TTF/MTTF extrapolation under different operational conditions and are essential to reduce systems’ reliability evaluation time. The proposed method is empirically evaluated on six applications suffering from failures caused by real-world data race bugs. Through analysis of the experimental results, we obtain several important findings: First, the reduction in the manifestation time to data race failures achieved by controlling the influencing factors is statistically significant. Second, Power model is the best-fitting model of the relationship between the MTTF and the influencing factors. Third, Power Weibull distribution is the best-fitting probability distribution between the TTF and the influencing factors. Finally, the TTF/MTTF can be accurately estimated with the approach proposed in this paper. [ABSTRACT FROM AUTHOR]

Owing to release of software in multiple releases, code changes take place in software. Because of this added complexity in software, the testing team may be unable to correct the fault upon detection, leaving the actual fault to reside in the software, termed as imperfect debugging or there may be replacement of original fault by other fault, leading to error generation. Many other factors exist that affect the testing phase of software like strategies of testing, test cases, skill, efficiency, and learning of testing team. All these factors cannot be kept stable during the whole process of testing. They may change at any time moment causing the background processes to experience change, which is known as change‐point. Keeping all these critical testing environment factors under consideration, a new software reliability growth model has been proposed, which is derived from an non homogenous Poisson process (NHPP)based unified scheme for multi‐release two‐stage fault detection/observation and correction/removal software reliability models. The developed model is numerically illustrated on tandem data set for four releases. [ABSTRACT FROM AUTHOR]

Generation of complex heap structures is required for many software testing and verification techniques, which are used to enhance software reliability. It enables them to work for programs dependent on such structures. Generating all the valid structures within a size bound is time-consuming, and it is hard to predict an input size completely generatable within a time bound. This is due to the depth-first traversal of the candidate space. However, an efficient breadth-first traversal requires a compact state encoding to store candidates for exploration in the next level. In this paper, we present present a novel state encoding technique that stores an incomplete representation of state that can be recovered during search. We build upon the Korat algorithm—demonstrated to effectively generate tests and find bugs for many software programs—and present iKorat, an incremental algorithm for breadth-first exploration of the search space. Our encoding enables a more efficient implementation of iterative deepening that avoids redundant work. Standard iterative deepening algorithms allow a breadth-first search when the underlying algorithm is depth-first by repeating the work of earlier iterations. iKorat, however, uses information from smaller sizes to avoid redundant work for larger sizes. It also enables a new technique for parallelizing Korat by communicating candidates using our encoding. piKorat generates structures of larger sizes in parallel as soon as incremental information is available from smaller sizes. Our evaluation shows that iKorat-based iterative deepening is more efficient than Korat and that piKorat naturally extends the technique for parallel generation of complex structures. [ABSTRACT FROM AUTHOR]

Over the last few decades, various software reliability growth models (SRGM) have been proposed, and in recent years, a gradual but marked shift has led to a focus on the balance between acceptable software reliability and affordable software testing costs. Chiu et al.(2008) proposed an SRGM from the perspective of learning effects that is more flexible in terms of fitting various software error data although it has a restrictive assumption of a constant number of potential errors. In this paper, we consider a software reliability growth model in which the number of potential errors varies over the debugging period since wrongly fixing an error may cause more errors, while correctly debugging one error may resolve others. However, such variations will gradually converge as the testing staff becomes more familiar with the software system. In order to describe this phenomenon, a sine function is introduced in this study to describe the time-dependent behavior of the number of potential errors with imperfect debugging, and the expected testing cost is thus evaluated to assist in the determination of an optimal software release policy. A numerical example is illustrated to verify the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

Many security and software testing applications require checking whether certain properties of a program hold for any possible usage scenario. For instance, a tool for identifying software vulnerabilities may need to rule out the existence of any backdoor to bypass a program's authentication. One approach would be to test the program using different, possibly random inputs. As the backdoor may only be hit for very specific program workloads, automated exploration of the space of possible inputs is of the essence. Symbolic execution provides an elegant solution to the problem, by systematically exploring many possible execution paths at the same time without necessarily requiring concrete inputs. Rather than taking on fully specified input values, the technique abstractly represents them as symbols, resorting to constraint solvers to construct actual instances that would cause property violations. Symbolic execution has been incubated in dozens of tools developed over the past four decades, leading to major practical breakthroughs in a number of prominent software reliability applications. The goal of this survey is to provide an overview of the main ideas, challenges, and solutions developed in the area, distilling them for a broad audience. [ABSTRACT FROM AUTHOR]

This is an extensive tutorial introducing the state of art in Software Reliability Engineering and associated metrics. Software Reliability Engineering (SRE) encompasses many tasks such as Reliability Growth Testing (RGT) based on software reliability growth modeling, assessment of operational reliability, software release time determination and determination of estimated testing time or required testing efforts and resources. The foremost activity in SRE is Software Reliability growth modeling to plan and execute RGT which is carried out as final testing activity in the software development life cycle (SDLC) and is known in different names such as system testing. This phase in SDLC requires extensive use of Computational Mathematics to estimate parameters of the software reliability growth model (SRGM). In this paper, we will briefly discuss the various SRGM parameter estimation methodologies and conclude that the Least Squares methodology provides a satisfactory solution in all situations. [ABSTRACT FROM AUTHOR]

Testing is a pivotal activity in ensuring the quality of software. Code coverage is a common metric used as a yardstick to measure the efficacy and adequacy of testing. However, does higher coverage actually lead to a decline in postrelease bugs? Do files that have higher test coverage actually have fewer bug reports? The direct relationship between code coverage and actual bug reports has not yet been analyzed via a comprehensive empirical study on real bugs. Past studies only involve a few software systems or artificially injected bugs (mutants). In this empirical study, we examine these questions in the context of open-source software projects based on their actual reported bugs. We analyze 100 large open-source Java projects and measure the code coverage of the test cases that come along with these projects. We collect real bugs logged in the issue tracking system after the release of the software and analyze the correlations between code coverage and these bugs. We also collect other metrics such as cyclomatic complexity and lines of code, which are used to normalize the number of bugs and coverage to correlate with other metrics as well as use these metrics in regression analysis. Our results show that coverage has an insignificant correlation with the number of bugs that are found after the release of the software at the project level, and no such correlation at the file level. [ABSTRACT FROM PUBLISHER]

In software testing, optimal testing resource allocation problems (OTRAPs) are important when seeking a good tradeoff between reliability, cost, and time with limited resources. There have been intensive studies of OTRAPs using multiobjective evolutionary algorithms (MOEAs), but little attention has been paid to the constraint handling. This paper comprehensively investigates the effect of the constraint handling on the performance of nondominated sorting genetic algorithm II (NSGA-II) for solving OTRAPs, from both theoretical and empirical perspectives. The heuristics for individual repairs are first proposed to handle constraint violations in NSGA-II, based on which several properties are derived. Additionally, the Z-score based Euclidean distance is adopted to estimate the difference between solutions. Finally, the above methods are evaluated and the experiments show several results. 1) The developed heuristics for constraint handling are better than the Existing Strategy in terms of the capacity and coverage values. 2) The Z-score operation obtains better diversity values and reduces repeated solutions. 3) The modified NSGA-II for OTRAPs (called NSGA-II-TRA) performs significantly better than the existing MOEAs in terms of capacity and coverage values, which suggests that NSGA-II-TRA could obtain more and higher quality testing-time-allocation schemes, especially for large, complex datasets. 4) NSGA-II-TRA is robust according to the sensitivity analysis results. [ABSTRACT FROM PUBLISHER]

Software product lines (SPLs) are families of software systems sharing common assets and exhibiting variabilities specific to each product member of the family. Commonalities and variabilities are often represented as features organized in a feature model. Due to combinatorial explosion of the number of products induced by possible features combinations, exhaustive testing of SPLs is intractable. Therefore, sampling and prioritization techniques have been proposed to generate sorted lists of products based on coverage criteria or weights assigned to features. Solely based on the feature model, these techniques do not take into account behavioural usage of such products as a source of prioritization. In this paper, we assess the feasibility of integrating usage models into the testing process to derive statistical testing approaches for SPLs. Usage models are given as Markov chains, enabling prioritization of probable/rare behaviours. We used featured transition systems, compactly modelling variability and behaviour for SPLs, to determine which products are realizing prioritized behaviours. Statistical prioritization can achieve a significant reduction in the state space, and modelling efforts can be rewarded by better automation. In particular, we used MaTeLo, a statistical test cases generation suite developed at ALL4TEC. We assess feasibility criteria on two systems: Claroline, a configurable course management system, and Sferion™, an embedded system providing helicopter landing assistance. [ABSTRACT FROM AUTHOR]

When to stop testing is an important aspect in software test management. This paper discusses a new approach to this problem. There are two major improvements in this approach in comparison to the earlier studies. It uses failure size proportional model to estimate the number of failures and the corresponding cost of repair. Instead of reliability it uses a new measure called attained failure size. Failure size is the probability of finding a failure causing input in the input domain. Attained failure size is the failure size at the termination of testing. The advantages of these improvements are also discussed. [ABSTRACT FROM AUTHOR]

It is essential to regularly test the usability of a learning management system, in order to ensure a fast adoption by new users and rapidly shift the focus from the platform to the content and the learning experience. Quantitative testing yields the most reliable results due to the large number of data points acquired, but lacks the in-depth analysis of the qualitative research from a controlled testing setup. We are proposing in this paper an integrated usability testing tool, which can replace a certain type of laboratory testing, where the users' actions on the real platform are measured and analyzed. We conducted tests with the tool and compared the results with a small scale laboratory test using the same scenarios. The results seem to confirm the proposed tool as a viable alternative to the laboratory test. [ABSTRACT FROM AUTHOR]

Software testing began as an empirical activity, and remains part of engineering practice without a widely accepted theoretical foundation. The overwhelming majority of test methods are designed to find software errors, termed faults, in program source code, but not to assess software operational quality. To go beyond fault-seeking requires a theory that relates static program properties to executions. In the 1970s and 1980s, Gerhart, Howden, and others developed a sound functional theory of program testing. Then Duran and others used this theory to precisely define the notions of random testing and operational reliability. In the Gerhart-Howden-Duran theory, a program's behavior is a pure input-output mapping. This paper extends the theory to include persistent state, by adding a state space to the input space, and a state mapping to a program's output mapping. The extended theory is significantly different because test states, unlike inputs, cannot be chosen arbitrarily. The theory is used to analyze state-based testing methods, to examine the practicality of reliability assessment, and to suggest experiments that would increase understanding of the statistical properties of software. [ABSTRACT FROM AUTHOR]

Standard software testing might not catch important defects, and formal methods can be difficult to use. But, there's a middle ground between the two. This middle ground involves adding five steps to standard testing and employing test randomization, model-based testing, and a more aggressive use of assertions (also called self-tests). [ABSTRACT FROM PUBLISHER]

Testing is a key activity of software development and maintenance that determines the level of reliability. Traceability is the ability to describe and follow the life of software artifacts, and has been promoted as a means for supporting various activities, most importantly testing. Traceability information facilitates the testing and debugging of complex software by modeling the dependencies between code and tests. Actively supplementing traceability to testing enables rectifying defects more reliably and efficiently. Despite its importance, the development of test-to-code traceability has not been sufficiently addressed in the literature, and even worse there is currently no organized review of traceability studies in this field. In this work, we have investigated the main conferences, workshops, and journals of the requirements engineering, testing, and reliability, and identified those contributions that refer to traceability topics. From that starting point, we characterized and analyzed the chosen contributions against three research questions by utilizing a comparative framework including nine criteria. As a result, our study arrives to some interesting points, and outlines a number of potential research directions. This, in turn, can pave the way for facilitating and empowering traceability research in this domain to assist software engineers and testers in test management. [ABSTRACT FROM AUTHOR]

Open source software (OSS) approach has become increasingly prevalent for software development. As the widespread utilization of OSS, the reliability of OSS products becomes an important issue. By simulating the testing and debugging processes of software life cycle, the rate-based queueing simulation model has shown its feasibility for closed source software (CSS) reliability assessment. However, the debugging activities of OSS projects are different in many ways from those of CSS projects and thus the simulation approach needs to be calibrated for OSS projects. In this paper, we first characterize the debugging activities of OSS projects. Based on this, we propose a new rate-based queueing simulation framework for OSS reliability assessment including the model and the procedures. Then a decision model is developed to determine the optimal version-updating time with respect to two objectives: minimizing the time for version update, and maximizing OSS reliability. To illustrate the proposed framework, three real datasets from Apache and GNOME projects are used. The empirical results indicate that our framework is able to effectively approximate the real scenarios. Moreover, the influences of the core contributor staffing levels are analyzed and the optimal version-updating times are obtained. [ABSTRACT FROM PUBLISHER]

This paper presents a general testing coverage software reliability modeling framework that covers imperfect debugging and considers not only fault detection processes (FDP) but also fault correction processes (FCP). Numerous software reliability growth models have evaluated the reliability of software over the last few decades, but most of them attached importance to modeling the fault detection process rather than modeling the fault correction process. Previous studies analyzed the time dependency between the fault detection and correction processes and modeled the fault correction process as a delayed detection process with a random or deterministic time delay. We study the quantitative dependency between dual processes from the viewpoint of fault amount dependency instead of time dependency, then propose a generalized modeling framework along with imperfect debugging and testing coverage. New models are derived by adopting different testing coverage functions. We compared the performance of these proposed models with existing models under the context of two kinds of failure data, one of which only includes observations of faults detected, and the other includes not only fault detection but also fault correction data. Different parameter estimation methods and performance comparison criteria are presented according to the characteristics of different kinds of datasets. No matter what kind of data, the comparison results reveal that the proposed models generally give improved descriptive and predictive performance than existing models. [ABSTRACT FROM AUTHOR]

Abstract: This paper studies the fault detection process (FDP) and fault correction process (FCP) with the incorporation of testing effort function and imperfect debugging. In order to ensure high reliability, it is essential for software to undergo a testing phase, during which faults can be detected and corrected by debuggers. The testing resource allocation during this phase, which is usually depicted by the testing effort function, considerably influences not only the fault detection rate but also the time to correct a detected fault. In addition, testing is usually far from perfect such that new faults may be introduced. In this paper, we first show how to incorporate testing effort function and fault introduction into FDP and then develop FCP as delayed FDP with a correction effort. Various specific paired FDP and FCP models are obtained based on different assumptions of fault introduction and correction effort. An illustrative example is presented. The optimal release policy under different criteria is also discussed. [Copyright &y& Elsevier]

In this paper we discuss the application of a range of techniques to the verification of mission-critical flight software at NASA's Jet Propulsion Laboratory. For this type of application we want to achieve a higher level of confidence than can be achieved through standard software testing. Unfortunately, given the current state of the art, especially when efforts are constrained by the tight deadlines and resource limitations of a flight project, it is not feasible to produce a rigorous formal proof of correctness of even a well-specified stand-alone module such as a file system (much less more tightly coupled or difficult-to-specify modules). This means that we must look for a practical alternative in the area between traditional testing and proof, as we attempt to optimize rigor and coverage. The approaches we describe here are based on testing, model checking, constraint-solving, monitoring, and finite-state machine learning, in addition to static code analysis. The results we have obtained in the domain of file systems are encouraging, and suggest that for more complex properties of programs with complex data structures, it is possibly more beneficial to use constraint solvers to guide and analyze execution (i.e., as in testing, even if performed by a model checking tool) than to translate the program and property into a set of constraints, as in abstraction-based and bounded model checkers. Our experience with non-file-system flight software modules shows that methods even further removed from traditional static formal methods can be assisted by formal approaches, yet readily adopted by test engineers and software developers, even as the key problem shifts from test generation and selection to test evaluation. [ABSTRACT FROM AUTHOR]

We propose a method to determine how to sequentially allocate test cases among partitions of the software to minimize the expected loss incurred by the Bayes estimator of the overall reliability when the total number of software test cases is fixed. In contrast to fixed sampling schemes, where the proportion of test cases taken from each partition is determined before reliability testing begins, we make allocation decisions dynamically throughout the testing process. Using a fully Bayesian approach, we can take advantage of information from previous functional testing and insights from developers. We then refine these estimates in an iterative manner as we sample. We also compare the results from a multistage sampling scheme with the optimal fixed sampling scheme, and demonstrate its superiority in terms of the expected loss incurred when the overall reliability is estimated by its Bayes estimator both theoretically and through Monte Carlo simulations. [ABSTRACT FROM AUTHOR]

The article focuses on the role of traceability in establishing evidence regarding the hazards and risks related to specifications and implementations of safety critical software devices such as pacemakers and train signals. Topics discussed include strategic practices to provide cost-effective support to evaluate a system's safety, traceability standards in safety-critical projects and analysis of traceability project documents by the U.S. Food and Drug Administration for medical devices.

The article offers information related to the DO-178C, software considerations in airborne systems and equipment certification document published by Radio Technical Commission for Aeronautics (RTCA) Inc. in 2011. It discusses the testing-based verification activities under the DO-178C which includes two processes that involves requirements-based and structural coverage analyses.

The article focuses on the role of model-driven approaches in the development of high-quality and cost effective safety-critical software for air traffic control (ATC). Topics discussed include interoperability of the model driven architecture (MDA) development paradigm, integration of MDA into the verification and validation (V-model) and MDA and Model-Driven testing in a V-Model.

The article focuses on the need of statistical model checking (SMC) of safety-critical softwares to validate software reliability at an early stage. It mentions that the softwares should be checked to prevent the propagation of reliability allocation errors and design errors at later stages. It further discusses a case study related to SMC of fault-tolerant fuel control system (FFCS) a safety-critical component of an automobile's engine controller.

Random testing is a basic software testing technique that can be used to assess the software reliability as well as to detect software failures. Adaptive random testing has been proposed to enhance the failure-detection capability of random testing. Previous studies have shown that adaptive random testing can use fewer test cases than random testing to detect the first software failure. In this paper, we evaluate and compare the performance of adaptive random testing and random testing from another perspective, that of code coverage. As shown in various investigations, a higher code coverage not only brings a higher failure-detection capability, but also improves the effectiveness of software reliability estimation. We conduct a series of experiments based on two categories of code coverage criteria: structure-based coverage, and fault-based coverage. Adaptive random testing can achieve higher code coverage than random testing with the same number of test cases. Our experimental results imply that, in addition to having a better failure-detection capability than random testing, adaptive random testing also delivers a higher effectiveness in assessing software reliability, and a higher confidence in the reliability of the software under test even when no failure is detected. [ABSTRACT FROM PUBLISHER]

Determining the software quality and release time of the software is an important role of software reliability. Software reliability growth models (SRGMs) are applications of software reliability. The change-point is defined as the prominent change in the software testing time, in connection with the quality of a software error occurrence or error-detection phenomenon. As a result, the software reliability growth process, based on the change-point, affects the precision of the software reliability prediction, based on the SRGMs. To find the accurate total expected delivery cost with a suitable warranty period of the software system, a new cost model for the software system with the warranty and change-point phenomenon is proposed in this paper. The entire expected delivery cost and the reliability of the software system is calculated using the change-point SRGM. The optimal release time is calculated by reducing the overall estimated delivery cost for various desired reliability levels. Based on the proposed warranty cost model and the reliability of the software system, we have derived some optimal release policies. Numerical illustrations and interconnected discussed data are itemized. From the testing outcome, we get software release policies that provide a comprehensive analysis of software based on the total expected delivery cost with a suitable warranty period, desired level of reliability, and change-point. Moreover, these policies will be helpful for project managers to decide where to stop the testing for customer release at the exact time with a suitable warranty period. [ABSTRACT FROM AUTHOR]