Search

Exploratory Testing technique is well applicable to software development projects, where test basis is not available (or at least not complete and consistent to the extent allowing the creation of efficient test cases). The key factor for the efficiency of this technique is a structured process for the recording of explored path in the system under test. This approach also allows the creation of the test cases during exploratory testing process. These test cases can be used in the following re-testing of the system. If performed manually, the efficiency of such process strongly depends on the team organization and systematic work of the individuals in the team. This process can be aided by an automated support. In the paper, a framework, which automatically records selected tester's actions in the system under test is presented. From these recordings, a model of the screen and action flows is reengineered and test cases are prepared. Tester is also able to define more meta-data in the test cases during this process. The recorded model and defined test cases are then available for the next rounds of testing. The performed case study shows that Exploratory Testing aided by this machine support is less resource demanding than Exploratory Testing performed manually only. Also, larger part of SUT was explored during the tests, when this systematic support was available to testers. [ABSTRACT FROM AUTHOR]

In this paper we discuss the application of a range of techniques to the verification of mission-critical flight software at NASA's Jet Propulsion Laboratory. For this type of application we want to achieve a higher level of confidence than can be achieved through standard software testing. Unfortunately, given the current state of the art, especially when efforts are constrained by the tight deadlines and resource limitations of a flight project, it is not feasible to produce a rigorous formal proof of correctness of even a well-specified stand-alone module such as a file system (much less more tightly coupled or difficult-to-specify modules). This means that we must look for a practical alternative in the area between traditional testing and proof, as we attempt to optimize rigor and coverage. The approaches we describe here are based on testing, model checking, constraint-solving, monitoring, and finite-state machine learning, in addition to static code analysis. The results we have obtained in the domain of file systems are encouraging, and suggest that for more complex properties of programs with complex data structures, it is possibly more beneficial to use constraint solvers to guide and analyze execution (i.e., as in testing, even if performed by a model checking tool) than to translate the program and property into a set of constraints, as in abstraction-based and bounded model checkers. Our experience with non-file-system flight software modules shows that methods even further removed from traditional static formal methods can be assisted by formal approaches, yet readily adopted by test engineers and software developers, even as the key problem shifts from test generation and selection to test evaluation. [ABSTRACT FROM AUTHOR]

Testing of a software product is the key activity before deploying it in the real-time environment. Therefore testing should be introduced into the product development process as early as possible in order to decrease the costs of repairing the damage in later phases. In this paper we use Coloured Petri Nets (CPN) for the system specification and also as a system to be tested instead of its implementation. This specification also serves as the basis for the test suite generation. Test cases are provided in the language of Testing and Test Control Notation version 3 (TTCN-3) due to its general application area and platform independence. Stop-and-wait communication protocol has been used as a tutorial example of our approach. [ABSTRACT FROM AUTHOR]

Dynamic software updating (DSU) systems patch programs on the fly without incurring downtime. To avoid failures due to the updating process itself, many DSU systems employ timing restrictions. However, timing restrictions are theoretically imperfect, and their practical effectiveness is an open question. This paper presents the first significant empirical evaluation of three popular timing restrictions: activeness safety (AS), which prevents updates to active functions, con-freeness safety (CFS), which only allows modifications to active functions when doing so is provably type-safe, and manual identification of the event-handling loops during which an update may occur. We evaluated these timing restrictions using a series of DSU patches to three programs: OpenSSH, vsftpd, and ngIRCd. We systematically applied updates at each distinct update point reached during execution of a suite of system tests for these programs to determine which updates pass and which fail. We found that all three timing restrictions prevented most failures, but only manual identification allowed none. Further, although CFS and AS allowed many more update points, manual identification still supported updates with minimal delay. Finally, we found that manual identification required the least developer effort. Overall, we conclude that manual identification is most effective. [ABSTRACT FROM AUTHOR]

Claims that there are several programs available to test and benchmark drives and media and these are available as freeware, shareware and off-the-shelf programs. Testing a compact disc-read only memory (CD-ROM) or CD-Recordable drive; Some of the testing and benchmarking programs; Recommended tool for those who only need CD-ROM testing and do not need a full-featured software-based tester. INSETS: What happened to disc detective?.;Benchmarking and testing--the same thing?..

In this article we introduce the problem of computer software reliability and discuss a probabilistic model for describing the failure of software. We suggest a procedure for estimating the parameters of the model and propose a stopping rule for debugging the software. We apply our procedures to some published data on software failures. [ABSTRACT FROM AUTHOR]

This paper explores the testing complexity of several classes of programs, where the testing complexity is measured in terms of the number of test data required for demonstrating program correctness by testing. It is shown that even for very restrictive classes of programs, none of the commonly used test criteria, namely, having every statement, branch, and path executed at least once, is nearly sufficient to guarantee absence of errors. Based on the study of testing complexity, this paper proposes two new test criteria, one for testing a path and the other for testing a program. These new criteria suggest how to select test data to obtain confidence in program correctness beyond the requirement of having each statement, branch, or path tested at least once. [ABSTRACT FROM AUTHOR]

This paper presents a testing strategy designed to detect errors in the control flow of a computer program, and the conditions under which this strategy is reliable are given and characterized. The control flow statements in a computer program partition the input space into a set of mutually exclusive domains, each of which corresponds to a particular program path and consists of input data points which cause that path to be executed. The testing strategy generates test points to examine the boundaries of a domain to detect whether a domain error has occurred, as either one or more of these boundaries will have shifted or else the corresponding predicate relational operator has changed. If test points can be chosen within ∈ of each boundary, under the appropriate assumptions, the strategy is shown to be reliable in detecting domain errors of magnitude greater than ∈. Moreover, the number of test points required to test each domain grows only linearly with both the dimensionality of the input space and the number of predicates along the path being tested. [ABSTRACT FROM AUTHOR]

Two approaches to the study of program testing arc described. One approach is theoretical and the other empirical. In the theoretical approach situations are characterized in which it is possible to use testing to formally prove the correctness of programs or the correctness of properties of programs. In the empirical approach statistics are collected which record the frequency with which different testing strategies reveal the errors in a collection of programs. A summary of the results of two research projects which investigated these approaches are presented. The differences between the two approaches are discussed and their relative advantages and disadvantages are compared. [ABSTRACT FROM AUTHOR]