Search

Though Deep Neural Networks (DNNs) have been widely deployed and achieved great success in many domains, they have severe safety and reliability concerns. To provide testing evidence for DNNs' reliable behaviors, various coverage testing techniques inspired by traditional software testing have been proposed. However, the coverage criteria in these techniques are either not fine enough to capture subtle behaviors of DNNs, or too time-consuming to be applied on large-scale DNNs. In this paper, we propose a coverage testing framework named HashC, which makes mainstream coverage criteria (e.g., NC and KMNC) much finer. Meanwhile, HashC reduces the time complexity of combinatorial coverage testing from polynomial time to linear time. We also develop the corresponding test sample selection method. Our experiments show that, (1) the existing mainstream coverage criteria are becoming finer after being equipped with HashC, (2) HashC greatly accelerates combinatorial coverage testing and can handle the testing of large-scale DNNs. [ABSTRACT FROM AUTHOR]

• A custom made measurement system for on-line testing of electromechanical relays. • Measurement of essential magnetic and contact properties in less than two seconds. • Measurement algorithms based on simultaneous waveform generation and acquisition. • The comprehensive relay test software application. An automatic measurement system has been developed for high-speed testing of the most important magnetic and contact properties for elementary monostable and bistable miniature electromechanical relays. The system design is based on the virtual instrumentation approach, which is a combination of software and hardware. The comprehensive test application was developed using the LabVIEW graphical programming environment, while the used hardware is based on commercial measurement equipment and a custom developed device. The presented measurement methods are based on simultaneous generation and acquisition of analog and digital waveforms with the high-speed data acquisition device. The implemented automatic relay measurement system was validated metrologically and tested successfully with all supported relay types. The results show that the developed system is capable of performing accurate measurements of the coil resistance, contact resistance, time parameters, and operate/release values in less than two seconds. [ABSTRACT FROM AUTHOR]

During the unit testing of model-driven tools, a large number of models and test classes needs to be managed and maintained. Typically, some of these artifacts are specified manually, some are generated automatically. Existing approaches to test management rely on the available visual and textual modeling notations. As these notations are not tailored to unit testing, distinct maintainability trade-offs arise. In this paper, we propose a notation that aims to combine the benefits of visual and text-based approaches. The notation is at the same time visual and text-based, as it uses ASCII characters to emulate the familiar graphical notations. In our evaluation based on real models, we identify problematic model shapes challenging the scalability our notation, while finding that it is well-suited to capture typical test models. [ABSTRACT FROM AUTHOR]

Software testing enables development teams to maintain the quality of a software system while it evolves. The database manipulation code requires special attention in this context. However, it is often neglected and suffers from software maintenance problems. In this paper, we study the current state-of-the-practice in testing database manipulation code. We first analysed the tests of 72 open-source projects to gain insight into the coverage of database access code. The database was poorly tested: 46% of the projects did not cover with tests half of their database access methods, and 33% did not cover the database code at all. This poor coverage motivated us to study developers' challenges and best practices. (i) First, we analysed 532 questions on Stack Exchange sites and deduced a taxonomy of issues. Developers mostly looked for general best practices to test database access code. Their technical questions were related to database management, mocking, parallelisation, or framework/tool usage. (ii) Next, we examined the answers to these questions. We manually labelled 598 answers to 255 questions. We distinguished 363 solutions and organised them in a taxonomy of best practices. Most of the suggestions considered the testing environment and recommended various tools or configurations. The second largest category was database management, where many addressed database initialisation and clean-up between tests. Other categories pertained to code structure or design, concepts, performance, processes, test characteristics, test code, and mocking. We illustrate the two taxonomies through intriguing examples. • A study on the coverage of database access code in the tests of open-source projects. • A qualitative assessment of developers' issues when testing database access code. • An investigation of developers' best practices when testing database access code. • Taxonomies of issues and best practices when testing database access code. • The full dataset of the studies is publicly available as an online appendix. [ABSTRACT FROM AUTHOR]

We show how to write concise and readable specifications of linked data structures that are applicable for both formal deductive verification and testing. A singly linked list and a binary search tree are provided as examples. The main characteristic of the specifications is the use of observer methods, in particular to express reachability of elements in a data structure. The specifications are written in the Java Modeling Language (JML) and do not require extensions of that language. This paper addresses a mixed audience of users and developers in the fields of formal verification, testing, and specification language design. We provide an in-depth description of the proposed specifications and analyze the implications both for verification and testing. Based on this analysis we have developed verification techniques that are implemented in the deductive verification tool KeY and enable fully automatic verification of the linked list example. This techniques are also described in this paper. [ABSTRACT FROM AUTHOR]

Abstract: When computational methods are developed, the efficiency of the novel methods should be compared to the existing ones. This can be done using, e.g., analytical methods and benchmark test patterns. In addition, the comparison of the best and the worst case performance is commonly of interest. In this paper, methodologies of genetic algorithm based software testing are adopted to the comparative computational testing of three varieties of dynamic two-dimensional nearest point algorithms. The extreme performances of the algorithms are searched for by optimizing the shape of two-dimensional Gaussian distributions, from which the test patterns are drawn. In particular, an approach to pairwise comparisons of computational complexities of algorithms is proposed. The test case algorithms can be sorted with respect to their computational complexity by the proposed method. [Copyright &y& Elsevier]

Abstract: Models are abstractions of reality that have predetermined limits (often not consciously thought through) on what problem domains the models can be used to explore. These limits are determined by the range of observed data used to construct and validate the model. However, it is important to remember that operating the model beyond these limits, one of the reasons for building the model in the first place, potentially brings unwanted behaviour and thus reduces the usefulness of the model. Our experience with the Agricultural Production Systems Simulator (APSIM), a farming systems model, has led us to adapt techniques from the disciplines of modelling and software development to create a model development process. This process is simple, easy to follow, and brings a much higher level of stability to the development effort, which then delivers a much more useful model. A major part of the process relies on having a range of detailed model tests (unit, simulation, sensibility, validation) that exercise a model at various levels (sub-model, model and simulation). To underline the usefulness of testing, we examine several case studies where simulated output can be compared with simple relationships. For example, output is compared with crop water use efficiency relationships gleaned from the literature to check that the model reproduces the expected function. Similarly, another case study attempts to reproduce generalised hydrological relationships found in the literature. This paper then describes a simple model development process (using version control, automated testing and differencing tools), that will enhance the reliability and usefulness of a model. [ABSTRACT FROM AUTHOR]