Search

Manual testing is a machine testing method in which testing cases are carried out manually rather than utilizing an automatic process. Both test cases were handled manually by the tester from the perspective of the end consumer. It guarantees that the program performs, regardless of whether or not it is specified in the prerequisite text. In order to complete approximately 100% of the application, test cases are planned and implemented. Manually produced test case reports are common. Since it can spot both visible and hidden program bugs, manual testing is one of the most effective testing techniques. A defect is described by the software as the difference between expected and actual performance. The bugs were corrected and turned over to the tester by the creator. Manual training is expected before automatic testing for any newly created software. This research requires a lot of time and resources, but it ensures the entire program is bug-free. Manual research entails knowledge of manual testing methods but does not require the use of an automated evaluation method. Manual checking is necessary since one of the fundamental software testing concepts is that "100% automation is not feasible." [ABSTRACT FROM AUTHOR]

This article proposes a hybrid test scheme based on interleaved test applications to increase the efficiency of multisite testing of integrated circuits (ICs) by solving problems related to test coverage, which is an important feature in automotive IC, and test costs, including the number of test pins and the volume of test data. To solve the problems, a hybrid test scheme that combines the merits of broadcast-based scan compression and built-in self-test (BIST) is proposed. The proposed scheme not only generates deterministic test patterns, similar to scan compression, but also requires fewer test pins, similar to BIST, thereby increasing the efficiency of multisite testing. With appropriately designed linear-feedback shift registers (LFSRs) and seeds periodically inserted through the reduced test pins, the test application process combines deterministic testing and random testing. The random testing stage can compensate the seed-insertion time for the subsequent deterministic testing stage and reduce the number of required deterministic patterns by detecting easy-to-detect faults. For further reduction of the volume of test data and hardware overhead, some compatible scan chains (SCs) are grouped and receive test data from an LFSR. The experimental results show that the proposed method increases the efficiency of multisite testing with a reduced pin count. [ABSTRACT FROM AUTHOR]

Recurrent neural networks (RNNs) have been applied to a broad range of applications, including natural language processing, drug discovery, and video recognition. Their vulnerability to input perturbation is also known. Aligning with a view from software defect detection, this article aims to develop a coverage-guided testing approach to systematically exploit the internal behavior of RNNs, with the expectation that such testing can detect defects with high possibility. Technically, the long short-term memory network (LSTM), a major class of RNNs, is thoroughly studied. A family of three test metrics are designed to quantify not only the values but also the temporal relations (including both stepwise and bounded-length) exhibited when LSTM processing inputs. A genetic algorithm is applied to efficiently generate test cases. The test metrics and test case generation algorithm are implemented into a tool testRNN, which is then evaluated on a set of LSTM benchmarks. Experiments confirm that testRNN has advantages over the state-of-the-art tool DeepStellar and attack-based defect detection methods, owing to its working with finer temporal semantics and the consideration of the naturalness of input perturbation. Furthermore, testRNN enables meaningful information to be collected and exhibited for users to understand the testing results, which is an important step toward interpretable neural network testing. [ABSTRACT FROM AUTHOR]

In an era of connectivity and automation, the vehicle industry is adopting numerous technologies to transform driver-centric vehicles into intelligent mechanical devices driven by software components. Software integration and network connectivity inherit numerous security issues that open the door for malicious attacks. Software security testing is a scalable and practical approach to identify systems’ weaknesses and vulnerabilities at an early stage and throughout their life-cycle. Security specialists recommend fuzz testing to identify vulnerabilities within vehicle software systems. Nevertheless, the randomness and blindness of fuzzing hinder it from becoming a reliable security tool. This article presents a vulnerability-oriented fuzz (VulFuzz) testing framework that utilizes security vulnerability metrics designed particularly for connected and autonomous vehicles to direct and prioritize the fuzz testing toward the most vulnerable components. While most gray-box fuzzing techniques aim solely to expand code coverage, the proposed approach assigns weights to ensure a thorough examination of the most vulnerable components. Moreover, we employ an input structure-aware mutation technique that can bypass vehicle software systems’ input formats to boost test performance and avoid dropped test cases. Such a testing technique will contribute to the quality assurance of vehicle software engineering. We implemented the proposed approach on OpenPilot, a driver assistance system, and compared our results to American fuzzy lop (AFL) and an unguided mutation-based fuzzer. Within 16.8 h, VulFuzz exposed 335 crashes, 41 times more than AFL and two times more than an unguided mutation-based fuzzer. VulFuzz is explicitly efficient for automotive systems, reaching the same code coverage as AFL but with more exposed crashes and fewer dropped messages. [ABSTRACT FROM AUTHOR]

Symbolic execution is a powerful technique for automating software testing to detect many types of errors such as memory errors and assertion violations. However, it encounters the problem of path explosion, and by using only assertions, it still lacks the capability of going deep into checking the functional correctness of a path based on corresponding formal specifications. To address these problems, we propose a specification-based incremental testing method with symbolic execution, called SIT-SE, providing a much more rigorous way to automatically check the functional correctness of all the discovered program paths, by introducing theorems (instead of assertions) for path correctness and branch sequence coverage algorithm for guiding a moderate path exploration. Compared with Hoare logic for proving the correctness of an entire program, a theorem in the SIT-SE is made for verifying the correctness of a program path. The proposed method carefully treats the relationship between a path condition and the specification in a theorem to restrict the monotonous path exploration, whereas traditional concolic testing methods roughly use one test data to determine the path correctness by assertions during long path searching. We use a classic case to demonstrate how the method works and conduct an experiment to evaluate the performance of both the proposed method and the commonly used well-known concolic testing tool KLEE. The experimental results show that our method SIT-SE is effective and outperforms KLEE in detecting faulty paths based on specifications. [ABSTRACT FROM AUTHOR]

Though Deep Neural Networks (DNNs) have been widely deployed and achieved great success in many domains, they have severe safety and reliability concerns. To provide testing evidence for DNNs' reliable behaviors, various coverage testing techniques inspired by traditional software testing have been proposed. However, the coverage criteria in these techniques are either not fine enough to capture subtle behaviors of DNNs, or too time-consuming to be applied on large-scale DNNs. In this paper, we propose a coverage testing framework named HashC, which makes mainstream coverage criteria (e.g., NC and KMNC) much finer. Meanwhile, HashC reduces the time complexity of combinatorial coverage testing from polynomial time to linear time. We also develop the corresponding test sample selection method. Our experiments show that, (1) the existing mainstream coverage criteria are becoming finer after being equipped with HashC, (2) HashC greatly accelerates combinatorial coverage testing and can handle the testing of large-scale DNNs. [ABSTRACT FROM AUTHOR]

Software testing enables development teams to maintain the quality of a software system while it evolves. The database manipulation code requires special attention in this context. However, it is often neglected and suffers from software maintenance problems. In this paper, we study the current state-of-the-practice in testing database manipulation code. We first analysed the tests of 72 open-source projects to gain insight into the coverage of database access code. The database was poorly tested: 46% of the projects did not cover with tests half of their database access methods, and 33% did not cover the database code at all. This poor coverage motivated us to study developers' challenges and best practices. (i) First, we analysed 532 questions on Stack Exchange sites and deduced a taxonomy of issues. Developers mostly looked for general best practices to test database access code. Their technical questions were related to database management, mocking, parallelisation, or framework/tool usage. (ii) Next, we examined the answers to these questions. We manually labelled 598 answers to 255 questions. We distinguished 363 solutions and organised them in a taxonomy of best practices. Most of the suggestions considered the testing environment and recommended various tools or configurations. The second largest category was database management, where many addressed database initialisation and clean-up between tests. Other categories pertained to code structure or design, concepts, performance, processes, test characteristics, test code, and mocking. We illustrate the two taxonomies through intriguing examples. • A study on the coverage of database access code in the tests of open-source projects. • A qualitative assessment of developers' issues when testing database access code. • An investigation of developers' best practices when testing database access code. • Taxonomies of issues and best practices when testing database access code. • The full dataset of the studies is publicly available as an online appendix. [ABSTRACT FROM AUTHOR]

Modern vehicles have evolved to support connected and self-driving capabilities. The concepts such as connected driving, cooperative driving, and intelligent transportation systems have resulted in an increase in the connectivity of vehicles and subsequently created new information security risks. The original vehicular ad-hoc network term is now emerged to a new term, Internet of Vehicles (IoV), which is a typical application of symmetry of Internet of Things (IoT). Vehicle manufacturers address some critical issues such as software bugs or security issues through remote updates, and this gives rise to concerns regarding the security of updated components. Moreover, aftermarket units such as those imposed by transportation authorities or insurance companies expose vehicles to high risk. Software testing aims to ensure that software products are reliable and behave as expected. Many commercial and open-source software products undergo formal certifications to increase users' confidence in their accuracy, reliability, and security. There are different techniques for software certification, including test-based certification. Testcase repositories are available to support software testing and certification, such as the Linux Test Project for Linux kernel testing. Previous studies performed various testing and experimental evaluation of different parts of modern vehicles to assess the security risks. Due to the lack of trusted testcase repositories and a common approach for testing, testing efforts are performed individually. In this paper, we propose a blockchain-based approach for a testcase repository to support test-based software and security testing and overcome the lack of trusted testcase repositories. The novel concept Proof-of-Validation to manage global state is proposed to manage updates to the repository. The initial work in this study considers the LTP test suite as a use case for the testcase repository. This research work is expected to contribute to the further development in including evidence generation for testing verification. [ABSTRACT FROM AUTHOR]