Search

The types of the testing methods were analyzed from the side of development. If changes are made to the code, even minor ones, bugs can appear anywhere in the system. The cost of fixing bugs increases with the time they are discovered. Using effective web testing can prevent unexpected costs. Modern web applications are indeed a place where all the type of the testing is vital for the high-quality product. At the moment, traditional front and back office applications are increasingly moving from desktop computers to web interfaces, so testing web applications is becoming extremely relevant. Thus, learning what to automate and how is an important component of successful web application testing. Web application testing is a software practice that is designed to ensure quality by verifying that the functionality of a particular web application works correctly or according to fixed requirements. Web testing makes it possible to find bugs at any point in time, before release or on a daily basis. On the one hand there are wide range of tools that could be used to pursue this goal by means of using best decisions present for now but on the other hand there are essential advantages and disadvantages present in these tools, which are all in the approach to it, so there are always cons and pros of using one or another. For now, despite on having the ability to test using both black and white box testing it looks like the second one is not the best choice. There are several points on each board for both of them, but black box approach that is being represented with react testing library is more successful and efficient way to cover and application with high- and low-level tests, that could be easily maintained and understood. But white box testing is now the most used decision due to the historic development of the industry. So, it also has some great features and could be chosen to be used on the project, but it should be precise choice with the understanding of all the consequences standing behind., Види методів тестування були проаналізовані з боку розробки. Якщо в код вносяться зміни, навіть незначні, помилки можуть виникнути будь-де системі. Вартість виправлення помилок зростає з часом їх виявлення. Використання ефективного веб тестування може запобігти непередбаченим витратам. Сучасні веб застосування дійсно є місцем, де весь тип тестування є життєво важливим для якісного продукту. На даний момент традиційні фронт та бек-офісні програми все частіше переходять з настільних комп’ютерів на веб інтерфейси, тому тестування веб застосувань стає вкрай актуальним. Таким чином, вивчення того, що і як автоматизувати, є важливим компонентом успішного тестування веб застосувань. Тестування веб застосувань є практикою програмного забезпечення, яка призначена для забезпечення якості за рахунок перевірки того, що функціональні можливості певного веб програми працюють правильним чином або відповідно до зафіксованих вимог. Веб тестування дає можливість знаходити помилки в будь-який час, до релізу або щодня. З одного боку, є широкий спектр інструментів, які можна було б використовувати для досягнення цієї мети за допомогою використання найкращих рішень, присутніх на даний момент, але з іншого боку, є суттєві переваги і недоліки, присутні в цих інструментах, які всі в підході до нього, тому завжди є мінуси і плюси використання того чи іншого. Поки що, незважаючи на можливість тестування з використанням як чорної, так і білої скриньки, останній метод має багато недоліків та не рекомендується до використання. Ці два методи мають місце бути у окремих проєктах, але підхід чорної скриньки, який представлений бібліотекою тестування React є найбільш успішним і ефективним способом покрити і застосування з високими і низькими тестами, які можна легко підтримувати і розуміти. Але тестування білої скриньки в даний час є найбільш вживаним рішенням у зв’язку з історичним розвитком галузі. Отже, воно також має деякі позитивні риси і може бути обрано, щоб бути використаним на проєкті, але повинно бути точним вибором з розумінням всіх наслідків, що за ним слідують.

We propose a novel technique based on recurrent artificial neural networks to generate test cases for black-box testing of reactive systems. We combine functional testing inputs that are automatically generated from a model together with manually-applied test cases for robustness testing. We use this combination to train a long short-term memory (LSTM) network. As a result, the network learns an implicit representation of the usage behavior that is liable to failures. We use this network to generate new event sequences as test cases. We applied our approach in the context of an industrial case study for the black-box testing of a digital TV system. LSTM-generated test cases were able to reveal several faults, including critical ones, that were not detected with existing automated or manual testing activities. Our approach is complementary to model-based and exploratory testing, and the combined approach outperforms random testing in terms of both fault coverage and execution time.

An online partitioning strategy is presented, in which test cases are selected based on feedback information collected during the testing process. The strategy differs from conventional approaches because the partitioning is performed online rather than off-line and because the partitioning is not based on program code or specifications. It can, therefore, be implemented in the absence of the source code or specification of the program under test. The cost-effectiveness of the proposed strategy has been empirically investigated with a set of subject programs, namely, SPACE, SED, GREP, and the Siemens Suite of Programs. The results demonstrate that the proposed strategy constantly achieves large savings in terms of the total number of test case executions needed to detect all faults.

Software testing is the process of verifying and validating the user’s requirements. Testing is ongoing process during whole software development. Software testing is characterized into three main types. That is, in Black box testing, user doesn’t know domestic knowledge, internal logics and design of system. In white box testing, Tester knows the domestic logic of code. In Grey box testing, Tester has little bit knowledge about the internal structure and working of the system. It is commonly used in case of Integration testing.Load testing helps us to analyze the performance of the system under heavy load or under Zero load. This is achieved with the help of a Load Testing Tool. The intention for writing this research is to carry out a comparison of four load testing tools i.e. Apache JMeter, LoadRunner, Microsoft Visual Studio (TFS), Siege based on certain criteria i.e. test scripts generation , result reports, application support, plug-in supports, and cost . The main focus is to study these load testing tools and identify which tool is better and more efficient . We assume this comparison can help in selecting the most appropriate tool and motivates the use of open source load testing tools.

Object-oriented software systems contain large number of modules which make unit testing, integration testing, and system testing very difficult and challenging. While the aim of unit testing is to show that individual modules are working properly and the aim of the system testing is to determine whether the whole system meets its specifications, the aim of integration testing is to uncover errors in the interactions between system modules. However, it is generally impossible to test all connections between modules because of time and budget constraints. Thus, it is important to focus the testing on the connections presumed to be more error-prone. The goal of this work is to guide software testers where in a software system to focus when performing integration testing to save time and resources. This paper proposes a new approach to predict and rank error-prone connections. We use method level metrics that capture both dependencies and internal complexity of methods. We performed experiments on several Java applications and used error seeding techniques for evaluation. The experimental results showed that our approach is effective for selecting the test focus in integration testing.

Objective: An electronic system developed for specific application with the integration of hardware and software is known as Embedded System. Due to complexity of hardware and software in a single system, it requires specific technique for testing before deployment of the device. This paper proposes the comparative study of testing techniques of embedded system. Methods/Analysis: In this paper, a test methodology of embedded system using PCBA has been proposed, that covers all the testing aspects of the embedded system from electrical board level hardware to the embedded system software. Three tests namely power rail test, interconnect test and the infrastructure tests are used for hardware functionality for stuck at and bridging faults. The functionality test has been used for programming validation using port interface. Findings: This paper proposed automatic testing of embedded board, which reduces manual errors, increases test coverage and reduces the test time in the production line. The proposed method showed that it can be applied to any board in any form with the same test hardware barring the input-output cables. The software in host machine has been used for testing purpose of different boards. This testing technique is useful for embedded systems implemented in control systems.

Computer forensic techniques are important for the prevention, detection, and investigation of electronic crime. Computer forensic investigators need computer forensic tools to produce reliable results that meet legal requirements and are acceptable in the courts. Most of these tools are closed-source, making the software a black-box for testing purposes. This paper illustrates a different black box testing method for experimenting computer forensic tools based on functional scenarios.

Static testing is used to detect software defects in the earlier phases of the software development lifecycle, which makes the total costs caused by defects lower and the software development project less risky. Different types of static testing have been introduced and are used in software projects. In this paper, we focus on static testing related to data consistency in a software system. In particular, we propose extensions to contemporary static testing techniques based on CRUD matrices, employing cross-verifications between various types of CRUD matrices made by different parties at various stages of the software project. Based on performed experiments, the proposed static testing technique significantly improves the consistency of Data Cycle Test cases. Together with this trend, we observe growing potential of test cases to detect data consistency defects in the system under test, when utilizing the proposed technique.

Software testing being one of the major phases of the software development life cycle is critical in delivering reliable software products. Traditional manual GUI testing has severe limitations like insufficient test coverage, labor intensiveness, complexity involved and cost ineffectiveness. “How much testing is enough or sufficient?” still remains a challenging question. Coverage analysis helps to guide the test engineers regarding test coverage and is used extensively to determine the effectiveness of selected testing methodology. The problems mentioned above related to manual GUI testing prompted the need for automation of GUI testing and coverage analysis. With the rapid emergence of GUI based context free applications, automated testing tools seems less effective. Recently, it has been learnt that concurrent and distributed machines based cluster can be used to reduce the required effort to test GUI in context of time. Proposed system use two different evolutionary algorithms concurrently to gain multiple objectives, based on the fitness functions of maximization of GUI test path coverage (measured as the number of events in an event sequence tested by a particular test case) and simultaneously minimizing the number of test cases. For coverage analysis, two of the very well-known multi objective algorithms; NSGA II and MOPSO from evolutionary domain have been employed. Experiments results of coverage analysis show that accuracy to the tune of 85% was achieved in MOPSO and NSGA II. This high level of accuracy is an indicator of usefulness of proposed fitness function. Both algorithms execute more than 90% of test paths.

The test oracle —a judge of the correctness of the system under test (SUT)—is a major component of the testing process. Specifying test oracles is challenging for some domains, such as real-time embedded systems, where small changes in timing or sensory input may cause large behavioral differences. Models of such systems, often built for analysis and simulation, are appealing for reuse as test oracles. These models, however, typically represent an idealized system, abstracting away certain issues such as non-deterministic timing behavior and sensor noise. Thus, even with the same inputs, the model's behavior may fail to match an acceptable behavior of the SUT, leading to many false positives reported by the test oracle. We propose an automated steering framework that can adjust the behavior of the model to better match the behavior of the SUT to reduce the rate of false positives. This model steering is limited by a set of constraints (defining the differences in behavior that are acceptable) and is based on a search process attempting to minimize a dissimilarity metric. This framework allows non-deterministic, but bounded, behavioral differences, while preventing future mismatches by guiding the oracle—within limits—to match the execution of the SUT. Results show that steering significantly increases SUT-oracle conformance with minimal masking of real faults and, thus, has significant potential for reducing false positives and, consequently, testing and debugging costs while improving the quality of the testing process.

Video games comprise a multi-billion-dollar industry. Companies invest huge amounts of money for the release of their games. A part of this money is invested in testing the games. Current game test...

Currently, when a technology is defined (Bluetooth, GSM, UMTS or LTE), it is mandatory to think about an infrastructure with an abstract test suite, codecs, timeframes, etc. that will support the design of a new testing architecture. If, in addition, the testing methodology used is for different purposes, such as protocol conformance, radio conformance, radio performance or network conformance, the result is a redundant effort in the design and development of different test systems, increasing costs, development time and reducing reusability. In this paper it is shown how a framework for formal languages such as TTCN can be realized in a common adapter layer capable of sharing a model for any technology and testing methodology, reducing costs, improving time and avoiding redundant designing efforts.

Exploratory Testing technique is well applicable to software development projects, where test basis is not available (or at least not complete and consistent to the extent allowing the creation of efficient test cases). The key factor for the efficiency of this technique is a structured process for the recording of explored path in the system under test. This approach also allows the creation of the test cases during exploratory testing process. These test cases can be used in the following re-testing of the system. If performed manually, the efficiency of such process strongly depends on the team organization and systematic work of the individuals in the team. This process can be aided by an automated support. In the paper, a framework, which automatically records selected tester's actions in the system under test is presented. From these recordings, a model of the screen and action flows is reengineered and test cases are prepared. Tester is also able to define more meta-data in the test cases during this process. The recorded model and defined test cases are then available for the next rounds of testing. The performed case study shows that Exploratory Testing aided by this machine support is less resource demanding than Exploratory Testing performed manually only. Also, larger part of SUT was explored during the tests, when this systematic support was available to testers.

Currently, software tends to assume increasingly critical roles in our society so assuring its quality becomes ever more crucial. There are several tools and processes of software testing to help increase quality in virtually any type of software. One example is the so called model-based testing (MBT) tools, that generate test cases from models. Pattern Based Graphical User Interface Testing (PBGT) is an example of a MBT new methodology that aims at systematizing and automating the Graphical User Interface (GUI) testing process. It is supported by a Tool (PBGT Tool) which provides an integrated modeling and testing environment for crafting test models based on User Interface Test Patterns (UITP) using a GUI modeling Domain Specific Language (DSL) called PARADIGM. Most of the MBT tools have a configuration phase, where test input data is provided manually by the tester, which influences the quality of the test suite generated. By adding coverage analysis to MBT tools, it is possible to give feedback and help the tester to define the configuration data needed to achieve the most valuable test suite as possible and, ultimately, contribute for increasing the quality of the software. This paper presents a multidimensional test coverage analysis approach and tool (PARADIGM-COV), developed in the context of the PBGT project, that produces coverage information both over the PARADIGM model elements and during test case execution (to identify the parts of the model that were actually exercised). It also presents a case study illustrating the benefits of having multidimensional analysis and assessing the overall test coverage approach.

Recently, testing techniques based on dynamic exploration, which try to automatically exercise every possible user interface element, have been extensively used to facilitate fully testing web applications. Most of such testing tools are however not effective in reaching dynamic pages induced by form interactions due to their emphasis on handling client-side scripting. In this paper, we present a combinatorial strategy to achieve a full form test and build an automated test model. We propose an algorithm called pairwise testing with constraints (PTC) to implement the strategy. Our PTC algorithm uses pairwise coverage and handles the issues of semantic constraints and illegal values. We have implemented a prototype tool ComjaxTest and conducted an empirical study on five web applications. Experimental results indicate that our PTC algorithm generates less form test cases while achieving a higher coverage of dynamic pages than the general pairwise testing algorithm. Additionally, our ComjaxTest generates a relatively complete test model and then detects more faults in a reasonable amount of time, as compared with other existing tools based on dynamic exploration.

Software testing has a significant importance to achieve maximum quality to satisfy the customers and concerned stakeholders. A test case is designed to perform set of actions with intend of finding errors and verify some functions and features of an application. During design process, a huge number of test cases produced, some of them are of little or no use, which can be ignore or postponed, when there is budget and time constraints, or a need to decide which test cases to execute first and which to last. However, in black box testing, test cases are prioritized manually during planning phase and companies mostly experience schedule limitations, in that case, effective testing costs them badly. Test case prioritization’s main purpose is to effectively use time and budget to execute highest priority test cases first with customer’s satisfaction. To achieve this goal, we proposed a technique in which we use a customer assigned weight abstracted from business requirements to keep the customer’s preference first, based on that three main clusters formed. Then we calculate proposed cost and time percentage for each test case using function points and complexity measure, with in each cluster. Based on that, clusters further classified in to High, Medium and Low priorities clusters by K-Medoids algorithm. In our approach, test cases finally classified in to clusters and sub clusters based on the priority of the both stakeholders. Our approach shows 79.174% accuracy as compared to the actual data. To achieve maximum efficiency, considering user’s satisfaction, this method of mining test cases will be helpful in terms of saving time and cost.

A test harness, in automated test generation, defines the set of valid tests for a system, as well as their correctness properties. The difficulty of writing test harnesses is a major obstacle to the adoption of automated test generation and model checking. Languages for writing test harnesses are usually tied to a particular tool and unfamiliar to programmers, and often limit expressiveness. Writing test harnesses directly in the language of the software under test (SUT) is a tedious, repetitive, and error-prone task, offers little or no support for test case manipulation and debugging, and produces hard-to-read, hard-to-maintain code. Using existing harness languages or writing directly in the language of the SUT also tends to limit users to one algorithm for test generation, with little ability to explore alternative methods. In this paper, we present TSTL, the template scripting testing language, a domain-specific language (DSL) for writing test harnesses. TSTL compiles harness definitions into an interface for testing, making generic test generation and manipulation tools for all SUTs possible. TSTL includes tools for generating, manipulating, and analyzing test cases, including simple model checkers. This paper motivates TSTL via a large-scale testing effort, directed by an end-user, to find faults in the most widely used geographic information systems tool. This paper emphasizes a new approach to automated testing, where, rather than focus on developing a monolithic tool to extend, the aim is to convert a test harness into a language extension. This approach makes testing not a separate activity to be performed using a tool, but as natural to users of the language of the system under test as is the use of domain-specific libraries such as ArcPy, NumPy, or QIIME, in their domains. TSTL is a language and tool infrastructure, but is also a way to bring testing activities under the control of an existing programming language in a simple, natural way.

Testing Web Services has become the spotlight of software engineering as an important means to assure the quality of Web application. Due to lacking of graphic interface and source code, Web services need an automated testing method, which is an important part in efficiently designing and generating test suite. However, the existing testing methods may lead to the redundancy of test suite and the decrease of fault-detecting ability since it cannot handle scenarios where the strengths of the different interactions are not uniform. With the purpose of solving this problem, firstly the formal tree model based on WSDL is constructed and the actual interaction relationship of each node is made sufficient consideration into, then the combinatorial testing is proposed to generate variable strength combinatorial test suite based on One-test-at-a-time strategy. At last test cases are minimized according to constraint rules. The results show that compared with conventional random testing, the proposed approach can detect more errors with the same amount of test cases which turning out to be more ideal than existing ones in size.

Usage-based testing focuses quality assurance on highly used parts of the software. The basis for this are usage profiles based on which test cases are generated. There are two fundamental approaches in usage-based testing for deriving usage profiles: either the system under test (SUT) is observed during its operation and from the obtained usage data a usage profile is automatically inferred, or a usage profile is modeled by hand within a model-based testing (MBT) approach. In this article, we propose a third and combined approach, where we automatically infer a usage profile and create a test data repository from usage data. Then, we create representations of the generated tests and test data in the test model from an MBT approach. The test model enables us to generate executable Testing and Test Control Notation version 3 (TTCN-3) and thereby allows us to automate the test execution. Together with industrial partners, we adopted this approach in two pilot studies. Our findings show that usage-based testing can be applied in practice and greatly helps with the automation of tests. Moreover, we found that even if usage-based testing is not of interest, the incorporation of usage data can ease the application of MBT.

Software development in automotive industry has bestowed greater comforts and conveniences to mankind. A remarkable progress in this field often faces a setback due to minor defects in the software. So there is recurring need for standardization and implementation of testing strategies. But the process of creation of test scripts to check if the software created complies with its specifications and requirements is both time- and resource-consuming. Generating a short but effective test suite usually requires a lot of manual work and expert knowledge. Patronizing research work in this field is the need of the hour. This paper solves the problem by using Model-Based Testing where test harness and evaluation are performed economically through automation. Simulink Design Verifier and Reactis are the tools used to carry out this purpose in Adaptive Front Light System. The resulting outputs obtained from Simulink Design Verifier and Reactis using Model-Based Testing prove that short test suites can be generated for the model where full model coverage can be achieved easily through automation. The outputs of these test cases when compared with the expected outputs confirm that the model developed is working as per the requirements.

In this study, we propose a testing environment for cyber-physical systems (CPS). In system testing for CPS, many tests are difficult to design or implement because of these systems’ many product variations. The proposed environment executes the tests and guarantees that these systems operate reliably using two methods. The first method provides easy management of test cases by managing functions to be tested and configurations to be tested separately. The second method involves automatic testing of real devices based on model checking technologies. The authors have developed a horizontal prototype of the proposed environment and confirmed its feasibility and applicability.

Background/Objectives: When client wants some change in functionality or in look/design of software, then after implementation of requirement, tester has to retest it. For this tester has to perform regression testing. It is done to verify that any other issue is not pushed into the system, and also verify existing issue if any, So, Regression testing not only is very important process when software change made but also it is very costly too. Methods: This paper proposes a new method based on ratio between number of issues found and run time of Test cases. This paper gives predilection to the original test suite so that the new suite, which is run within a time-constrained execution environment, will have a superior rate of fault detection. Findings: This paper presents a technique that prioritizes regression test cases so that the new ordered test case produces a high productivity. Application/Improvements: This paper assumes data for pre-executed test cases in the form of number of issues found and run time.

FinTech, short for ``financial technology,'' has advanced the process of transforming financial business from a traditional manual-process-driven to an automation-driven model by providing various software platforms. However, the current FinTech-industry still heavily depends on manual testing, which becomes the bottleneck of FinTech industry development. To automate the testing process, we propose an approach of black-box testing for a FinTech system with effective tool support for both test generation and test oracles. For test generation, we first extract input categories from business-logic specifications, and then mutate real data collected from system logs with values randomly picked from each extracted input category. For test oracles, we propose a new technique of priority differential testing where we evaluate execution results of system-test inputs on the system's head (i.e., latest) version in the version repository (1) against the last legacy version in the version repository (only when the executed test inputs are on new, not-yet-deployed services) and (2) against both the currently-deployed version and the last legacy version (only when the test inputs are on existing, deployed services). When we rank the behavior-inconsistency results for developers to inspect, for the latter case, we give the currently-deployed version as a higher-priority source of behavior to check. We apply our approach to the CSTP subsystem, one of the largest data processing and forwarding modules of the China Foreign Exchange Trade System (CFETS) platform, whose annual total transaction volume reaches 150 trillion US dollars. Extensive experimental results show that our approach can substantially boost the branch coverage by approximately 40%, and is also efficient to identify common faults in the FinTech system.

A goal of performance testing is to find situations when applications unexpectedly exhibit worsened characteristics for certain combinations of input values. A fundamental question of performance testing is how to select a manageable subset of the input data faster in order to automatically find performance bottlenecks in applications. We propose FOREPOST, a novel solution, for automatically finding performance bottlenecks in applications using black-box software testing. Our solution is an adaptive, feedback-directed learning testing system that learns rules from execution traces of applications. Theses rules are then used to automatically select test input data for performance testing. We hypothesize that FOREPOST can find more performance bottlenecks as compared to random testing. We have implemented our solution and applied it to a medium-size industrial application at a major insurance company and to two open-source applications. Performance bottlenecks were found automatically and confirmed by experienced testers and developers. We also thoroughly studied the factors (or independent variables) that impact the results of FOREPOST.

Software testing is one of the most important process in the software development life cycle, since it ensures the delivery of good quality software to the user. In order to make this process more efficient, automated testing is required. Automated testing offers the advantages of reduced cost and time and it is also more reliable than manual means. In automated testing, test oracle process is required to test software applications. The software applications developed using aspect oriented programming paradigm can also be tested using automated testing. Aspect oriented programming offers the advantages over object oriented programming paradigm by centralizing various cross cutting concerns in a software code, thereby improving the modularity of the code. Since the crosscutting concern is separated into a single module, maintenance of such systems becomes easy. In this paper, the artificial neural network approach is used for testing the aspect oriented programming applications by developing test oracle.

Software test processes are complex and costly. To reduce testing effort without compromising effectiveness and product quality, automation of test activities has been adopted as a popular approach in software industry. However, because test automation usually requires substantial upfront investments, automation is not always more cost-effective than manual testing. To support decision-makers in finding the optimal degree of test automation in a given project, we recently proposed a process simulation model using the System Dynamics modeling technique and used the simulation model in the context of a case study with a software company in Calgary, Canada. With the help of the simulation model, we were able to evaluate the performance of test processes with varying degrees of automation of test activities and help testers choose the most optimal cases. The goal of the earlier study was to investigate how the simulation model can help decision-makers decide whether and to what degree the company should automate their test processes. In this article, we present further details of the System Dynamics model, its usage scenarios and examples of simulation experiments independent from a specific company context. Copyright © 2015 John Wiley & Sons, Ltd.

Model-based testing has gained widespread acceptance over the last decade, not only in academia but also in industry. Despite its powerful features of abstraction and automation, most existing approaches and tools provide only limited support for regression testing. Yet regression testing, the repeated execution of selected test cases after system modification, is vital, because changes may introduce new bugs or unwanted side effects that must be avoided at all costs. Model-based testing's potential for supporting regression testing has yet to be explored, even though syntactic and semantic abstractions within software models already allow identifying changes in software systems. This change information can easily be used for test case selection. In this article, we present a model-based regression testing method based on OCL. By means of a running example using the UML Testing Profile we show how our method supports regression testing on the basis of an existing model-based testing method.

Context Event-Driven Software (EDS) is a class of software whose behavior is driven by incoming events. Web and desktop applications that respond to user-initiated events on their Graphical User Interface (GUI), or embedded software responding to events and signals received from the equipment in its operating environment are examples of EDS. Testing EDS poses great challenges to software testers. One of these challenges is the need to generate a huge number of possible event sequences that could sufficiently cover the EDS’s state space. Objective In this paper, we introduce a new six-stage testing procedure for event-driven web applications to overcome EDS testing challenges. Method The stages of the testing procedure include dividing the application based on its structure, creating functional graphs for each section, creating mutants from functional graphs, choosing coverage criteria to produce test paths, merging event sequences to make longer ones, and deriving and running test cases. We have analyzed our proposed testing procedure with the help of four metrics consisting of Fault Detection Density (FDD), Fault Detection Effectiveness (FDE), Mutation Score, and Unique Fault. Results Using this procedure, we have prepared prioritized test cases and also discovered a list of unique faults by running the suggested test cases on a sample real-world web application called Academic E-mail System. Conclusion We propose that our suggested testing procedure has some advantages such as creating functional graphs with requirements document, resolving the problem of removing infeasible test cases with these graphs and conditions on the “add edge” operator before creating mutants. But the suggested testing procedure, like any other method, had some drawbacks. Because most of the stages in the approach were performed manually, the testing time was increased.

Many large-scale software systems must service thousands or millions of concurrent requests. These systems must be load tested to ensure that they can function correctly under load (i.e., the rate of the incoming requests). In this paper, we survey the state of load testing research and practice. We compare and contrast current techniques that are used in the three phases of a load test: (1) designing a proper load, (2) executing a load test, and (3) analyzing the results of a load test. This survey will be useful for load testing practitioners and software engineering researchers with interest in the load testing of large-scale software systems.

Software testing began as an empirical activity, and remains part of engineering practice without a widely accepted theoretical foundation. The overwhelming majority of test methods are designed to find software errors, termed faults, in program source code, but not to assess software operational quality. To go beyond fault-seeking requires a theory that relates static program properties to executions. In the 1970s and 1980s, Gerhart, Howden, and others developed a sound functional theory of program testing. Then Duran and others used this theory to precisely define the notions of random testing and operational reliability. In the Gerhart-Howden-Duran theory, a program's behavior is a pure input-output mapping. This paper extends the theory to include persistent state, by adding a state space to the input space, and a state mapping to a program's output mapping. The extended theory is significantly different because test states, unlike inputs, cannot be chosen arbitrarily. The theory is used to analyze state-based testing methods, to examine the practicality of reliability assessment, and to suggest experiments that would increase understanding of the statistical properties of software.

Testing was desired to ensure the quality of software. Testing seeks to deliver quality applications within budget and time constraints. Proper test management was required for a disciplined testing process. The reporting mechanism plays an important role for scrutinizing the testing progress, assessing test results for a regimented testing process during the testing life cycle. This paper presents a report generation using JavaScript Object Notation (JSON). JavaScript Object Notation is an alternative to Extensible Markup Language and simpler to use. Proper reporting could efficiently improve the communication between several testing groups.

A software testing process that tries to uncover new bugs for an existing system from the previous test suite due to expansion of the software is known as Regression Testing. The test suite will hold the same test cases that were tested for the system in its earlier version. For regression testing, prioritizing the test cases is always a complex as well as challenging task. In fact researchers have been proposing many approaches to arrange the test cases so that the cost of the software can be reduced in terms of human labor, time, and money as well. Many such approaches have shown quite good results too. In this paper, we have proposed a new approach of prioritizing the test cases that extends hamming distance based prioritization with code coverage based techniques. Our proposed method helps to unfold the previous bugs as well as the newly arrived bugs at the early cycle of the regression testing.

testing plays an important role for evaluation of the quality of the software. Quality of the software depends upon the kind of testing strategies applied by the software tester who generates valid and invalid test cases for evaluating the quality of software. For optimizing the testing procedure, testing paths are very valuable to judge the quality of the software. From the literature, it is revealed that it is big challenge to optimize the testing paths. In the present work, the concept of genetic algorithm is used for prioritization of test cases generated from Unified Modeling Language. Testing paths are generated from activity diagram is designed for generation of test cases from design specification which will further reduce the cost of software testing. Branch coverage technique and predicate coverage methods are used for prioritization of test cases by identifying the independent paths. Information flow metric and decision node based genetic function are used through a case study.

In this article we propose a technique, called abstract testing, which replaces traditional test cases by abstract test cases. By doing so, fewer test cases are needed, and they are linked more closely to the requirements. Abstract tests can be considered as verification scenarios on the source code level which are derived from the requirements. Checking verification scenarios against the source code is done automatically using a software model checker. We also suggest a migration path from traditional tests to abstract test cases, which provides a smooth transition towards this new technique. Finally, we demonstrate feasibility of abstract testing by a case study from the automotive systems domain.

Software testing is an important activity in software development life cycle. Testing includes running a program on a set of test cases and comparing seen results with expected results. Automated testing encompasses all automation efforts across software testing lifecycle, with focus on automating system testing efforts and integration. Automated testing brings plenty of benefits that speeding up test running time, increasing accuracy of testing process and minimizing costs in different parts of system are three superior features of it. Maintenance and development of test automation tools are not as easy as traditional testing due to unexplored issues which need more examinations. Automated test patterns have been presented to mitigate some problems happening by automated testing and improve efficiency. This paper aims to investigate into automatic testing and automated test patterns. Also, demonstrates behaviour of applying an automated test pattern on a complex object. Results show during choosing an automated pattern to run, we should consider test structure especially level of test object complexity otherwise inconsistency may happen.

Software Testing is one of the expensive activities in Software development lifecycle but it used in to assure the quality of the Software. One of the important focuses in software engineering process is lessen the effort of software testing and cut down the cost & time of testing. This paper presents a unified method for reducing the test cases by adopting the hypothesis of Genetic algorithm such as crossover, mutation and crowding distance to elucidate the given problem and results in the minimum set of test cases with the ability to cover all the faults in minimum time.

An execution system of distributed performance testing is designed in this paper, which provides a mechanism of node communication, test script deployment, test scheduling, execution-driving and test result collection in distributed environment. A workload model is established, by which testers can describe the performance testing requirement. A performance testing framework is given, which simulates user behaviors in real environment based on virtual users so as to generate workload from the system under test (SUT). It can control the execution of virtual users by TTCN-3 standard interface. After executing the performance testing, test report is generated by extracting log. A method of generating performance test-case is studied by reusing functional test scripts. By executing performance testing on an online bookstore, this paper demonstrates the availability of the method of reusing TTCN-3 functional test scripts and the capability of distributed performance testing system established.

 Abstract—Software testing plays a vital role in developing software that is freedom from bugs and defects. Manual test is a cost and time consuming process although it may find many defects in a software application. If the testing process could be automated, the cost of developing software could be reduced obviously. The most critical part of the testing process is generation of test cases. Model based test path generation approaches identify faults in the implementation at early design phase, reduce the software development time, and inspire developer to improve design quality. The proposed approach focuses on Model Based testing for automatic best test path generation. In this approach, the proposed system generates all possible test paths and optimizes a test path from these paths. The proposed system will reduce the processing time by using XMI based activity diagram, validate the generated test paths with Cyclomatic Complexity and choose the best test path without dependence on others.

This paper demonstrates a case study of combining two rigorous software specification and testing methods, namely sequence based software specification and Markov chain usage-based statistical testing and their supporting tools, all developed by the University of Tennessee Software Quality Research Laboratory, and integrating them with an automated testing tool selected for the chosen application to provide fully automated statistical testing and software certification as an economical and feasible means to achieve high product quality. Working on any non-trivial real world problem requires considerable efforts be made to work out all the details needed for fully automated testing with no human intervention. At the end of this process we have the ability of running large numbers of tests, as well as an automated testing facility for low-cost, quick-turnaround testing and re-testing. Our experiences demonstrate a pathway towards lowered cost of testing and improved product quality.

Summary This paper includes the importance of performance testing of web applications and analyses the application's bottleneck based on hardware, software and resource utilization. The main focus of this paper is performance testing based on load test. It has been seen that everyone wants a very fast application, but at the same time, reliability of the application takes an important role; so, customer's satisfaction is the push for performance testing of a given application. Performance testing determines how fast some aspects of the system perform under a pre-defined workload. The performance testing is calculated by analyzing the production, which comes from the application hosted on the server. The main performance is calculated when business is at its peak by its hits.

With the widespread use of embedded software, the embedded software testing has become an indispensable part of the development process. The current representative tools for testing embedded software include ADS2 made by TechSAT, RT-LAB developed by OpalRT, and GESTE developed by Beihang University. However, their degree of automation is inadequate. Therefore, this paper studies an automatic testing framework for embedded software. It through Interface Protocol Modeling Module, Test Profile Modeling Module, Test Data Generation Module and T est Script Generation Module, to achieve platform-related test script generation. The framework greatly reduces the time for testers to design test cases.

Testing is a fundamental task to ensure software quality. Regression testing aims to ensure that changes to software do not introduce new failures. As resources are often limited and testing comprises a vast amount of test cases, different regression strategies have been proposed to reduce testing effort by selecting or prioritizing important test cases, e.g., code coverage (to ensure a sufficient testing depth). However, in system testing, source code is often not available creating a black-box system. In this paper, we introduce an automated, multi-objective test case selection technique in black-box systems using genetic algorithms. We define seven different objectives, based on meta-data, allowing a flexible test case selection for a variety of systems. For evaluation, we apply our technique on two different subject systems assessing the feasibility and suitability of our test case selection approach. Results indicate that our approach is applicable based on different data available and is able to outperform random test case selection and retest-all.

The practitioner interested in reducing software verification effort may found herself lost in the many alternative definitions of Graphical User Interface (GUI) testing that exist and their relation to the notion of system testing. One result of these many definitions is that one may end up testing the same parts of the Software Under Test (SUT), specifically the application logic, twice. To clarify two important testing activities and avoid duplicate testing effort, this paper empirically evaluates to what extent GUI tests exercise the application logic of the software under test (and not only the GUI code). Experimental results show that GUI tests do not necessarily entirely exercise application logic functionality, at least not as much as system tests directly interacting with application logic code.

Statistical testing based on a Markov chain usage model, as a rigorous testing method, has been around for more than two decades. Through the comprehensive application of statistical science to the testing of software, it provides audit trails of evidence to support correctness arguments for a software-intensive system as well as a decision that the system is of requisite quality for its intended use. This paper reports a real-world case study in which we applied standard statistical testing practices to the phone flag copy testing problem at the site of our industrial collaborator, and presents our solution from problem formalization, usage modeling and model analysis to test case generation and analysis. Our results helped evaluate the coverage of a heuristically generated test suite, and shed light on what other test cases to craft/generate to improve test coverage and adequacy, as well as reliability estimates both at the arc (usage event) level and at the path (system) level.

Testing is a very essential activity for software development. It is the act of executing a software product in order to validate whether it behaves as intended and identify possible malfunctions. Studies have shown that testing makes up more than 50% of the development cost. Besides, much effort and emphasis have now been placed on tasks related to automation with the purpose of reducing cost and the participation of the human element in software testing activities. Testing is still a human-based activity. Therefore, efficient ways of testing software products for quality assurance will require a better and more comprehensive understanding of testers' feelings, perceptions and motivations, in this paper referred to as Tester Experience (TX). Thus, the better the tester's experience during the software testing process, the better the result. TX can be defined as a means of capturing how testers think and feel about their activities within the software testing environment, with the assumption that an improvement of the tester's experience has positive impact on quality assurance. This paper motivates the importance of TX, highlights related approaches from other domains, proposes a definition based on similar concepts in other domains, and proposes future research activities.

Testing is an important activity in engineering of industrial software. For such software, testing is usually performed manually by handcrafting test suites based on specific design techniques and domain-specific experience. To support developers in testing, different approaches for producing good test suites have been proposed. In the last couple of years combinatorial testing has been explored with the goal of automatically combining the input values of the software based on a certain strategy. Pairwise testing is a combinatorial technique used to generate test suites by varying the values of each pair of input parameters to a system until all possible combinations of those parameters are created. There is some evidence suggesting that these kinds of techniques are efficient and relatively good at detecting software faults. Unfortunately, there is little experimental evidence on the comparison of these combinatorial testing techniques with, what is perceived as, rigorous manually handcrafted testing. In this study we compare pairwise test suites with test suites created manually by engineers for 45 industrial programs. The test suites were evaluated in terms of fault detection, code coverage and number of tests. The results of this study show that pairwise testing, while useful for achieving high code coverage and fault detection for the majority of the programs, is almost as effective in terms of fault detection as manual testing. The results also suggest that pairwise testing is just as good as manual testing at fault detection for 64% of the programs.

Crowdsourced testing is gaining a lot of attention these days. Crowdsourced testing utilizes testers which subscribe to an external or internal crowdsourcing platform. Usually these testers are distributed across geographies. Thus, such testing can be treated as a form of distributed testing. Crowdsourced testing, quite often, is used to perform exploratory testing in which testers test the features as per their wish. Such testing results in redundant testing efforts due to the lack of awareness of other testers' activities, thus does not yield the full benefits of crowdsourced testing in terms of speed and coverage. Moreover, as each tester possesses a testing style, the present model of crowdsourced testing does not fully utilize tester's strength or style. In this paper, we study various redundancies involved in a distributed testing. We also study testers' behavior to understand various testing styles. The study finds that there exists a lot of redundancies in testing efforts. The study also observes that testers indeed have different testing styles which should be understood more deeply for engaging them better. Our study shows that there is need for tools and strategies for allocating testing tasks that leverages testers' testing style and creates more awareness among the testers.

This paper aims to present a new method for automated testing of industrial equipment, used primarily in the automotive field. The underlying concept of this work consists in developing an application to test the equipment (DUT — device under test) that automated tests need to be performed on, from the point of view of the hardware system, and in terms of the software loaded on such a device. The developed application also aims to function both as a concept HiL (Hardware in the Loop), and as a SiL (Software in the Loop), providing at the end of the testing routine, the test reports on the method, and tests that the equipment has passed. The application allows monitoring and controlling, in real time, tested devices, using a communication protocol — high speed CAN. Real-time monitoring is done both by the high-speed CAN protocol, for debugging the application, and through video capture, allowing users to perform tests both in testing laboratories and in special laboratories, such as EMC or climatic chambers, where the presence of a test engineer is, many times, restricted.