Search

Gamification is an established practice in Software Engineering to increase effectiveness and engagement in many practices. This manuscript provides a characterisation of the application of gamification to the Software Testing area. Such practice in fact reportedly suffers from low engagement by both personnel in industrial contexts and learners in educational contexts. Our goal is to identify the application areas and utilised gamified techniques and mechanics, the provided benefits and drawbacks, as well as the open challenges in the field. To this purpose, we conducted a Multivocal Literature Review to identify white and grey literature sources addressing gamified software testing. We analysed 73 contributions and summarised the most common gamified mechanics, concepts, tools and domains where they are mostly applied. We conclude that gamification in software testing is mostly applied to the test creation phase with simple white-box unit or mutation testing tools, and is mostly used to foster good behaviours by promoting the testers’ accomplishment. Key research areas and main challenges in the field are: careful design of tailored gamified mechanics for specific testing techniques; the need for technological improvements to enable crowdsourcing, cooperation, and concurrency; the necessity for empirical and large-scale evaluation of the benefits delivered by gamification mechanics.

Business Process Modeling is a skill that is becoming sought after for computer engineers, with Business Process Modeling Notation (BPMN) being one example of the tools used in modeling activities. Students of the Master of Computer Engineering course at Politecnico di Torino learn about BPMN in dedicated courses but often underperform on BPMN-related exercises due to difficulties understanding how to model processes. In recent years, there has been a surge of studies that employ gamification (using game elements in non-recreative contexts to obtain benefits) as a tool in Computer Engineering education to increase students’ engagement with the learning process. This study aims to use the principles of gamification to design a supplementary learning tool for the teaching of information systems technology. In particular, to improve student understanding and use of BPMN diagrams. This study also analyzes the usability and motivation of the participants in using different game elements in increasing student motivation and performance. As part of the study, a prototype web application was developed, which implemented three different designs, each incorporating different game elements relating to either progress, competition, or rewards. An evaluation was then conducted on the prototype to evaluate the performance of the practitioners in performing BPMN modeling tasks with the gamified tool, the usability of the proposed mechanics and the enjoyment of the individual game mechanics that were implemented. With the usage of the gamified tool, the users of the experimental sample were able to complete BPMN modeling tasks with performances compatible with estimates made through expert judgement (i.e., gamification had no negative effect on performance), and were motivated to check the correctness of their models many times during the task execution. The system was evaluated as highly usable (85.8 System Usability Score); the most enjoyed game elements were rewards, levels, progress bars and aesthetics.

Automated Graphical User Interface (GUI) testing is a fundamental part of the Verification and Validation process of every software, but it is often linked to notable maintenance costs, especially for mobile applications. The literature reports a general lack of automated testing adoption among mobile developers in the industry. In this article, we present the outcome of seven interviews centered on how companies automate the testing process of mobile applications. The interviews confirmed that automated testing is still not widely adopted and rarely formalized by industry, with manual testing being still the primary form of testing. Test fragility and evolution of the user interface are seen as a relevant issue by developers, with a cost of around 30% of the overall maintenance performed on test suites. Some clear shared needs emerged during our interviews that can be considered hints for the added research effort from academia in meeting the needs of industry.

Several techniques exist for mobile test automation, from script-based techniques to automated test generation based on GUI models. Most techniques fall short in being adopted extensively by practitioners because of the very costly definition (and maintenance) of test cases. We present a novel testing framework for Android apps that allows a developer to write effective test scripts without having to know the implementation details and the user interface of the app under test. The main goal of the framework is to generate adaptive tests that can be executed on a significant number of apps, or different releases of the same app, without manual editing of the tests. The frameworks consists of: (1) a Test Scripting Language, that allows the tester to write generic test scripts tailored to activity and app categories; (2) a State Graph Modeler, that creates a model of the app's GUI, identifying activities (i.e., screens) and widgets; (3) an app classifier that determines the type of application under test; (4) an activity classifier that determines the purpose of each screen; (5) a test adapter that executes test scripts that are compatible with the specific app and activity, automatically tailoring the test scripts to the classes of the app and the activities under test. We evaluated empirically the components of our testing framework. The classifiers were able to outperform available approaches in the literature. The developed testing framework was able to correctly adapt high-level test cases to 28 out of 32 applications, and to reduce the LOCs of the test scripts of around 90%. We conclude that machine learning can be fruitfully applied to the creation of high-level, adaptive test cases for Android apps. Our framework is modular in nature and allows expansions through the addition of new commands to be executed on the classified apps and activities.

Evidence from empirical studies suggests that mobile applications are not thoroughly tested as their desktop counterparts. In particular, GUI testing is generally limited. Like web-based applications, mobile apps suffer from GUI testing fragility, i.e., GUI test classes failing or needing interventions because of modifications in the AUT or in its GUI arrangement and definition. The objective of our study is to examine the diffusion of test classes created with a set of popular GUI Automation Frameworks for Android apps, the amount of changes required to keep test classes up to date, and the amount of code churn in existing test suites, along with the underlying modifications in the AUT that caused such modifications. We defined 12 metrics to characterize the evolution of test classes and test methods, and a taxonomy of 28 possible causes for changes to test code. To perform our experiments, we selected six widely used open-source GUI Automation Frameworks for Android apps. We evaluated the diffusion of the tools by mining the GitHub repositories featuring them, and computed our set of metrics on the projects. Applying the Grounded Theory technique, we then manually analyzed diff files of test classes written with the selected tools, to build from the ground up a taxonomy of causes for modifications of test code. We found that none of the considered GUI automation frameworks achieved a major diffusion among open-source Android projects available on GitHub. For projects featuring tests created with the selected frameworks, we found that test suites had to be modified often – specifically, about 8% of developers’ modified LOCs belonged to test code and that a relevant portion (around 50% on average) of those modifications were induced by modifications in GUI definition and arrangement. Test code written with GUI automation fromeworks proved to need significant interventions during the lifespan of a typical Android open-source project. This can be seen as an obstacle for developers to adopt this kind of test automation. The evaluations and measurements of the maintainance needed by test code wrtitten with GUI automation frameworks, and the taxonomy of modification causes, can serve as a benchmark for developers, and the basis for the formulation of actionable guidelines and the development of automated tools to help mitigating the issue.

System testing through the Graphical User Interface (GUI) is a valuable form of Verification & Validation for modern applications, especially in graphically-intensive domains like web and mobile applications. However, the practice is often overlooked by developers mostly because of its costly nature and the absence of immediate feedback about the quality of test sequence. This paper describes a proposal for the Gamification of exploratory GUI testing. We define - in a tool and domain-agnostic way - the basic concepts, a set of metrics, a scoring scheme and visual feedbacks to enable a gamified approach to the practice; we finally discuss the potential implications and envision a roadmap for the evaluation of the approach. &nbsp

open access

Layout-based (2nd Generation) and Visual (3rd Generation) GUI testing are two very common approaches for mobile application testing. The two techniques expose complementary advantages and drawbacks, and the literature on GUI Testing has highlighted the benefits of an approach based on a translation from one generation to the other. The objective of this work is to provide an improvement to our prototype tool, TOGGLE, designed to translate 2nd Generation test suites, written with the Espresso framework, to 3rd Generation ones that can be run by the EyeAutomate and Sikuli tool. We extended TOGGLE by adding (1) support for context-based gestures, performed through the scrollTo and onData commands, and (2) support for the combination of Layout-based locators with logical operators. We evaluated the new version of the tool on five different experimental subjects. For each of the applications, 30 test cases were developed and automatically translated with TOGGLE+. We observed an increase of 68% of translatable test cases when transitioning from the previous prototype to the current version of the tool. The generated Visual test cases also proved to have high robustness, with flakiness of just 2% (i.e., 98% correct executions).

Rust is an innovative programming language initially implemented by Mozilla, developed to ensure high performance, reliability, and productivity. The final purpose of this study consists of applying a set of common static software metrics to programs written in Rust to assess the verbosity, understandability, organization, complexity, and maintainability of the language. To that extent, nine different implementations of algorithms available in different languages were selected. We computed a set of metrics for Rust, comparing them with the ones obtained from C and a set of object-oriented languages: C++, Python, JavaScript, TypeScript. To parse the software artifacts and compute the metrics, it was leveraged a tool called rust-code-analysis that was extended with a software module, written in Python, with the aim of uniforming and comparing the results. The Rust code had an average verbosity in terms of the raw size of the code. It exposed the most structured source organization in terms of the number of methods. Rust code had a better Cyclomatic Complexity, Halstead Metrics, and Maintainability Indexes than C and C++ but performed worse than the other considered object-oriented languages. Lastly, the Rust code exhibited the lowest COGNITIVE complexity of all languages. The collected measures prove that the Rust language has average complexity and maintainability compared to a set of popular languages. It is more easily maintainable and less complex than the C and C++ languages, which can be considered syntactically similar. These results, paired with the memory safety and safe concurrency characteristics of the language, can encourage wider adoption of the language of Rust in substitution of the C language in both the open-source and industrial environments.

Context:Kotlin is a new programming language representing an alternative to Java; they both target the same JVM and can safely coexist in the same application. Kotlin is advertised as capable to solve several known limitations of Java. Recent surveys show that Kotlin achieved a relevant diffusion among Java developers. Goal:We planned to empirically assess a few typical promises of Kotlin w.r.t. known Java’s limitations, in terms of development effectiveness, maintainability, and ease of development. Method:Our experiment involved 27 teams of 4 people each that completed a set of maintenance tasks (both defect correction and feature addition) on Android apps written in either Java or Kotlin. In addition to the number of fixed defects, effort, and code size, we collected, though a questionnaire, the participants’ perceptions about the avoidance of known pitfalls. Results:We did not observe any significant difference in terms of maintainability between the two languages.We found a significant difference regarding the amount of code written, which constitutes evidence of better conciseness of Kotlin. Concerning ease of development, the frequency of NullPointerExceptions reported by the subjects was significantly lower when developing in Kotlin. On the other hand, no significant difference was found in the occurrence of other common Java pitfalls. Finally, the IDE support was deemed better for Java than Kotlin. Conclusions:Some of the promises of Kotlin to be a ”better Java” have been confirmed by our empirical assessment. Evidence suggests that the effort in transitioning to Kotlin can provide some advantages to Java developers, especially regarding code conciseness.Our results may serve as the basis for further investigations on the properties of the language.

Energy efficiency is an increasingly important non-functional property of software, especially when it runs on mobile or IoT devices. An engineering approach demands a reliable measurement of energy consumption of software while performing computational tasks. In this paper, we describe PowTrAn, an R package supporting the analysis of the power traces of a device executing software tasks. The tool analyzes traces with embedded markers, a non-invasive technique that enables gauging software efficiency based on the energy consumed by the whole device. The package effectively handles large power traces, detects work units, and computes correct energy measures, even in noisy conditions, such as those caused by multiple processes working simultaneously. PowTrAn was validated on applications in realistic conditions and multiple hardware configurations. PowTrAn also provides data visualization that helps the user to assess the measurement consistency, and it also helps to highlight possible energy outliers.

Software maintainability is a crucial property of software projects. It can be defined as the ease with which a software system or component can be modified to be corrected, improved, or adapted to its environment. The software engineering literature proposes many models and metrics to predict the maintainability of a software project statically. However, there is no common accordance with the most dependable metrics or metric suites to evaluate such nonfunctional property. The goals of the present manuscript are as follows: (i) providing an overview of the most popular maintainability metrics according to the related literature; (ii) finding what tools are available to evaluate software maintainability; and (iii) linking the most popular metrics with the available tools and the most common programming languages. To this end, we performed a systematic literature review, following Kitchenham’s SLR guidelines, on the most relevant scientific digital libraries. The SLR outcome provided us with 174 software metrics, among which we identified a set of 15 most commonly mentioned ones, and 19 metric computation tools available to practitioners. We found optimal sets of at most five tools to cover all the most commonly mentioned metrics. The results also highlight missing tool coverage for some metrics on commonly used programming languages and minimal coverage of metrics for newer or less popular programming languages. We consider these results valuable for researchers and practitioners who want to find the best selection of tools to evaluate the maintainability of their projects or to bridge the discussed coverage gaps for newer programming languages.