Search

This paper reviews existing work in software engineering that applies statistical causal inference methods. These methods aim at estimating causal effects from observational data. The review covers 32 papers published between 2010 and 2022. Our results show that the application of statistical causal inference methods is relatively recent and that the corresponding research community remains relatively fragmented., Comment: 38 pages, 12 tables, 9 figures, submitted to Information and Software Technology

Context: Requirements engineering researchers have been experimenting with machine learning and deep learning approaches for a range of RE tasks, such as requirements classification, requirements tracing, ambiguity detection, and modelling. However, most of today's ML/DL approaches are based on supervised learning techniques, meaning that they need to be trained using a large amount of task-specific labelled training data. This constraint poses an enormous challenge to RE researchers, as the lack of labelled data makes it difficult for them to fully exploit the benefit of advanced ML/DL technologies. Objective: This paper addresses this problem by showing how a zero-shot learning approach can be used for requirements classification without using any labelled training data. We focus on the classification task because many RE tasks can be framed as classification problems. Method: The ZSL approach used in our study employs contextual word-embeddings and transformer-based language models. We demonstrate this approach through a series of experiments to perform three classification tasks: (1)FR/NFR: classification functional requirements vs non-functional requirements; (2)NFR: identification of NFR classes; (3)Security: classification of security vs non-security requirements. Results: The study shows that the ZSL approach achieves an F1 score of 0.66 for the FR/NFR task. For the NFR task, the approach yields F1~0.72-0.80, considering the most frequent classes. For the Security task, F1~0.66. All of the aforementioned F1 scores are achieved with zero-training efforts. Conclusion: This study demonstrates the potential of ZSL for requirements classification. An important implication is that it is possible to have very little or no training data to perform classification tasks. The proposed approach thus contributes to the solution of the long-standing problem of data shortage in RE., Comment: 60 pages, 22 tables, 1 figure

Context: Boundary Artefacts (BAs) support software development activities in many aspects because it carries lots of information in the same object that can be used and interpreted by several social groups within an organisation. When the BAs are inconsistent regarding their content, such as many meanings or lack of contextual information, their efficiency is reduced because stakeholders will not trust them. Objective: This study aimed to understand the implications of differences in the perception of trust on software projects and their influence on stakeholders’ behaviour. Methods: We conducted an exploratory case study to observe the creation and utilisation of one specific BA and the implications of differences in trust and their influence on stakeholders’ behaviour. Results: Our investigation has shown that practitioners adding and adjusting existing content do not entirely understand the stakeholders’ needs. Together with the partial management of the content, trust is impacted. When the content of BAs does not meet the trust factors, specifically reliability and predictability, the stakeholders cannot execute their tasks appropriately, and several implications affect the software development project. Additionally, they create workarounds to supply their needs. Conclusion: The differences in trust in BAs affect software projects in different areas of the organisation and interfere with the task execution of various stakeholders. The decrease in trust results from inconsistencies in the content associated with the lack of management of the BA. A structured strategy for representing and managing a BA’s content seems appropriate to increase trust levels and efficiency.

Context: Double-counting in a literature review occurs when the same data, population, or evidence is erroneously counted multiple times during synthesis. Detecting and mitigating the threat of double-counting is particularly challenging in tertiary studies. Although this topic has received much attention in the health sciences, it seems to have been overlooked in software engineering. Objective: We describe issues with double-counting in tertiary studies, investigate the prevalence of the issue in software engineering, and propose ways to identify and address the issue. Method: We analyze 47 tertiary studies in software engineering to investigate in which ways they address double-counting and whether double-counting might be a threat to validity in them. Results: In 19 of the 47 tertiary studies, double-counting might bias their results. Of those 19 tertiary studies, only 5 consider double-counting a threat to their validity, and 7 suggest strategies to address the issue. Overall, only 9 of the 47 tertiary studies, acknowledge double-counting as a potential general threat to validity for tertiary studies. Conclusions: Double-counting is an overlooked issue in tertiary studies in software engineering, and existing design and evaluation guidelines do not address it sufficiently. Therefore, we propose recommendations that may help to identify and mitigate double-counting in tertiary studies. © 2023 The Author(s)

Context: Burnout is a work-related syndrome that, similar to many occupations, influences most software developers. For decades, studies in software engineering(SE) have explored the causes of burnout and its consequences among IT professionals. Objective: This paper is a systematic mapping study (SMS) of the studies on burnout in SE, exploring its causes and consequences, and how it is studied (e.g., choice of data). Method: We conducted a systematic mapping study and identified 92 relevant research articles dating as early as the early 1990s, focusing on various aspects and approaches to detect burnout in software developers and IT professionals. Results: Our study shows that early research on burnout was primarily qualitative, which has steadily moved to more quantitative, data-driven in the last decade. The emergence of machine learning (ML) approaches to detect burnout in developers has become a de-facto standard. Conclusion: Our study summarises what we now know about burnout, how software artifacts indicate burnout, and how machine learning can help its early detection. As a comprehensive analysis of past and present research works in the field, we believe this paper can help future research and practice focus on the grand challenges ahead and offer necessary tools.

Context Usability issues of security Application Programming Interfaces (APIs) are a main factor for mistakes programmers make that could result in introducing security vulnerabilities into applications they develop. This has become a common problem as there is no methodology to evaluate the usability of security APIs. A usability evaluation methodology for security APIs would allow API developers to identify usability issues of security APIs and fix them. A Cognitive Dimensions Framework (CDF) based usability evaluation methodology has been proposed in previous research to empirically evaluate the usability of security APIs. Objective In this research, we evaluated the proposed CDF based methodology through four security APIs (Google Authentication API, Bouncy Castle light weight Crypto API, Java Secure Socket Extension API, OWASP Enterprise Security API). Method We conducted four experiments where in each experiment we recruited programmers and they completed a programming task using one of the four security APIs. Participants’ feedback on each cognitive dimension of the particular API was collected using the cognitive dimensions questionnaire. Usability issues of each API was identified based on this feedback. Results Results of the four experiments revealed that over 83% of the usability issues in a security API could be identified by this methodology with a considerably good validity and reliability. Conclusion The proposed CDF based usability evaluation methodology provides a good platform to conduct usability evaluation for security APIs.

Context Over the past 50 years of Software Engineering, numerous studies have acknowledged the importance of human factors. However, software developers’ emotions are still an area under investigation and debate that is gaining relevance in the software industry. Objective In this study, a systematic literature review (SLR) was carried out to identify, evaluate, and synthesize research published concerning software developers’ emotions as well as the measures used to assess its existence. Method By searching five major bibliographic databases, authors identified 7172 articles related to emotions in Software Engineering. We selected 66 of these papers as primary studies. Then, they were analyzed in order to find empirical evidence of the intersection of emotions and software engineering. Results Studies report a total of 40 discrete emotions but the most frequent were: anger, fear, disgust, sadness, joy, love, and happiness. There are also 2 different dimensional approaches and 10 datasets related to this topic which are publicly available on the Web. The findings also showed that self-reported mood instruments (e.g., SAM, PANAS), physiological measures (e.g., heart rate, perspiration) or behavioral measures (e.g., keyboard use) are the least reported tools, although, there is a recognized intrinsic problem with the accuracy of current state of the art sentiment analysis tools. Moreover, most of the studies used software practitioners and/or datasets from industrial context as subjects. Conclusions The study of emotions has received a growing attention from the research community in the recent years, but the management of emotions has always been challenging in practice. Although it can be said that this field is not mature enough yet, our results provide a holistic view that will benefit researchers by providing the latest trends in this area and identifying the corresponding research gaps.

Context In the Model Driven Development (MDD) approach, model transformations are responsible for the semi-automation of software development process converting models between different abstraction levels. The development of model transformations involves a complexity inherent to the transformation domain, in addition to the complexity of software development in general. Therefore, the construction of model transformations requires software engineering feature such as processes and languages to facilitate its development and maintenance. Objective This paper presents a framework to develop unidirectional relational model transformation using the MDD approach itself, which integrates: (i) a software development process suitable for the model transformation domain (ii) a Domain specific language for transformation modeling (iii) a transformation chain, to (semi) automate the proposed process, and (iv) a development environment to support it. Methods The proposal systematizes the development of model transformation, following the MDD principles. An iterative and incremental process guides transformation development from requirement specification to transformation codification. The proposal has been evaluated through a case study and a controlled experiment. Results The framework enables model transformation specification at a high abstraction level and (semi) automatically transforms it into models at a low abstraction level until the transformation code. The results of the case study showed that people with different levels of knowledge of MDD, or without experience in transformation languages, were able to develop transformations through the framework and generated executable code. Conclusions The framework integrates the essential elements involved in the development of model transformation and enables the abstraction of technological details. The results of the case study and controlled experiment showed the feasibility of the proposal and its use in dealing with the complexity involved in model transformation development.

Context: Software maintenance is expensive and so anything that can be done to reduce its cost is potentially of huge benefit. However, it is recognised that some maintenance, especially refactoring, can be automated. Given the number of possible refactorings and combinations of refactorings, a search-based approach may provide the means to optimise refactorings.Objective: This paper describes the investigation of a many-objective genetic algorithm used to automate software refactoring, implemented as a Java tool, MultiRefactor. Method: The approach and tool is evaluated using a set of open source Java programs. The tool contains four separate measures of software looking at the software quality as well as measures of code priority, refactoring coverage and element recentness. The many-objective algorithm combines the four objectives to improve the software in a holistic manner. An experiment has been constructed to compare the many-objective approach against a mono-objective approach that only uses a single objective to measure software quality. Different permutations of the objectives are also tested and compared to see how well the different objectives can work together in a multi-objective refactoring approach. The eight approaches are tested on six different open source Java programs. Results: The many-objective approach is found to give better objective scores on average than the mono-objective approach and in less time. However, the priority and element recentness objectives are both found to be less successful in multi/many-objective setups when they are used together.Conclusion: A many-objective approach is suitable and effective for optimising automated refactoring to improve quality. Including other objectives does not unduly degrade the quality improvements, but is less effective for those objectives than if they were used in a mono-objective approach.

[Context] Complex and heterogeneous software systems need to be monitored as their full behavior often only emerges at runtime, e.g., when interacting with other systems or the environment. Software monitoring approaches observe and check properties or quality attributes of software systems during operation. Such approaches have been developed in diverse communities for various kinds of systems and purposes. For instance, requirements monitoring aims to check at runtime whether a software system adheres to its requirements, while resource or performance monitoring collects information about the consumption of computing resources by the monitored system. Many venues publish research on software monitoring, often using diverse terminology, and focusing on different monitoring aspects and phases. The lack of a comprehensive overview of existing research often leads to re-inventing the wheel. [Objective] We provide a domain model to structure and systematize the field of software monitoring, starting with requirements and resource monitoring. [Method] We developed an initial domain model based on (i) our extensive experiences with requirements and resource monitoring, (ii) earlier efforts to develop a comparison framework for monitoring approaches, and (iii) an earlier systematic literature review on requirements monitoring frameworks. We then systematically analyzed 47 existing requirements and resource monitoring approaches to iteratively refine the domain model and to develop a reference architecture for software monitoring approaches. [Results] Our domain model covers the key elements of monitoring approaches and allows analyzing their commonalities and differences. Together with the reference architecture, our domain model supports the development of integrated monitoring solutions. We provide details on 47 approaches we analyzed with the model to assess its coverage. We also evaluate the reference architecture by instantiating it for five different monitoring solutions. [Conclusions] We conclude that requirements and resource monitoring have more commonalities than differences, which is promising for the future integration of existing monitoring solutions.

Context In the last few years, the exigency of mobile devices has proliferated to prodigious heights. The process of developing the mobile software/application proceeds amidst testing phase to verify the correctness of the mobile app. The estimation of testing plays a vital role in the effective completion of testing. Objective To identify how estimation of test effort for mobile applications is distinct from other software via published literature and from mobile software organizations. Second is to recognize different issues in adapting traditional test estimation methods to the mobile domain and if suggestions from survey results could be helpful in providing an improved test estimation model for mobile applications. Method A systematic literature review is conducted followed by a survey through an online questionnaire filled from experienced mobile application developers and testers. Results The results from SLR cover identification of mobile app specific characteristics and reports test effort estimation techniques in the mobile domain. Findings from survey corroborate that a) Function Point/Test Point Analysis is highly adapted traditional test estimation technique to mobile domain; b) Challenges like uncertain requirements, no tool support for test estimation, complexity in testing, client miscommunication etc. are reported; c)Suggestions to improve test estimation process include proper test planning, adoption of agile methodology, healthier communication among client, developer, and tester etc.; d) On the basis of responses, Analytical Hierarchical Process (AHP) identifies “Diverse Devices and OS” along with “Type of App” as highly influential mobile app characteristic on the test estimation process. Conclusion Results conclude that the importance of identified mobile app characteristics from SLR cannot be ignored in the estimation process of mobile software testing. There might be a possibility to improve existing test estimation techniques for mobile apps by giving weight to mobile app specific characteristics and by considering suggestions from experienced developers and testers.

Context: Goal modeling is an important instrument for the elicitation, specification, analysis, and validation of early requirements. Goal models capture hierarchical representations of stakeholder objectives, requirements, possible solutions, and their relationships to help requirements engineers understand stakeholder goals and explore solutions based on their impact on these goals. To reuse a goal model and benefit from the strengths of goal modeling, we argue that it is necessary (i) to make sure that analysis and validation of goal models is possible through reuse hierarchies, (ii) to provide the means to delay decision making to a later point in the reuse hierarchy, (iii) to take constraints imposed by other modeling notations into account during analysis, (iv) to allow context dependent information to be modeled so that the goal model can be used in various reuse contexts, and (v) to provide an interface for reuse. Objective: In this two-part systematic literature review, we (i) evaluate how well existing goal modeling approaches support reusability with our five desired characteristics of contextual and reusable goal models, (ii) categorize these approaches based on language constructs for context modeling and connection to other modeling formalisms, and then (iii) draw our conclusions on future research themes. Method: Following guidelines by Kitchenham, the review is conducted on seven major academic search engines. Research questions, inclusion criteria, and categorization criteria are specified, and threats to validity are discussed. A final list of 146 publications and 34 comparisons/assessments of goal modeling approaches is discussed in more detail. Results: Five major research themes are derived to realize reusable goal models with context dependent information. Conclusion: The results indicate that existing goal modeling approaches do not fully address the required capabilities for reusability in different contexts and that further research is needed to fill this gap in the landscape of goal modeling approaches.

Context: Performance is a key quality consideration for large-scale software systems which supports thousands of concurrent users. Load testing is an integral part of the development lifecycle and is used to address performance issues before deploying the system in production. But, how do we validate the output reported by load testing tools? Little’s Law is useful for validating the accuracy of load testing output. Though IT industry is flooded with various enterprise and open source tools for load testing, but they do not offer support for validation of its result using Little’s Law. Manual validation is time intensive and infeasible with increase in the complexity of testing scripts. Objective: In this paper we provide a Little’s law-based validation framework which will enable the researchers and industry practitioners to validate load testing results. The implementation of the framework is also demonstrated. Method: To understand the constructs commonly used in load test scripts, we analysed scripts of two open source benchmark applications and eight large-scale software systems used in industry. We found that transactions are arranged using control flow patterns like sequential, loop and conditional. Based on the analysis, we devised the framework. Results: The efficacy of the proposed framework is successfully evaluated on two systems - an open source Dell DVD Store benchmarking application and a real-world large scale system used in industry. The framework is independent of load testing tool used and can be used with complex testing scripts. Conclusions There are no known frameworks or inbuilt support in existing load testing tools for guiding practitioners on applying Little’s Law using output generated by tools. We address this significant gap by providing a framework which combines information from test scripts/reports and Little’s Law to determine whether the results are valid. The provided implementation can be easily integrated with existing load testing tools.

Context Information Quality (IQ) is particularly important for organizations: they depend on information for managing their daily tasks and relying on low-quality information may negatively influence their overall performance. Despite this, the literature shows that most software development approaches do not consider IQ requirements during the system design, which leaves the system open to different kinds of vulnerabilities. Objective The main objective of this research is proposing a framework for modeling and analyzing IQ requirements for Socio-Technical Systems (STS). Method We propose STS-IQ, a goal-oriented framework for modeling and analyzing IQ requirements in their social and organizational context since the early phases of the system design. The framework extends and refines our previous work, and it consists of: (i) a modeling language that provides concepts and constructs for modeling IQ requirements; (ii) a set of analysis techniques that support the verification of the correctness and consistency of the IQ requirements model; (iii) a mechanism for deriving the final IQ specifications in terms of IQ policies; (iv) a methodology to assist software engineers during the system design; and (v) a CASE tool, namely STS-IQ Tool. Result We demonstrated the applicability, usefulness, and scalability of the modeling and reasoning techniques within a stock market case study, and we also evaluated the usability and utility of the framework with end-users. Conclusion We conclude that the STS-IQ framework supports the modeling and analysis of IQ requirements, and also the derivation of precise IQ specifications in terms of IQ policies. Therefore, we believe it has potential in practice.

Context: As a result of the growth of non-developer users of OSS applications, usability has over the last ten years begun to attract the interest of the open source software (OSS) community. The OSS community has some special characteristics (such as worldwide geographical distribution of both users and developers and missing resources) which are an obstacle to the direct adoption of many usability techniques as specified in the human-computer interaction (HCI) field. Objective: The aim of this research is to adapt and evaluate the feasibility of applying four usability techniques: user profiles, personas, direct observation and post-test information to four OSS projects from the viewpoint of the development team. Method: The applied research method was a multiple case study of the following OSS projects: Quite Universal Circuit Simulator, PSeInt, FreeMind and OpenOffice Writer. Results: We formalized the application procedure of each of the adapted usability techniques. We found that either there were no procedures for adopting usability techniques in OSS or they were not fully systematized. Additionally, we identified the adverse conditions that are an obstacle to their adoption in OSS and propose the special adaptations required to overcome the obstacles. To avoid some of the adverse conditions, we created web artefacts (online survey, wiki and forum) that are very popular in the OSS field. Conclusion: It is necessary to adapt usability techniques for application in OSS projects considering their idiosyncrasy. Additionally, we found that there are obstacles (for example, number of participant users, biased information provided by developers) to the application of the techniques. Despite these obstacles, it is feasible to apply the adapted techniques in OSS projects, This research was funded by the Secretariat of Higher Education, Science, Technology and Innovation (SENESCYT) of the Government of Ecuador as part of an academic scholarship granted for postgradu- ate training, and Quevedo State Technical University through doctoral scholarships for university professors. Also this research was funded by the Spanish Ministry of Education, Culture and Sports FLEXOR ( TIN2014-52129-R ) and TIN2014-60490-P projects and the eMadrid- CM project ( S2013/ICE-2715 ). Finally, this research received funding from the University of Atacama “DIUDA 22316 ”project

Context Software development is increasingly dependent on existing libraries. Developers need help to find suitable library APIs. Although many studies have been proposed to recommend relevant functional APIs that can be invoked for implementing a functionality, few studies have paid attention to an orthogonal need associated with event-driven programming frameworks, such as the Android framework. In addition to invoking functional APIs, Android developers need to know where to place functional code according to various events that may be triggered within the framework. Objective This paper aims to develop an API recommendation engine for Android application development that can recommend both (1) functional APIs for implementing a functionality and (2) the event callback APIs that are to be overridden to contain the functional code. Method We carry out an empirical study on actual Android programming questions from StackOverflow to confirm the need of recommending callbacks. Then we build Android-specific API databases to contain the correlations among various functionalities and APIs, based on customized parsing of code snippets and natural language processing of texts in Android tutorials and SDK documents, and then textual and code similarity metrics are adapted for recommending relevant APIs. Results We have evaluated our prototype recommendation engine, named LibraryGuru, with about 1500 questions on Android programming from StackOverflow, and demonstrated that our top-5 results on recommending callbacks and functional APIs can on estimate achieve up to 43.5% and 50.9% respectively in precision, 24.6% and 32.5% respectively in mean average precision (MAP) scores, and 51.1% and 44.0% respectively in recall. Conclusion We conclude that it is important and possible to recommend both functional APIs and callbacks for Android application development, and future work is needed to take more data sources into consideration to make more relevant recommendations for developers’ needs.

Context: Software Product Line (SPL) development requires at least concepts for variability implementation and variability modeling for deriving products from a product line. These variability implementation concepts are not required for the development of single systems and, thus, are not considered in traditional software engineering. Metrics are well established in traditional software engineering, but existing metrics are typically not applicable to SPLs as they do not address variability management. Over time, various specialized product line metrics have been described in literature, but no systematic description of these metrics and their characteristics is currently available. Objective: This paper describes and analyzes variability-aware metrics, designed for the needs of software product lines. More precisely we restrict the scope of our study explicitly to metrics designed for variability models, code artifacts, and metrics taking both kinds of artifacts into account. Further, we categorize the purpose for which these metrics were developed. We also analyze to what extent these metrics were evaluated to provide a basis for researchers for selecting adequate metrics. Method: We conducted a systematic literature review to identify variability-aware implementation metrics. We discovered 42 relevant papers reporting metrics intended to measure aspects of variability models or code artifacts. Results: We identified 57 variability model metrics, 34 annotation-based code metrics, 46 code metrics specific to composition-based implementation techniques, and 10 metrics integrating information from variability model and code artifacts. For only 31 metrics, an evaluation was performed assessing their suitability to draw any qualitative conclusions. Conclusions: We observed several problematic issues regarding the definition and the use of the metrics. Researchers and practitioners benefit from the catalog of variability-aware metrics, which is the first of its kind. Also, the research community benefits from the identified observations in order to avoid those problems when defining new metrics.

Context: The growth in the number of non-technical open source software (OSS) application users and the escalating use of these applications have redoubled the need for, and interest in, developing usable OSS. OSS communities are unclear about which techniques to use in each development process activity. Objective: The aim of our research is to adapt a usability technique (visual brainstorming) to an OSS project and evaluate the feasibility of its application. Method: We used the case study research method to investigate technique application and participation in a project. To do this, we participated as volunteers in the HistoryCal project. Results: We identified adverse conditions that were an obstacle to technique application (like it was not easy to recruit OSS users to participate) and modified the technique to make it applicable. Conclusion: We conclude that these changes were helpful for applying the technique using web artifacts like blogs, This research was funded by the SENESCYT , Quevedo State Tech- nical University, TIN2014-52129-R and TIN2014-60490-P projects and the e-Madrid project (S2013/ICE-2715)

Context A Multivocal Literature Review (MLR) is a form of a Systematic Literature Review (SLR) which includes the grey literature (e.g., blog posts, videos and white papers) in addition to the published (formal) literature (e.g., journal and conference papers). MLRs are useful for both researchers and practitioners since they provide summaries both the state-of-the art and –practice in a given area. MLRs are popular in other fields and have recently started to appear in software engineering (SE). As more MLR studies are conducted and reported, it is important to have a set of guidelines to ensure high quality of MLR processes and their results. Objective There are several guidelines to conduct SLR studies in SE. However, several phases of MLRs differ from those of traditional SLRs, for instance with respect to the search process and source quality assessment. Therefore, SLR guidelines are only partially useful for conducting MLR studies. Our goal in this paper is to present guidelines on how to conduct MLR studies in SE. Method To develop the MLR guidelines, we benefit from several inputs: (1) existing SLR guidelines in SE, (2), a literature survey of MLR guidelines and experience papers in other fields, and (3) our own experiences in conducting several MLRs in SE. We took the popular SLR guidelines of Kitchenham and Charters as the baseline and extended/adopted them to conduct MLR studies in SE. All derived guidelines are discussed in the context of an already-published MLR in SE as the running example. Results The resulting guidelines cover all phases of conducting and reporting MLRs in SE from the planning phase, over conducting the review to the final reporting of the review. In particular, we believe that incorporating and adopting a vast set of experience-based recommendations from MLR guidelines and experience papers in other fields have enabled us to propose a set of guidelines with solid foundations. Conclusion Having been developed on the basis of several types of experience and evidence, the provided MLR guidelines will support researchers to effectively and efficiently conduct new MLRs in any area of SE. The authors recommend the researchers to utilize these guidelines in their MLR studies and then share their lessons learned and experiences.

Context Empirical research in software engineering contributes towards developing scientific knowledge in this field, which in turn is relevant to inform decision-making in industry. A number of em ...

Context: Smart contracts play a vital role in many fields. Despite being called smart, the development of smart contracts is a tedious task beyond defining a set of contractual rules. In addition to business knowledge, coding a smart contract requires strong technical knowledge in a multiplex of new and rapidly changing domain-specific languages and blockchain platforms. Objectives: The goal of this paper is to assist developers in building smart contracts independently from the language or the target blockchain platform. In which, we present our second-generation smart contract language iContractML 2.0. Methods: We follow a feature-oriented approach to analyze three different blockchain platforms and propose an enhanced reference model and a modeling framework for smart contracts (iContractML 2.0). Then, we evaluate the coverage and extensibility of iContractML 2.0, first through mapping the concepts of the reference models to the constructs within each of the platforms used in devising the reference model, and second through mapping its concepts to a new smart contract language not previously considered. Finally, we demonstrate the capabilities of iContractML 2.0 using five case studies from different business domains. Results: iContractML 2.0 extends our first generation language to support DAML, which is another standardized language for smart contracts. This makes iContractML 2.0 supports the platforms that DAML support by extension. Moreover, iContractML 2.0 supports generating the structural and deployment artifacts in addition to the smart contract behaviour by implementing templates for some of the common functions. The results of evaluating the generality of the iContractML 2.0 reference model show that it is 91.7% lucid and 72.2% laconic. Moreover, the reference model is able to capture all the elements of the new language with 83.3% of the components which have a direct one-to-one mapping. Conclusion: iContractML 2.0 is an extensible framework that empowers developers to model and generate functional smart contract code that can be deployed onto multiple blockchain platforms.

Context: Organizations increasingly develop software in a distributed manner. The Cloud provides an environment to create and maintain software-based products and services. Currently, it is unknown which software processes are suited for Cloud-based development and what their effects in specific contexts are. Objective: We aim at better understanding the software process applied to distributed software development using the Cloud as development environment. We further aim at providing an instrument, which helps project managers comparing different solution approaches and to adapt team processes to improve future project activities and outcomes. Method: We provide a simulation model, which helps analyzing different project parameters and their impact on projects performed in the Cloud. To evaluate the simulation model, we conduct different analyses using a Scrumban process and data from a project executed in Finland and Spain. An extra adaptation of the simulation model for Scrum and Kanban was used to evaluate the suitability of the simulation model to cover further process models. Results: A comparison of the real project data with the results obtained from the different simulation runs shows the simulation producing results close to the real data, and we could successfully replicate a distributed software project. Furthermore, we could show that the simulation model is suitable to address further process models. Conclusion: The simulator helps reproducing activities, developers, and events in the project, and it helps analyzing potential tradeoffs, e.g., regarding throughput, total time, project size, team size and work-in-progress limits. Furthermore, the simulation model supports project managers selecting the most suitable planning alternative thus supporting decision-making processes.

Background Human-centric software engineering activities, such as requirements engineering, are prone to error. These human errors manifest as faults. To improve software quality, developers need methods to prevent and detect faults and their sources. Aims Human error research from the field of cognitive psychology focuses on understanding and categorizing the fallibilities of human cognition. In this paper, we applied concepts from human error research to the problem of software quality. Method We performed a systematic literature review of the software engineering and psychology literature to identify and classify human errors that occur during requirements engineering. Results We developed the Human Error Taxonomy (HET) by adding detailed error classes to Reason's well-known human error taxonomy of Slips, Lapses, and Mistakes. Conclusion The process of identifying and classifying human error identification provides a structured way to understand and prevent the human errors (and resulting faults) that occur during human-centric software engineering activities like requirements engineering. Software engineering can benefit from closer collaboration with cognitive psychology researchers.

Context. Non-functional requirements (NFRs) are not easy to elicit and formulate. Therefore, some experts advocate using templates, i.e., statement patterns with parameters and optional parts. Unfortunately, there is still scarcity of evidence showing the benefits of this approach. Objective. We aim at evaluating the usefulness of catalog of NFR templates in the context of inexperienced requirements elicitors and the effort required to maintain such catalog. Method. To investigate the usefulness of NFR templates, an experiment was conducted with 107 participants. The participants, individually or in teams, elicited NFRs based on a business case concerning an e-commerce system. To study the maintenance effort, we analyzed 2231 NFRs, 41 industrial projects to simulate the development of a catalog of NFR templates. We investigated how the characteristics of the catalog, essential from the maintenance perspective, change over a series of projects (a counterpart of elapsing time). Results. The participants using NFR templates provided NFRs that were more complete, less ambiguous, more detailed, and better from the point of view of verifiability than their counterparts using the ad hoc approach. However, the catalog of templates did not speed up the elicitation process. As regards the maintenance effort, we introduced the notion of mature catalog. In our case, ca. 40 projects were needed to make the catalog mature and then it contained 400 templates but less than 10% of them were used by a single project. The mature catalog subjected to the Pareto principle—about 20% of templates resulted in almost 80% of NFRs. Moreover, when updating the catalog after each project, less than 10% of templates had to be modified or added. Conclusions. Catalog of NFR templates seems useful. It increases the quality of NFRs and does not hinder elicitation speed. However, it takes time to make such catalog mature.

Context. Developers often improve software quality through refactorings—the practice of behavior-preserving changes to existing code. Recent studies showed that, despite their awareness of tool support for automated refactorings, developers prefer manual refactorings. This practice can be often error-prone and increase testing cost. Objective. To address the problem, we present the Refactorings Investigation and Testing technique, called R it . R it improves the testing efficiency for validating refactoring changes and providing confidence that changed parts behave as intended. As testing is expensive for developers of high-assurance software, R it reduces a considerable amount of its costs by only identifying dependent statements on a failure in each test and by detecting specific refactoring edits responsible for testing failures. Method. Our approach identifies refactorings by analyzing original and edited versions of a program. It then uses the semantic impact of a set of identified refactoring changes to detect tests whose behavior may have been affected and modified by refactoring edits. Given each failed asserts after running regression test suites, R it helps developers focus their attention on logically related program statements by applying program slicing for minimizing each test. For debugging purposes, R it determines specific failure-inducing refactoring edits, separating from other changes that only affect other asserts or tests. Results. We evaluated R it on three open source projects, and found that R it detected tests affected by refactorings with 80.9% accuracy on average. Furthermore, it identified and formed partitions relating program statements only dependent on failed asserts with 97.2% accuracy on average. Conclusion. R it , which combines a refactoring reconstruction technique with change impact analysis to localize failure-inducing program edits, helps developers localize fault causes by focusing on refactoring changes as opposed to all the code fragments in the new version.

Context Critical infrastructures (CIs) are mission-critical, large-scale systems that provide essential products and services for everyday life. Modeling and simulation (M&S) technique is one of useful methods for understanding the complex and emergent behavior of CIs. However, the several characteristics of CIs, such as interdependence, adaptability, dependability, etc., can disturb modelers’ activities to develop the quality models for CIs. The quality of these models can affect the reliability of simulation results as well as the reusability of the models in further simulations. Objective In this paper, we propose a M&S methodology that aims to improve the quality of CI models by means of focusing on dependability, which is one of important characteristics of CIs. Method First, we propose a system to model and simulate CIs based on discrete event system specification, a belief-desire-intention model, and a role-oriented command hierarchy model. Next, we integrate several modeling methods, such as goal modeling, agent-based modeling, and object-oriented modeling, into a unified M&S methodology (UMAS) to seamlessly develop CI models from initial requirements to final modeling artifacts. Finally, we propose a dependability-enhanced UMAS (DUMAS) that can deal with the dependability of CI models. Results In our case study of applying the DUMAS into the M&S of a smart grid, we show how the DUMAS reflects the characteristics of CIs on the overall modeling artifacts. Conclusion The DUMAS provides a novel method that can improve the quality of CI models through elaborating on the activities of requirements engineering with regard to the dependability of CIs. Therefore, modelers can systematically develop quality models from requirements analysis to implementation in compliance with the DUMAS.

Context Pragmatic software reuse is a common activity in industry, involving the reuse of software artifacts not designed to anticipate that reuse. Objective There are two key issues in such tasks that have not been previously explored. (1) Subtle bugs can be inserted due to mistakes on the part of a developer performing the pragmatic reuse. The reused code, integrated in the target system, should be (re-)validated there. But it is not clear what validation strategies would be employed by professional developers, and which of these strategies would be most effective to detect and to repair these inserted bugs. (2) Although semi-automated reuse of the associated test suite has been previously proposed as a strategy to detect such inserted bugs, it is unknown if the reused test suite would be maintainable in practice and how its maintenance characteristics would compare against alternative strategies. Method We present two empirical studies with industrial developers to address these open issues. Results We find that industrial developers use a few strategies including test suite reuse, but that test suite reuse is more reliably effective at discovering and repairing bugs inserted during pragmatic reuse. We also find that, in general, semi-automatically reused test suites are slightly more maintainable than manually reused test suites, in pragmatic reuse scenarios; specific situations can vary wildly however. Participants suggested specific extensions to tool support for semi-automated reuse of test suites. Conclusions While various validation strategies are employed by industrial developers in the context of pragmatic reuse, none is as reliable and effective as test case reuse at discovering and repairing bugs inserted during pragmatic reuse. Despite the fact that semi-automatically reused test cases contain non-trivial adaptive code, their maintainability is equivalent to or exceeds that of manually reused test suites. The approach could be improved, however, by adopting the suggestions of our participants to increase usability.

Context: Standardised specifications for sophisticated technologies are subdivided in multiple documents maintained by different working groups, typically accompanied by models and other formalised artefacts. As the specifications and the models evolve, ensuring their consistency at scale becomes challenging. Objective: While previous work developed a methodology for facilitating the co-evolution of models and standards, based on the Network Function Virtualisation (NFV) Information Model (IM) and models extracted from the related standardised specifications, the methodology focused on structural aspects only. This article refines the methodology, enabling the alignment of semantic descriptions of information elements and attributes, both across specifications and across information elements. Method: To enable the alignment of semantic descriptions, we extend the methodology by using statistical and visual analyses of terms used in the specifications. The underlying meta-model for the information extracted from the specifications is extended to accommodate the capturing of additional semantic information. Results: We report on our experiences with the application of a prototypical implementation of the methodology during the continued alignment and maintenance of the IM and the related standardised specifications. More than 400 potential inconsistencies were identified, leading to more than 100 contributions, some of which addressed multiple findings. Feedback from the working group provided insights on how to refine the methodology further. Conclusions: Models shall play a more central role and be better integrated throughout the specification development and implementation processes, helping to ensure and maintain consistency among specifications. Our experiences may provide useful insights into ongoing and future initiatives where similar challenges are faced.

Context: The advent of modeling in software engineering, like other engineering fields, has revolutionized the formalism and pace of software development. However, software applications are not built from scratch, instead, other existing software artifacts are reused and combined with new artifacts. This notion of software reuse has been in existence for decades. When structural models such as class diagrams are reused, the reusing and reused models often need to be merged and the result visualized to the modeler. However, layout mechanisms such as GraphViz, JGraphX, and other related layout tools do not retain the original layout and rather arbitrarily layout the merged models. Therefore, important information that corresponds to the mental map of a modeler and is conveyed by the specific layout is currently lost. Objective: This paper aims to establish layout algorithms to retain the original layout information from a set of individual but interrelated models after they are merged during software reuse to preserve a modeler’s mental map of the models. Method: In this work, rpGraph uses the relative positioning of model elements to retain the general layout of a single reusing model and a single reused model (two-model merge). Additionally, rpGraph integrates its two-model merge approach into a multi-model merge in a reuse hierarchy to preserve the general topology of several interrelated models. Our findings are evaluated with 20 example single-model reuses from a library of reusable software model artifacts. We further carry out a case study in a reuse hierarchy framework, Concern Oriented Reuse (CORE), where rpGraph is applied to the layout of reusable artifacts, which result from a merge of several individual models. Result: A comparison of the merged layouts of rpGraph, GraphViz, and JGraphX shows that rpGraph performs better in terms of retaining the original layouts. Conclusion: Considering relative positioning during model merge increases the degree with which original layouts can be preserved.

Context In embedded software industry, stakeholders usually promote run-time properties (e.g., performance, energy efficiency, etc.) as quality drivers, which in many cases leads to a compromise at the levels of design-time qualities (e.g., maintainability, reusability, etc.). Such a compromise does not come without a cost; since embedded systems need heavy maintenance cycles. To assure effective bug-fixing, shorten the time required for releasing updates, a refactoring of the software codebase needs to take place regularly. Objective: This study aims to investigate how refactorings are applied in ES industry; and propose a systematic approach that can guide refactoring through a 3-step process for refactoring: (a) planning; (b) design; and (c) evaluation. Method The aforementioned goals were achieved by conducting a single case study in a company that develops medical applications for bio-impedance devices; and follows a rather systematic refactoring process in periodic timestamps. Three data collection approaches have been used: surveys, interviews (10 practitioners), and artifact analysis (51 refactoring activities). Results The results of the study suggest that: (a) maintainability and reusability are the design-time quality attributes that motivate the refactoring of Embedded Software (ES), with 30% of the participants considering them as of “Very High” importance; (b) the refactorings that are most frequently performed are “Extract Method”, “Replace Magic Number with Constant” and “Remove Parameter”. We note that the “Extract Method” refactoring has an applicability of more than over 80%; and (c) to evaluate the refactoring process engineers use tools producing structural metrics, internal standards, and reviews. Conclusions The outcomes of this study can be useful to both researchers and practitioners, in the sense that the former can focus their efforts on aspects that are meaningful to industry, whereas the latter are provided with a systematic refactoring process.

Context: Learning-based automatic program repair techniques are showing promise to provide quality fix suggestions for detected bugs in the source code of the software. These tools mostly exploit historical data of buggy and fixed code changes and are heavily dependent on bug localizers while applying to a new piece of code. With the increasing popularity of code review, dependency on bug localizers can be reduced. Besides, the code review-based bug localization is more trustworthy since reviewers’ expertise and experience are reflected in these suggestions. Objective: The natural language instructions scripted on the review comments are enormous sources of information about the bug’s nature and expected solutions. However, none of the learning-based tools has utilized the review comments to fix programming bugs to the best of our knowledge. In this study, we investigate the performance improvement of repair techniques using code review comments. Method: We train a sequence-to-sequence model on 55,060 code reviews and associated code changes. We also introduce new tokenization and preprocessing approaches that help to achieve significant improvement over state-of-the-art learning-based repair techniques. Results: We boost the top-1 accuracy by 20.33% and top-10 accuracy by 34.82%. We could provide a suggestion for stylistics and non-code errors unaddressed by prior techniques. Conclusion: We believe that the automatic fix suggestions along with code review generated by our approach would help developers address the review comment quickly and correctly and thus save their time and effort.

Context: Code review is a crucial step of the software development life cycle in order to detect possible problems in source code before merging the changeset to the codebase. Although there is no consensus on a formally defined life cycle of the code review process, many companies and open source software (OSS) communities converge on common rules and best practices. In spite of minor differences in different platforms, the primary purpose of all these rules and practices leads to a faster and more effective code review process. Non-conformance of developers to this process does not only reduce the advantages of the code review but can also introduce waste in later stages of the software development. Objectives: The aim of this study is to provide an empirical understanding of the bad practices followed in the code review process, that are code review (CR) smells. Methods: We first conduct a multivocal literature review in order to gather code review bad practices discussed in white and gray literature. Then, we conduct a targeted survey with 32 experienced software practitioners and perform follow-up interviews in order to get their expert opinion. Based on this process, a taxonomy of code review smells is introduced. To quantitatively demonstrate the existence of these smells, we analyze 226,292 code reviews collected from eight OSS projects. Results: We observe that a considerable number of code review smells exist in all projects with varying degrees of ratios. The empirical results illustrate that 72.2% of the code reviews among eight projects are affected by at least one code review smell. Conclusion: The empirical analysis shows that the OSS projects are substantially affected by the code review smells. The provided taxonomy could provide a foundation for best practices and tool support to detect and avoid code review smells in practice.

API deprecation, which enables API developers to assist API users in migration tasks, has been widely employed in API removal management. However, mismanaged API deprecation will cause unnecessary cost and bring negligible benefit to API users. Cost-effective investments in API deprecation become challenges for API developers. In this work, an iterative model for cost-effective investments in API deprecation is developed. The model provides a data-driven mechanism for API developers to iteratively make investments in API deprecation. A tool named AWARE (A Win–win Assistant for API REmoval management) is also developed for API developers to accurately assess the benefit from the perspective of API usage statistics. Based on the prioritized benefit, API developers can allocate appropriate resources on API deprecation. A case study is performed to evaluate the effectiveness of the iterative model with AWARE. The evaluation result shows that the cost paid by API developers can be reduced significantly while the benefit brought to API users can be increased. A win–win strategy for API deprecation can be achieved.

Context: Behavior-driven development (BDD) is a variant of test-driven development where specifications are described in a structured domain-specific natural language. Although refactoring is a crucial activity of BDD, little research is available on the topic. Objective: To support practitioners in refactoring BDD specifications by (1) proposing semi-automated approaches to identify refactoring candidates; (2) defining refactoring techniques for BDD specifications; and (3) evaluating the proposed identification approaches in an industry context. Method: Using Action Research, we have developed an approach for identifying refactoring candidates in BDD specifications based on two measures of similarity and applied the approach in two projects of a large software organization. The accuracy of the measures for identifying refactoring candidates was then evaluated against an approach based on machine learning and a manual approach based on practitioner perception. Results: We proposed two measures of similarity to support the identification of refactoring candidates in a BDD specification base; (1) normalized compression similarity (NCS) and (2) similarity ratio (SR). A semi-automated approach based on NCS and SR was developed and applied to two industrial cases to identify refactoring candidates. Our results show that our approach can identify candidates for refactoring 6o times faster than a manual approach. Our results furthermore showed that our measures accurately identified refactoring candidates compared with a manual identification by software practitioners and outperformed an ML-based text classification approach. We also described four types of refactoring techniques applicable to BDD specifications; merging candidates, restructuring candidates, deleting duplicates, and renaming specification titles. Conclusion: Our results show that NCS and SR can help practitioners in accurately identifying BDD specifications that are suitable candidates for refactoring, which also decreases the time for identifying refactoring candidates.

Context : The study of cognitive aspects around software activities can provide valuable insights to improve software engineering practices. Objective: This paper presents an approach based on distributed cognition and sequential analysis to explore cognitive activities in the software development context by analyzing the interactions between software practitioners and the resources used to support them. Method: We conducted nine laboratory-based observation sessions to record qualitative audio/video data of interactions between the study participants and at-hand resources during the planning and managing of software analytics tasks. Results: The interaction strategies of the resources model included 21 emergent actions, and the sequential analysis revealed two different patterns of interaction over time. Conclusion: Our approach has been useful for evaluating how well an artifact works to support a team in software analytics activities. Furthermore, it can be applied to explore and discover interaction patterns in different software activities.

Context: Ontologies are known as a formal and explicit conceptualization of entities, their interfaces, behaviors, and relationships. They have been applied in various application domains such as autonomous driving where ontologies are used for decision making, traffic description, auto-pilot etc. It has always been a challenge to test the corresponding safety-critical software systems in autonomous driving that have been playing an increasingly important role in our daily routines. Objective: Failures in these systems potentially not only cause great financial loss but also the loss of lives. Therefore, it is vital to obtain and cover as many as critical driving scenarios during auto drive testing to ensure that the system can always reach a fail-safe state under different circumstances. Method: We outline a general framework for testing, verification, and validation for automated and autonomous driving functions. The introduced method makes use of ontologies for describing the environment of autonomous vehicles and convert them to input models for combinatorial testing. The combinatorial test suite comprises abstract test cases that are mapped to concrete test cases that can be executed using simulation environments. Results: We discuss in detail on how to automatically convert ontologies to the corresponding combinatorial testing input models. Specifically, we present two conversion algorithms and compare their applicability using ontologies with different sizes. We also carried out a case study to further demonstrate the practical value of applying ontology-based test generation in industrial settings. Conclusion: The proposed approach for testing autonomous driving takes ontologies describing the environment of autonomous vehicles, and automatically converts it to test cases that are used in a simulation environment to verify automated driving functions. The conversion relies on combinatorial testing. The first experimental results relying on an example from the automotive industry indicates that the approach can be used in practice.

Context We observed a special type of bug reopen that has no direct impact on the user experience or the normal operation of the system being developed. We refer to these as non-negative bug reopens. Objective Non-negative bug reopens are novel and somewhat contradictory to popular conceptions. Therefore, we thoroughly explored these phenomena in this study. Method We begin with a novel approach that preliminarily characterizes non-negative bug reopens. Based on bug reports extracted from Eclipse Bugzilla, we then examined a case study to compare non-negative and regular bug reopens using the Wilcoxon-Mann-Whitney test. Results The results show that non-negative bug reopens are statistically significantly different than regular bug reopens, based on their survival times and the number of developers involved in the entire debugging process. Conclusion Taking into account the significant differences, we suggest that the effects of non-negative bug reopens should be considered in future research in related areas, such as bug triage and reopened bug prediction.

Context Code smells are suboptimal design or implementation choices made by programmers during the development of a software system that possibly lead to low code maintainability and higher maintenance costs. Objective Previous research mainly studied the characteristics of code smell instances affecting a source code file, while only few studies analyzed the magnitude and effects of smell co-occurrence, i.e., the co-occurrence of different types of smells on the same code component. This paper aims at studying in details this phenomenon. Method We analyzed 13 code smell types detected in 395 releases of 30 software systems to firstly assess the extent to which code smells co-occur, and then we analyze (i) which code smells co-occur together, and (ii) how and why they are introduced and removed by developers. Results 59% of smelly classes are affected by more than one smell, and in particular there are six pairs of smell types (e.g., Message Chains and Spaghetti Code) that frequently co-occur. Furthermore, we observed that method-level code smells may be the root cause for the introduction of class-level smells. Finally, code smell co-occurrences are generally removed together as a consequence of other maintenance activities causing the deletion of the affected code components (with a consequent removal of the code smell instances) as well as the result of a major restructuring or scheduled refactoring actions. Conclusions Based on our findings, we argue that more research aimed at designing co-occurrence-aware code smell detectors and refactoring approaches is needed.

Context Experiment replications play a central role in the scientific method. Although software engineering experimentation has matured a great deal, the number of experiment replications is still relatively small. Software engineering experiments are composed of complex concepts, procedures and artefacts. Laboratory packages are a means of transferring knowledge among researchers to facilitate experiment replications. Objective This paper investigates the experiment replication process to find out what information is needed to successfully replicate an experiment. Our objective is to propose the content and structure of laboratory packages for software engineering experiments. Method We evaluated seven replications of three different families of experiments. Each replication had a different experimenter who was, at the time, unfamiliar with the experiment. During the first iterations of the study, we identified experimental incidents and then proposed a laboratory package structure that addressed these incidents, including document usability improvements. We used the later iterations to validate and generalize the laboratory package structure for use in all software engineering experiments. We aimed to solve a specific problem, while at the same time looking at how to contribute to the body of knowledge on laboratory packages. Results We generated a laboratory package for three different experiments. These packages eased the replication of the respective experiments. The evaluation that we conducted shows that the laboratory package proposal is acceptable and reduces the effort currently required to replicate experiments in software engineering. Conclusion We think that the content and structure that we propose for laboratory packages can be useful for other software engineering experiments.

Context The increased use of Open Source Software (OSS) affects how software-intensive product development organizations (SIPDO) innovate and compete, moving them towards Open Innovation (OI). Specifically, software engineering tools have the potential for OI, but require better understanding regarding what to develop internally and what to acquire from outside the organization, and how to cooperate with potential competitors. Aim This paper aims at synthesizing a theory of openness for software engineering tools in SIPDOs, that can be utilized by managers in defining more efficient strategies towards OSS communities. Method We synthesize empirical evidence from a systematic mapping study, a case study, and a survey, using a narrative method. The synthesis method entails four steps: (1) Developing a preliminary synthesis, (2) Exploring the relationship between studies, (3) Assessing the validity of the synthesis, and (4) Theory formation. Result We present a theory of openness for OSS tools in software engineering in relation to four constructs: (1) Strategy, (2) Triggers, (3) Outcomes, and (4) Level of openness. Conclusion The theory reasons that openness provides opportunities to reduce the development cost and development time. Furthermore, OI positively impacts on the process and product innovation, but it requires investment by organizations in OSS communities. By betting on openness, organizations may be able to significantly increase their competitiveness.

Context In attempt to simulate the factors that affect the software evolution behaviour and possibly predict it, several simulation models have been developed recently. The current system dynamic (SD) simulation model of software evolution process was built based on actor-network theory (ANT) of software evolution by using system dynamic environment, which is not a suitable environment to reflect the complexity of ANT theory. In addition the SD model has not been investigated for its ability to represent the real-world process of software evolution. Objectives This paper aims to re-implements the current SD model to an agent-based simulation environment ‘Repast’ and checks the behaviour of the new model compared to the existing SD model. It also aims to investigate the ability of the new Repast model to represent the real-world process of software evolution. Methods a new agent-based simulation model is developed based on the current SD model's specifications and then tests similar to the previous model tests are conducted in order to perform a comparative evaluation between of these two results. In addition an investigation is carried out through an interview with an expert in software development area to investigate the model's ability to represent real-world process of software evolution. Results The Repast model shows more stable behaviour compared with the SD model. Results also found that the evolution health of the software can be calibrated quantitatively and that the new Repast model does have the ability to represent real-world processes of software evolution. Conclusion It is concluded that by applying a more suitable simulation environment (agent-based) to represent ANT theory of software evolution, that this new simulation model will show more stable bahaviour compared with the previous SD model; And it will also shows the ability to represent (at least quantatively) the real-world aspect of software evolution.

Context There are many techniques for performing authentication, such as text passwords and biometrics. Combining two factors into one technique is known as multi-factor authentication. The lack of a proper method for comparing and selecting these techniques for their implementation in software development processes is observed. Objective The article presents a recommendation Framework proposal for comparing and selecting authentication techniques in a software development process. Method Knowledge from academy is obtained through a systematic literature review and experience from industry is gathered using a survey and interviews. The results of these two techniques are used to generate a Framework proposal, which is validated afterwards, through an expert panel and the case study method. Results A recommendation Framework is generated, which recommends the most appropriate authentication schemes and methods for software applications based on criteria identified in literature and industry, categorized by usability, security and costs, plus the context for which the application is intended. The Framework's validity is ascertained by confirming that its recommendations are on line with those on industry, based in the results from the developed case studies. A tool prototype was created in order to help using the Framework in software development processes. Conclusion The proposed Framework helps to cover the observed gap in literature, helping software developers to compare and select the most appropriate authentication techniques for their applications.

Context Gamification has been proven to increase engagement and motivation in multiple and different non-game contexts such as healthcare, education, workplace, and marketing, among others. However, many of these applications fail to achieve the desired benefits of gamification, mainly because of a poor design. Objective This paper explores the conceptualization, implementation and monitoring phases of meaningful gamification strategies and proposes a solution for strategy experts that hides the implementation details and helps them focus only on what is crucial for the success of the strategy. The solution makes use of Model-Driven Engineering (MDE) and Complex Event Processing (CEP) technology. Method An easy-to-use graphical editor is used to provide the high-level models that represent the design of the gamification strategy and its deployment and monitoring. These models contain the event pattern definitions to be automatically transformed into code. This code is then deployed both in a CEP engine to detect the conditions expressed in such patterns and in an enterprise service bus to execute the corresponding pattern actions. Results The paper reports on the use of both a graphical modeling editor for gamification domain definition and a graphical modeling editor for gamification strategy design, monitoring and code generation in event-based systems. It also shows how the proposal can be used to design and automate the implementation and monitoring of a gamification strategy in an educational domain supported by a well-known Learning Management System (LMS) such as Moodle. Conclusion It can be concluded that this unprecedented model-driven approach leveraging gamification and CEP technology provides strategy experts with the ability to graphically define gamification strategies, which can be directly transformed into code executable by event-based systems. Therefore, this is a novel solution for bringing CEP closer to any strategy expert, positively influencing the gamification strategy design, implementation and real-time monitoring processes.

Context:Pre-publication peer review of scientific articles is considered a key element of the research process in software engineering, yet it is often perceived as not to work fully well. Objective:We aim at understanding the perceptions of and attitudes towards peer review of authors and reviewers at one of software engineering’s most prestigious venues, the International Conference on Software Engineering (ICSE). Method:We invited 932 ICSE 2014/15/16 authors and reviewers to participate in a survey with 10 closed and 9 open questions. Results:We present a multitude of results, such as: Respondents perceive only one third of all reviews to be good, yet one third as useless or misleading; they propose double-blind or zero-blind reviewing regimes for improvement; they would like to see showable proofs of (good) reviewing work be introduced; attitude change trends are weak. Conclusion:The perception of the current state of software engineering peer review is fairly negative. Also, we found hardly any trend that suggests reviewing will improve by itself over time; the community will have to make explicit efforts. Fortunately, our (mostly senior) respondents appear more open for trying different peer reviewing regimes than we had expected.

Context Software refactoring is a common practice aimed at addressing requirements or fixing bugs during the software development. While refactoring related to functional requirements has been widely studied in the last few years, non-functional-driven refactoring is still critical, mostly because non-functional characteristics of software are hard to assess and appropriate refactoring actions can be difficult to identify. In the context of performance, which is the focus of this paper, antipatterns represent effective instruments to tackle this issue, because they document common mistakes leading to performance problems as well as their solutions. Objective In order to effectively reuse the knowledge beyond performance antipatterns, automation is required to detect and remove them. In this paper we introduce a framework that enables, in an unique tool context, the refactoring of software models driven by performance antipattern detection and removal. Method We have implemented, within the EPSILON platform, detection rules and refactoring actions on UML models for a set of well-known performance antipatterns. By exploiting the EPSILON languages to check properties and apply refactoring on models, we enable three types of refactoring sessions. Results We experiment our framework on a Botanical Garden Management System to show, on one side, that antipatterns can effectively drive software refactoring towards models that satisfy performance requirements and, on the other side, that the automation introduced by EPSILON-based sessions enables to inspect multiple paths and to propose a variety of solutions. Conclusion This work demonstrates that automation in performance-driven software model refactoring can be beneficial, and that performance antipatterns can be powerful instruments in the hands of software engineers for detecting (and solving) performance problems usually hidden to traditional bottleneck analysis. This work also opens the road to the integration of well-known techniques for software refactoring driven by functional requirements with novel techniques addressing non-functional requirements like performance.

© 2017 Elsevier B.V. Context: Security, in digitally connected organizational environments of today, involves many different perspectives, including social, physical, and technical factors. In order to understand the interactions among these correlated aspects and elicit potential threats geared towards a given organization, different security requirements analysis approaches are proposed in the literature. However, the body of knowledge is yet to unleash its full potential due to the complex nature of security problems, and inadequate ways to improve security awareness of key players in the organization. Objective: Objective(s) of the research study is to improve the security awareness of players utilizing serious games via: (i) Know-how of security concepts and security protection; (ii) guided process of identifying valuable assets and vulnerabilities in a given organizational setting; (iii) guided process of defining successful security attacks to the organization. Method: Important methods used to address the above objectives include: (i) a comprehensive review of the literature to better understand security and game design elements; (ii) designing a serious game using cyber security knowledge and game-based techniques combined with security requirements engineering concepts; (iii) using empirical evaluation (observation and survey) to verify the effectiveness of the proposed game design. Result: The solution proposed is a serious game for security requirements education, which: (i) can be an effective and fun way of learning security related concepts; (ii) mimics a real life problem setting in a presentable and understandable way; (iii) motivates players to learn more about security related concepts in future. Conclusion: From this study, we conclude that the proposed Security Requirement Education Game (SREG) has positive results and is helpful for players of the game to get an understanding of security attacks and vulnerabilities.

Context Refactoring is a technique for improving the internal structure of software systems. It has a solid theoretical background while being used in development practice also. However, we lack empirical research results on the real effect of code refactoring and its application. Objective This paper presents a manually validated subset of a previously published dataset containing the refactorings extracted by the RefFinder tool, code metrics, and maintainability of 7 open-source systems. We found that RefFinder had around 27% overall average precision on the subject systems, thus our manually validated subset has substantial added value. Using the dataset, we studied several aspects of the refactored and non-refactored source code elements (classes and methods), like the differences in their maintainability and source code metrics. Method We divided the source code elements into a group containing the refactored elements and a group with non-refactored elements. We analyzed the elements’ characteristics in these groups using correlation analysis, Mann–Whitney U test and effect size measures. Results Source code elements subjected to refactorings had significantly lower maintainability than elements not affected by refactorings. Moreover, refactored elements had significantly higher size related metrics, complexity, and coupling. Also these metrics changed more significantly in the refactored elements. The results are mostly in line with our previous findings on the not validated dataset, with the difference that clone metrics had no strong connection with refactoring. Conclusions Compared to the preliminary analysis using a not validated dataset, the manually validated dataset led to more significant results, which suggests that developers find targets for refactorings based on some internal quality properties of the source code, like their size, complexity or coupling, but not clone related metrics as reported in our previous studies. They do not just use these properties for identifying targets, but also control them with refactorings.

Context: The performance assessment of complex software systems is not a trivial task since it depends on the design, code, and execution environment. All these factors may affect the system quality and generate negative consequences, such as delays and system failures. The identification of bad practices leading to performance flaws is of key relevance to avoid expensive rework in redesign, reimplementation, and redeployment. Objective: The goal of this manuscript is to provide a systematic process, based on load testing and profiling data, to identify performance issues with runtime data. These performance issues represent an important source of knowledge as they are used to trigger the software refactoring process. Software characteristics and performance measurements are matched with well-known performance antipatterns to document common performance issues and their solutions. Method: We execute load testing based on the characteristics of collected operational profile, thus to produce representative workloads. Performance data from the system under test is collected using a profiler tool to create profiler snapshots and get performance hotspot reports. From such data, performance issues are identified and matched with the specification of antipatterns. Software refactorings are then applied to solve these performance antipatterns. Results: The approach has been applied to a real-world industrial case study and to a representative laboratory study. Experimental results demonstrate the effectiveness of our tool-supported approach that is able to automatically detect two performance antipatterns by exploiting the knowledge of domain experts. In addition, the software refactoring process achieves a significant performance gain at the operational stage in both case studies. Conclusion: Performance antipatterns can be used to effectively support the identification of performance issues from load testing and profiling data. The detection process triggers an antipattern-based software refactoring that in our two case studies results in a substantial performance improvement.

ContextSoftware project plan is the basis of the project execution, and its quality depends on completeness of software scope definition. A method is required that should gauge the completeness of different aspects of scope definition, thus, providing guidance to practitioners to reconsider areas that have not been defined well. ObjectiveThis paper aims to evaluate completeness of software project scope definition. It identifies a detailed list of different aspects of software project scope definition, and builds a method where numerical score can be assigned to a scope definition. MethodA detailed list of different elements of software project scope definition is identified through literature. These elements are then empirically evaluated through an electronic survey, conducted with the industrial experts. Once finalized, these elements are ranked and assigned weights to systematically build a scorecard that is used to calculate scope definition score. Evaluation of the proposed method is done through a series of formal experiments. ResultsEvaluation results suggest that the proposed method is useful not only in calculating completeness of software projects scope definition, but it also serves as a guide for practitioners to determine specific aspects that require further consideration.

ContextAutomated test generation promises to improve the effectiveness of software testing and to reduce the involved manual effort. While automated test generation has been successfully applied for code-level API testing, it has not found widespread adoption in practice for testing of graphical user interfaces. Tools for test generation do not support GUI testing out-of-the-box but require dedicated extensions. ObjectiveThis paper explores the applicability of automated test generation for testing GUIs of industry applications. We propose a test adapter approach to bridge the gap between automated test generation tools and industry applications. MethodA multiple case study was conducted in which automated test generation with test adapters has been applied at the unit, integration, and system test level in three industry projects from two different companies. ResultsAutomated test generation via test adapters could be applied at all test levels. It has led to an increase of coverage as well as the detection of new defects that were not found by preceding testing activities in the projects. While test adapters can easily be implemented at the unit test level, their complexity and the corresponding effort for providing adapter implementations rises at higher test levels. ConclusionTest adapters can be used for applying automated test generation for testing GUIs of industry applications. They bridge the gap between automated test generation tools and industry applications. The development of test adapters requires no tool-specific knowledge and can be performed by members of the development team.