Search

Context: The relevance of Requirements Engineering (RE) research to practitioners is vital for a long-term dissemination of research results to everyday practice. Some authors have speculated about a mismatch between research and practice in the RE discipline. However, there is not much evidence to support or refute this perception. Objective: This paper presents the results of a study aimed at gathering evidence from practitioners about their perception of the relevance of RE research and at understanding the factors that influence that perception. Method: We conducted a questionnaire-based survey of industry practitioners with expertise in RE. The participants rated the perceived relevance of 435 scientific papers presented at five top RE-related conferences. Results: The 153 participants provided a total of 2,164 ratings. The practitioners rated RE research as essential or worthwhile in a majority of cases. However, the percentage of non-positive ratings is still higher than we would like. Among the factors that affect the perception of relevance are the paper?s links to industry, the research method used, and respondents? roles. The reasons for positive perceptions were primarily related to the relevance of the problem and the soundness of the solution, while the causes for negative perceptions were more varied. The respondents also provided suggestions for future research, including topics researchers have studied for decades, like elicitation or requirement quality criteria. Conclusions: The study is valuable for both researchers and practitioners. Researchers can use the reasons respondents gave for positive and negative perceptions and the suggested research topics to help make their research more appealing to practitioners and thus more prone to industry adoption. Practitioners can benefit from the overall view of contemporary RE research by learning about research topics that they may not be familiar with, and compare their perception with those of their c, open access

Authorship ethics is a central topic of discussion in research ethics fora. There are various guidelines for authorship (i.e., naming and order). It is not easy to decide the authorship in the presence of varying authorship guidelines. This paper gives an overview of research on authorship practices and issues. It presents a review of 16 empirical research papers published between 2014 - 2020. The objective is to learn how various research disciplines handle authorship. What are the authorship practices in various research disciplines, and what are the issues associated with these practices © 2021 IEEE.

Technical Debt management decisions always imply a trade-off among outcomes at different points in time. In such intertemporal choices, distant outcomes are often valued lower than close ones, a phenomenon known as temporal discounting. Technical Debt research largely develops prescriptive approaches for how software engineers should make such decisions. Few have studied how they actually make them. This leaves open central questions about how software practitioners make decisions. This paper investigates how software practitioners discount uncertain future outcomes and whether they exhibit temporal discounting. We adopt experimental methods from intertemporal choice, an active area of research. We administered an online questionnaire to 33 developers from two companies in which we presented choices between developing a feature and making a longer-term investment in architecture. The results show wide-spread temporal discounting with notable differences in individual behavior. The results are consistent with similar studies in consumer behavior and raise a number of questions about the causal factors that influence temporal discounting in software engineering. As the first empirical study on intertemporal choice in SE, the paper establishes an empirical basis for understanding how software developers approach intertemporal choice and provides a blueprint for future studies. © 2019 IEEE.

Software development ecosystems vary significantly among different industrial domains. Therefore, it is challenging to establish quality assurance processes that can be deployed seamlessly to multiple domains. In this paper, we extend our previous work on performance and scalability assessment by identifying the architecture variability points in our PPTAM tooling infrastructure. The goal is to design a modifiable software architecture that enables low cost deployment of our performance and scalability assessment approach. We present the scalability assessment context, architecture modifiability, and lessons learned that were derived from our experience with scalability assessment in several business domains. Specifically, we describe our experience with the application of the proposed approach to a large complex telecom system at Ericsson. © 2021 IEEE.

This papers surveys current stakeholder identification methods and techniques from the Requirements Engineering (RE) discipline. Upon critical analysis of available models and techniques, we identify the techniques that should be prioritized when identifying stakeholders that should be involved in the decision making processes about software components or assets. Next, we analyzed industrial decision scenarios and match the most prominent issues with stakeholder identification against the perceived benefits that the identified techniques offer. We conclude this paper with a research agenda in stakeholder identification for decision making about software components. © 2017 IEEE.

This papers surveys current stakeholder identification methods and techniques from the Requirements Engineering (RE) discipline. Upon critical analysis of available models and techniques, we identify the techniques that should be prioritized when identifying stakeholders that should be involved in the decision making processes about software components or assets. Next, we analyzed industrial decision scenarios and match the most prominent issues with stakeholder identification against the perceived benefits that the identified techniques offer. We conclude this paper with a research agenda in stakeholder identification for decision making about software components. © 2017 IEEE.

Collecting and using user feedback as a method to support requirements engineering, might undermine user rights. This becomes apparent when looking at related areas, e.g., research in user experience, where collecting user feedback also plays an important role. In such settings, researchers need to ensure that the stakeholders' rights and integrity are respected. This paper identifies and discusses some of the ethical challenges and issues a researcher can face, using an example case. Focusing on user feedback, this case can serve as an example for CrowdRE, i.e. several of our findings might apply to CrowdRE in general. However, further research is needed as our work mainly reflects the challenges experienced by the authors of this paper. © 2017 IEEE.

This papers surveys current stakeholder identification methods and techniques from the Requirements Engineering (RE) discipline. Upon critical analysis of available models and techniques, we identify the techniques that should be prioritized when identifying stakeholders that should be involved in the decision making processes about software components or assets. Next, we analyzed industrial decision scenarios and match the most prominent issues with stakeholder identification against the perceived benefits that the identified techniques offer. We conclude this paper with a research agenda in stakeholder identification for decision making about software components. © 2017 IEEE.

Collecting and using user feedback as a method to support requirements engineering, might undermine user rights. This becomes apparent when looking at related areas, e.g., research in user experience, where collecting user feedback also plays an important role. In such settings, researchers need to ensure that the stakeholders' rights and integrity are respected. This paper identifies and discusses some of the ethical challenges and issues a researcher can face, using an example case. Focusing on user feedback, this case can serve as an example for CrowdRE, i.e. several of our findings might apply to CrowdRE in general. However, further research is needed as our work mainly reflects the challenges experienced by the authors of this paper. © 2017 IEEE.

This papers surveys current stakeholder identification methods and techniques from the Requirements Engineering (RE) discipline. Upon critical analysis of available models and techniques, we identify the techniques that should be prioritized when identifying stakeholders that should be involved in the decision making processes about software components or assets. Next, we analyzed industrial decision scenarios and match the most prominent issues with stakeholder identification against the perceived benefits that the identified techniques offer. We conclude this paper with a research agenda in stakeholder identification for decision making about software components. © 2017 IEEE.

This papers surveys current stakeholder identification methods and techniques from the Requirements Engineering (RE) discipline. Upon critical analysis of available models and techniques, we identify the techniques that should be prioritized when identifying stakeholders that should be involved in the decision making processes about software components or assets. Next, we analyzed industrial decision scenarios and match the most prominent issues with stakeholder identification against the perceived benefits that the identified techniques offer. We conclude this paper with a research agenda in stakeholder identification for decision making about software components. © 2017 IEEE.

This papers surveys current stakeholder identification methods and techniques from the Requirements Engineering (RE) discipline. Upon critical analysis of available models and techniques, we identify the techniques that should be prioritized when identifying stakeholders that should be involved in the decision making processes about software components or assets. Next, we analyzed industrial decision scenarios and match the most prominent issues with stakeholder identification against the perceived benefits that the identified techniques offer. We conclude this paper with a research agenda in stakeholder identification for decision making about software components. © 2017 IEEE.

Collecting and using user feedback as a method to support requirements engineering, might undermine user rights. This becomes apparent when looking at related areas, e.g., research in user experience, where collecting user feedback also plays an important role. In such settings, researchers need to ensure that the stakeholders' rights and integrity are respected. This paper identifies and discusses some of the ethical challenges and issues a researcher can face, using an example case. Focusing on user feedback, this case can serve as an example for CrowdRE, i.e. several of our findings might apply to CrowdRE in general. However, further research is needed as our work mainly reflects the challenges experienced by the authors of this paper. © 2017 IEEE.

Collecting and using user feedback as a method to support requirements engineering, might undermine user rights. This becomes apparent when looking at related areas, e.g., research in user experience, where collecting user feedback also plays an important role. In such settings, researchers need to ensure that the stakeholders' rights and integrity are respected. This paper identifies and discusses some of the ethical challenges and issues a researcher can face, using an example case. Focusing on user feedback, this case can serve as an example for CrowdRE, i.e. several of our findings might apply to CrowdRE in general. However, further research is needed as our work mainly reflects the challenges experienced by the authors of this paper. © 2017 IEEE.

Collecting and using user feedback as a method to support requirements engineering, might undermine user rights. This becomes apparent when looking at related areas, e.g., research in user experience, where collecting user feedback also plays an important role. In such settings, researchers need to ensure that the stakeholders' rights and integrity are respected. This paper identifies and discusses some of the ethical challenges and issues a researcher can face, using an example case. Focusing on user feedback, this case can serve as an example for CrowdRE, i.e. several of our findings might apply to CrowdRE in general. However, further research is needed as our work mainly reflects the challenges experienced by the authors of this paper. © 2017 IEEE.

Collecting and using user feedback as a method to support requirements engineering, might undermine user rights. This becomes apparent when looking at related areas, e.g., research in user experience, where collecting user feedback also plays an important role. In such settings, researchers need to ensure that the stakeholders' rights and integrity are respected. This paper identifies and discusses some of the ethical challenges and issues a researcher can face, using an example case. Focusing on user feedback, this case can serve as an example for CrowdRE, i.e. several of our findings might apply to CrowdRE in general. However, further research is needed as our work mainly reflects the challenges experienced by the authors of this paper. © 2017 IEEE.

Technical Debt (TD) is a metaphor used to explain the negative impacts that sub-optimal design decisions have in the long-term perspective of a software project. Although TD is acknowledged by both researchers and practitioners to have strong negative impact on Software development, its study on Testware has so far been very limited. A gap in knowledge that is important to address due to the growing popularity of Testware (scripted automated testing) in software development practice.In this paper we present a mapping analysis that connects 21 well-known, Software, object-oriented TD items to Testware, establishing them as Testware Technical Debt (TTD) items. The analysis indicates that most Software TD items are applicable or observable as TTD items, often in similar form and with roughly the same impact as for Software artifacts (e.g. reducing quality of the produced artifacts, lowering the effectiveness and efficiency of the development process whilst increasing costs). In the analysis, we also identify three types of connections between software TD and TTD items with varying levels of impact and criticality. Additionally, the study finds support for previous research results in which specific TTD items unique to Testware were identified. Finally, the paper outlines several areas of future research into TTD. © 2017 IEEE.

This papers surveys current stakeholder identification methods and techniques from the Requirements Engineering (RE) discipline. Upon critical analysis of available models and techniques, we identify the techniques that should be prioritized when identifying stakeholders that should be involved in the decision making processes about software components or assets. Next, we analyzed industrial decision scenarios and match the most prominent issues with stakeholder identification against the perceived benefits that the identified techniques offer. We conclude this paper with a research agenda in stakeholder identification for decision making about software components. © 2017 IEEE.

Software needs to be secure, in particular, when deployed to critical infrastructures. Secure coding guidelines capture practices in industrial software engineering to ensure the security of code. This study aims to assess the level of awareness of secure coding in industrial software engineering, the skills of software developers to spot weaknesses in software code, avoid them, and the organizational support to adhere to coding guidelines. The approach draws on well-established theories of policy compliance, neutralization theory, and security-related stress and the authors' many years of experience in industrial software engineering and on lessons identified from training secure coding in the industry. The paper presents the questionnaire design for the online survey and the first analysis of data from the pilot study. © 2020 IEEE.

The cause-effect recognition (CEREC) system provides an API for causality extraction tailored to the requirements engineering context. The library is written in Java and is released under the MIT open source license. In this paper, the underlying algorithm is described, and a demonstration of the active learning component for causality extraction is outlined. The results are promising and strengthen the confidence in exploring automation approaches for model-based testing. © 2020 IEEE., open access

The cause-effect recognition (CEREC) system provides an API for causality extraction tailored to the requirements engineering context. The library is written in Java and is released under the MIT open source license. In this paper, the underlying algorithm is described, and a demonstration of the active learning component for causality extraction is outlined. The results are promising and strengthen the confidence in exploring automation approaches for model-based testing. © 2020 IEEE., open access

The cause-effect recognition (CEREC) system provides an API for causality extraction tailored to the requirements engineering context. The library is written in Java and is released under the MIT open source license. In this paper, the underlying algorithm is described, and a demonstration of the active learning component for causality extraction is outlined. The results are promising and strengthen the confidence in exploring automation approaches for model-based testing. © 2020 IEEE., open access

Companies adopting agile development tend to face challenges in complying with security norms. Existing research either focuses on how to integrate security into agile methods or on discussing compliance issues of agile methods but independently of the regulation type, in particular of security standards. A comprehensive overview of this scattered field is still missing and we know little about how to achieve security compliance in agile software development. Existing secondary studies (mapping studies and literature reviews) analyze publications on secure agile development, but they do not analyze implications of security standard compliance, e.g., integration of specific standard requirements or compliance assessments. To close this gap, we report on a systematic mapping study. Starting with a set of 2,383 papers, our work distills 11 relevant publications addressing security compliance in agile software development. With this study, we contribute by describing the maturity of the field, as well as domains where security compliant agile software engineering was investigated. Moreover, we make explicit which phases of a secure development process are covered by the field and which agile principles are analyzed when aiming at compliance with international security standards, country-specific security regulations, industry-specific security standards, and other well-known security frameworks. © 2020 IEEE.

Platforms are emerging that allow data scientists, software and hardware engineers to collaborate through organisational boundaries to develop systems of Artificial Intelligence (AI). Such collaboration involves the exchange of assets representing Intellectual Property (IP) of the collaborators. The tension between permitting access and protecting IP is thus one of the critical challenges faced by organisations willing to innovate through collaboration. Licensing is a common way to address the issue, but the influence of the licensing rules on the intended form of collaboration is still unclear.In this paper, we identify and analyse the rules that are used to regulate IP exchanges in two common forms of collaboration: a) competitions where one customer benchmarks and selects among multiple suppliers and b) consortium projects where multiple parties collaborate to product joint results. Due to our interest in AI, we have chosen to analyse the terms and conditions of competitions hosted on KaggleTM a leading online platform for Competitions. For consortium projects, we have analysed the DESCA Consortium Agreement template. DESCA is often used for European projects, an increasing number of which are used to fund AI innovation projects. We have applied In Vivo Coding and Concept Coding coding techniques to highlight rules applicable to IP exchange. We structured the findings in the form of tree graphs consisting of interdependent textual phrases to extract, group and compare the terms and conditions of IP sharing in each collaboration form and how they relate to the characteristics of the studied collaborations.The results indicate that each form of collaboration has its own set of rules that address comparable concerns but have different content. Practitioners, both platform providers and collaborators, can utilise our results to implement licensing for IP exchange that fits the desired type of collaboration. For researchers, our results represent a step towards the au

In cross-version defect prediction (CVDP), historical data is derived from the prior version of the same project to predict defects of the current version. Recent studies in CVDP focus on subset selection to deal with the changes of the data distributions. No prior study has focused on training data arriving in streaming fashion across the versions where the significant differences between versions make the prediction unreliable. We refer to this situation as Drift Propensity (DP). By identifying DP, necessary steps can be taken (e.g., updating or retraining the model) to improve the prediction performance. In this paper, we investigate the chronological defect datasets and identify DP in the datasets. The no-memory data management technique is employed to manage the data distributions and a DP detection technique is proposed. The idea behind the proposed DP detection technique is to monitor the algorithm's error-rate. The DP detector triggers DP, warning, and control flags to take necessary steps. The proposed technique is significantly superior in identifying the distribution differences (p-value < 0.05). The DP's identified in the data distributions achieve large effect sizes (Hedges' g ≥ 0.80) during the pair-wise comparisons. We observe that if the error-rate exponentially increases, it causes DP, resulting in prediction performance deterioration. We thus recommend researches and practitioners to address DP in the chronological datasets. Due to its potential effects in the datasets, the prediction models could be enhanced to get the best results in CVDP. © 2020 IEEE.

Noise, defined as an unwanted sound, is one of the most common factors people have to deal with when performing their daily working tasks. Researchers have marginally investigated the effect of noise on software engineers' performance. In this paper, we present the results of two replicated experiments whose main goal was to increase the body of knowledge, by confirming or not the results of the baseline experiments, on the effect of noise while comprehending functional requirements specifications and fixing faults in source code. The results of the replicated experiments suggest that: (i) noise does not significantly affect the comprehension of functional requirements specifications and (ii) noise significantly and negatively affects fixing faults if this task lasts 30 minutes, while it does not have a significant impact if the task lasts 60 minutes. The results of the replications confirm to a large extent those of the baseline experiments and allow us to postulate, as done for the baseline experiments, that fixing faults is more vulnerable to noise than comprehending the specifications of functional requirements. © 2020 IEEE.

The cause-effect recognition (CEREC) system provides an API for causality extraction tailored to the requirements engineering context. The library is written in Java and is released under the MIT open source license. In this paper, the underlying algorithm is described, and a demonstration of the active learning component for causality extraction is outlined. The results are promising and strengthen the confidence in exploring automation approaches for model-based testing. © 2020 IEEE., open access

The cause-effect recognition (CEREC) system provides an API for causality extraction tailored to the requirements engineering context. The library is written in Java and is released under the MIT open source license. In this paper, the underlying algorithm is described, and a demonstration of the active learning component for causality extraction is outlined. The results are promising and strengthen the confidence in exploring automation approaches for model-based testing. © 2020 IEEE., open access

The cause-effect recognition (CEREC) system provides an API for causality extraction tailored to the requirements engineering context. The library is written in Java and is released under the MIT open source license. In this paper, the underlying algorithm is described, and a demonstration of the active learning component for causality extraction is outlined. The results are promising and strengthen the confidence in exploring automation approaches for model-based testing. © 2020 IEEE., open access

[Context] Systematic Literature Reviews (SLRs) have been adopted within the Software Engineering (SE) domain for more than a decade to provide meaningful summaries of evidence on several topics. Many of these SLRs are now outdated, and there are no standard proposals on how to update SLRs in SE. [Objective] The goal of this paper is to provide recommendations on how to best to search for evidence when updating SLRs in SE. [Method] To achieve our goal, we compare and discuss outcomes from applying different search strategies to identifying primary studies in a previously published SLR update on effort estimation. [Results] The use of a single iteration forward snowballing with Google Scholar, and employing the original SLR and its primary studies as a seed set seems to be the most cost-effective way to search for new evidence when updating SLRs. [Conclusions] The recommendations can be used to support decisions on how to update SLRs in SE. © 2019 IEEE.

App store mining has proven to be a promising technique for requirements elicitation as companies can gain valuable knowledge to maintain and evolve existing apps. However, despite first advancements in using mining techniques for requirements elicitation, little is yet known how to distill requirements for new apps based on existing (similar) solutions and how exactly practitioners would benefit from such a technique. In the proposed work, we focus on exploring information (e.g. app store data) provided by the crowd about existing solutions to identify key features of applications in a particular domain. We argue that these discovered features and other related influential aspects (e.g. ratings) can help practitioners(e.g. software developer) to identify potential key features for new applications. To support this argument, we first conducted an interview study with practitioners to understand the extent to which such an approach would find champions in practice. In this paper, we present the first results of our ongoing research in the context of a larger road-map. Our interview study confirms that practitioners see the need for our envisioned approach. Furthermore, we present an early conceptual solution to discuss the feasibility of our approach. However, this manuscript is also intended to foster discussions on the extent to which machine learning can and should be applied to elicit automated requirements on crowd generated data on different forums and to identify further collaborations in this endeavor. © 2019 IEEE.

This paper presents our initial experiences with employing option theory and NPV techniques for optimizing waste reduction in requirements scoping. Inspired by financial market theories, we analyze a large requirements scoping decision making history from the mobile handset domain. We outline how we can optimize waste reduction in requirements scoping by modeling the neutral, positive and negative scenarios, giving each of the scenarios appropriate budget and development team commitment. © 2019 IEEE.

Requirements prioritization plays a vital role in software release planning, and the process requires good interaction between stakeholders. Requirement prioritization process is challenging in global software development (GSD) context, where communication and coordination are core bottlenecks. Based on the issues found in the literature and industry survey, this paper proposes a requirements prioritization technique for GSD settings based on stakeholders' weight, vote, and priority (SWVP). The technique has two phases: (1) a calculation of stakeholders' profile weights and setting up communication parameters and (2) a voting process supporting the actual prioritization, based on the weights from phase (1). The proposed technique was applied to three industrial projects. The feedback of the practitioners after using SWVP was encouraging, and the feedback results indicate that SWVP could be a practical solution for requirements prioritization in GSD contexts. © 2019 IEEE.

Integrating value-oriented perspectives into the principles and practices of software engineering is critical to ensure that software development and management activities address all key stakeholders' views and also balance short-and-long-term goals. This is put forward in the discipline of Value-Based Software Engineering (VBSE). In this paper, a mapping study of VBSE is detailed. We classify evidence on VBSE principles and practices, research methods, and the research types. This mapping study includes 134 studies located from online searches, and backward snowballing of references. Our results show that VB Requirements Engineering (22%) and VB Planning and Control (19%) were the two principles and practices mostly investigated in the VBSE literature, whereas VB Risk Management, VB People Management and Value Creation (3% respectively) were the three less researched. In terms of the research method, the most commonly employed method is case-study research. In terms of research types, most of the studies (28%) proposed solution technique(s) without empirical validation. © 2019 IEEE.

The most common way of performing regression testing is by executing all test cases associated with a software system. However, this approach is not scalable since time and cost to execute the test cases increase together with the system's size. A way to address this consists of prioritizing the existing test cases, aiming to maximize a test suite's fault detection rate. To address the limitations of existing approaches, in this paper we propose a new approach to maximize the rate of fault detection of test suites. Our proposal has three steps: I) infer code components' criticality values using a fuzzy inference system; ii) calculate test cases' criticality; iii) prioritize the test cases using ant colony optimization. The test cases are prioritized considering criticality, execution time and history of faults, and the resulting test suites are evaluated according to their fault detection rate. The evaluation was performed in eight programs, and the results show that the fault detection rate of the solutions was higher than in the non-ordered test suites and ones obtained using a greedy approach, reaching the optimal value when possible to verify. A sanity check was performed, comparing the obtained results to the results of a random search. The approach performed better at significant levels of statistic and practical difference, evidencing its true applicability to the prioritization of test cases. © 2019 IEEE.

Large-scale distributed software projects with long life cycles often involve a considerable amount of complex legacy code. The combination of scale and distribution challenges and the difficulty in acquiring knowledge about massive amounts of complex legacy code may make the onboarding of new developers/teams problematic. These problems may lead to extended periods of low performance. The primary objective of this paper is to investigate the performance evolution of offshore newcomers onboarded in a large-scale globally distributed project and how it relates to the employed onboarding strategy. To achieve our objective, we conducted a case study in Ericsson. We identified that the following aspects in the onboarding strategy employed in the investigated case seem to be related to the unexpectedly low performance evolution: i) the distance to mentors; ii) the used formal training approach, which did not fit the sociocultural background of the newcomers; iii) allocation of large and distributed tasks in the early stages of the onboarding process; and iv) team instability. We conclude that the onboarding of newcomers in globally distributed projects must be planned well ahead and should consider avoiding the aspects mentioned above. © 2019 IEEE.

Offshore outsourcing of software development has been both famous for the promises of great cost reductions, and infamous for the hidden costs associated with the challenges of organizing software work over distance. Experience shows that many of these costs do not receive the deserved attention and are often excluded when making offshoring decisions. As a result, there is often a significant deviation between the expected and the realized costs of offshoring. In this paper, we investigate the awareness of the extra costs when making an offshoring decision, and the significance of the actual cost deviations. We conducted a single case study of a company that carried out an offshore outsourcing pilot project. We collected qualitative data from interviews, observations and a retrospective, and quantitative data on the costs and effort associated with the project. We conclude that the company was aware of the hidden cost factors, but largely underestimated their significance. The costs that surfaced in the studied project accounted for a total deviation of 181% and several individual cost categories with more than 400% overrun. The two main cost drivers in our study were the distance and poor process fit, which escalated the investments needed to make the collaboration work. Our results suggest that pilots are useful to understand the key problem areas in an offshoring collaboration, but too limited to shed light on all potential problems (e.g. turnover) due to the short timeframe. We also conclude that results of pilot projects shall not be the only data source when calculating the true costs of offshoring, since the start-up phase of an offshoring relationship carries large investments. Finally, we provide recommendations for companies in a similar situation on how to run and learn from offshore outsourcing pilot projects. © 2019 IEEE.

Manual testing is the most commonly used approach in the industry today for acceptance-and system-testing of software applications. Test automation has been suggested to address drawbacks with manual testing but both test automation and manual testing have several challenges that limit their return of investment for system-and acceptance-test automation. Hence, there is still an industrial need for another approach to testing that can mitigate the challenges associated with system-and acceptance-testing and make it more efficient and cost effective for the industry. In this paper we present a novel technique we refer to as Augmented Testing (AT). AT is defined as testing through a visual layer between the tester and the System Under Test (SUT) that superimposes information on top of the GUI. We created a prototype for AT and performed an industrial workshop study with 10 software developers to get their perceived benefits and drawbacks of AT. The benefits and drawbacks will be useful for further development of the technique and prototype for AT. The workshop study identified more benefits than drawbacks with AT. Two of the identified benefits were: 'Know what to test and what has been tested' and 'Less manual work'. Due to these results, we believe that AT is a promising technique that deserves more research since it may provide industry with new benefits that current techniques lack. © 2019 IEEE.

This paper presents background, the basic steps and an example for a testability analysis framework for non-functional properties.

Continuous integration (CI) is growing in industrial popularity, spurred on by market trends towards faster delivery and higher quality software. A key facilitator of CI is automated testing that should be executed, automatically, on several levels of system abstraction. However, many systems lack the interfaces required for automated testing. Others lack test automation coverage of the system under test's (SUT) graphical user interface (GUI) as it is shown to the user. One technique that shows promise to solve these challenges is Visual GUI Testing (VGT), which uses image recognition to stimulate and assert the SUT's behavior. Research has presented the technique's applicability and feasibility in industry but only limited support, from an academic setting, that the technique is applicable in a CI environment. In this paper we presents a study from an industrial design research study with the objective to help bridge the gap in knowledge regarding VGT's applicability in a CI environment in industry. Results, acquired from interviews, observations and quantitative analysis of 17.567 test executions, collected over 16 weeks, show that VGT provides similar benefits to other automated test techniques for CI. However, several significant drawbacks, such as high costs, are also identified. The study concludes that, although VGT is applicable in an industrial CI environment, its severe challenges require more research and development before the technique becomes efficient in practice. © 2018 IEEE.

Business process integration (BPI) is a crucial technique for supporting inter-organizational business interoperability. BPI allows automation of business processes and the integration of systems across numerous organizations. The integration of organizations' process models is one of the most addressed and used approach to achieve BPI. However, this model integration is complex and requires that designers have extensive experience in particular when organizations' business processes are incompatible. This paper considers the issue of modeling cross-organization processes out of a collection of organizations' private process models. To this end, we propose six adaptation patterns to resolve incompatibilities when combining organizations' processes. Each pattern is formalized with workflow net. © 2017 IEEE.

Crowd-based requirements engineering (CrowdRE) is promising to derive requirements by gathering and analyzing information from the crowd. Setting up CrowdRE in practice seems challenging, although first solutions to support CrowdRE exist. In this paper, we report on a German software company's experience on crowd involvement by using feedback communication channels and a monitoring solution for user-event data. In our case study, we identified several problem areas that a software company is confronted with to setup an environment for gathering requirements from the crowd. We conclude that a CrowdRE process cannot be implemented ad-hoc and that future work is needed to create and analyze a continuous feedback and monitoring data stream. © 2017 IEEE.

Offshoring software development activities to a remote site in another country continues to be one of the key strategies to save development cost. However, the assumed economic benefits of offshoring are often questionable, due to a large number of hidden costs and too simple cost calculations. This study is a continuation of our work on calculating the true hourly cost that includes the extra direct and indirect costs on top of the salary-based hourly rates. We collected data from an empirical case study conducted in a large international corporation. This corporation develops software-intensive systems and has offshored its ongoing product development from Sweden to a recently on-boarded captive company site in India. In this paper, we report a number of extra costs and their impact on the resulting hourly cost as well as the bottom-line cost per work unit. Our analysis includes quantitative data from corporate archives, and expert-based estimates gathered through focus groups and workshops with company representatives from both the onshore and the offshore sites. Our findings show that there is additional cost that can be directly or at least strongly attributed to the transfer of work, working on a distance, and immaturity of the offshore site. Consideration of extra costs increases the hourly cost several times, while the performance gaps between the mature sites and the immature site leads to an even higher difference. As a result, two years after on-boarding of the offshore teams, the mature teams in high-cost locations continue to be 'cheaper' despite the big salary differences, and the most positive hypothetical scenario, in which the company could break even, is unrealistic. The implications of our findings are twofold. First, offshoring of complex ongoing products does not seem to lead to short-term bottom-line economic gains, and may not even reach breakeven within five years. Second, offshoring in the studied case can be justified but merely when initia

Collaboration between industry and academia in software testing leads to improvement and innovation in industry, and it is the basis for achieving transferable and empirically evaluated results. Thus, the aim of TAIC PART is to forge collaboration between industry and academia on the challenging and exciting problem of real-world software testing. The workshop is promoted by representatives of both industry and academia, bringing together industrial software engineers and testers with researchers working on theory and practice of software testing. We present an overview of the 12th Workshop on Testing: Academia-Industry Collaboration, Practice and Research Techniques (TAIC PART 2017) and its contributions. © 2017 IEEE., Conference code: 127387; Export Date: 18 May 2017; Conference Paper

One form of robustness in a software system is its ability to handle, in an appropriate manner, inputs that are unexpected compared to those it would experience in normal operation. In this paper we investigate a generic approach to generating such unexpected test inputs by extending a framework that we have previously developed for the automated creation of complex and high-structured test data. The approach is applied to the generation of valid inputs that are atypical as well as inputs that are invalid. We demonstrate that our approach enables control of the 'degree' to which the test data is invalid or atypical, and show empirically that this can alter the extent to which the robustness of a software system is exercised during testing. © 2017 IEEE.

Context: Software is an important part in safety- critical system (SCS) development since it is becoming a major source of hazards. Requirements-related hazards have been as- sociated with many accidents and safety incidents. Requirements issues tend to be mitigated in companies with high processes maturity levels since they do their business in a systematic, consistent and proactive approach. However, requirements en- gineers need systematic guidance to consider safety concerns early in the development process. Goal: the paper investigates which safety practices are suitable to be used in the Requirements Engineering (RE) process for SCS and how to design a safety maturity model for this area. Method: we followed the design science methodology to propose Uni-REPM SCS, a safety module for Unified Requirements Engineering Process Maturity Model (Uni-REPM). We also conducted a static validation with two practitioners and nine academic experts to evaluate its coverage, correctness, usefulness and applicability. Results: The module has seven main processes, fourteen sub-processes and 148 practices that form the basis of safety processes maturity. Moreover, we describe its usage through a tool. Conclusions: The validation indicates a good coverage of practices and well receptivity by the experts. Finally, the module can help companies in evaluating their current practices. IEEE

Software development is governed by guidelines that aim to improve the code's qualities, such as maintainability. However, whilst coding guidelines are commonplace for software, guidelines for testware are much less common. In particular, for GUI-based tests driven with image recognition, also referred to as Visual GUI Testing (VGT), explicit coding guidelines are missing.In this industrial case study, performed at the Swedish defence contractor Saab AB, we propose a set of coding guidelines for VGT and evaluate their impact on test scripts for an industrial, safety-critical system. To study the guidelines' effect on maintenance costs, five representative manual test cases are each translated with and without the proposed guidelines in the two VGT tools SikuliX and EyeAutomate. As such, 20 test scripts were developed, with a combined development cost of more than 100 man-hours. Three of the tests are then maintained by one researcher and two practitioners for another version of the system and costs measured to evaluate return on investment. This analysis is complemented with observations and interviews to elicit practitioners' perceptions and experiences with VGT.Results show that scripts developed with the guidelines had higher maintenance costs than scripts developed without guidelines. This is supported by qualitative results that many of the guidelines are considered inappropriate, superfluous or unnecessary due to the inherent properties of the scripts, e.g. their natural small size, linear flows, natural separation of concerns, and more. We conclude that there are differences between VGT scripts and software that prohibit direct translation of guidelines between the two. As such, we consider our study as a failure but argue that several lessons can be drawn from our results to guide future research into guidelines for VGT and GUI-based test automation. © 2021 IEEE., open access

Software development is governed by guidelines that aim to improve the code's qualities, such as maintainability. However, whilst coding guidelines are commonplace for software, guidelines for testware are much less common. In particular, for GUI-based tests driven with image recognition, also referred to as Visual GUI Testing (VGT), explicit coding guidelines are missing.In this industrial case study, performed at the Swedish defence contractor Saab AB, we propose a set of coding guidelines for VGT and evaluate their impact on test scripts for an industrial, safety-critical system. To study the guidelines' effect on maintenance costs, five representative manual test cases are each translated with and without the proposed guidelines in the two VGT tools SikuliX and EyeAutomate. As such, 20 test scripts were developed, with a combined development cost of more than 100 man-hours. Three of the tests are then maintained by one researcher and two practitioners for another version of the system and costs measured to evaluate return on investment. This analysis is complemented with observations and interviews to elicit practitioners' perceptions and experiences with VGT.Results show that scripts developed with the guidelines had higher maintenance costs than scripts developed without guidelines. This is supported by qualitative results that many of the guidelines are considered inappropriate, superfluous or unnecessary due to the inherent properties of the scripts, e.g. their natural small size, linear flows, natural separation of concerns, and more. We conclude that there are differences between VGT scripts and software that prohibit direct translation of guidelines between the two. As such, we consider our study as a failure but argue that several lessons can be drawn from our results to guide future research into guidelines for VGT and GUI-based test automation. © 2021 IEEE., open access

Software development is governed by guidelines that aim to improve the code's qualities, such as maintainability. However, whilst coding guidelines are commonplace for software, guidelines for testware are much less common. In particular, for GUI-based tests driven with image recognition, also referred to as Visual GUI Testing (VGT), explicit coding guidelines are missing.In this industrial case study, performed at the Swedish defence contractor Saab AB, we propose a set of coding guidelines for VGT and evaluate their impact on test scripts for an industrial, safety-critical system. To study the guidelines' effect on maintenance costs, five representative manual test cases are each translated with and without the proposed guidelines in the two VGT tools SikuliX and EyeAutomate. As such, 20 test scripts were developed, with a combined development cost of more than 100 man-hours. Three of the tests are then maintained by one researcher and two practitioners for another version of the system and costs measured to evaluate return on investment. This analysis is complemented with observations and interviews to elicit practitioners' perceptions and experiences with VGT.Results show that scripts developed with the guidelines had higher maintenance costs than scripts developed without guidelines. This is supported by qualitative results that many of the guidelines are considered inappropriate, superfluous or unnecessary due to the inherent properties of the scripts, e.g. their natural small size, linear flows, natural separation of concerns, and more. We conclude that there are differences between VGT scripts and software that prohibit direct translation of guidelines between the two. As such, we consider our study as a failure but argue that several lessons can be drawn from our results to guide future research into guidelines for VGT and GUI-based test automation. © 2021 IEEE., open access

Software development is governed by guidelines that aim to improve the code's qualities, such as maintainability. However, whilst coding guidelines are commonplace for software, guidelines for testware are much less common. In particular, for GUI-based tests driven with image recognition, also referred to as Visual GUI Testing (VGT), explicit coding guidelines are missing.In this industrial case study, performed at the Swedish defence contractor Saab AB, we propose a set of coding guidelines for VGT and evaluate their impact on test scripts for an industrial, safety-critical system. To study the guidelines' effect on maintenance costs, five representative manual test cases are each translated with and without the proposed guidelines in the two VGT tools SikuliX and EyeAutomate. As such, 20 test scripts were developed, with a combined development cost of more than 100 man-hours. Three of the tests are then maintained by one researcher and two practitioners for another version of the system and costs measured to evaluate return on investment. This analysis is complemented with observations and interviews to elicit practitioners' perceptions and experiences with VGT.Results show that scripts developed with the guidelines had higher maintenance costs than scripts developed without guidelines. This is supported by qualitative results that many of the guidelines are considered inappropriate, superfluous or unnecessary due to the inherent properties of the scripts, e.g. their natural small size, linear flows, natural separation of concerns, and more. We conclude that there are differences between VGT scripts and software that prohibit direct translation of guidelines between the two. As such, we consider our study as a failure but argue that several lessons can be drawn from our results to guide future research into guidelines for VGT and GUI-based test automation. © 2021 IEEE., open access