Search

Weak alignment of requirements engineering (RE) with verification and validation (VV) may lead to problems in delivering the required products in time with the right quality. For example, weak communication of requirements changes to testers may result in lack of verification of new requirements and incorrect verification of old invalid requirements, leading to software quality problems, wasted effort and delays. However, despite the serious implications of weak alignment research and practice both tend to focus on one or the other of RE or VV rather than on the alignment of the two. We have performed a multi-unit case study to gain insight into issues around aligning RE and VV by interviewing 30 practitioners from 6 software developing companies, involving 10 researchers in a flexible research process for case studies. The results describe current industry challenges and practices in aligning RE with VV, ranging from quality of the individual RE and VV activities, through tracing and tools, to change control and sharing a common understanding at strategy, goal and design level. The study identified that human aspects are central, i.e. cooperation and communication, and that requirements engineering practices are a critical basis for alignment. Further, the size of an organisation and its motivation for applying alignment practices, e.g. external enforcement of traceability, are variation factors that play a key role in achieving alignment. Our results provide a strategic roadmap for practitioners improvement work to address alignment challenges. Furthermore, the study provides a foundation for continued research to improve the alignment of RE with VV.

[Context and motivation] When developing software, coordination between different organizational units is essential in order to develop a good quality product, on time and within budget. Particularly, the synchronization between requirements and verification processes is crucial in order to assure that the developed software product satisfies customer requirements. [Question/problem] Our research question is: what are the current challenges in aligning the requirements and verification processes? [Principal ideas/results] We conducted an interview study at a large software development company. This paper presents preliminary findings of these interviews that identify key challenges in aligning requirements and verification processes. [Contribution] The result of this study includes a range of challenges faced by the studied organization grouped into the categories: organization and processes, people, tools, requirements process, testing process, change management, traceability, and measurement. The findings of this study can be used by practitioners as a basis for investigating alignment in their organizations, and by scientists in developing approaches for more efficient and effective management of the alignment between requirements and verification., Comment: Requirements Engineering: Foundation for Software Quality: 16th International Working Conference, REFSQ 2010, Essen, Germany, June 30-July 2, 2010. Proceedings 16 (pp. 128-142). Springer Berlin Heidelberg

Testing autonomous driving systems for safety and reliability is extremely complex. A primary challenge is identifying the relevant test scenarios, especially the critical ones that may expose hazards or risks of harm to autonomous vehicles and other road users. There are several proposed methods and tools for critical scenario identification, while the industry practices, such as the selection, implementation, and limitations of the approaches, are not well understood. In this study, we conducted 10 interviews with 13 interviewees from 7 companies in autonomous driving in Sweden. We used thematic modeling to analyse and synthesize the interview data. We found there are little joint efforts in the industry to explore different approaches and tools, and every approach has its own limitations and weaknesses. To that end, we recommend combining different approaches available, collaborating among different stakeholders, and continuously learning the field of critical scenario identification and testing. The contributions of our study are the exploration and synthesis of the industry practices and related challenges for critical scenario identification and testing, and the potential increase of the industry relevance for future studies in related topics., Comment: 29 pages, 3 figures, submitted to a journal

Continuous experimentation (CE) is used by many companies with internet-facing products to improve their software based on user data. Some companies deliberately adopt an experiment-driven approach to software development while some companies use CE in a more ad-hoc fashion. The goal of the study is to identify factors that explain the variations in the utility and efficacy of CE between different companies. We conducted a multi-case study of 12 companies involved with CE and performed 27 interviewees with practitioners at these companies. Based on that empirical data, we then built a theory of factors at play in CE. We introduce a theory of Factors Affecting Continuous Experimentation (FACE). The theory includes three factors, namely 1) processes and infrastructure for CE, 2) the user problem complexity of the product offering, and 3) incentive structures for CE. It explains how these factors affect the effectiveness of CE and its ability to achieve problem-solution and product-market fit. Our theory can be used by practitioners to assess an organisation's potential for adopting CE, as well as, identifying factors which pose challenges in gaining value from CE practices. Our results also provide a starting point for further research on how contextual factors affect CE and how these may be mitigated.

Motivation. Digital commons is an emerging phenomenon and of increasing importance, as we enter a digital society. Open data is one example that makes up a pivotal input and foundation for many of today's digital services and applications. Ensuring sustainable provisioning and maintenance of the data, therefore, becomes even more important. Aim. We aim to investigate how such provisioning and maintenance can be collaboratively performed in the community surrounding a common. Specifically, we look at Open Data Ecosystems (ODEs), a type of community of actors, openly sharing and evolving data on a technological platform. Method. We use Elinor Ostrom's design principles for Common Pool Resources as a lens to systematically analyze the governance of earlier reported cases of ODEs using a theory-oriented software engineering framework. Results. We find that, while natural commons must regulate consumption, digital commons such as open data maintained by an ODE must stimulate both use and data provisioning. Governance needs to enable such stimulus while also ensuring that the collective action can still be coordinated and managed within the frame of available maintenance resources of a community. Subtractability is, in this sense, a concern regarding the resources required to maintain the quality and value of the data, rather than the availability of data. Further, we derive empirically-based recommended practices for ODEs based on the design principles by Ostrom for how to design a governance structure in a way that enables a sustainable and collaborative provisioning and maintenance of the data. Conclusion. ODEs are expected to play a role in data provisioning which democratize the digital society and enables innovation from smaller commercial actors. Our empirically based guidelines intend to support this development., Comment: Accepted for publication at the 18th International Symposium on Open Collaboration (OpenSym 2022)

Context. Innovation is promoted in companies to help them stay competitive. Four types of innovation are defined: product, process, business, and organizational. Objective. We want to understand the perception of the innovation concept in industry, and particularly how the innovation types relate to each other. Method. We launched a survey at a branch of a multi-national corporation. Results. From a qualitative analysis of the 229 responses, we see that the understanding of the innovation concept is somewhat narrow, and mostly related to product innovation. A majority of respondents indicate that product innovation triggers process, business, and organizational innovation, rather than vice versa. However, there is a complex interdependency between the types. We also identify challenges related to each of the types. Conclusion. Increasing awareness and knowledge of different types of innovation, may improve the innovation. Further, they cannot be handled one by one, but in their interdependent relations.

Background: Open innovation highlights the potential benefits of external collaboration and knowledge-sharing, often exemplified through Open Source Software (OSS). The public sector has thus far mainly focused on the sharing of Open Government Data (OGD), often with a supply-driven approach with limited feedback-loops. We hypothesize that public sector organizations can extend the open innovation benefits by also creating platforms, where OGD, related OSS, and open standards are collaboratively developed and shared. Objective: The objective of this study is to explore how public sector organizations in the role of platform providers facilitate such collaboration in the form of OGD ecosystems and how the ecosystem's governance may be structured to support the collaboration. Method: We conduct an exploratory multiple-case study of two such ecosystems, focused on OGD related to the Swedish labor market and public transport sector, respectively. Data is gathered through interviews, document studies, and prolonged engagement at one of the platform providers. Results: The study presents governance structure and collaboration practices of the two ecosystems and discusses how these contribute to the platform providers' goals. The case studies highlight the need for platform providers to take an active and multi-functional role in enabling the sharing of data and software from and between the members of the ecosystem. Conclusions: We conclude that OGD ecosystems offer public sector organizations a possibility to catalyze the potential innovation output of OGD, but that it requires investment and adoption of an open and collaborative mindset., Comment: arXiv admin note: substantial text overlap with arXiv:2208.00305

Background: By creating ecosystems around platforms of Open Source Software (OSS) and Open Data (OD), and adopting open collaborative development practices, platform providers may exploit open innovation benefits. However, adopting such practices in a traditionally closed organization is a maturity process that we hypothesize cannot be undergone without friction. Objective: This study aims to investigate what challenges may occur for a newly-turned platform provider in the public sector, aiming to adopt open collaborative practices to create an ecosystem around the development of the underpinning platform. Method: An exploratory case-study is conducted at a Swedish public sector platform provider, which is creating an ecosystem around OSS and OD, related to the labor market. Data is collected through interviews, document studies, and prolonged engagement. Results: Findings highlight a fear among developers of being publicly questioned for their work, as they represent a government agency undergoing constant scrutiny. Issue trackers, roadmaps, and development processes are generally closed, while multiple channels are used for communication, causing internal and external confusion. Some developers are reluctant to communicate externally as they believe it interferes with their work. Lack of health metrics limits possibilities to follow ecosystem growth and for actors to make investment decisions. Further, an autonomous team structure is reported to complicate internal communication and enforcement of the common vision, as well as collaboration. A set of interventions for addressing the challenges are proposed, based on related work. Conclusions: We conclude that several cultural, organizational, and process-related challenges may reside, and by understanding these early on, platform providers can be preemptive in their work of building healthy ecosystems.

High-quality data has become increasingly important to software engineers in designing and implementing today's software, for example, as an input to machine-learning algorithms and visualisation- and analytics-based features. Open data - i.e., data shared under a licence that gives users the right to study, process, and distribute the data to anyone and for any purpose - offers a mechanism to address this need. Data may originate from multiple sources, whether crowdsourced, shared by government agencies, or shared between commercial entities, and is undoubtedly inherent to all business and revenue models across the public sector, business and industry today. In this guest editorial for the Special Issue on Collaborative Aspects of Open Data in Software Engineering, we explore the collaborative aspects of open data in software engineering. We highlight how these aspects can benefit organisations, what challenges may exist and how these may be addressed based on current practice, and introduce the four papers included in this special issue.

Despite growing interest of Open Innovation (OI) in Software Engineering (SE), little is known about what triggers software organizations to adopt it and how this affects SE practices. OI can be realized in numerous of ways, including Open Source Software (OSS) involvement. Outcomes from OI are not restricted to product innovation but also include process innovation, e.g. improved SE practices and methods. This study explores the involvement of a software organization (Sony Mobile) in OSS communities from an OI perspective and what SE practices (requirements engineering and testing) have been adapted in relation to OI. It also highlights the innovative outcomes resulting from OI. An exploratory embedded case study investigates how Sony Mobile use and contribute to Jenkins and Gerrit; the two central OSS tools in their continuous integration tool chain. Quantitative analysis was performed on change log data from source code repositories in order to identify the top contributors and triangulated with the results from five semi-structured interviews to explore the nature of the commits. The findings of the case study include five major themes: i) The process of opening up towards the tool communities correlates in time with a general adoption of OSS in the organization. ii) Assets not seen as competitive advantage nor a source of revenue are made open to OSS communities, and gradually, the organization turns more open. iii) The requirements engineering process towards the community is informal and based on engagement. iv) The need for systematic and automated testing is still in its infancy, but the needs are identified. v) The innovation outcomes included free features and maintenance, and were believed to increase speed and quality in development. Adopting OI was a result of a paradigm shift of moving from Windows to Linux.

Objective: Our objective is to explore how public entities in the role of platform providers can address this issue by enabling collaboration within their OGD ecosystems, both in terms of the OGD published on the underpinning platform, as well as any related Open Source Software (OSS) and standards. Method: We conducted an exploratory multiple-case study of four OGD ecosystems with diverse characteristics. Data was collected through semi-structured interviews, and in one of the cases through a prolonged engagement. The data was then coded using a set of \textit{apriori} codes. Results: The study descriptively presents each case based on the coding, along with synthesis in the form of a conceptual model that highlights different attributes of OGD ecosystems. For example, we observe how collaboration can be enabled through different types of ownership of the platform provider, how the ecosystem's scope can vary, what roles the platform provider may undertake, how to enable open collaboration, and how to collaborate in terms of data sharing, OSS development, and standards. For each aspect, we provide recommendations based on the explored cases that, together with the model, may help public entities in designing and orchestrating new or existing OGD ecosystems. Conclusions: We conclude that enabling and facilitating collaboration in an OGD ecosystem is a complex exercise, yet believe that it offers new ways for public entities in how they can leverage the power of open innovation to address their goals and directives.

Software systems are increasingly depending on data, particularly with the rising use of machine learning, and developers are looking for new sources of data. Open Data Ecosystems (ODE) is an emerging concept for data sharing under public licenses in software ecosystems, similar to Open Source Software (OSS). It has certain similarities to Open Government Data (OGD), where public agencies share data for innovation and transparency. We aimed to explore open data ecosystems involving commercial actors. Thus, we organized five focus groups with 27 practitioners from 22 companies, public organizations, and research institutes. Based on the outcomes, we surveyed three cases of emerging ODE practice to further understand the concepts and to validate the initial findings. The main outcome is an initial conceptual model of ODEs' value, intrinsics, governance, and evolution, and propositions for practice and further research. We found that ODE must be value driven. Regarding the intrinsics of data, we found their type, meta-data, and legal frameworks influential for their openness. We also found the characteristics of ecosystem initiation, organization, data acquisition and openness be differentiating, which we advise research and practice to take into consideration.

Objective: The purpose of this paper is to identify the largest cognitive challenges faced by novices developing software in teams. Method: Using grounded theory, we conducted an ethnographic study for two months following four ten person novice teams, consisting of computer science students, developing software systems. Result: This paper identifies version control and merge operations as the largest challenge faced by the novices. The literature studies reveal that little research appears to have been carried out in the area of version control from a user perspective. Limitations: A qualitative study on students is not applicable in all contexts, but the result is credible and grounded in data and substantiated by extant literature. Conclusion: We conclude that our findings motivate further research on cognitive perspectives to guide improvement of software engineering and its tools., Comment: 11 pages

Testing of autonomous systems is extremely important as many of them are both safety-critical and security-critical. The architecture and mechanism of such systems are fundamentally different from traditional control software, which appears to operate in more structured environments and are explicitly instructed according to the system design and implementation. To gain a better understanding of autonomous systems practice and facilitate research on testing of such systems, we conducted an exploratory study by synthesizing academic literature with a focus group discussion and interviews with industry practitioners. Based on thematic analysis of the data, we provide a conceptualization of autonomous systems, classifications of challenges and current practices as well as of available techniques and approaches for testing of autonomous systems. Our findings also indicate that more research efforts are required for testing of autonomous systems to improve both the quality and safety aspects of such systems., Comment: 8 pages, 5 figures, conference

Context: Continuous experimentation and A/B testing is an established industry practice that has been researched for more than 10 years. Our aim is to synthesize the conducted research. Objective: We wanted to find the core constituents of a framework for continuous experimentation and the solutions that are applied within the field. Finally, we were interested in the challenges and benefits reported of continuous experimentation. Method: We applied forward snowballing on a known set of papers and identified a total of 128 relevant papers. Based on this set of papers we performed two qualitative narrative syntheses and a thematic synthesis to answer the research questions. Results: The framework constituents for continuous experimentation include experimentation processes as well as supportive technical and organizational infrastructure. The solutions found in the literature were synthesized to nine themes, e.g. experiment design, automated experiments, or metric specification. Concerning the challenges of continuous experimentation, the analysis identified cultural, organizational, business, technical, statistical, ethical, and domain-specific challenges. Further, the study concludes that the benefits of experimentation are mostly implicit in the studies. Conclusions: The research on continuous experimentation has yielded a large body of knowledge on experimentation. The synthesis of published research presented within include recommended infrastructure and experimentation process models, guidelines to mitigate the identified challenges, and what problems the various published solutions solve.

Background: Assessing and communicating software engineering research can be challenging. Design science is recognized as an appropriate research paradigm for applied research but is seldom referred to in software engineering. Applying the design science lens to software engineering research may improve the assessment and communication of research contributions. Aim: The aim of this study is 1) to understand whether the design science lens helps summarize and assess software engineering research contributions, and 2) to characterize different types of design science contributions in the software engineering literature. Method: In previous research, we developed a visual abstract template, summarizing the core constructs of the design science paradigm. In this study, we use this template in a review of a set of 38 top software engineering publications to extract and analyze their design science contributions. Results: We identified five clusters of papers, classifying them according to their alignment with the design science paradigm. Conclusions: The design science lens helps emphasize the theoretical contribution of research output---in terms of technological rules---and reflect on the practical relevance, novelty, and rigor of the rules proposed by the research., Comment: 32 pages, 10 figures

Exploratory testing (ET) is a powerful and efficient way of testing software by integrating design, execution, and analysis of tests during a testing session. ET is often contrasted with scripted testing, and seen as a choice between black and white. We pose that there are different levels of exploratory testing from fully exploratory to fully scripted and propose a scale for the degree of exploration for ET. The degree is defined through levels of ET, which correspond to the way test charters are formulated. We have evaluated the classification through focus groups at four companies and identified factors that influence the level of exploratory testing. The results show that the proposed ET levels have distinguishing characteristics and that the levels can be used as a guide to structure test charters. Our study also indicates that applying a combination of ET levels can be beneficial in achieving effective testing., Comment: Submitted to IEEE Software

Software engineers working in large projects must navigate complex information landscapes. Change Impact Analysis (CIA) is a task that relies on engineers' successful information seeking in databases storing, e.g., source code, requirements, design descriptions, and test case specifications. Several previous approaches to support information seeking are task-specific, thus understanding engineers' seeking behavior in specific tasks is fundamental. We present an industrial case study on how engineers seek information in CIA, with a particular focus on traceability and development artifacts that are not source code. We show that engineers have different information seeking behavior, and that some do not consider traceability particularly useful when conducting CIA. Furthermore, we observe a tendency for engineers to prefer less rigid types of support rather than formal approaches, i.e., engineers value support that allows flexibility in how to practically conduct CIA. Finally, due to diverse information seeking behavior, we argue that future CIA support should embrace individual preferences to identify change impact by empowering several seeking alternatives, including searching, browsing, and tracing., Comment: Accepted for publication in the proceedings of the 25th International Conference on Program Comprehension