Search

Machine Learning (ML) algorithms are widely used in building software-intensive systems, including safety-critical ones. Unlike traditional software components, Machine-Learned Components (MLC)s, software components built using ML algorithms, learn their specifications through generalizing the common features that they find in a limited set of collected examples. While this inductive nature overcomes the limitations of programming hard-to-specify concepts, the same feature becomes problematic for verifying safety in ML-based software systems. One reason is that, due to MLCs data-driven nature, there is often no set of explicitly written and pre-defined specifications, against which the MLC can be verified. In this regard, we propose to partially specify hard-to-specify domain concepts, which MLCs tend to classify, instead of fully relying on their inductive learning ability from arbitrarily-collected datasets. In this paper, we propose a semi-automated approach to construct a multi-level semantic web to partially outline the hard-to-specify, yet crucial, domain concept "pedestrian" in automotive domain. We evaluate the applicability of the generated semantic web in two ways: first, with a reference to the web, we augment a pedestrian dataset for a missing feature, wheelchair, to show training a state-of-the-art ML-based object detector on the augmented dataset improves its accuracy in detecting pedestrians; second, we evaluate the coverage of the generated semantic web based on multiple state-of-the-art pedestrian and human datasets. [ABSTRACT FROM AUTHOR]

Requirements engineering is one of the most important areas of software engineering because it enables practitioners to determine the characteristics and constraints of the software to be developed. Therefore, the requirements elicitation process aims to obtain a preliminary version of the requirements before their specification, but it is difficult to know if the elicited requirements have the desired attributes (i.e., these requirements are complete, correct, consistent, etc.) due to the nature of this process. There are some measures that allow requirements engineers to quantify certain aspects of the requirements elicitation process and its final product (i.e., the preliminary version of the requirements, also known as preliminary requirements), but there is no way of measuring and controlling the quality of the elicited preliminary requirements. With the aim of understanding, evaluating, and improving the activities of the requirements elicitation process, some of the existing measures for this process were analyzed and, consequently, a case study was conducted to define and implement a measurement program with a set of eight measures that are proposed in this study. This program is based on the ISO/IEC/IEEE 15939:2017—systems and software engineering—measurement process, an international standard that establishes a common process and framework for the measurement of systems and software. The measurement program corresponds to a lightweight implementation of the standard in the context of a small-sized software organization (where the size of the software staff is 11–50 people, the size of projects is 50,000–100,000 LOC, and the time spent on the projects is six to twelve months) and the obtained results showed a preliminary positive influence when obtaining high-quality preliminary requirements. [ABSTRACT FROM AUTHOR]

Recommender systems for requirements are typically built on the assumption that similar requirements can be used as proxies to retrieve similar software. When a stakeholder proposes a new requirement, natural language processing (NLP)-based similarity metrics can be exploited to retrieve existing requirements, and in turn, identify previously developed code. Several NLP approaches for similarity computation between requirements are available. However, there is little empirical evidence on their effectiveness for code retrieval. This study compares different NLP approaches, from lexical ones to semantic, deep-learning techniques, and correlates the similarity among requirements with the similarity of their associated software. The evaluation is conducted on real-world requirements from two industrial projects from a railway company. Specifically, the most similar pairs of requirements across two industrial projects are automatically identified using six language models. Then, the trace links between requirements and software are used to identify the software pairs associated with each requirements pair. The software similarity between pairs is then automatically computed with JPLag. Finally, the correlation between requirements similarity and software similarity is evaluated to see which language model shows the highest correlation and is thus more appropriate for code retrieval. In addition, we perform a focus group with members of the company to collect qualitative data. Results show a moderately positive correlation between requirements similarity and software similarity, with the pre-trained deep learning-based BERT language model with preprocessing outperforming the other models. Practitioners confirm that requirements similarity is generally regarded as a proxy for software similarity. However, they also highlight that additional aspect comes into play when deciding software reuse, e.g., domain/project knowledge, information coming from test cases, and trace links. Our work is among the first ones to explore the relationship between requirements and software similarity from a quantitative and qualitative standpoint. This can be useful not only in recommender systems but also in other requirements engineering tasks in which similarity computation is relevant, such as tracing and change impact analysis. [ABSTRACT FROM AUTHOR]

Many software users give feedback online about the applications they use. This feedback often contains valuable requirements information that can be used to guide the effective maintenance and evolution of a software product. Yet, not all software users give online feedback. If the demographics of a user-base aren't fairly represented, there is a danger that the needs of less vocal users won't be considered in development. This work investigates feedback on three prominent online channels: app stores, product forums, and social media. We directly survey software users about their feedback habits, as well as what motivates and dissuades them from providing feedback online. In an initial survey of 1040 software users, we identify statistically significant differences in the demographics of users who give feedback (gender, age, etc.), and key differences in what motivates them to engage with each of the three studied channels. In a second survey of 936 software users, we identify the top reasons users don't give feedback, including significant differences between demographic groups. We also present a detailed list of user-rated methods to encourage their feedback. This work provides meaningful context for requirements sourced from online feedback, identifying demographic groups who are underrepresented. Findings on what motivates and discourages user feedback give insight into how feedback channels and developers can increase engagement with their user-base. [ABSTRACT FROM AUTHOR]

A correction to this paper has been published: https://doi.org/10.1007/s00766-021-00354-4 [ABSTRACT FROM AUTHOR]

Expert judgement is a common method for software effort estimations in practice today. Estimators are often shown extra obsolete requirements together with the real ones to be implemented. Only one previous study has been conducted on if such practices bias the estimations. We conducted six experiments with both students and practitioners to study, and quantify, the effects of obsolete requirements on software estimation. By conducting a family of six experiments using both students and practitioners as research subjects ( N = 461 ), and by using a Bayesian Data Analysis approach, we investigated different aspects of this effect. We also argue for, and show an example of, how we by using a Bayesian approach can be more confident in our results and enable further studies with small sample sizes. We found that the presence of obsolete requirements triggered an overestimation in effort across all experiments. The effect, however, was smaller in a field setting compared to using students as subjects. Still, the over-estimations triggered by the obsolete requirements were systematically around twice the percentage of the included obsolete ones, but with a large 95% credible interval. The results have implications for both research and practice in that the found systematic error should be accounted for in both studies on software estimation and, maybe more importantly, in estimation practices to avoid over-estimations due to this systematic error. We partly explain this error to be stemming from the cognitive bias of anchoring-and-adjustment, i.e. the obsolete requirements anchored a much larger software. However, further studies are needed in order to accurately predict this effect. [ABSTRACT FROM AUTHOR]