Search

[Context] Controlled experiments are an important empirical method to generate and validate theories. Many software engineering experiments are conducted with students. It is often claimed that the use of students as participants in experiments comes at the cost of low external validity while using professionals does not. [Objective] We believe a deeper understanding is needed on the external validity of software engineering experiments conducted with students or with professionals. We aim to gain insight about the pros and cons of using students and professionals in experiments. [Method] We performed an unconventional, focus group approach and a follow-up survey. First, during a session at ISERN 2014, 65 empirical researchers, including the seven authors, argued and discussed the use of students in experiments with an open mind. Afterwards, we revisited the topic and elicited experts’ opinions to foster discussions. Then we derived 14 statements and asked the ISERN attendees excluding the authors, to provide their level of agreement with the statements. Finally, we analyzed the researchers’ opinions and used the findings to further discuss the statements. [Results] Our survey results showed that, in general, the respondents disagreed with us about the drawbacks of professionals. We, on the contrary, strongly believe that no population (students, professionals, or others) can be deemed better than another in absolute terms. [Conclusion] Using students as participants remains a valid simplification of reality needed in laboratory contexts. It is an effective way to advance software engineering theories and technologies but, like any other aspect of study settings, should be carefully considered during the design, execution, interpretation, and reporting of an experiment. The key is to understand which developer population portion is being represented by the participants in an experiment. Thus, a proposal for describing experimental participants is put forward. [ABSTRACT FROM AUTHOR]

Several factors make empirical research in software engineering particularly challenging as it requires studying not only technology but its stakeholders' activities while drawing concepts and theories from social science. Researchers, in general, agree that selecting a research design in empirical software engineering research is challenging, because the implications of using individual research methods are not well recorded. The main objective of this article is to make researchers aware and support them in their research design, by providing a foundation of knowledge about empirical software engineering research decisions, in order to ensure that researchers make well-founded and informed decisions about their research designs. This article provides a decision-making structure containing a number of decision points, each one of them representing a specific aspect on empirical software engineering research. The article provides an introduction to each decision point and its constituents, as well as to the relationships between the different parts in the decision-making structure. The intention is the structure should act as a starting point for the research design before going into the details of the research design chosen. The article provides an in-depth discussion of decision points in relation to the research design when conducting empirical research. [ABSTRACT FROM AUTHOR]

There is a plethora of studies investigating object-oriented measures and their link with external quality attributes, but usefulness of the measures may differ across empirical studies. This study aims to aggregate and identify useful object-oriented measures, specifically those obtainable from the source code of object-oriented systems that have gone through such empirical evaluation. By conducting a systematic literature review, 99 primary studies were identified and traced to four external quality attributes: reliability, maintainability, effectiveness and functionality. A vote-counting approach was used to investigate the link between object-oriented measures and the attributes, and to also assess the consistency of the relation reported across empirical studies. Most of the studies investigate links between object-oriented measures and proxies for reliability attributes, followed by proxies for maintainability. The least investigated attributes were: effectiveness and functionality. Measures from the C&K measurement suite were the most popular across studies. Vote-counting results suggest that complexity, cohesion, size and coupling measures have a better link with reliability and maintainability than inheritance measures. However, inheritance measures should not be overlooked during quality assessment initiatives; their link with reliability and maintainability could be context dependent. There were too few studies traced to effectiveness and functionality attributes; thus a meaningful vote-counting analysis could not be conducted for these attributes. Thus, there is a need for diversification of quality attributes investigated in empirical studies. This would help with identifying useful measures during quality assessment initiatives, and not just for reliability and maintainability aspects. [ABSTRACT FROM AUTHOR]

Many organizations nowadays strive for utilization of benefits offered by global software engineering (GSE) and sourcing strategies are thus discussed more often. Since there are so many variations of the attributes associated with global software projects a large amount of new terms has been introduced. The diversity in sourcing jargon however has caused difficulties in determining which term to use in which situation, and thus causing further obstacles to searching and finding relevant research during e.g. systematic literature reviews. The inability of judging the applicability of the research in an industrial context is another important implication on the transferability of research into practice. Thus the need for accurate terminology and definitions for different global sourcing situations emerges as a way for the community to build upon each other's work and hence making progress more quickly. In this paper we first investigate the state of the use of the GSE jargon concluding that terminology is very diverse (many synonyms used to describe the same phenomena), often confusing (same terms used to describe different phenomena) and occasionally ambiguous (few terms used to describe several phenomena). In order to address the identified problems, we conducted a Delphi-inspired study with ten well-established researchers in GSE and developed an empirically based glossary for the key concepts in global software engineering. We then propose a taxonomy for GSE by categorizing the selected terms based on generalization-specialization relationships and illustrate how the taxonomy can be used to categorize and map existing knowledge. The contribution targets future researchers, who will publish or synthesize further empirical work and practitioners, who are interested in published empirical cases. Therefore this work is expected to make a contribution to the future development of research in the GSE field, and alleviate understandability and transferability of existing and future knowledge into practice. [ABSTRACT FROM AUTHOR]

Context: Code hardening is meant to fight malicious tampering with sensitive code executed on client hosts. Code splitting is a hardening technique that moves selected chunks of code from client to server. Although widely adopted, the effective benefits of code splitting are not fully understood and thoroughly assessed. Objective: The objective of this work is to compare non protected code vs. code splitting protected code, considering two levels of the chunk size parameter, in order to assess the effectiveness of the protection - in terms of both attack time and success rate - and to understand the attack strategy and process used to overcome the protection. Method: We conducted an experiment with master students performing attack tasks on a small application hardened with different levels of protection. Students carried out their task working at the source code level. Results: We observed a statistically significant effect of code splitting on the attack success rate that, on the average, was reduced from 89% with unprotected clear code to 52% with the most effective protection. The protection variant that moved some small-sized code chunks turned out to be more effective than the alternative moving fewer but larger chunks. Different strategies were identified yielding different success rates. Moreover we discovered that successful attacks exhibited different process w.r.t. failed ones. Conclusions: We found empirical evidence of the effect of code splitting, assessed the relative magnitude, and evaluated the influence of the chunk size parameter. Moreover we extracted the process used to overcome such obfuscation technique. [ABSTRACT FROM AUTHOR]

The problem of inconsistency in requirements engineering has been in the spotlight of the RE community from the early 1990s. In the early years, inconsistency was perceived in the literature as a problem that needs to be eliminated on sight. More recently, it has become recognized that maintaining consistency at all times is not only infeasible, but also even counterproductive. Based on this recognition, paradigms and tools have been proposed in the RE literature for managing inconsistency. However, over the same period, inconsistency as perceived and managed in practice has not received much attention. Our research aims to better understand the phenomenon of inconsistency and the strategies to address it in RE practice. This paper describes an empirical study investigating practitioners' perceptions of inconsistency manifestations in RE, their attitudes towards these manifestations, and strategies they apply to address them. The findings of this research led to the two contributions: (a) an explanation of how the ideas of the RE field about managing RE inconsistency are reflected in practitioners' perceptions of the inconsistency that they encounter in their daily work, and (b) the identification of some barriers that appear to be hindering practitioners' adoption of the RE field's inconsistency management strategies, together with possible reasons underlying these barriers. [ABSTRACT FROM AUTHOR]