Search

The International Conference on Agile Software Development (XP) was established almost sixteen years ago. Based on data from Scopus database, a total of 789 papers have been published in between years of 2002 and 2018. We employed bibliometrics analysis and topic modeling with R/RStudio to analyze these published papers from various dimensions, including the most active authors, collaboration of authorship, most cited papers, used keywords and trends of probable topics from the titles and abstracts of those papers. The results show that the first five years of XP conference cover nearly 40% of the papers published until now and almost 62% of the XP papers have been cited at least once. Mining of XP conference paper titles and abstracts result in these hot research topics: “Coordination”, “Technical Debt”, “Teamwork”, “Startups” and “Agile Practices”, thus strongly focusing on practical issues and problems faced by the practitioners in the industry. The results highlight the most influential researchers and institutions, and the collaboration between the authors in the conference papers. The approach applied in this study can be extended to other software engineering venues and can be applied to large-scale studies.

To better prepare the next generation of software professional, it is important to provide opportunities for them to work on real software project along with real customer during their studies. This is the reason universities around the world offer project-based capstone course. Such courses help students to understand what they will face in the industry and experience real customer interaction and challenges in collaborative work. In regards, University of Oulu, Finland offers a software factory (SWF) course to enhance the learning and experience multicultural teamwork. This paper presents the design of the SWF course and student and teacher experiences. It discusses the importance of reflective learning diaries and serious games. Additionally, this paper examines factors in the SWF learning environment that affect student learning in the SWF course. Survey data were collected from the last 6 years of SWF projects. The results show that students consider the SWF to be a good collaborative learning environment that helps them achieve academic triumphs and enhances various professional skills. The learning diaries are effective for increasing students’ learning experiences as well as providing an opportunity for teaching staff to monitor students’ progress and offer better facilitation. These results are helpful for academic institutions and industry when developing such a learning environment.

Aesthetic image analysis is essential for improving the performance of multimedia image retrieval systems, especially from a repository of social media and multimedia content stored on mobile devices. This paper presents a novel method for classifying aesthetic natural scene images by studying the naturalness of image content using statistical features, and reading text in the images using semantic features. Unlike existing methods that focus only on image quality with human information, the proposed approach focuses on image features as well as text-based semantic features without human intervention to reduce the gap between subjectivity and objectivity in the classification. The aesthetic classes considered in this work are (i) Very Pleasant, (ii) Pleasant, (iii) Normal and (iv) Unpleasant. The naturalness is represented by features of focus, defocus, perceived brightness, perceived contrast, blurriness and noisiness, while semantics are represented by text recognition, description of the images and labels of images, profile pictures, and banner images. Furthermore, a deep learning model is proposed in a novel way to fuse statistical and semantic features for the classification of aesthetic natural scene images. Experiments on our own dataset and the standard datasets demonstrate that the proposed approach achieves 92.74%, 88.67% and 83.22% average classification rates on our own dataset, AVA dataset and CUHKPQ dataset, respectively. Furthermore, a comparative study of the proposed model with the existing methods shows that the proposed method is effective for the classification of aesthetic social media images.

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link. Software product quality models have evolved in their abilities to capture and describe the abstract notion of software quality since the 1970’s. Many models constructed deal with a specific part of software quality only which makes them ineligible to assess the quality of software products as a whole. Former publications failed to thoroughly examine and list all the available models which attempt to describe each known property of software product quality. This paper discovers such complete software product quality models published since 2000; moreover, it endeavours to measure the relevance of each model quantitatively by introducing indicators with regard to the scientific and industrial community. The identified 23 software product quality model classes differ significantly in terms of publication intensity, publication range, quality score average, relevance score and the 12-month average of the Google Relative Search Index. The results offer a foundation for selecting the appropriate software product quality model for use or for extension if newly identified quality properties need to be connected to a general context. Furthermore, the experiences accumulated on the field of software product quality modelling motivated researchers to successfully transfer the concepts to other areas where abstract entities need to be compared or assessed including the quality of higher educational teaching and business processes, which is also briefly highlighted in the paper.

Software engineering (SE) research should be relevant to industrial practice. There have been regular discussions in the SE community on this issue since the 1980’s, led by pioneers such as Robert Glass. As we recently passed the milestone of “50 years of software engineering”, some recent positive efforts have been made in this direction, e.g., establishing “industrial” tracks in several SE conferences. However, many researchers and practitioners believe that we, as a community, are still struggling with research relevance and utility. The goal of this paper is to synthesize the evidence and experience-based opinions shared on this topic so far in the SE community, and to encourage the community to further reflect and act on the research relevance. For this purpose, we have conducted a Multi-vocal Literature Review (MLR) of 54 systematically-selected sources (papers and non peer-reviewed articles). Instead of relying on and considering the individual opinions on research relevance, mentioned in each of the sources, the MLR aims to synthesize and provide the “holistic” view on the topic. The highlights of our MLR findings are as follows. The top three root causes of low relevance, discussed in the community, are: (1) Researchers having simplistic views (or wrong assumptions) about SE in practice; (2) Lack of connection with industry; and (3) Wrong identification of research problems. The top three suggestions for improving research relevance are: (1) Using appropriate research approaches such as action-research; (2) Choosing relevant (practical) research problems; and (3) Collaborating with industry. By synthesizing all the discussions on this important topic so far, this paper aims to encourage further discussions and actions in the community to increase our collective efforts to improve the research relevance. Furthermore, we raise the need for empirically-grounded and rigorous studies on the relevance problem in SE research, as carried out in other fields such as management science.

Research collaboration between industry and academia supports improvement and innovation in industry and helps ensure the industrial relevance of academic research. However, many researchers and practitioners in the community believe that the level of joint industry-academia collaboration (IAC) projects in Software Engineering (SE) research is relatively low, creating a barrier between research and practice. The goal of the empirical study reported in this paper is to explore and characterize the state of IAC with respect to industrial needs, developed solutions, impacts of the projects and also a set of challenges, patterns and anti-patterns identified by a recent Systematic Literature Review (SLR) study. To address the above goal, we conducted an opinion survey among researchers and practitioners with respect to their experience in IAC. Our dataset includes 101 data points from IAC projects conducted in 21 different countries. Our findings include: (1) the most popular topics of the IAC projects, in the dataset, are: software testing, quality, process, and project managements; (2) over 90% of IAC projects result in at least one publication; (3) almost 50% of IACs are initiated by industry, busting the myth that industry tends to avoid IACs; and (4) 61% of the IAC projects report having a positive impact on their industrial context, while 31% report no noticeable impacts or were “not sure”. To improve this situation, we present evidence-based recommendations to increase the success of IAC projects, such as the importance of testing pilot solutions before using them in industry. This study aims to contribute to the body of evidence in the area of IAC, and benefit researchers and practitioners. Using the data and evidence presented in this paper, they can conduct more successful IAC projects in SE by being aware of the challenges and how to overcome them, by applying best practices (patterns), and by preventing anti-patterns.

Process standardization allows to optimize ERP systems and is a nec-essary step prior to ERP implementation projects. Traditional approaches to standardizing business processes are based on manually created "de-jure" process models, which are distorted, error-prone, simplistic, and often deviating from process reality. Theoretically embedded in the organizational contingency theory as kernel theory, this paper employs a design science approach to design a process mining-enabled decision support system (DSS) which combines bottom-up process mining models with manually added top-down standardization infor-mation to recommend a suitable standard process specification from a repository. Extended process models of the as-is process are matched against a repository of best-practice standard process model using an attributebased process similarity matching algorithm. Thus, the DSS aims to reduce the overall costs of process standardization, to optimize the degree of fit between the organization and the implemented processes, and to minimize the degree of organizational change re-quired in standardization and ERP implementation projects. This paper imple-ments a working prototype instantiation in the open-source process analytics platform Apromore based on a real-life event log and standardization attributes for the Purchase-to-Pay and Order-to-Cash processes from three SAP R/3 ERP systems at the industry partner.

The use of models and formal analysis techniques at runtime is fundamental to address safety assurance during the system operational stage, when all relevant uncertainties and unknowns can be resolved. This paper presents a novel approach to runtime safety enforcement of software systems based on the MAPE-K control loop architecture for system monitoring and control, and on the Abstract State Machine as runtime model representing the enforcement strategy aimed at preserving or eventually restoring safety. The enforcer software is designed as an autonomic manager that wraps around the software system to monitor and manage unsafe system changes using probing and effecting interfaces provided by the system, so realising grey-box safety enforcement. The proposed approach is supported by a component framework that is here illustrated by means of a case study in the health-care domain.

Journal of grid computing 16(3), 381 - 408 (2018). doi:10.1007/s10723-018-9453-3, This paper describes the achievements of the H2020 project INDIGO-DataCloud. The project has provided e-infrastructures with tools, applications and cloud framework enhancements to manage the demanding requirements of scientific communities, either locally or through enhanced interfaces. The middleware developed allows to federate hybrid resources, to easily write, port and run scientific applications to the cloud. In particular, we have extended existing PaaS (Platform as a Service) solutions, allowing public and private e-infrastructures, including those provided by EGI, EUDAT, and Helix Nebula, to integrate their existing services and make them available through AAI services compliant with GEANT interfederation policies, thus guaranteeing transparency and trust in the provisioning of such services. Our middleware facilitates the execution of applications using containers on Cloud and Grid based infrastructures, as well as on HPC clusters. Our developments are freely downloadable as open source components, and are already being integrated into many scientific applications., Published by Springer Nature, Dordrecht

The increasing complexity and autonomy of modern automotive systems, together with the safety-sensitive nature of many vehicle information flows require a careful analysis of the security requirements and adequate mechanisms for ensuring integrity and confidentiality of data. This is especially true for semi-autonomous vehicle systems, in which user intervention is limited or absent, and information must be trusted. This paper provides a proposal for the representation of high-level security properties in the specification of application components according to the AUTOSAR standard AUTomotive Open System ARchitecture. An automatic generation of security components from security-annotated AUTOSAR specifications is also proposed. It provides for the automatic selection of the adequate security mechanisms based on a high-level specification, thus avoiding complex and error-prone manual encodings by the designer. These concepts and tools are applied to a paradigmatic example in order to show their simplicity and efficacy.

The RAST (Rapid Annotation using Subsystem Technology) annotation engine was built in 2008 to annotate bacterial and archaeal genomes. It works by offering a standard software pipeline for identifying genomic features (i.e., protein-encoding genes and RNA) and annotating their functions. Recently, in order to make RAST a more useful research tool and to keep pace with advancements in bioinformatics, it has become desirable to build a version of RAST that is both customizable and extensible. In this paper, we describe the RAST tool kit (RASTtk), a modular version of RAST that enables researchers to build custom annotation pipelines. RASTtk offers a choice of software for identifying and annotating genomic features as well as the ability to add custom features to an annotation job. RASTtk also accommodates the batch submission of genomes and the ability to customize annotation protocols for batch submissions. This is the first major software restructuring of RAST since its inception. United States National Institute of Allergy and Infectious Diseases, National Institutes of Health, Department of Health and Human Service [HHSN272201400027C]; United States Department of Energy, DOE Systems Biology Knowledgebase [DE-AC02-06CH11357]; United States National Science Foundation [CNS-1305112]; Experimental and Computational Determination of Microbial Genotypes and Phenotypes [MCB-1330800]; National Aeronautics and Space Administration through the NASA Astrobiology Institute [NNA13AA91A] We thank Emily Dietrich for her helpful comments. This work was supported by the United States National Institute of Allergy and Infectious Diseases, National Institutes of Health, Department of Health and Human Service [Contract No. HHSN272201400027C]; the United States Department of Energy [DE-AC02-06CH11357], as part of the DOE Systems Biology Knowledgebase; R.A.E. was supported by United States National Science Foundation Grants grants II-EN: Computational Enhancement of Analytical Metagenomics Systems CNS-1305112, and Experimental and Computational Determination of Microbial Genotypes and Phenotypes MCB-1330800; and G.J.O. was supported by the National Aeronautics and Space Administration through the NASA Astrobiology Institute under Cooperative Agreement No. NNA13AA91A issued through the Science Mission Directorate. United States Department of Energy: National Institute of Allergy and Infectious Diseases.

This paper offers a service-oriented architecture (SOA) for ontology-based multi-agent system (MAS) negotiations in the context of virtual enterprises (VEs). The objective of this paper is fourfold. First, it is to design a SOA which utilizes ontology and MAS to provide a distributed and interoperable environment for automated negotiations in VE. In this architecture, individual ontologies for both the VE initiator and its potential partners are constructed to describe and store resources and service knowledge. Second, a series of semantic ontology matching methods are developed to reach agents' interoperability during the negotiation process. Third, correspondence-based extended contract net protocol is presented, which provides basic guidelines for agents' reaching mutual understandings and service negotiation. Last, a fuzzy set theory based knowledge reuse approach is proposed to evaluate the current negotiation behaviors of the VE partners. A walkthrough example is presented to illustrate the methodologies and system architecture proposed in this paper.

Traceability is recognized to be important for supporting agile development processes. However, after analyzing many of the existing traceability approaches it can be concluded that they strongly depend on traditional development process characteristics. Within this paper it is justified that this is a drawback to support adequately agile processes. As it is discussed, some concepts do not have the same semantics for traditional and agile methodologies. This paper proposes three features that traceability models should support to be less dependent on a specific development process: (1) user-definable traceability links, (2) roles, and (3) linkage rules. To present how these features can be applied, an emerging traceability metamodel (TmM) will be used within this paper. TmM supports the definition of traceability methodologies adapted to the needs of each project. As it is shown, after introducing these three features into traceability models, two main advantages are obtained: 1) the support they can provide to agile process stakeholders is significantly more extensive, and 2) it will be possible to achieve a higher degree of automation. In this sense it will be feasible to have a methodical trace acquisition and maintenance process adapted to agile processes.

Cloud computation technologies open a new perspective for scientific computing. Sophisticated software can now be made available as an on-demand service, reducing costs of maintenance and improving availability of the software to end-users. The paper presents a web based service system for electromagnetic computations. The back-end of the system is built of open-source scientific code. The service is targeted on medical staff and electromagnetic safety engineers which usually are not familiar with electromagnetic field simulation methods but its design allows it to be extended for more demanding, scientific-oriented users. The primary goal of the system is user friendliness for medical and engineering staff combined with flexibility, scalability and extendibility for scientists developing simulation software. These goals were achieved with help of carefully designed concept of configurable usage-scenarios (workflows). The purpose of this paper is to present in detail the concept and implementation of scientific scenarios.

Models play a crucial role in the development and maintenance of software systems, but are often neglected during the development process due to the considerable manual effort required to produce them. In response to this problem, numerous techniques have been developed that seek to automate the model generation task with the aid of increasingly accurate algorithms from the domain of Machine Learning. From an empirical perspective, these are extremely challenging to compare; there are many factors that are difficult to control (e.g. the richness of the input and the complexity of subject systems), and numerous practical issues that are just as troublesome (e.g. tool availability). This paper describes the StaMinA (State Machine Inference Approaches) competiton, that was designed to address these problems. The competition attracted numerous submissions, many of which were improved or adapted versions of techniques that had not been subjected to extensive empirical evaluations, and had not been evaluated with respect to their ability to infer models of software systems. This paper shows how many of these techniques substantially improve on the state of the art, providing insights into some of the factors that could underpin the success of the best techniques. In a more general sense it demonstrates the potential for competitions to act as a useful basis for empirical software engineering by (a) spurring the development of new techniques and (b) facilitating their comparative evaluation to an extent that would usually be prohibitively challenging without the active participation of the developers.

Software Product Lines (SPLs) play an essential role in contemporary software development, improving program quality and reducing the time to market. However, despite its importance, several questions concerning SPL dependability did not get enough attention yet, such as: how the exception handling code has been implemented in SPLs? The characteristics of the exception handling code may lead to faulty SPL products? The Exception Handling (EH) is a widely used mechanism for building robust systems and is embedded in most of mainstream programming languages. In SPL context we can find exception signalers and handlers spread over code assets associated to common and variable SPL features. If exception signalers and handlers are added to a SPL in an unplanned way, products can be generated on which variable features may signal exceptions that remain uncaught or are mistakenly caught by common features or other variable features. This paper describes an empirical study that categorizes the possible ways exceptions flow through SPL features and investigates whether some of their characteristics can lead to faulty exception handling behavior. The study outcomes presented in this paper are helpful in several ways, such as: (i) enhancing the general understanding of how exceptions flow through mandatory and variable features; (ii) providing information about the potential problems related to specific kinds of flows detected in this study; and (iii) presenting how a static analysis tool can be used to support the identification of potentially faulty exception handling flows.

Packages are re-usable components for most of object-oriented systems. To promote reuse in object-oriented systems and to make deployment and maintenance tasks easy, packages in object-oriented systems must be cohesive. Quantification of cohesion of packages can be very useful in assessing their reusability, quality etc. In this paper, a new measure for the measurement of package cohesion is proposed. The cohesion of a package is measured in terms of the degree of intra-package dependencies among its elements. The hierarchical structure of packages has also been taken into account during the measurement. The proposed measure has been validated theoretically as well as empirically. An empirical study has been conducted using 25 packages taken from six open-source software projects developed in Java. The proposed package cohesion measure is found to be a useful indicator of external quality factors such as the reusability of packages. The proposed metric is also established as a better predictor of code reusability than the existing cohesion measures.

Software organizations have increased their interest in software process improvement (SPI). Nowadays, there are several frameworks that support SPI implementation. Some of them, such as CMMI (Capability Maturity Model Integration), propose to implement SPI in levels. At high maturity levels, such as CMMI levels 4 and 5, SPI involves carrying out statistical process control (SPC), which requires measures and data suitable for this context. However, measurement problems have been pointed in the literature as one of the main obstacles for a successful implementation of SPC in SPI efforts. With this scenario in mind, we developed a strategy to help software organizations prepare themselves regarding measurement aspects in order to implement SPC. The strategy is made up of three components: a Reference Software Measurement Ontology, an Instrument for Evaluating the Suitability of a Measurement Repository for SPC, and a Body of Recommendations for Software Measurement Suitable for SPC. In this paper we present the strategy as a whole and describe each one of its components.

We investigate quality improvement in QVT operational mappings (QVTo) model transformations, one of the languages defined in the OMG standard on model-to-model transformations. Two research questions are addressed. First, how can we assess quality of QVTo model transformations? Second, how can we develop higher-quality QVTo transformations? To address the first question, we utilize a bottom---up approach, starting with a broad exploratory study including QVTo expert interviews, a review of existing material, and introspection. We then formalize QVTo transformation quality into a QVTo quality model. The quality model is validated through a survey of a broader group of QVTo developers. We find that although many quality properties recognized as important for QVTo do have counterparts in general purpose languages, a number of them are specific to QVTo or model transformation languages. To address the second research question, we leverage the quality model to identify developer support tooling for QVTo. We then implemented and evaluated one of the tools, namely a code test coverage tool. In designing the tool, code coverage criteria for QVTo model transformations are also identified. The primary contributions of this paper are a QVTo quality model relevant to QVTo practitioners and an open-source code coverage tool already usable by QVTo transformation developers. Secondary contributions are a bottom---up approach to building a quality model, a validation approach leveraging developer perceptions to evaluate quality properties, code test coverage criteria for QVTo, and numerous directions for future research and tooling related to QVTo quality.

Royal Navy Junior Warfare Officers (JWO) undergo a comprehensive training package in order to prepare them to be officers of the watch. One aspect of this training relates to their knowledge of the ‘Rules of the Road’ or ‘COLREGS’; the rules for the manoeuvring and signalling that approaching vessels make in order to avoid collision. The training and assessment exercises undertaken predominantly use non-interactive static materials. These do not exercise the required skill in reconciling information from maritime charts, radar displays and ‘out-of-the-window’ monitoring. Consequently, performance during assessment on the VR-based bridge simulator falls short. This paper describes The Rules of the Road SIMulator (RORSIM)—a proof of concept interactive 3D (i3D) simulator developed to bridge the training gap between classroom teaching and VR bridge simulator assessment. RORSIM’s differentiation and its key functionality in terms of visualisaton, physics/interaction and game mechanics are influenced by the consideration of pedagogical learning models during requirements capture. This capture is formalised by a ‘Training Gap Use Case’—a graphical viewpoint using the Universal Modelling Language which can assist developers in requirements capture and development of i3D tools for existing training programmes. Key functionality, initial JWO feedback and a planned pilot study design are reported.

This paper presents a model, called the StoryToCode, which allows designing iTV programs focusing on using software components. First, StoryToCode allows transforming a storyboard into an abstract description of an element set. After this, this model transforms these elements into a specific programming language source code. In StoryToCode a software component is treated as a special element that can be reused in other contexts (web, mobile, and so on). StoryToCode is based on Model Driven Architecture (MDA) and allows designing and implementing applications, with context free, considering iTV program specific characteristics.

The increasing complexity of modern System-on-Chip (SoC) platforms has revealed the need for methodologies that enable a rigorous engineering design process, based on a combination of Electronic System Level (ESL) description languages, and IP-core modeling and reuse. On the other hand, ESL modeling has faced designers with the same methodology problems encountered in the design of large computer programs. In this paper, we describe a SystemC-only IP-core design process, called IP PROCESS (IPP). IPP is inspired on two rigorous software engineering processes (RUP and XP), and on well-known hardware design standards (VSIA and RMM). The IPP Verification Methodology (IPV) is based on a careful refinement of the SystemC behavioral description towards RTL. Such approach enabled a continuous co-simulation against the behavioral reference model, while allowed for a SystemC-only environment. As a result, we have experienced a considerable reduction in design time and an improvement in early bug detection. The IPP process has been used by over 70 designers of the BrazilIP Network, a SystemC collaborative partnership, in the design of the Fenix system. An intermediate step in the Fenix design is a real-world multimedia platform called CINE-IP (demo available at http://www.brazilip.org.br/cine-ip), composed of MPEG4, MP3 decoders and an 8051 microcontroller. The application of the IPP methodology in the design of CINE-IP, and its impact in design productivity is thoroughly analyzed.

The Service Component Architecture (SCA) makes it possible to combine existing and new services based on a variety of technologies with components built using a component-based development approach. However, when asynchronous service compositions are executed, one or more errors can occur, possibly at the same time, affecting the dependability of the composition. To guarantee that the composition succeeds or at least fails in a controlled manner, fault tolerance mechanisms must be employed. In this paper, we propose a novel exception handling model that targets the needs of dependable SCA applications. The model is applicable to service-oriented systems and allows the creation of fault-tolerant asynchronous service compositions. The EH-SCA framework instantiates the proposed model as an extension of the Apache Tuscany SCA infrastructure. Developers can apply this instantiation of the model to both new and existing applications by using a simple and flexible aspect-oriented programming model. Finally, a case study of the EH-SCA framework shows how it can be used to build dependable distributed applications.

Progress in analytical ultracentrifugation (AUC) has been hindered by obstructions to hardware innovation and by software incompatibility. In this paper, we announce and outline the Open AUC Project. The goals of the Open AUC Project are to stimulate AUC innovation by improving instrumentation, detectors, acquisition and analysis software, and collaborative tools. These improvements are needed for the next generation of AUC-based research. The Open AUC Project combines on-going work from several different groups. A new base instrument is described, one that is designed from the ground up to be an analytical ultracentrifuge. This machine offers an open architecture, hardware standards, and application programming interfaces for detector developers. All software will use the GNU Public License to assure that intellectual property is available in open source format. The Open AUC strategy facilitates collaborations, encourages sharing, and eliminates the chronic impediments that have plagued AUC innovation for the last 20 years. This ultracentrifuge will be equipped with multiple and interchangeable optical tracks so that state-of-the-art electronics and improved detectors will be available for a variety of optical systems. The instrument will be complemented by a new rotor, enhanced data acquisition and analysis software, as well as collaboration software. Described here are the instrument, the modular software components, and a standardized database that will encourage and ease integration of data analysis and interpretation software. Electronic supplementary material The online version of this article (doi:10.1007/s00249-009-0438-9) contains supplementary material, which is available to authorized users.

Visual analytics is the science of analytical reasoning facilitated by interactive visual interfaces. In this paper, we present an adaptation of the visual analytics framework to the context of software understanding for maintenance. We discuss the similarities and differences of the general visual analytics context with the software maintenance context, and present in detail an instance of a visual software analytics application for the build optimization of large-scale code bases. Our application combines and adapts several data mining and information visualization techniques in answering several questions that help developers in assessing and reducing the build cost of such code bases by means of user-driven, interactive analysis techniques.

Visual GUI Testing (VGT) is a tool-driven technique for automated GUI-based testing that uses image recognition to interact with and assert the correctness of the behavior of a system through its GUI as it is shown to the user. The technique's applicability, e.g. defect-finding ability, and feasibility, e.g. time to positive return on investment, have been shown through empirical studies in industrial practice. However, there is a lack of studies that evaluate the usefulness and challenges associated with VGT when used long-term (years) in industrial practice. This paper evaluates how VGT was adopted, applied and why it was abandoned at the music streaming application development company, Spotify, after several years of use. A qualitative study with two workshops and five well chosen employees is performed at the company, supported by a survey, which is analyzed with a grounded theory approach to answer the study's three research questions. The interviews provide insights into the challenges, problems and limitations, but also benefits, that Spotify experienced during the adoption and use of VGT. However, due to the technique's drawbacks, VGT has been abandoned for a new technique/framework, simply called the Test interface. The Test interface is considered more robust and flexible for Spotify's needs but has several drawbacks, including that it does not test the actual GUI as shown to the user like VGT does. From the study's results it is concluded that VGT can be used long-term in industrial practice but it requires organizational change as well as engineering best practices to be beneficial. Through synthesis of the study's results, and results from previous work, a set of guidelines are presented that aim to aid practitioners to adopt and use VGT in industrial practice. However, due to the abandonment of the technique, future research is required to analyze in what types of projects the technique is, and is not, long-term viable. To this end, we also present Spotify's Test interface solution for automated GUI-based testing and conclude that it has its own benefits and drawbacks.

The OWL Reasoner Evaluation competition is an annual competition (with an associated workshop) that pits OWL 2 compliant reasoners against each other on various standard reasoning tasks over naturally occurring problems. The 2015 competition was the third of its sort and had 14 reasoners competing in six tracks comprising three tasks (consistency, classification, and realisation) over two profiles (OWL 2 DL and EL). In this paper, we discuss the design, execution and results of the 2015 competition with particular attention to lessons learned for benchmarking, comparative experiments, and future competitions.

Background Subject recruitment for medical research is challenging. Slow patient accrual leads to increased costs and delays in treatment advances. Researchers need reliable tools to manage and predict the accrual rate. The previously developed Bayesian method integrates researchers’ experience on former trials and data from an ongoing study, providing a reliable prediction of accrual rate for clinical studies. Methods In this paper, we present a user-friendly graphical user interface program developed in R. A closed-form solution for the total subjects that can be recruited within a fixed time is derived. We also present a built-in Android system using Java for web browsers and mobile devices. Results Using the accrual software, we re-evaluated the Veteran Affairs Cooperative Studies Program 558— ROBOTICS study. The application of the software in monitoring and management of recruitment is illustrated for different stages of the trial. Conclusions This developed accrual software provides a more convenient platform for estimation and prediction of the accrual process. Electronic supplementary material The online version of this article (doi:10.1186/s13063-016-1457-3) contains supplementary material, which is available to authorized users.

Monitoring and repair are two sides of the on-the-fly maintenance of IT systems. Monitoring is well supported by automatic tools. In contrast, repairs involve much higher human intervention which negatively affects reliability and efficiency. The paper introduces a method of automating repairs of IT systems which can be integrated with any of the existing monitoring mechanisms. The method is described as a collection of models and algorithms. The monitored system is modelled as a multi-hierarchy of components, and predicates are used to detect problems and to trigger the corresponding repairs. Repair procedures are represented using flow diagrams. Graphical notations are used to specify the architecture of the runnable component of the proposed method and the corresponding repair API. Formal notation (the Z language) is used to provide for precision and unambiguity of the specifications. The formal specification facilitates reuse of the method in different target environments. For validation purposes, the method has been instantiated (using the PERL language) in an industrial helpdesk supporting the operation of several hundred IT systems. The validation data demonstrate the significant positive impact of the proposed method on the efficiency of the performed repairs.

Establishing representative samples for Software Engineering surveys is still considered a challenge. Specialized literature often presents limitations on interpreting surveys’ results, mainly due to the use of sampling frames established by convenience and non-probabilistic criteria for sampling from them. In this sense, we argue that a strategy to support the systematic establishment of sampling frames from an adequate source of sampling can contribute to improve this scenario. A conceptual framework for supporting large scale sampling in Software Engineering surveys has been organized after performing a set of experiences on designing such strategies and gathering evidence regarding their benefits. The use of this conceptual framework based on a sampling strategy developed for supporting the replication of a survey on characteristics of agility and agile practices in software processes is depicted in this paper. A professional social network (Linkedln) was established as the source of sampling and its groups of interest as the units for searching members to be recruited. It allowed to deal with a sampling frame composed by more than 110,000 members (prospective subjects) distributed over 19 groups of interest. Then, through the similarity levels observed among these groups, eight strata were organized and 7745 members were invited, from which 291 have confirmed participation and answered the questionnaire. The heterogeneity and number of participants in this replication contributed to improve the strength of original survey’s results. Therefore, we believe the sharing of this experience, the instruments and plan can be helpful for those researchers and practitioners interested on executing large scale surveys in Software Engineering.

The Academia-Industry partnership has been increasingly encouraged in the software development field. The main focus of the initiatives is driven by the collaborative work where the scientific research work meets the real needs of the Industry. Aiming to contribute to this effort we have proposed an approach called SoftCoDeR (Software Cooperative Design Research) that combines CMD (Cooperative Method Development), a method of Action Research, to concepts of DSR (Design Science Research). We have applied the SoftCoDeR approach in a software development company to support a research and development project aiming to integrate User Experience practices into the agile software development process. In this paper, we present new findings about this approach that has been extended with the use of Experimental Software Engineering (ESE) practices to conduct experimental validations of artifacts before putting them in industrial practice.

As cloud application platforms (CAPs) are reaching the stage where the human effort required to maintain them at an operational level is unsupportable, one of the major challenges faced by the cloud providers is to develop appropriate mechanisms for run-time monitoring and adaptation, to prevent cloud application platforms from quickly dissolving into a non-reliable environment. In this context, the application of intelligent approaches to Autonomic Clouds may offer promising opportunities. In this paper we present an approach to providing cloud platforms with autonomic capabilities, utilising techniques from the Semantic Web and Stream Reasoning research fields. The main idea of this approach is to encode values, monitored within cloud application platforms, using Semantic Web languages, which then allows us to integrate semantically-enriched observation streams with static ontological knowledge and apply intelligent reasoning. Using such run-time reasoning capabilities, we have developed a conceptual architecture for an autonomous framework and describe a prototype solution we have constructed which implements this architecture. Our prototype is able to perform analysis and failure diagnosis, and suggest further adaptation actions. We report our experience in utilising the Stream Reasoning technique in this context as well as further challenges that arise out of our work.

This paper presents an empirical study based on a set of measures to evaluate the usability of mobile applications running on different mobile operating systems, including Android, iOS and Symbian. The aim is to evaluate empirically a framework that we have developed on the use of the Software Quality Standard ISO 9126 in mobile environments, especially the usability characteristic. To do that, 32 users had participated in the experiment and we have used ISO 25062 and ISO 9241 standards for objective measures by working with two widely used mobile applications: Google Apps and Google Maps. The QUIS 7.0 questionnaire have been used to collect measures assessing the users’ level of satisfaction when using these two mobile applications. By analyzing the results we highlighted a set of mobile usability issues that are related to the hardware as well as to the software and that need to be taken into account by designers and developers in order to improve the usability of mobile applications.

Continued progress in geoinformatics research and applications depends on educating new generations of theorists and practitioners. Effective education and training in turn depend on the availability of appropriate tools, including software. The OpenDragon Project was established in 2004 to help satisfy this requirement. OpenDragon provides a simple yet powerful platform for hands-on lab exercises and student projects. The fact that it is available in multiple languages and can be internationalized without changing any code gives the system a world-wide audience. It is simple, portable, easy to use, and modest in its hardware requirements. OpenDragon can be used as a turnkey platform to teach undergraduates and even secondary school students about geography, agriculture, geology, meteorology, forestry and other natural resource disciplines. OpenDragon is unusual in that it also provides support, through the Programmer’s Toolkit, for students and researchers who need to program their own algorithms or who want to understand the internal workings of remote sensing and GIS software. This paper describes OpenDragon, its history, and its capabilities. It also explains the system architecture and the organization of the Programmer’s Toolkit, and includes some examples of how we have used these capabilities to support student research and to teach geoinformatics programming.

This paper evaluates the usage of analytical programming and different fitness functions for software effort estimation. Analytical programming and differential evolution generate regression functions. These functions are evaluated by the fitness function which is part of differential evolution. The differential evolution requires a proper fitness function for effective optimization. The problem is in proper selection of the fitness function. Analytical programming and different fitness functions were tested to assess insight to this problem. Mean magnitude of relative error, prediction 25 %, mean squared error (MSE) and other metrics were as possible candidates for proper fitness function. The experimental results shows that means squared error performs best and therefore is recommended as a fitness function. Moreover, this work shows that analytical programming method is viable method for calibrating use case points method. All results were evaluated by standard approach: visual inspection and statistical significance. © 2015, Urbanek et al., Tomas Bata University in Zlin [IGA/CebiaTech/2015/034]

This paper provides an overview of the state of the art technologies for software development in cloud environments. The surveyed systems cover the whole spectrum of cloud-based development including integrated programming environments, code repositories, software modeling, composition and documentation tools, and application management and orchestration. In this work we evaluate the existing cloud development ecosystem based on a wide number of characteristics like applicability (e.g. programming and database technologies supported), productivity enhancement (e.g. editor capabilities, debugging tools), support for collaboration (e.g. repository functionality, version control) and post-development application hosting and we compare the surveyed systems. The conducted survey proves that software engineering in the cloud era has made its initial steps showing potential to provide concrete implementation and execution environments for cloud-based applications. However, a number of important challenges need to be addressed for this approach to be viable. These challenges are discussed in the article, while a conclusion is drawn that although several steps have been made, a compact and reliable solution does not yet exist.

Background: Software reference architectures are becoming widely adopted by organizations that need to support the design and maintenance of software applications of a shared domain. For organizations that plan to adopt this architecture-centric approach, it becomes fundamental to know the return on investment and to understand how software reference architectures are designed, maintained, and used. Unfortunately, there is little evidence-based support to help organizations with these challenges. Methods: We have conducted action research in an industry-academia collaboration between the GESSI research group and everis, a multinational IT consulting firm based in Spain. Results: The results from such collaboration are being packaged in order to create guidelines that could be used in similar contexts as the one of everis. The main result of this paper is the construction of empirically-grounded guidelines that support organizations to decide on the adoption of software reference architectures and to gather evidence to improve RA-related practices. Conclusions: The created guidelines could be used by other organizations outside of our industry-academia collaboration. With this goal in mind, we describe the guidelines in detail for their use.

Use case templates can be used to describe functional requirements of a Software Product Line. However, to the best of our knowledge, no efforts have been made to collect and summarize these existing templates and no empirical evaluation of the use cases’ comprehensibility provided by these templates has been addressed yet. The contributions of this paper are twofold. First, we present a systematic mapping study about the SPL variability description using textual use cases. From this mapping, we found twelve SPL use case templates and observed the need not only for the application of these templates in real SPL but also for supporting tools. Secondly, this work presents an evaluation of the comprehensibility of SPL use cases specified in these templates through a controlled experiment with 48 volunteers. The results of this experiment show that the specification of variabilities in the steps’ numeric identifiers of the textual use cases is better to the use case understanding than the other approaches identified. We also found evidence that the specification of variabilities at the end of the use cases favors the comprehension of them and the use of questions associated to the variation points in the use cases improves the understanding of use cases. We conclude that each characteristic of the existing templates has an impact on the SPL use case understanding and this should be taken into account when choosing one.

Writing patches to fix bugs or implement new features is an important software development task, as it contributes to raise the quality of a software system. Not all patches are accepted in the first attempt, though. Patches can be rejected because of problems found during code review, automated testing, or manual testing. A high rejection rate, specially later in the lifecycle, may indicate problems with the software development process. Our objective is to better understand the relationship among different forms of patch rejection and to characterize their frequency within a project. This paper describes one step towards this objective, by presenting an analysis of a large open source project, Firefox. In order to characterize patch rejection, we relied on issues and source code commits from over four years of the project’s history. We computed monthly metrics on the occurrence of three indicators of patch rejection—negative code reviews, commit backouts, and bug reopening—and measured the time it takes both to submit a patch and to reject inappropriate patches. In Firefox, 20 % of the issues contain rejected patches. Negative reviews, backouts, and issue reopening are relatively independent events; in particular, about 70 % of issue reopenings are premature; 75 % of all inappropriate changes are rejected within four days. Patch rejection is a frequent event, occurring multiple times a day. Given the relative independence of rejection types, existing studies that focus on one single rejection type fail to detect many rejections. Although inappropriate changes cause rework, they have little effect on the quality of released versions of Firefox.

The key objective of No-Reference (NR) visual metrics (indicators) is to predict the end-user experience concerning remotely delivered video content. Rapidly increasing demand for easily accessible, high quality video material makes it crucial for service providers to test the user experience without the need for comparison with reference material. In this paper, we present a versatile measurement system and describe various optimisation strategies utilised to reach real-time operation. Furthermore, several calculation automation scripts are described, along with a dedicated graphical user interface, which gives a more comprehensive insight into the presented system. On top of that, we show the results of crowd-sourcing experiments used to estimate subjective threshold values for quality indicators. Additionally, integration with the IMCOP system is introduced.

Software libraries and frameworks play an important role in software system development. The appropriate usage of their functionalities/components through their APIs, however, is a challenge for developers. Usually, API documentation, when it exists, is insufficient to assist them in their programming tasks. There are few API documentation writers for the many potential readers, resulting in the lack of explanations and examples concerning different scenarios and perspectives. The interaction of developers on the Web, on the other hand, generates content concerning APIs from different perspectives, which can be used to document APIs, also known as crowd documentation. In this paper, we present a study regarding the knowledge generated by the crowd on the Stack Overflow question-and-answer website. Our main goal is to understand how the crowd can contribute for API documentation on two programming tasks: how to implement a scenario using an API (how-to-do-it), and how to fix domain-independent bugs in an existing code where there was a misunderstanding regarding the usage of an API (debug-corrective). We classified questions available on Stack Overflow by the main concerns of askers, and we used those classified as how-to-do-it and debug-corrective to analyze the coverage of API elements on the discussions related to such questions. Our cases included the well-known and popular Swing and Android APIs. Our main findings showed that the crowd provides more content for debug-corrective tasks than for how-to-do-it tasks, regardless of the API. Android API elements are more discussed by the crowd compared to Swing. Moreover, we observed that some API elements are frequently mentioned together in discussions, and that there is a strong association between API coverage on Stack Overflow and its usage in real software systems. Crowd documentation may not be a complete substitute for official documentation because of its partial coverage, especially for how-to-do-it tasks. However, it can still significantly enhance the existent documentation, especially for the most commonly used API elements, providing code samples and explanations on a large variety of usage nuances. Finally, taking advantage of the high coverage for debug-corrective tasks, a new kind of debugging assistant may be conceived.

The well-orchestrated use of distilled experience, domain-specific knowledge, and well-informed trade-off decisions is imperative if we are to design effective architectures for complex software-intensive systems. In particular, designing modern self-adaptive systems requires intricate decision-making over a remarkably complex problem space and a vast array of solution mechanisms. Nowadays, a large number of approaches tackle the issue of endowing software systems with self-adaptive behavior from different perspectives and under diverse assumptions, making it harder for architects to make judicious decisions about design alternatives and quality attributes trade-offs. It has currently been claimed that search-based software design approaches may improve the quality of resulting artifacts and the productivity of design processes, as a consequence of promoting a more comprehensive and systematic representation of design knowledge and preventing design bias and false intuition. To the best of our knowledge, no empirical studies have been performed to provide sound evidence of such claim in the self-adaptive systems domain. This paper reports the results of a quasi-experiment performed with 24 students of a graduate program in Distributed and Ubiquitous Computing. The experiment evaluated the design of self-adaptive systems using a search-based approach proposed by us, in contrast to the use of a non-automated approach based on architectural styles catalogs. The goal was to investigate to which extent the adoption of search-based design approaches impacts on the effectiveness and complexity of resulting architectures. In addition, we also analyzed the approach’s potential for leveraging the acquisition of distilled design knowledge. Our findings show that search-based approaches can improve the effectiveness of resulting self-adaptive systems architectures and reduce their design complexity. We found no evidence regarding the approach’s potential for leveraging the acquisition of distilled design knowledge by novice software architects. This study contributes to reveal empirical evidence on the benefits of search-based approaches when designing self-adaptive systems architectures. The results presented herein increase our belief that the systematic representation of distilled design knowledge and the adoption of search-based design approaches indeed lead to improved architectures.

Background Systems are considered legacy when their maintenance costs raise to unmanageable levels, but they still deliver valuable benefits for companies. One intrinsic problem of this kind of system is the presence of crosscutting concerns in their architecture, hindering its comprehension and evolution. Architecture-driven modernization (ADM) is the new generation of reengineering in which models are used as main artifacts during the whole process. Using ADM, it is possible to modernize legacy systems by remodularizing their concerns in a more modular shape. In this sense, the first step is the identification of source code elements that contribute to the implementation of those concerns, a process known as concern mining. Although there exist a number of concern mining approaches in the literature, none of them are devoted to ADM, leading individual groups to create their own ad hoc proprietary solutions. In this paper, we propose an approach called crosscutting-concern knowledge discovery meta-model (CCKDM) whose goal is to mine crosscutting concerns in ADM context. Our approach employs a combination of a concern library and a K-means clustering algorithm. Methods We have conducted an experimental study composed of two analyses. The first one aimed to identify the most suitable levenshtein values to apply the clustering algorithm. The second one aimed to check the recall and precision of our approach when compared to oracles and also to two other existing mining techniques (XScan and Timna) found in literature. Results The main result of this work is a combined mining approach for KDM that enables a concern-oriented modernization to be performed. As a secondary and more general result, this work shows that it is possible to adapt existing concern mining code-level approaches for being used in ADM processes and maintain the same level of precision and recall. Conclusions By using the approach herein presented, it was possible to conclude the following: (i) it is possible to automate the identification of crosscutting concerns in KDM models and (ii) the results are similar or equal to other approaches.

Cloud computing offers various computational resources via convenient on-demand service provision. Currently, heterogeneous services and cloud resources are usually utilized and managed through diverse service portals. This significantly limits the effectiveness and efficiency for tasks implementation. Fundamentally, it is due to the lack of adequate specifications for service concepts, operations and interfaces from diverse cloud service models and types. This paper proposes a service management operation semantic description framework for comprehensive cloud service operation specification. Relying on ontological modelling techniques, cloud service operations are specified via entity classification, attribute assertion, relationship assertion and annotation assertion. Further, the proposed framework benefits from operation reasoning application. It enables intelligent assistances for multiple operation preparation and remote execution tasks. Based on the approach, a cloud service operation ontology and a unified service access and manipulation system prototype are implemented. Extensive experiments are conducted over different cloud service providers and for distinct service models. Obtained results demonstrate that the approach outperforms existing practices by facilitating reliable and effective service access, manipulation and interaction tasks.

Among research opportunities in software engineering for cloud computing model, interoperability stands out. We found that the dynamic nature of cloud technologies and the battle for market domination make cloud applications locked-id, i.e, proprietary, non-portable and non-interoperable. In general context of cloud computing, interoperability goes beyond communication between systems like in other fields, it goes in direction of more dynamic, heterogeneous, complex and composed applications that take advantage of best features from different providers and services simultaneously. Interoperability in cloud constitutes a great challenge that must be overcome for that, in the future, software be more dynamic and improved. Objective: This paper aims at identifying how interoperability in cloud computing has been addressed in the existing literature, offering an up-to-date view of concepts relate to how to develop interoperable software that takes advantage of different cloud models. Thus, providing a basis for further research in the field and consolidating e better exploring existing concepts. Method: To fulfill this objective, we surveyed literature. We defined six research questions and conducted the study according to a protocol that included planning, and execution. Results: A first result of the review is that there is no well established definition for cloud interoperability. This study also identified cloud interoperability concepts (e.g., cloud brokers, multi-cloud and cloud federation), requirements for interoperable applications and existing cloud interoperability solutions, showing that these are either too specific for particular situations. Finally, the survey found no evaluation models for cloud interoperability solutions. We also present a discussion on the findings of this study. Conclusion: Since the study observed that there are no well-established cloud interoperability solutions yet, we conclude that the issues raised by lack of interoperability persist. Selecting one interoperable solution or even a cloud standard can free the system from the underlying providers, but it would still be locked into the selected particular solution.

In this paper we address the problem of testing complex computer models for infrastructure protection and emergency response based on detailed and realistic application scenarios using advanced computational methods and tools. Specifically, we focus here on testing situation analysis decision support models for marine safety & security operations as a sample application domain. Arguably, methodical approaches for analyzing and validating situation analysis methods, decision support models, and information fusion algorithms require realistic vignettes that describe in great detail how a situation unfolds over time depending on initial configurations, dynamic environmental conditions and uncertain operational aspects. Meaningful results from simulation runs require appropriate test cases, the production of which is in itself a complex activity. To simplify this task, we introduce here the conceptual design of a Vignette Generator that has been developed and tested in an industrial research project. We also propose a framework for composing vignettes from reusable vignette elements together with a formal representation for vignettes using the Abstract State Machine method and illustrate the approach by means of various practical examples.