Search

The continuous software engineering paradigm is gaining popularity in modern development practices, where the interleaving of design and runtime activities is induced by the continuous evolution of software systems. In this context, performance assessment is not easy, but recent studies have shown that architectural models evolving with the software can support this goal. In this paper, we present a mapping study aimed at classifying existing scientific contributions that deal with the architectural support for performance-targeted continuous software engineering. We have applied the systematic mapping methodology to an initial set of 215 potentially relevant papers and selected 66 primary studies that we have analyzed to characterize and classify the current state of research. This classification helps to focus on the main aspects that are being considered in this domain and, mostly, on the emerging findings and implications for future research

Software quality estimation is a challenging and time-consuming activity, and models are crucial to face the complexity of such activity on modern software applications. In this context, software refactoring is a crucial activity within development life-cycles where requirements and functionalities rapidly evolve. One main challenge is that the improvement of distinctive quality attributes may require contrasting refactoring actions on software, as for trade-off between performance and reliability (or other non-functional attributes). In such cases, multi-objective optimization can provide the designer with a wider view on these trade-offs and, consequently, can lead to identify suitable refactoring actions that take into account independent or even competing objectives. In this paper, we present an approach that exploits NSGA-II as the genetic algorithm to search optimal Pareto frontiers for software refactoring while considering many objectives. We consider performance and reliability variations of a model alternative with respect to an initial model, the amount of performance antipatterns detected on the model alternative, and the architectural distance, which quantifies the effort to obtain a model alternative from the initial one. We applied our approach on two case studies: a Train Ticket Booking Service, and CoCoME. We observed that our approach is able to improve performance (by up to 42\%) while preserving or even improving the reliability (by up to 32\%) of generated model alternatives. We also observed that there exists an order of preference of refactoring actions among model alternatives. We can state that performance antipatterns confirmed their ability to improve performance of a subject model in the context of many-objective optimization. In addition, the metric that we adopted for the architectural distance seems to be suitable for estimating the refactoring effort., Comment: Accepted for publication in Information and Software Technologies. arXiv admin note: substantial text overlap with arXiv:2107.06127

Microbenchmarking is a widely used form of performance testing in Java software. A microbenchmark repeatedly executes a small chunk of code while collecting measurements related to its performance. Due to Java Virtual Machine optimizations, microbenchmarks are usually subject to severe performance fluctuations in the first phase of their execution (also known as warmup). For this reason, software developers typically discard measurements of this phase and focus their analysis when benchmarks reach a steady state of performance. Developers estimate the end of the warmup phase based on their expertise, and configure their benchmarks accordingly. Unfortunately, this approach is based on two strong assumptions: (i) benchmarks always reach a steady state of performance and (ii) developers accurately estimate warmup. In this paper, we show that Java microbenchmarks do not always reach a steady state, and often developers fail to accurately estimate the end of the warmup phase. We found that a considerable portion of studied benchmarks do not hit the steady state, and warmup estimates provided by software developers are often inaccurate (with a large error). This has significant implications both in terms of results quality and time-effort. Furthermore, we found that dynamic reconfiguration significantly improves warmup estimation accuracy, but still it induces suboptimal warmup estimates and relevant side-effects. We envision this paper as a starting point for supporting the introduction of more sophisticated automated techniques that can ensure results quality in a timely fashion., Comment: Published in Empirical Software Engineering (EMSE)

Refactoring aims at improving the maintainability of source code without modifying its external behavior. Previous works proposed approaches to recommend refactoring solutions to software developers. The generation of the recommended solutions is guided by metrics acting as proxy for maintainability (e.g., number of code smells removed by the recommended solution). These approaches ignore the impact of the recommended refactorings on other non-functional requirements, such as performance, energy consumption, and so forth. Little is known about the impact of refactoring operations on non-functional requirements other than maintainability. We aim to fill this gap by presenting the largest study to date to investigate the impact of refactoring on software performance, in terms of execution time. We mined the change history of 20 systems that defined performance benchmarks in their repositories, with the goal of identifying commits in which developers implemented refactoring operations impacting code components that are exercised by the performance benchmarks. Through a quantitative and qualitative analysis, we show that refactoring operations can significantly impact the execution time. Indeed, none of the investigated refactoring types can be considered “safe” in ensuring no performance regression. Refactoring types aimed at decomposing complex code entities (e.g., Extract Class/Interface, Extract Method) have higher chances of triggering performance degradation, suggesting their careful consideration when refactoring performance-critical code.

Microservices are quite widely impacting on the software industry in recent years. Rapid evolution and continuous deployment represent specific benefits of microservice-based systems, but they may have a significant impact on non-functional properties like performance. Despite the obvious relevance of this property, there is still a lack of systematic approaches that explicitly take into account performance issues in the lifecycle of microservice-based systems. In such a context of evolution and re-deployment, Model-Driven Engineering techniques can provide major support to various software engineering activities, and in particular they can allow managing the relationships between a running system and its architectural model. In this paper, we propose a model-driven integrated approach that exploits traceability relationships between the monitored data of a microservice-based running system and its architectural model to derive recommended refactoring actions that lead to performance improvement. The approach has been applied and validated on two microservice-based systems, in the domain of e-commerce and ticket reservation, respectively, whose architectural models have been designed in UML profiled with MARTE.

Replication package "Architectural support for Software Performance in Continuous Software Engineering: a Systematic Mapping Study"

Performance debugging in production is a fundamental activity in modern service-based systems. The diagnosis of performance issues is often time-consuming, since it requires thorough inspection of large volumes of traces and performance indices. In this paper we present DeLag, a novel automated search-based approach for diagnosing performance issues in service-based systems. DeLag identifies subsets of requests that show, in the combination of their Remote Procedure Call execution times, symptoms of potentially relevant performance issues. We call such symptoms Latency Degradation Patterns. DeLag simultaneously searches for multiple latency degradation patterns while optimizing precision, recall and latency dissimilarity. Experimentation on 700 datasets of requests generated from two microservice-based systems shows that our approach provides better and more stable effectiveness than three state-of-the-art approaches and general purpose machine learning clustering algorithms. DeLag is more effective than all baseline techniques in at least one case study (with p $\leq$ 0.05 and non-negligible effect size). Moreover, DeLag outperforms in terms of efficiency the second and the third most effective baseline techniques on the largest datasets used in our evaluation (up to 22%)., Accepted for publication in IEEE Transactions on Software Engineering (TSE)

Context: Software architecture refactoring can be induced by multiple reasons, such as satisfying new functional requirements or improving non-functional properties. Multi-objective optimization approaches have been widely used in the last few years to introduce automation in the refactoring process, and they have revealed their potential especially when quantifiable attributes are targeted. However, the effectiveness of such approaches can be heavily affected by configuration characteristics of the optimization algorithm, such as the composition of solutions. Objective: In this paper, we analyze the behavior of E A S I E R , which is an Evolutionary Approach for Software archItecturE Refactoring, while varying its configuration characteristics, with the objective of studying its potential to find near-optimal solutions under different configurations. Method: In particular, we use two different solution space inspection algorithms (i.e., N S G A − I I and S P E A 2 ) while varying the genome length and the solution composition. Results: We have conducted our experiments on a specific case study modeled in AEmilia ADL, on which we have shown the ability of E A S I E R to identify performance-critical elements in the software architecture where refactoring is worth to be applied. Beside this, from the comparison of multi-objective algorithms, N S G A − I I has revealed to outperform S P E A 2 in most of cases, although the latter one is able to induce more diversity in the proposed solutions. Conclusion: Our results show that the E A S I E R thoroughly automated process for software architecture refactoring allows to identify configuration contexts of the evolutionary algorithm in which multi-objective optimization more effectively finds near-optimal Pareto solutions.

Refactoring is often needed to ensure that software systems meet their performance requirements in deployments with different operational profiles, or when these operational profiles are not fully known or change over time. This is a complex activity in which software engineers have to choose from numerous combinations of refactoring actions. Our paper introduces a novel approach that uses performance antipatterns and stochastic modelling to support this activity. The new approach computes the performance antipatterns present across the operational profile space of a software system under development, enabling engineers to identify operational profiles likely to be problematic for the analysed design, and supporting the selection of refactoring actions when performance requirements are violated for an operational profile region of interest. We demonstrate the application of our approach for a software system comprising a combination of internal (i.e., in-house) components and external third-party services.

The extent of epistemic uncertainty in modeling and analysis of complex systems is ever growing, mainly due to increasing levels of the openness, heterogeneity and versatility in cloud-based applications that are being adopted in critical sectors, like banking and finance. State-of-the-art approaches for model-based performance assessment do not embed such uncertainty in analytic models, hence the predicted results do not account for the parametric uncertainty. In this paper, we develop a method for incorporating epistemic uncertainty of the input parameters (i.e., the arrival rate λ and the service rate μ ) to the M/M/1 queueing models, that are commonly used to analyze system performance. We consider two steady state and average output measures: the number of entities in the system and the response time. We start with closed-form solutions for these measures that enable us to study the propagation of epistemic uncertainty in input parameters to these output measures. We demonstrate the suitability of our method for the performance analysis of a cloud-based system, where the epistemic uncertainty comes from continuous re-deployment of applications across servers of different computational capabilities. System simulation results validate the ability of our models to produce satisfactorily accurate predictions of system performance indices under epistemic uncertainty.

Context Software refactoring is a common practice aimed at addressing requirements or fixing bugs during the software development. While refactoring related to functional requirements has been widely studied in the last few years, non-functional-driven refactoring is still critical, mostly because non-functional characteristics of software are hard to assess and appropriate refactoring actions can be difficult to identify. In the context of performance, which is the focus of this paper, antipatterns represent effective instruments to tackle this issue, because they document common mistakes leading to performance problems as well as their solutions. Objective In order to effectively reuse the knowledge beyond performance antipatterns, automation is required to detect and remove them. In this paper we introduce a framework that enables, in an unique tool context, the refactoring of software models driven by performance antipattern detection and removal. Method We have implemented, within the EPSILON platform, detection rules and refactoring actions on UML models for a set of well-known performance antipatterns. By exploiting the EPSILON languages to check properties and apply refactoring on models, we enable three types of refactoring sessions. Results We experiment our framework on a Botanical Garden Management System to show, on one side, that antipatterns can effectively drive software refactoring towards models that satisfy performance requirements and, on the other side, that the automation introduced by EPSILON-based sessions enables to inspect multiple paths and to propose a variety of solutions. Conclusion This work demonstrates that automation in performance-driven software model refactoring can be beneficial, and that performance antipatterns can be powerful instruments in the hands of software engineers for detecting (and solving) performance problems usually hidden to traditional bottleneck analysis. This work also opens the road to the integration of well-known techniques for software refactoring driven by functional requirements with novel techniques addressing non-functional requirements like performance.

Context awareness is a first-class attribute of today software systems. Indeed, many applications need to be aware of their context in order to adapt their structure and behavior for offering the best quality of service even in case the software and hardware resources are limited. Modeling the context, its evolution, and its influence on the services provided by (possibly resource constrained) applications are becoming primary activities throughout the whole software life cycle, although it is still difficult to capture the multidimensional nature of context. We propose a framework for modeling and reasoning on the context and its evolution along multiple dimensions. Our approach enables (1) the representation of dependencies among heterogeneous context attributes through a formally defined semantics for attribute composition and (2) the stochastic analysis of context evolution. As a result, context can be part of a model-based software development process, and multidimensional context analysis can be used for different purposes, such as non-functional analysis. We demonstrate how certain types of analysis, not feasible with context-agnostic approaches, are enabled in our framework by explicitly representing the interplay between context evolution and non-functional attributes. Such analyses allow the identification of critical aspects or design errors that may not emerge without jointly taking into account multiple context attributes. The framework is shown at work on a case study in the eHealth domain.

Context: While the performance analysis of a software architecture is a quite well-assessed task nowadays, the issue of interpreting the performance results for providing feedback to software architects is still very critical. Performance antipatterns represent effective instruments to tackle this issue, because they document common mistakes leading to performance problems as well as their solutions.Objective: Up today performance antipatterns have been only studied in the context of software modeling languages like UML, whereas in this manuscript our objective is to catch them in the context of ADL-based software architectures to investigate their effectiveness.Method: We have implemented a model-driven approach that allows the automatic detection of four performance antipatterns in AEmilia, that is a stochastic process algebraic ADL for performance-aware component-oriented modeling of software systems.Results: We evaluate the approach by applying it to three case studies in different application domains. Experimental results demonstrate the effectiveness of our approach to support the performance improvement of ADL-based software architectures.Conclusion: We can conclude that the detection of performance antipatterns, from the earliest stages of software development, represents an effective instrument to tackle the issue of identifying flaws and improving system performance.

Model-Driven Engineering techniques may achieve a major support to the software development when they allow to manage relationships between a running system and its architectural model. These relationships can be exploited for different goals, such as the software evolution due to new functional requirements. In this paper, we define and use relationships that work as support to the performance improvement of a running system. In particular, we combine: (i) a bidirectional model transformation framework tailored to define relationships between performance monitoring data and an architectural model, with (ii) a technique for detecting performance antipatterns and for suggesting architectural changes, aimed at removing performance problems identified on the basis of runtime information. The result is an integrated approach that exploits traceability relationships between the monitoring data and the architectural model to derive recommended refactoring solutions for the system performance improvement. The approach has been applied to an e-commerce application based on microservices that has been designed by means of UML software models profiled with MARTE.

Multi-objective optimization has demonstrated, in the last few years, to be an effective paradigm to tackle different architectural problems, such as service selection, composition and deployment. In particular, multi-objective approaches for searching architectural configurations that optimize quality properties (such as performance, reliability and cost) have been introduced in the last decade. However, a relevant amount of complexity is introduced in this context when performance are considered, often due to expensive iterative generation and solution of performance models. In this paper we introduce EASIER (Evolutionary Approach for multi-objective Software archItecturE Refactoring), that is an approach for applying architecture refactoring based on performance aspects and on the cost of architectural changes. In order to mitigate the complexity related to performance, we exploit the knowledge dwelling in performance antipatterns for more effectively driving the evolutionary algorithm towards optimal solutions. We have implemented our approach on AEmilia ADL, so to carry out performance analysis, antipatterns detection and architecture refactoring within the same environment. We demonstrate the effectiveness and applicability of our approach through a non-trivial experimentation on a case study.

Model-Driven Engineering techniques may achieve a major support to the software development when they allow to manage relationships between a running system and its architectural model. These relationships can be exploited for different goals, such as the software evolution due to new functional requirements. In this paper, we define and use relationships that work as support to the performance improvement of a running system. In particular, we combine: (i) a bidirectional model transformation framework tailored to define relationships between performance monitoring data and an architectural model, with (ii) a technique for detecting performance antipatterns and for suggesting architectural changes, aimed at removing performance problems identified on the basis of runtime information. The result is an integrated approach that exploits traceability relationships between the monitoring data and the architectural model to derive recommended refactoring solutions for the system performance improvement. The approach has been applied to an e-commerce application based on microservices that has been designed by means of UML software models profiled with MARTE.

International audience; Context: Managing Non-Functional Requirements (NFRs) in software projects is challenging, and projects that adopt Model-Driven Development (MDD) are no exception. Although several methods and techniques have been proposed to face this challenge, there is still little evidence on how NFRs are handled in MDD by practitioners. Knowing more about the state of the practice may help researchers to steer their research and practitioners to improve their daily work. Objective: In this paper, we present our findings from an interview-based survey conducted with practitioners working in 18 different companies from 6 European countries. From a practitioner's point of view, the paper shows what barriers and benefits the management of NFRs as part of the MDD process can bring to companies, how NFRs are supported by MDD approaches, and which strategies are followed when (some) types of NFRs are not supported by MDD approaches. Results: Our study shows that practitioners perceive MDD adoption as a complex process with little to no tool support for NFRs, reporting productivity and maintainability as the types of NFRs expected to be supported when MDD is adopted. But in general, companies adapt MDD to deal with NFRs. When NFRs are not supported, the generated code is sometimes changed manually, thus compromising the maintainability of the software developed. However, the interviewed practitioners claim that the benefits of using MDD outweight the extra effort required by these manual adaptations. Conclusion: Overall, the results indicate that it is important for practitioners to handle NFRs in MDD, but further research is necessary in order to lower the barrier for supporting a broad spectrum of NFRs with MDD. Still, much conceptual and tool implementation work seems to be necessary to lower the barrier of integrating the broad spectrum of NFRs in practice.

Cyber-foraging is a technique to enable mobile de- vices to extend their computing power and storage by offloading computation or data to more powerful servers located in the cloud or in single-hop proximity. While there is a large amount of research in this area, the reality is that there are not many deployed, operational cyber-foraging systems. As these systems become more prevalent due to their proven benefits, combined with the emergence of micro data centers and edge clouds, a need will arise for guidance on their architecture and development. To start providing this guidance, we present a decision model based on a mapping of functional and non-functional requirements for cyber-foraging systems to a set of architectural tactics. The decision model was validated by developers to obtain an expert opinion on its correctness and usefulness for guiding the architecture, design, and evolution of cyber-foraging systems that meet their intended functional and non-functional requirements, while understanding the effects of decisions.