Search

International audience; Causal inference (CI) is one of the popular performance diagnosis methods, which infers the anomaly propagation from the observed data for locating the root causes. Although some specific CI methods have been employed in the literature, the overall performance of this class of methods on microservice performance diagnosis is not well understood. To this end, we select six representative CI methods from three categories and evaluate their performance against the challenges of microservice operations, including the large-scale observable data, heterogeneous anomaly symptoms, and a wide range of root causes. Our experimental results show that 1) CI techniques must be integrated with anomaly detection or anomaly scores to differentiate the causality in normal and abnormal data; 2) CI techniques are more robust to false positives in anomaly detection than knowledge-based non-CI method; 3) To get the fine-grained root causes, an effective way with CI techniques is to identify the faulty service first and infer the detailed explanation of the service abnormality. Overall, this work broadens the understanding of how CI methods perform on microservice performance diagnosis and provides recommendations for an efficient application of CI methods.

Today’s cloud data center infrastructures are not even near being able to cope with the enormous and rapidly varying capacity demands that will be reality in a near future. So far, very little is understood about how to transform today’s data centers (being large, power-hungry facilities, and operated through heroic efforts by numerous administrators) into a self-managed, dynamic, and dependable infrastructure, constantly delivering expected QoS with reasonable operation costs and acceptable carbon footprint for large-scale services with sometimes dramatic variations in capacity demands. In this paper, we discuss some of the major challenges for resource-optimized cloud data center. We propose a new research area called Cloud Control, which is a control theoretic approach to a range of cloud management problems, aiming to transform today´s static and energy consuming cloud data centers into self-managed, dynamic, and dependable infrastructures, constantly delivering expected quality of service with acceptable operation costs and carbon footprint for large-scale services with varying capacity demands.

The emergence of Grid computing infrastructures enables researchers to shareresources and collaborate in more efficient ways than before, despite belongingto different organizations and being geographically distributed. While the Gridcomputing paradigm offers new opportunities, it also gives rise to newdifficulties. This thesis investigates methods, architectures, and algorithmsfor a range of topics in the area of Grid resource management. One studiedtopic is how to automate and improve resource selection, despite heterogeneityin Grid hardware, software, availability, ownership, and usage policies.Algorithmical difficulties for this are, e.g., characterization of jobs andresources, prediction of resource performance, and data placementconsiderations. Investigated Quality of Service aspects of resource selectioninclude how to guarantee job start and/or completion times as well as how tosynchronize multiple resources for coordinated use through coallocation.Another explored research topic is architectural considerations for frameworksthat simplify and automate submission, monitoring, and fault handling for largeamounts of jobs. This thesis also investigates suitable Grid interactionpatterns for scientific workflows, studies programming models that enable dataparallelism for such workflows, as well as analyzes how workflow compositiontools should be designed to increase flexibility and expressiveness. We today have the somewhat paradoxical situation where Grids, originally aimed tofederate resources and overcome interoperability problems between differentcomputing platforms, themselves struggle with interoperability problems causedby the wide range of interfaces, protocols, and data formats that are used indifferent environments. This thesis demonstrates how proof-of-concept softwaretools for Grid resource management can, by using (proposed) standard formatsand protocols as well as leveraging state-of-the-art principles fromservice-oriented architectures, be made independent of current Gridinfrastructures. Further interoperability contributions include an in-depthstudy that surveys issues related to the use of Grid resources in scientificworkflows. This study improves our understanding of interoperability amongscientific workflow systems by viewing this topic from three differentperspectives: model of computation, workflow language, and executionenvironment. A final contribution in this thesis is the investigation of how the design ofGrid middleware tools can adopt principles and concepts from softwareengineering in order to improve, e.g., adaptability and interoperability.

The emergence of Grid computing infrastructures enables researchers to share resources and collaborate in more efficient ways than before, despite belonging to different organizations and being distanced geographically. While the Grid computing paradigm offers new opportunities, it also gives rise to new difficulties. One such problem is the selection of resources for user applications. Given the large and disparate set of Grid resources, manual resource selection becomes impractical, even for experienced users. This thesis investigates methods, algorithms and software for a Grid resource broker, i.e., a scheduling agent that automates the resource selection process for the user. The development of such a component is a non-trivial task as Grid resources are heterogeneous in hardware, software, availability, ownership and usage policies. A wide range of algorithmically difficult issues must also be solved, including characterization of jobs, prediction of resource performance, data placement considerations, and, how to provide Quality of Service guarantees. One contribution of this thesis is the development of resource brokering algorithms that enable resource selection based on Grid job performance predictions and use advance reservations to provide Quality of Service guarantees. The thesis also includes an algorithm for coallocation of sets of jobs. This algorithm guarantees a simultaneous start of each subjob, as required e.g., when running larger-than-supercomputer simulations that involve multiple resources. We today have the somewhat paradoxal situation where Grids, originally aimed to overcome interoperability problems between different computing platforms, themselves struggle with interoperability problems caused by the wide range of interfaces, protocols and data formats that are used in different environments. The reasons for this situation are obvious, expected and almost impossible to avoid, as the task of defining appropriate standards, models and best-practices must be preceded by basic research, proof-of-concept implementations and real-world testing. We address the interoperability problem with a generic Grid resource brokering architecture and job submission service. By using (proposed) standard formats and protocols, the service acts as an interoperability-bridge that translates job requests between clients and resources running different Grid middlewares. This concept is demonstrated by the integration of the service with three different Grid middlewares. The service also enables users to both fine-tune the existing resource selection algorithms and plug in custom brokering algorithms tailored to their requirements.

Microservices have emerged as a popular pattern for developing large-scale applications in cloud environments for its benefits of flexibility, scalability, and agility. A microservices-based cloud system (named as \emph{cloud microservices}) comprises hundreds or thousands of disparate services that communicate via lightweight messaging protocols, share a finite set of hardware and software resources, and are frequently updated to meet customer requirements. In such a complex and dynamic environment, the occurrence of performance problems (e.g., slow application responses) has become the norm rather than the exception, resulting in decreased revenue, damaged reputation, and significant human effort spent on performance diagnosis and recovery. Moreover, manual operation and maintenance of cloud microservices tend to be error-prone or even impracticable. Therefore, there is an urgent need for an automatic performance problem management system that can not only detect anomalous behaviors (performance anomalies) but also uncover the root causes and recommend recovery actions. In this thesis, we investigate methods for automatic performance diagnosis and recovery in cloud microservices. The core objectives of this thesis are to identify \emph{where} and \emph{why} a performance anomaly occurs, and further to decide \emph{how} to mitigate it. To this end, this thesis contributes: (1) a method for locating the faulty service from which a performance anomaly originates, including a graphical model for capturing the propagation of the anomaly in the system; (2) two methods for identifying the anomalous metrics that cause a performance anomaly, using deep learning and Spatio-temporal causal inference (CI). In addition, we evaluate the performance of CI techniques on performance diagnosis in cloud microservices through extensive experiments; (3) a method for selecting the most appropriate recovery action to mitigate an identified performance anomaly. Overall, the methods presented in this thesis diagnose root causes and recommend remedies in real-time without requiring any application instrumentation and historical failure instances. Our methods were implemented in prototypes and experimentally evaluated on representative microservices benchmarks. The evaluations show that they can support automatic performance diagnosis and recovery in cloud microservices and improve service reliability and end-user experience. The results have been peer-reviewed and published at renowned international conferences., Microservices haben sich aufgrund ihrer Flexibilität, Skalierbarkeit und Agilität zu einem beliebten muster für die Entwicklung umfangreicher Anwendungen in Cloud-Umgebungen entwickelt. Ein auf Microservices basierendes Cloud-System (auch als \emph{Cloud-Microservices bezeichnet}) besteht aus Hunderten oder Tausenden von unterschiedlichen Diensten, die über leichtgewichtige Nachrichtenprotokolle miteinander kommunizieren, häufig aktualisiert werden, und sich eine begrenzte Anzahl von Hardware- und Softwareressourcen teilen. In einer derart komplexen und dynamischen Umgebung ist das Auftreten von Leistungsproblemen (z. B. langsame Anwendungsreaktionen) eher die Regel als die Ausnahme. Darüber hinaus sind der manuelle Betrieb und die Wartung von Cloud-Microservices oft fehleranfällig. Daher besteht ein dringender Bedarf an einem automatischen System zur Verwaltung von Leistungsproblemen, das abnormales Verhalten erkennen, die Ursachen aufdecken und Abhilfemaßnahmen empfehlen kann. In dieser Arbeit werden Methoden zur automatischen Ursachenlokalisierung und Behebung von Leistungsproblemen in Cloud-Microservices erforscht. Das Hauptziel dieser Arbeit ist es, zu identifizieren, wo und warum ein Leistungsproblem auftritt, und dann zu entscheiden, wie es behoben werden kann. Zu diesem Zweck bietet diese Arbeit (1) eine Methode zur Lokalisierung des fehlerhaften Dienstes, von dem ein Leistungsproblem ausgeht; (2) zwei Methoden zur Identifizierung der anomalen Metriken, die ein Leistungsproblem verursachen, unter Verwendung von Deep Learning bzw. räumlich-zeitlicher kausaler Inferenz (CI); (3) eine Methode zur Auswahl der am besten geeigneten Wiederherstellungsmaßnahmen zur Entschärfung des Leistungsproblems. Insgesamt diagnostizieren die in dieser Arbeit vorgestellten Methoden die Grundursachen und empfehlen Abhilfemaßnahmen in Echtzeit, ohne dass eine Anwendungsinstrumentierung und historische Fehlerfälle erforderlich sind. Unsere Methoden wurden in Prototypen implementiert und experimentell an repräsentativen Microservices-Benchmarks evaluiert. Die Auswertungen zeigen, dass sie zur Unterstützung der automatischen Leistungsdiagnose und -wiederherstellung in Cloud-Microservices, zur Verbesserung der Servicezuverlässigkeit und der Endbenutzererfahrung eingesetzt werden können. Die Ergebnisse wurden auf renommierten internationalen Konferenzen veröffentlicht.

Cloud computing is widely applied by modern software development companies. Providing digital services in a cloud environment offers both the possibility of cost-efficient usage of computation resources and the ability to dynamically scale applications on demand. Based on this flexibility, more and more complex software applications are being developed leading to increasing maintenance efforts to ensure the reliability of the entire system infrastructure. Furthermore, highly available cloud service requirements (99.999% as industry standards) are difficult to guarantee due to the complexity of modern systems and can therefore just be ensured by great effort. Due to these trends, there is an increasing demand for intelligent applications that automatically detect anomalies and provide suggestions solving or at least mitigating problems in order not to cascade a negative impact on the service quality. This thesis focuses on the detection of degraded abnormal system states in cloud environments. A holistic analysis pipeline and infrastructure is proposed, and the applicability of different machine learning strategies is discussed to provide an automated solution. Based on the underlying assumptions, a novel unsupervised anomaly detection algorithm called CABIRCH is presented and its applicability is analyzed and discussed. Since the choice of hyperparameters has a great influence on the accuracy of the algorithm, a hyperparameter selection procedure with a novel fitness function is proposed, leading to further automation of the integrated anomaly detection. The method is generalized and applicable for a variety of unsupervised anomaly detection algorithms, which will be evaluated including a comparison to recent publications. The results show the applicability for the automated detection of degraded abnormal system states and possible limitations are discussed. The results show that detection of system anomaly scenarios achieves accurate detection rates but comes with a false alarm rate of more than 1%., Cloud Computing Paradigmen, werden in der modernen Softwareentwicklung bereits von den meisten Unternehmen angewendet. Die Bereitstellung von digitalen Diensten in einer Cloudumgebung bietet sowohl die Möglichkeit der kosteneffizienten Nutzung von Ressourcen als auch die Möglichkeit auf Bedarf dynamisch die Anwendungen zu skalieren. Basierend auf dieser Flexibilität werden immer komplexere Softwareanwendungen entwickelt, welches zu anspruchsvollen Wartungsarbeiten der Gesamtinfrastruktur führen. Ebenfalls werden immer höhere Ansprüche an die Verfügbarkeit von Softwarediensten gestellt (99,999% im Industriekontext), was durch die Komplexität moderner Systeme nur noch schwieriger und unter großer Mühe gewährleistet werden kann. Aufgrund dieser Trends steigt der Bedarf an intelligenten Anwendungen, die automatisiert Anomalien erkennen und Vorschläge erarbeiten, um Probleme zu erkennen, zu beheben oder zumindest zu mindern um keinen negativen Einfluss auf die Servicequalität zu kaskadieren. Diese Arbeit beschäftigt sich mit der Erkennung von degradierten abnormalen Systemzuständen in Cloudumgebungen. Hierbei wird sowohl eine holistische Analysepipeline und -infrastruktur beschrieben als auch die Anwendbarkeit von verschiedenen Strategien des maschinellen Lernens diskutiert, um möglichst eine voll automatisierte Lösung bereitzustellen. Basierend auf den zugrunde liegenden Annahmen, wird ein neuartiger unsupervised Anomalieerkennungsalgorithmus namens CABIRCH vorgestellt und dessen Anwendbarkeit analysiert und diskutiert. Da die Wahl der Hyperparameter einen wichtigen Einfluss auf die Genauigkeit des Algorithmus hat, wird zudem ein Hyperparameterauswahlverfahren mit einer neuartigen Fitness-Funktion vorgestellt, welches zur Vollautomatisierung der Anomalieerkennung führen soll. Hierbei ist das Verfahren generalisiert anwendbar für eine Vielzahl von unsupervised Anomalieerkennungsalgorithmen, welche basierend auf jüngsten Veröffentlichungen umfassend evaluiert werden. Dabei wird die Anwendbarkeit zur automatisierten Erkennung von degradierten abnormalen Systemzuständen gezeigt und mögliche Limitierungen diskutiert. Die Ergebnisse zeigen, dass eine Erkennung der verschiedenen Anomalien gewährleistet werden kann, jedoch mit einer Fehlalarmrate von über 1%.