Search

A presente pesquisa tem por objetivo propor um sistema de Web Scraping, baseado na nuvem computacional da AWS, utilizando as máquinas EC2 burstable. O framework define um cluster misto, com instâncias burstable fixas e temporárias, que pode variar o número de instâncias, adicionando ou removendo VMs, para garantir o SLA das mensagens e minimização dos custos. O framework proposto foi avaliado na nuvem AWS, comparado a uma abordagem apenas com instâncias on-demand não expansíveis e também a outra solução baseada em Function as a Service (FaaS). Os resultados mostraram que todos os testes atendem o SLA definido, alcançando uma redução de 96% de custo financeiro, em seu melhor caso quando comparado à abordagem FaaS, e redução de 95,59%, em seu melhor caso quando comparado à abordagem on-demand. Além disso, houve redução de custos de no mínimo 93,26% em todos os demais casos de teste, mostrando que máquinas burstable podem ser um ótimo recurso para esse problema.

Serviços da nuvem oferecem facilidades computacionais bem estabelecidas. Na busca de eficiência para executar aplicações do tipo MapReduce, que lidam com grandes volumes de dados, baixos custos monetários também são almejados. Para delinear os benefícios de diferentes serviços cloud, realizamos uma análise exploratória dos tempos e custos para a execução de uma aplicação MapReduce na nuvem pública da Amazon, a AWS. A partir de implementações com os frameworks Spark e MARLA sob os serviços EC2 e Lambda, respectivamente, apresentamos os impactos associados às quantidades e tipos de recursos escolhidos. Os resultados sugeriram o ambiente MARLA Lambda como o mais rápido e o Spark EC2 como o mais econômico.

In this paper, we developed a Spark application, named Diff Sequences Spark, which compares 540 SARS-CoV-2 sequences from South America in Amazon EC2 Cloud, generating as output the positions where the differences occur. We analyzed the performance of the proposed application on selected memory and storage optimized virtual machines (VMs) at on-demand and spot markets. The execution times and financial costs of the memory optimized VMs outperformed the storage optimized ones. Regarding the markets, Diff Sequences Spark reduced the average execution times and monetary costs when using spot VMs compared to their respective on-demand VMs, even in scenarios with several spot revocations, benefiting from the low overhead fault tolerance Spark framework.

Scientific workflows are abstractions composed of activities, data and dependencies that model a computer simulation and are managed by complex engines named scientific workflow management system (SWfMS). Many workflows demand many computational resources once their executions may involve a number of different programs processing a massive volume of data. Thus, the use of high-performance computing (HPC) and data-intensive scalable computing environments allied to parallelization techniques provides the necessary support for the execution of such workflows. Clouds are environments that already offer HPC capabilities and workflows can explore them. Although clouds offer advantages such as elasticity and availability, failures are a reality rather than a possibility in this environment. Thus, existing SWfMS must be fault-tolerant. There are several types of fault tolerance techniques used in SWfMS such as Checkpoint/Restart, Re-Execution and Over-provisioning, but it is far from trivial to choose the suitable fault tolerance technique for a workflow execution that is not going to jeopardize the parallel execution. The major problem is that the suitable fault tolerance technique may be different for each workflow, activity or activation since programs associated with activities may present different behaviors. This article aims at analyzing several fault-tolerance techniques in a cloud-based SWfMS named SciCumulus, and recommend the suitable one for user’s workflow activities and activations using machine learning techniques and provenance data, thus aiming at improving resiliency.

Heterogeneous systems employing CPUs and GPUs are becoming increasingly popular in large-scale data centers and cloud environments. In these platforms, sharing a GPU across different applications is an important feature to improve hardware utilization and system throughput. However, under scenarios where GPUs are competitively shared, some challenges arise. The decision on the simultaneous execution of different kernels is made by the hardware and depends on the kernels resource requirements. Besides that, it is very difficult to understand all the hardware variables involved in the simultaneous execution decisions, in order to describe a formal allocation method. In this work, we studied the impact that kernel resource requirements have in concurrent execution and used machine learning (ML) techniques to infer the interference caused by the concurrent execution, and to classify the slowdown that results from this interference. The ML techniques were analyzed over the GPU benchmark suites, Rodinia, Parboil and SHOC. Our results showed that, from the features selected in the analysis, the number of blocks per grid, number of threads per block, and number of registers are the resource consumption features that most affect the performance of the concurrent execution.

There has been an increasing interest in running High Performance Computing (HPC) applications in the cloud, mainly due to rapid resource provisioning and significant reduction of operational costs. Biological sequence comparison is an important HPC application that compares sequences in search of similarities. MASA-OpenMP is a highly optimized sequence comparison tool that obtains optimal results. Yet, it can take a long time, depending on the number of sequences compared and their lengths. The Covid-19 pandemic study is of particular interest nowadays, and the comparison of SARS-CoV-2 sequences is crucial to understanding this disease. In this paper, we compare SARS-CoV-2 sequences with MASA-OpenMP in the Amazon Elastic Compute Cloud (Amazon EC2), using both spot and on-demand instances. To efficiently execute a MASA-OpenMP application composed of more than 22,000 tasks on EC2 respecting a given deadline, we propose an execution modeling for MASA-OpenMP on top of the Burst-HADS framework. Burst-HADS is a spot instance-based dynamic scheduler for Bag-of-Tasks applications in the cloud, which minimizes both execution time and financial costs regarding a given deadline even in the presence of spot interruptions. Performance results reveal that, by using spots, our Burst-HADS strategy considerably reduces the monetary cost for executing 22,600 SARS-CoV-2 sequence comparisons with MASA-OpenMP when contrasted to the on-demand only approach. We also show that our strategy can meet the deadlines, even in scenarios with several spot interruptions.

Os provedores de nuvem oferecem vários serviços para processamento e armazenamento de dados. Alguns desses recursos são propensos a falhas ou revogações. Neste caso, a adoção de técnicas de tolerância a falhas é essencial para minimizar o impacto e o custo que estas falhas podem exercer na execução das aplicações e no orçamento dos usuários. Uma das soluções amplamente adotadas são as técnicas de checkpoint and recovery (CR), que gravam o estado atual da aplicação para que, caso ocorra falhas, esta seja reiniciada a partir do último estado gravado. Assim, essas abordagens exigem um local de armazenamento seguro e disponível ao longo de toda a execução. Felizmente, os serviços de nuvens também podem ser utilizados para este fim, uma vez que diferentes opções de armazenamento também são oferecidos. Neste trabalho, queremos avaliar três desses serviços: o Amazon Simple Storage Service (S3), o Amazon Elastic Block Store (EBS) e o Amazon Elastic File System (EFS). O objetivo é caracterizar e avaliar o desempenho desses serviços em relação a gravação e recuperação de checkpoints no contexto de falhas induzidas pela revogação de máquinas virtuais.

Leading Cloud providers offer several types of Virtual Machines (VMs) in diverse contract models, with different guarantees in terms of availability and reliability. Among them, the most popular contract models are the on-demand and the spot models. In the former, on-demand VMs are allocated for a fixed cost per time unit, and their availability is ensured during the whole execution. On the other hand, in the spot market, VMs are offered with a huge discount when compared to the on-demand VMs, but their availability fluctuates according to the cloud's current demand that can terminate or hibernate a spot VM at any time. Furthermore, in order to cope with workload variations, cloud providers have also introduced the concept of burstable VMs which are able to burst up their respective baseline CPU performance during a limited period of time with an up to 20% discount when compared to an equivalent non-burstable on-demand VMs. In the current work, we present the Burst Hibernation-Aware Dynamic Scheduler (Burst-HADS), a framework that schedules and executes tasks of Bag-of-Tasks applications with deadline constraints by exploiting spot and on-demand burstable VMs, aiming at minimizing both the monetary cost and the execution time. Based on ILS metaheuristics, Burst-HADS defines an initial scheduling map of tasks to VMs which can then be dynamically altered by migrating tasks of a hibernated spot VM or by performing work-stealing when VMs become idle. Performance results on Amazon EC2 cloud with different applications show that, when compared to a solution that uses only regular on-demand instances, Burst-HADS reduces the monetary cost of the execution and meet the application deadline even in scenarios with high spot hibernation rates. It also reduces the total execution time when compared to a solution that uses only spot and non-burstable on-demand instances.

This work aims to predict the level of interference caused by concurrent access to shared resources, such as cache and main memory, that can drastically affect the performance of HPC applications executed in clouds, by using some well-known machine learning techniques. As the user does not know the exact number of resource accesses in practice, we propose a human-readable categorization of these accesses. The used dataset contains information about synthetic and real HPC applications, and, to reflect the uncertainty of the user categorization, we inserted some noisy data in it. Our results showed that our approach could correctly predict the level of interference in most cases, indicating that it can be a practical solution.

Graphical Processing Units (GPUs) became an important platform to general purpose computing, thanks to their high performance and low cost when compared to CPUs. Modern GPU architectures are constantly evolving with growing resources. In order to take advantage of all the resources available and increase the GPU efficiency, new generation GPUs include support for concurrent kernel execution. Different kernels can be executed at the same time and share the GPU resources. Thus, benchmark suites developed to evaluate GPU performance and scalability should take this aspect into account that could be quite different from traditional CPU benchmarks. Nowadays, SHOC, Parboil, and Rodinia are the main benchmark suites for evaluating GPUs. This work analyzes these benchmark suites in a novel way. We propose to categorize the kernels of each application of these benchmarks by multiple criteria, built on their behavior in terms of computation type (integer or float), usage of memory hierarchy, efficiency and hardware occupancy. Based on the characterization results, we analyze kernel concurrency opportunities. The focus is on disclosing the resource requirements of the kernels of these benchmarks and to explain their behavior when executed concurrently.

International audience; One challenge for social network researchers is to evaluate balance in a social network. The degree of balance in a social group can be used as a tool to study whether and how this group evolves to a possible balanced state. The solution of clustering problems defined on signed graphs can be used as a criterion to measure the degree of balance in social networks and this measure can be obtained with the optimal solution of the Correlation Clustering (CC) problem, as well as a variation of it, the Relaxed Correlation Clustering (RCC) problem. However, solving these problems is no easy task, especially when large network instances need to be analyzed. In this work, we contribute to the efficient solution of both problems by developing sequential and parallel ILS metaheuristics. Then, by using our algorithms, we solve the problem of measuring the structural balance on large real-world social networks.

Em um ambiente de nuvem computacional, aplicacoes de altodesempenho podem sofrer interferencia ao serem executadas em maquinasvirtuais que estejam alocadas em uma mesma maquina fisica. Emboraalguns trabalhos tenham proposto estrategias de alocacao de maquinas virtuaiscientes deste problema, nenhuma dessas estrategias empregou ummetodo adequado para predizer a interferencia nem considerou, ao mesmotempo, tanto a minimizacao da interferencia quanto do numero de maquinasfisicas ativas na nuvem. Nesta tese, define-se o Problema de Alocacaode Maquinas Virtuais ciente da Interferencia para Aplicacoes de Alto Desempenho de Baixa Escalabilidade, um problema que tem a finalidade deminimizar, simultaneamente, (i) a interferencia sofrida por aplicacoes dealto desempenho que estejam sendo executadas em uma mesma maquinafisica e (ii)o numero de maquinas fisicas necessarias para alocar essas aplicacoesna nuvem. Este trabalho apresenta uma formulacao matematicapara o problema, alem de propor uma estrategia baseada na metaheuristicaBusca Local Iterada para resolve-lo. Para predizer a interferencia,esta estrategia utiliza um modelo quantitativo e multivariado que leva emconta a quantidade e similaridade de acesso aos recursos compartilhados eo numero de aplicacoes co-alocadas. Uma analise experimental, utilizandoaplicacoes reais da area de petroleo e o benchmark HPCC, mostraram que ometodo proposto foi capaz de superar,em termos de reducao de interferencia,varias heuristicas da literatura. Os resultados revelaram que, mesmousando o numero de maquinas fisicas indicados por tais heuristicas, a estrategiaproposta conseguiu reduzir o nivel de interferencia sofrido pelasaplicacoes alocadas na nuvem.

By contracting Virtual Machines on Amazon EC2, the user can opt for on-demand instances, which have high availability, or instances spot (preemptive), which are idle resources offered at a lower price, whose availability may vary throughout the run. Despite the economic advantage, if the demand for resources increases, the preemptive instances can be revoked by the provider without warning, and put in hibernation until the demand decreases. This work presents initial results and the formulation of the scheduling problem which considers hibernation-prone preemptive instances to minimize the monetary cost of execution.

International audience; Nowadays, cloud platforms usually offer several types of Virtual Machines (VMs) which have different guarantees in terms of availability and volatility, provisioning the same resource through multiple pricing models. For instance, in the Amazon EC2 cloud, the user pays per hour for on-demand VMs while spot VMs are unused instances available for a lower price. Despite the monetary advantages, a spot VM can be terminated or hibernated by EC2 at any moment. In this work, we propose the Hibernation-Aware Dynamic Scheduler (HADS), to schedule applications composed of independent tasks (bag-of-tasks) with deadline constraints in both hibernation-prone spot VMs (for cost sake) and on-demand VMs. We also consider the problem of temporal failures, that occurs when a spot VM hibernates, and does not resume within a time that guarantees the application's deadline. Our dynamic scheduling approach aims at minimizing the monetary costs of bag-of-tasks applications execution, respecting its deadline even in the presence of hibernation. It is also able to avoid temporal failures, by using task migration and work-stealing techniques. Experimental results with real executions using Amazon EC2 VMs confirm the effectiveness of our scheduling when compared with on-demand VM only based approaches, in terms of monetary costs and execution times. It is also shown that our strategy can tolerate temporal failures.

Usually, scientists need to execute experiments that demand high performance computing environments and parallel techniques. This is the scenario found in many bioinformatics experiments modeled as scientific workflows, such as phylogenetic and phylogenomic analyses. To execute these experiments, scientists have adopted virtual machines (VMs) instantiated in clouds. Estimating the number of VMs to instantiate is a crucial task to avoid negative impacts on the execution performance and on the financial costs with under or overestimations. Previously, the necessary number of VMs to execute bioinformatics workflows have been estimated by a GRASP heuristic and have been coupled to a Cloud-based Parallel Scientific Workflow Management System. Although this work was a step forward, this approach only provided a static dimensioning. If the characteristics of the environment change (processing capacity, network speed), this static dimensioning may not be suitable. In this way, it is of interest that the dimensioning is adjusted at runtime. To achieve this, we developed a novel framework for monitoring and dynamically dimensioning resources during the execution of parallel scientific workflows in clouds, called Dynamic Dimensioning of Cloud Computing Framework (DDC-F). We have evaluated DDC-F in real executions of bioinformatics workflows. Experiments showed that DDC-F is able to efficiently calculate the number of VMs necessary to execute bioinformatics workflows of Comparative Genomics (CG), also reducing the financial costs, when compared with other works of the related literature.