Search

Workload characteristics are vital for both data center operation and job scheduling in co-located data centers, where online services and batch jobs are deployed on the same production cluster. In this paper, a comprehensive analysis is conducted on Alibaba‘s cluster-trace-v2018 of a production cluster of 4034 machines. The findings and insights are the following: (1) The workload on the production cluster poses a daily cyclical fluctuation, in terms of CPU and disk I/O utilization, and the memory system has become the performance bottleneck of a co-located cluster. (2) Batch jobs including their tasks and derived instances can be approximated as Zipf distribution. However, for all batch jobs with directed acyclic graph dependency, they suffer from co-location with online services since the online services are highly prioritized. (3) The resource usages of containers have similar cyclical fluctuation consistent with the whole cluster, while their memory usages remain approximately constant. (4) The number of batch jobs co-located with online services is dependent on the mispredictions per kilo instructions of online services. In order to guarantee the QoS of online services, when the MPKI of online services rises, the number of batch jobs to be co-located on the same machine should decrease.

International audience

The traditional centralized network architecture can lead to a bandwidth bottleneck in the core network. In contrast, in the information-centric network, decentralized in-network caching can alleviate the traffic flow pressure from the network center to the edge. In this paper, a popularity-aware in-network caching policy, namely, Pop, is proposed to achieve an optimal caching of network contents in the resource-constrained edge networks. Specifically, Pop senses content popularity and distributes content caching without adding additional hardware and traffic overhead. We conduct extensive performance evaluation experiments by using ndnSIM. The experiments showed that the Pop policy achieves 54.39% cloud service hit reduction ratio and 22.76% user request average hop reduction ratio and outperforms other policies including Leave Copy Everywhere, Leave Copy Down, Probabilistic Caching, and Random choice caching. In addition, we proposed an ideal caching policy (Ideal) as a baseline whose popularity is known in advance; the gap of Pop and Ideal in cloud service hit reduction ratio is 4.36%, and the gap in user request average hop reduction ratio is only 1.47%. More simulation results further show the accuracy of Pop in perceiving popularity of contents, and Pop has good robustness in different request scenarios.

Today, the Network Slicing technology is massively addressed by the research community. However, Network Slice (NS) modelling details from Standards Developing Organizations (SDOs) are not yet well considered for End-to-End (E2E) NS implementations. In addition, each SDO develops standards targeting only a specific part of the NS architecture. Therefore, based on a profound analysis of the major existing works, this article explains first (i) a general architecture that clarifies the basic E2E network slicing functionality before diving deep into domain-specific visions. Then, (ii) it focuses at providing a survey stitching together the NS modelling works in Radio Access Networks (RAN), Core Networks (CN) and also Transport Networks (TN). The end goal is to clarify the E2E Network Slicing process from the service order request to the NS deployment and life-cycle management. Last, as there is no consensus on a specific information model in the Transport network domains (iii) we provide our vision on how several data models, developed by IETF working groups, can be integrated together in the context of the ACTN architecture in order to provision and manage Transport NSs.

This paper presents a new opportunistic scheduling and resource consolidation system based on an economic model related to different service level agreements (SLAs) classes. The goal is to address the problem of companies that manage a private infrastructure of machines, i.e., a cloud platform and would like to optimize the scheduling of several requests submitted online by users. For the sake of simplicity of the presentation, the proposed economic model has two SLAs classes (qualitative and quantitative) with three Quality of Service for each SLA class (Premium, Advanced and Best effort). The consequence of this choice as well as the need to serve requests as they come have an impact on the algorithmic ways to consolidate an infrastructure. Indeed, our system proposes a new allocation heuristic that adapts the number of active machines in the cloud according to the global resources usage of all machines inside the infrastructure. This heuristic can be examined as a consolidation heuristic, based on the idea that the system can make reasonable choices, based on the SLAs, for the placement and the allocation of resources for each request. Experimentation with our system is conducted on Prezi (Web workload) and Google Cloud Data (HPC-oriented workload) traces, and they demonstrate the potential of our approach under different scenarios. From a methodological point of view, we propose a general framework which is limited in scope, for the sake of simplicity in reading the paper, with a small number of SLAs, but the idea can be extended to many more SLAs and performance metrics. In this way, the user or the provider operating the cloud have more latitude, thanks to our multi-criteria approach, to control the workload without a sacrifice on performance.

Edge computing is an emerging paradigm to meet the ever-increasing computation demands from pervasive devices such as sensors, actuators, and smart things. Though the edge devices can execute complex applications, it is necessary for some applications to migrate to centralized servers. By offloading the computation from the edge nodes to the edge servers or cloud servers, the quality of computation experience could be greatly improved. However, it may cause delay and increase network overheads, and energy consumption eventually. Therefore, an optimal offloading strategy should take into account what task should be offloaded, when to offload and where to offload to avoid the overheads. Thus, it is important to tradeoff between energy consumption, computation delay and throughput when the system makes the computation offloading to achieve high energy efficiency. In this paper, we conduct a survey of energy aware edge computing, including the existing work on computation offloading frameworks and strategies in edge computing. Specifically, we describe the strategies from the perspective of energy aware offloading, energy optimization offloading and offloading algorithms.

Infrastructure-as-a-service container-based virtualization is gaining interest as a platform for running distributed applications. With increasing scale of cloud architectures, faults are becoming a frequent occurrence, which makes availability true challenge. Replication is a method to survive failures whether of checkpoints, containers or data to increase their availability. In fact, following a node failure, fault-tolerant cloud systems restart failed containers on a new node from distributed images of containers (or checkpoints). With a high failure rate, we can lose some replicas. It is interesting to increase the replication factor in some cases and finding the trade-off between restarting all failed containers and storage overhead. This paper addresses the issue of adapting the replication factor and contributes with a novel replication factor modeling approach, which is able to predict the right replication factor using prediction techniques. These techniques are based on experimental modeling, which analyze collected data related to different executions. We have used regression technique to find the relation between availability and replicas number. Experiments on the Grid’5000 testbed demonstrate the benefits of our proposal to satisfy the availability requirement, using a real fault-tolerant cloud system.

We introduce a new parallel and distributed algorithm for the solution of the satisfiability problem. It is based on an algorithm portfolio and is intended to be used for servicing requests in a distributed cloud. The core of our contribution is the modeling of the optimal resource sharing schedule in parallel executions and the proposition of heuristics for its approximation. For this purpose, we reformulate a computational problem introduced in a prior work. The main assumption is that it is possible to learn optimal resource sharing from traces collected on past executions on a representative set of instances. We show that the learning can be formalized as a set coverage problem. Then we propose to solve it by approximation and dynamic programming algorithms based on classical greedy algorithms for the maximum coverage problem. Finally, we conduct an experimental evaluation for comparing the performance of the various algorithms proposed. The results show that some algorithms become more competitive if we intend to determine the trade-off between their quality and the runtime required for their computation.

In this paper, we share our vision to study a complex Information Technology (IT) system handling a massive amount of data in the context of 'smart buildings.' One technique for analyzing complex IT systems relies on emulation, where the final software system is fully deployed on real architectures, and is evaluated in considering "small" instances of situations the system is supposed to solve. We propose a software architecture for studying the ecosystem of 'smart buildings'. This software architecture is built: 1) on top of ontologies for the description of smart buildings; 2) on a special tool for mastering the life cycle of data produced by sensors and actuators inside the buildings. We assume that it is equally important to model both the building's components and the flow of data produced inside the building. We use existing software components for both goals and to make real our concerns. According to a translational methodology, we also discuss use cases for illustrating the potential of our approach and the particular challenges associated with making the two main components of our emulation tool inter-operate. Therefore, our main contribution is to propose a comprehensive, ambitious and realistic research plan to guide communities. The paper illustrates how computer scientists and smart buildings domain scientists may communicate to address and solve specific research problems related to Big Data in emergent distributed environments. We are also guessing that experimental results that can demonstrate the practicality of the proposed combination of tools could be devised in the future, based on our broad vision. The paper is, first and foremost, a visionary paper.

Cloud services are now well established, thanks to some providers' pioneering work that currently offer on-premise the advantage of the predictability, continuity, and quality of service delivered by virtualization technologies. In this context, SDN (Software Defined Networking) aims to provide tenant-controlled management of forwarding and different abstractions of the underlying network infrastructure to the applications. The scheduling and placement of network functions in the cloud is a challenging task. One reason is that it also requires tedious provisioning and configuration steps. Even if we consider in this paper only the placement of network functions and not their configurations, we are faced with the general problem of defining, in an 'optimal' way, the placement of network functions to be executed so that some criteria are preserved. In this paper, we formulate an approach to schedule network functions according to their dependencies.

Faced to the diversity of the public in using clouds, computer scientists should take care to usages and facilitate access to non-experts i.e. to hiding internal artifacts and complexity of such distributed systems. At the opposite, the cloud provider wants to optimize resource usage, for instance for billing or to keep the cloud as green as possible. These two conflicting objectives are addressed in this paper. We combine a smart view for specifying the Service Level Agreements (SLA) of a user with a strategy of placing the user’s jobs, which represents the cloud provider’s point of view. We put in place a multi-criteria approach and the experimental results demonstrate the potential of our approach under a different scenario.

In cloud computing, elasticity is defined as "the degree to which a system is able to adapt to workload changes by provisioning and de-provisioning resources in an autonomic manner, such that at each point in time the available resources match the current demand as closely as possible". Adding elasticity to HPC (High Performance Computing) clusters management systems remains challenging even if we deploy such HPC systems in today's cloud environments. This difficulty is caused by the fact that HPC jobs scheduler needs to rely on a fixed set of resources. Every change of topology (adding or removing computing resources) leads to a global restart of the HPC jobs scheduler. This phenomenon is not a major drawback because it provides a very effective way of sharing a fixed set of resources but we think that it could be complemented by a more elastic approach. Moreover, the elasticity issue should not be reduced to the scaling of resources issues. Clouds also enable access to various technologies that enhance the services offer to users. In this paper, our approach is to use containers technology to instantiate a tailored HPC environment based on the user's reservation constraints. We claim that the introduction and use of containers in HPC job schedulers allow better management of resources, in a more economical way. From the use case of SLURM, we release a methodology for 'containerization' of HPC jobs schedulers which is pervasive i.e. spreading widely throughout any layers of job schedulers. We also provide initial experiments demonstrating that our containerized SLURM system is operational and promising.

Infrastructure-as-a-Service container-based virtualization is gaining interest as a platform for running distributed applications. With increasing scale of Cloud architectures, faults are becoming a frequent occurrence, which makes availability a challenge. LXCloudFT is a fault tolerant Cloud system, which is composed of LXCloud-CR, a Checkpoint–Restart model and GC-CR, a garbage collector component that eliminates old snapshots of containers. LXCloudFT is designed, originally, for scientific applications and all its components are decentralized. We want to adapt it to serve stateless loosely coupled applications such as web applications. Replication is a method to survive failures for such applications. This paper addresses the issue of replication and contributes with a novel replication model, LXCloud-Rep, in LXCloudFT. LXCloud-Rep is a replication model with versioning and garbage collection, which is able to replicate Linux Container instances on several nodes in a decentralized manner. Following a node failure, LXCloud-Rep restarts failed containers on a new node from distributed images of containers not from snapshots. It optimizes the use of storage space. Large-scale experiments on Grid’5000 improve the performance of applications.

Infrastructure-as-a-Service (IaaS) container-based virtualization technology is gaining, over these years, significant interest in industry as an alternative platform for running distributed applications. They present an interesting alternative to virtual machines in the Cloud due to newer advances in container-based virtualization as a new technology that simplify the deployment of applications. As the cloud architectures continue to grow in scale and complexity, faults become very recurrent which make reliability a true challenge. Given the dynamic nature of IaaS clouds and the pay-as-you-go cloud model where the costs are directly proportional to the resource usage, a Checkpoint Restart mechanism is essential in this context. We propose LXCloud-CR our new decentralized Checkpoint-Restart model based on a distributed checkpoints repository using key–value store on a Distributed Hash Table (DHT). It is able to take snapshots of the whole Linux Container (LXC) instances. LXCloud-CR aims to reduce the runtime and storage overheads of checkpointing. Large scale experiments on the Grid’5000 testbed demonstrate the benefits of our proposal. Obtained results validate our model and improve the performance of applications.

This paper presents an accelerated Promethee (Preference Ranking Organization METHod for Enrichment Evaluations) multi-criteria algorithm based on dimensionality reduction in large scale environments. In our context, the Promethee algorithm is used to select from a large set of objects, one or a small set of objects with a good compromise between several qualitative and quantitative criteria. The exact solution can be used by applying the exact multi-criteria Promethee algorithm. However, the drawback, with this type of exact algorithm, is the long execution time due to the combinatorial aspect of the problem. The exact Promethee computing time is linked both to the dimension of the problem (number of qualitative and quantitative criteria) and the size of the problem (number of objects). To address the previous drawback, we propose to accelerate the Promethee algorithm in combining the exact Promethee algorithm with an algorithm inherited from the Machine Learning (ML) field. The experiments demonstrate the potential of our approach under different scenarios to accelerate the respond time.

In this paper, we present a new allocation and resource consolidation system based on a scalability metric. According to cloud computing principles, the end users rent computing and big data analytic services with a pay-as-you-go cost model. However, when users’ data size increases or when the application stresses the memory or requires more computing power, they need to scale their rental to achieve approximately the same performance, such as task completion time and normalized system throughput. In this paper, we propose to delegate the responsibility to scale-up and scale-out the cloud system to a new component of the cloud orchestrator. The decision is taken on a metric that quantifies the scalability of the cloud system consistently under different system expansion configurations. The scalability metric is defined as a ratio between the new and the current situation, over the size of the i-th workload in terms of CPU cores and a performance metric. The considered metrics are the waiting time in the queue and the average resource (cores) usage rate during task execution. To validate our approach, we conduct experiments by emulation, on a real platform. The experimental results demonstrate the validity of the proposed general automatic strategies for cloud allocation and consolidation.

Network Slicing is becoming a mature technology in recent years due to the continuous efforts conducted by the standards organizations, academic units and operators. However, current Network Slice (NS) demonstrations are not fully aligned with the ongoing standardization activities, particularly with the Network Slice Subnet Management Function (NSSMF) defined by 3GPP. This paper provides a novel Framework for NS Subnet Management, called NESSMA, that satisfies a list of requirements derived from an exhaustive analysis of up-to-date NS standards. The Framework is designed to jointly manage NS Subnets and their supported services having in mind its integration with NFV-MANO.

Currently, the intrinsic nature of cloud computing offers new opportunities for scheduling unit of works as containers. In this paper we present a new Scheduling and Allocation Framework for Containers (SAFC) based on an economic model. The novelty of our framework is to gives the possibility to specify ranges on the resources demand for containers instead of specifying a fixed amount of resource. Thus, the goal and the new developments in this paper are to propose a general framework which dynamically schedules containers and decides, on the fly, the number of resources that must be allocated to optimize the use of cloud resources and to maximize the number of containers executed. The key idea of the paper is in relaxing strict constraints on the requested number of resources and on letting the system to adjust the amount of resources according to the execution context, such as the peaks of activities. The SAFC framework is evaluated using the Linux Containers (LXC) inside the Grid’5000 testbed.

The swarm robot network is the widely used wireless communication technology, and significantly, it has higher uplink and downlink throughput than the conventional communication networks. Apart from the appreciable advantages, there are various challenges like minimized data retention rate, low data transmission rate, countless handovers over a short time, and a short transmission range due to higher data transmission usage. To resolve these challenges, Scalable Network Architecture with Dual Connectivity Application (SN-DCA) in control and data plane is proposed to improve data retention rates and minimize network usage by controlling unwanted access in the distributive clouds during transmission. Here, Two-stage control data plane splitting (TSCDP) is introduced to enhance network usage by optimizing unwanted access. It further improves the data retention rate during transmission. The analytical performance of the proposed techniques has been validated computationally and examined through simulation studies. The Simulation results show that these techniques achieve a higher performance ratio in the network with a minimum transmission delay and a high data retention rate.

Recently, containers have been used extensively in the cloud computing field, and several frameworks have been proposed to schedule containers using a scheduling strategy. The main idea of the different scheduling strategies consist to select the most suitable node, from a set of nodes that forms the cloud platform, to execute each new submitted container. The Spread scheduling strategy, used as the default strategy in the Docker Swarmkit container scheduling framework, consists to select, for each new container, the node with the least number of running containers. In this paper, we propose to improve the Spread strategy by presenting a new container scheduling strategy based on the power consumption of heterogeneous cloud nodes. The novelty of our approach consists to make the best compromise that allows to reduce the global power consumption of an heterogeneous cloud infrastructure. The principle of our strategy is based on learning and scheduling steps which are applied each time a new container is submitted by a user. Our proposed strategy is implemented in Go language inside Docker Swarmkit. Experiments demonstrate the potential of our strategy under different scenarios.

In this paper we target the class of modal clustering methods where clusters are defined in terms of the local modes of the probability density function which generates the data. The most well-known modal clustering method is the k-means clustering. Mean Shift clustering is a generalization of the k-means clustering which computes arbitrarily shaped clusters as defined as the basins of attraction to the local modes created by the density gradient ascent paths. Despite its potential, the Mean Shift approach is a computationally expensive method for unsupervised learning. Thus, we introduce two contributions aiming to provide clustering algorithms with a linear time complexity, as opposed to the quadratic time complexity for the exact Mean Shift clustering. Firstly we propose a scalable procedure to approximate the density gradient ascent. Second, our proposed scalable cluster labeling technique is presented. Both propositions are based on Locality Sensitive Hashing (LSH) to approximate nearest neighbors. These two techniques may be used for moderate sized datasets. Furthermore, we show that using our proposed approximations of the density gradient ascent as a pre-processing step in other clustering methods can also improve dedicated classification metrics. For the latter, a distributed implementation, written for the Spark/Scala ecosystem is proposed. For all these considered clustering methods, we present experimental results illustrating their labeling accuracy and their potential to solve concrete problems., Comment: Algorithms are available at https://github.com/Clustering4Ever/Clustering4Ever

This paper presents a new resource allocation framework based on SLA (Service Level Agreements) classes for cloud computing environments. Our framework is proposed in the context of containers with two qualitative and two quantitative SLAs classes to meet the needs of users. The two qualitative classes represent the satisfaction time criterion, and the reputation criterion. Moreover, the two quantitative classes represent the criterion over the number of resources that must be allocated to execute a container and the redundancy (number of replicas) criterion. The novelty of our work is based on the possibility to adapt, dynamically, the scheduling and the resources allocation of containers according to the different qualitative and quantitative SLA classes and the activities peaks of the nodes in the cloud. This dynamic adaptation allows our framework a flexibility for efficient global scheduling of all submitted containers and for efficient management, on the fly, of the resources allocation. The key idea is to make the specification on resources demand less rigid and to ask the system to decide on the precise number of resources to allocate to a container. Our framework is implemented in C++ and it is evaluated using Docker containers inside the Grid’5000 testbed. Experimental results show that our framework gives expected results for our scenario and provides with good performance regarding the balance between objectives.

Cloud computing is regarded as the best generation of computation that relies on shared computing resources rather than having local servers or personal devices to handle different applications. Knowing that, from one side, computing resources can be dynamically allocated according to consumer requirements and from the other side, virtual machine deployment preferences has an important role in Cloud computing for improving resource use. In this paper, we propose an online placement strategy of services on dynamic availability resources in order to reduce energy consumption. Dynamic deployment of services is a key issue of Cloud computing; there are still many areas to be studied, especially for dynamic environments. Unlike the infrastructure topology of the traditional Clouds, i.e. the data centers to which users are connected, in the volunteer Cloud, users and data centers can process multiple requests. Indeed, the volunteer nodes are useful for the Cloud elasticity. However, they are not usually available. The dynamic availability is one of the biggest concerns for the consumers. In addition, live migration of services is a widely used technique for dynamically allocating resources in a volunteer Cloud to overcome such issue. In this work, we investigate the problem of energy efficient online task allocation in volunteer Cloud environment, where the dynamic services can be allocated to a set of dynamic availability resources. We propose through this paper an online heuristic called Dynamic Shortest Path Strategy (DSPS) in order to generate a good planning that minimizes the energy consumption of applications. Our contribution respects various constraints such as availability, capacity of machines and duplications factor of applications. A series of experiments are illustrated to validate the potential of our approach.

This paper presents two practical Large Scale Multi-Objectives Scheduling (LSMOS) strategies, proposed for Cloud Computing environments. The goal is to address the problems of companies that manage a large cloud infrastructure with thousands of nodes, and would like to optimize the scheduling of several requests submitted online by users. In our context, requests submitted by users are configured according to multi-objectives criteria, as the number of used CPUs and the used memory size, to take an example. The novelty of our strategies is to select effectively, from a large set of nodes forming the Cloud Computing platform, a node that execute the user request such that this node has a good compromise among a large set of multi-objectives criteria. In this paper, first we show the limit, in terms of performance, of exact solutions. Second, we introduce approximate algorithms in order to deal with high dimensional problems in terms of nodes number and criteria number. The proposed two scheduling strategies are based on exact Kung multi-objectives decision algorithm and k-means clustering algorithm or LSH hashing (random projection based) algorithm. The experiments of our new strategies demonstrate the potential of our approach under different scenarios.

Smart buildings present challenging opportunities and issues regarding the intelligence we push in the framework that manage the building. Among them the scheduling and allocation of tasks/jobs. Based on the expertises of Qarnot Computing and LIPN Laboratory, we address three relevant matters of concern for smart buildings. First of all, we explain the economic model that the Qarnot middleware implements for managing, in a distributed way, smart buildings. The key idea is to consider the building as a data-center, managed by a cloud middleware. Then we introduce the key properties we consider as important in the context of the construction of a middleware for buildings. At last, this paper presents an efficient scheduling and allocation system according to an innovative economic model. The idea is to presents a new container scheduling system based on SLA (Service Level Agreements) classes and which is used in smart building with cloud computing environment. The novelty of our system is based on the possibility to adapt, dynamically, the scheduling and the resources allocation of containers according to the different SLA classes and the activities peaks of the nodes in the cloud i.e. the building. Experimental results show that our system gives expected results for our scenario and provides with good performance regarding the balance between objectives.

Servers have different energy efficiency and energy proportionality (EP) due to their hardware configuration (i.e., CPU generation and memory installation) and workload. However, current virtual machine (VM) scheduling in virtualized environments will saturate servers without considering their energy efficiency and EP differences. This article will discuss EASE, the energy efficiency and proportionality aware VM scheduling approach. EASE first executes customized computing intensive, memory intensive, and hybrid benchmarks to calculate a server's energy efficiency and EP. Then it schedules VMs to servers to keep them working at their peak energy efficiency point (or optimal working range). This step improves the overall energy efficiency of the cluster and the data center. For performance guarantee, EASE migrates VMs from servers under highly contending conditions. The experimental results on real clusters show that power consumption can be saved 37.07% ~ 49.98% in the homogeneous cluster. The average completion time of the computing intensive VMs increases only 0.31 % ~ 8.49%. In the heterogeneous nodes, the power consumption of the computing intensive VMs can be reduced by 44.22 %. The job completion time can be saved by 53.80%.

The paper discusses various aspects of interest regarding the data management in the context of executing scientific workflows in the cloud. We offer a proper service-based solution e.g. Workflow as a Service, considering additional factors such as the control of data propagation and service level agreements. One important aspect is the optimization of resource allocation, and to accomplish this a dynamic allocation approach, based on certain heuristics, is adopted and tested. The aim of this paper is to introduce the basic concepts and some implementation details for a two layers based scheduling system that globally takes into account data access and space requirements. At the cloud level, we propose a scheduling strategy based on different Service Level Agreement (SLA) classes. The novelty of our strategy consists in using the SLA class of the user to provision a container that will execute the service, based on a dynamic computation of the number of resources. The user do not specify the exact amount of resources but a range. The key idea of the paper is in relaxing strict constraints on the number of resources for data management and on letting the "System" to adjust the resources number according to the execution context. The first and second layers are validated through simulation and emulation conducted on the Grid'5000 testbed and for an heterogeneous context. The results of our experiments demonstrate the potential of our approaches, as general approaches for a better control of data movement and placement in cloud to execute scientific workflows. We also provide with insights to coupling the two proposed layers in a coherent way.

Nowadays, the adoption of Cloud Computing platforms and Service Computing technologies are almost natural for the different e-Science communities. Cost benefits for data-intensive applications, ease of access, rich and varied offers for services are examples of positive returns by users. However, beyond this favorable welcome for the technology, some research problems remain and are still challenging. In this paper, we focus on the problems of automatically deploying IaaS for computing and for data management, using the SlapOS Cloud. The core of the system is a distributed protocol for orchestrating data and compute nodes. Using this interaction scheme, users are able to deploy, without any system administrator intervention, a PaaS inside the IaaS basically a Desktop Grid middleware. The aim of this paper is to demonstrate that the Desktop Grid and Cloud paradigms may merge and may be widely used by non-experts in the different areas of e-Science. We propose a fully self-organized volunteer Cloud for researchers where they can carry out e-Science experiments and process large amounts of data in a coherent way.

Traditional Cloud computing has emerged as a new paradigm for providing computing resources on demand and outsourcing software and hardware infrastructures. Cloud computing is rapidly changing the way IT services are made available and managed. These services can be requested by several Cloud providers, hence the need for networking between IT service components distributed in geographically diverse locations. Like the traditional Cloud computing, the volunteer computing paradigm has become increasingly important. For this paradigm, the resources on each personal machine are shared, thanks to the will of their owners. Cloud and volunteer paradigms have been recently seen as complementary technologies to better exploit the use of local resources. Besides execution time and cost, energy consumption is also becoming more important in the Cloud computing environments. Thus, it has become a major concern for the widespread deployment of Cloud data centers. Among methods that can overcome this problem, we are interested in planning services that improve the use of data center resources in a dynamic environment. In this context, we propose throughout this paper a heuristic that predicts the allocation of dynamic and independent services to reduce the total energy consumption. Our proposal respects various constraints: availability, capacity of machines and the number of applications duplications. A series of experiments illustrates and validates the potential of our approach.

The exponential increment in data size poses new challenges for computer scientists, giving rise to a new set of methodologies under the term Big Data. Many efficient algorithms for machine learning have been proposed, facing up time and memory requirements. Nevertheless, with hardware acceleration, multiple software instructions can be integrated and executed into a single hardware die. Current researches aim at eliminating the burden for the user in using multiple processor types. In this paper we propose our return of experience on a new way of implementing machine learning algorithms on heterogeneous hardware. To explore our vision, we use a parallel Mean-shift algorithm, developed at LIPN as our case study to investigate issues in building efficient Machine Learning libraries for heterogeneous systems. The ultimate goal is to provide a core set of building blocks for Machine Learning programming that could serve either to build new applications on heterogeneous architectures or to control the evolution of the underlying platform. We thus examine the difficulties encountered during the implementation of the algorithm with the aim to discover methodologies for building systems based on heterogeneous hardware. We also discover issues and building blocks for solving concrete machine learning (ML) problems on the Chisel software stack we use for this purpose.

Cloud computing is one of the most popular technologies nowadays because of its wide utilities and various benefits in several IT companies all over the world. However, in front of the increasing users' requests for computing services, cloud providers are encouraged to deploy large data centers, which consumes very large amount of energy and contribute to,,,, high operational costs. Among the effects, carbon dioxide emission rate is growing each day due to the huge amount of power consumption. This energy efficiency is an important issue in cloud computing, mainly due to the required electrical power to run these systems and to cool them. Therefore, energy consumption has become a major concern for the widespread deployment of Cloud data centers. The growing importance for parallel applications in the Cloud introduces significant challenges in,,,, reducing energy consumption from hosted servers. This paper addresses the problem of placing independent applications on the physical servers (hosts) of a Cloud infrastructure. We proposed a novel heuristic to allocate applications so that total energy consumption is reduced. Our proposal respects various constraints e.g. the machines availability, capability and the duplication of applications. Experiments are illustrated to validate the potential of our approach.

This paper presents a new scheduling and resource management allocation system based on an economic model related to different classes for SLAs (Service Level Agreements). The goal is to address the problems of companies that manage a private infrastructure of machines, and would like to optimize the scheduling of several requests submitted online by users. Each request is an application which is executed using a set of computing resources. Our economic model has two SLAs classes (a qualitative one and a quantitative one). The qualitative class represents the satisfaction time criteria, i.e. the user waiting time before the execution of its requests. Moreover, the quantitative class represents the number of resources that must be allocated to execute the user request. As a first contribution, our system allocates dynamically, for each selected request, a set of computing cores according to the quantitative SLA class and the load of the parallel machines across the infrastructure. To choose the machine that will execute a selected request, we propose to use a Bin Packing heuristic to minimize the number of used machines and reduce the cost of the infrastructure. As a second contribution, simulations of our system are conducted on Prezi and Google Cloud Data traces and they demonstrate the potential of our approach under different scenario.

In this paper, we consider the energy-efficient coordination of a set of appliances in a smart-building. We introduce a theoretical formulation of the coordination problem and an Integer Linear Programming model for its resolution. Our formalization is complemented by an analysis of the properties and limits of the model. We also define a practical smart-building setting in which our formalization holds. Finally, we conduct several experiments with realistic data and we make a scalability and sensitivity analysis. The experimental results correspond to emulation on an industrial infrastructure for smart-buildings, the one of Qarnot computing. They are obtained in using the API of the company. They show that we can quickly solve the problem for small and medium-sized buildings and for realistic settings. They also open interesting questions regarding the optimal model of control in future intelligent buildings. Should residents let a collective intelligence to decide on the optimal control of its appliance or is it more appropriate that the user decides by itself?

This paper presents our initial ideas for a new scheduling strategy integrated in the Docker Swarm scheduler. The aim of this paper is to introduce the basic concepts and the implementation details of a new scheduling strategy based on different Service Level Agreement (SLA) classes. This strategy is proposed to answer to the problems of companies that manage a private infrastructure of machines, and would like to optimize the scheduling of several requests submitted online by users. Each request is a demand of creating a container. Currently, Docker Swarm has three basic scheduling strategies (spread, binpack and random), each one executes a container with a fixed number of resources. However, the novelty of our new strategy consists in using the SLA class of the user to provision a container that must execute the service, based on a dynamic computation of the number of CPU cores that must be allocated to the container according to the user SLA class and the load of the parallel machines in the infrastructure. Testing of our new strategy is conducted, by emulation, on different part of our general framework and it demonstrates the potential of our approach for further development.

Infrastructure-as-a-Service container-based virtualization technology is gaining significant interest in industry as an alternative platform for running distributed applications. With increasing scale of Cloud Computing architectures, faults are becoming a frequent occurrence. Checkpoint-Restart is a key method to survive to failures in this context. However, there is a need to reduce the amount of checkpointing data as the Cloud is based on the pay-as-you-go model. This paper addresses the issue of garbage collection in LXCloud-CR and contributes with a novel decentralized garbage collection component GC-CR. LXCloud-CR, a decentralized Checkpoint-Restart model, is able to take snapshots of Linux Container instances and it uses replication to increase snapshots availability. LXCloud-CR contains a versioning scheme for each replica. The disadvantage refers to snapshots availability issues with versioning as the number of useless files grows. GC-CR is a decentralized garbage collector (checkpoint deletion) component that attempts to identify and eliminate old snapshots versions from the system in order to free storage space. Large scale experiments on the Grid5000 testbed demonstrate the benefits of our proposal. Obtained results validate our model and show significant reduction of storage space consumption

In this paper we would like to share our experience for transforming a parallel code for a Computational Fluid Dynamics (CFD) problem into a parallel version for the RedisDG workflow engine. This system is able to capture heterogeneous and highly dynamic environments, thanks to opportunistic scheduling strategies. We show how to move to the field of "HPC as a Service" in order to use heterogeneous platforms. We mainly explain, through the CFD use case, how to transform the parallel code and we exhibit challenges to 'unfold' the task graph dynamically in order to improve the overall performance (in a broad sense) of the workflow engine. We discuss in particular of the impact on the workflow engine of such dynamic feature. This paper states that new models for High Performance Computing are possible, under the condition we revisit our mind in the direction of the potential of new paradigms such as cloud, edge computing.

We introduce the Qarnot platform, a new edge computing platform for the design of smart-buildings. Our proposition is based on a new model of servers that also serve as heaters. As a use case, we consider the recognition of acoustic events. We describe a reference architecture for the processing of acoustic flows in the Qarnot platform. We also present some experimental results on the recognition of alarm sounds.

This paper deals with the reduction of the number of comatose servers. The characteristic of such a server is to consume electricity while not delivering useful information services. According to recent studies, up to 30% of the servers (including those in datacenters) are comatose. The existence of these servers lowers the interest in clouds for green computing. Our paper assumes a cloud provider whose services are operated on a minimal number of dedicated servers in a datacenter, we also assume that this provider delivers software as a service (SaaS). In order to reduce the number of dedicated servers that could be comatose, we propose to automatically generate and run auto-tuning tasks that, in servers idle time, will learn to calibrate the execution of the software delivered by the provider. Our proposition follows the goal of delivering auto-tuning as an opportunistic cloud-service. For this purpose we introduce an opportunistic auto-tuning service (QTuning) that exploits servers idle time. The service is based on a theoretical model that includes two computational problems: an exploration and exploitation problem. In the former problem, the goal is to evaluate the performance of a software in different configurations. In the latter, assuming a set of performance results for a given software, the goal is to compute the best configurations to run the software with. Our second contribution is to propose and evaluate solutions for the exploitation problem.

We consider the implementation of an in-situ machine learning system with the computing model promoted by Qarnot computing. Qarnot introduced an utility computing model in which servers are distributed in homes and offices where they serve as heaters. The Qarnot servers also embed several sensors for temperature, humidity, CO 2 etc. Qarnot offers an adequate platform to develop in-situ workflows for smart-homes problems. To demonstrate this point, we consider a typical problem: the detection of alarm sounds. Our paper introduces a new orchestration system for in-situ workflows, in the Qarnot platform. We also consider a general parallel framework for training alarm sound classifiers and decline an implementation that makes use of our orchestrator. Finally, we evaluate the implemented framework on different aspects including: the accuracy (of the resulting classifiers) and the runtime gain of the parallelization.

The rapid growth in demand for Information Technologies (IT) resources has led to the creation of large-scale data centers. These latter consume an enormous amount of electrical energy, resulting in high operating costs and carbon dioxide emissions. Moreover, modern Cloud computing environments should provide a high quality of service (QoS) for their customers. Thus, energy consumption has become a major concern for the widespread deployment of Cloud data centers. The growing importance for parallel applications in the Cloud introduces significant challenges in reducing energy consumption from hosted servers. With Cloud, customers can transparently access to virtually unlimited resources. The volunteer computing paradigm has become increasingly important, where the saved resources on each personal machine are shared thanks to the will of their customers. Cloud and volunteer paradigms have recently been seen as complementary technologies to better exploit the use of local resources. This paper presents the most well-known research in the literature on the minimization of energy consumption in traditional Cloud computing as well as in the volunteer one. A novel approach, based on searching shortest path in a graph, is also introduced.

In order to run Computational Fluid Dynamics (CFD) codes on large scale infrastructures, parallel computing has to be used because of the computational intensive nature of the problems. In this paper we investigate the 3D version of the ADAPT platform where we do a coupling between flow partial differential equations and a Poisson equation. This coupling leads to a linear system that we solve using direct methods. The implementation is conducted with the MUMPS parallel multi-frontal direct solver and with METIS for mesh partitioning methods to improve the overall performance of the framework. We also investigate, specifically in this paper, how the mesh partitioning methods are able to optimize the mesh cell distribution of the ADAPT solver.

The global objective of this study is to propose solutions for improving the quality of online search services. We consider the special case where the processing of requests submitted to such services consists of: the querying of several types of sub-services followed by the composition of the output produced the sub-services. This could be the case for instance of a booking service that proposes travel packs including: hotel reservation, car rental and flight booking. We propose to improve the quality of such services with the multi-selection problem. The goal in this problem is to select the subset of sub-services of each type to use in the the processing of search request. The selection must ensure that we maximize the quality of the results we could expect from the search. The multi-selection problem is close to the service selection problem. However, while in the latter, we are interested in a unique sub-service per type, in the former, we want to choose a subset of sub-services. Our paper introduces a theoretical formulation of the problem and demonstrates its NP-hardness. We also propose two approaches for the resolution. The first approach is based on Integer Linear Programming. The second approach combines parallel algorithm portfolio and sampling techniques. Finally, we make a comparative evaluation of the approaches based on practical scenarios.

International audience; Traditional automatic tuning systems are based on an exploration-exploitation tradeoff that consists of: learning the behavior of the algorithm to tune on several benchmarks (exploration) and then using the learned behavior for solving new problem instances. On NP-hard algorithms, this vision is questionable because of the potential huge runtime of the exploration phase. In this paper, we introduce QTuning, a new automatic tuning system specially designed for NP-hard algorithms. Like traditional tuning systems, QTuning uses benchmarks. But, during the learning process, new benchmark entries can always be introduced or existing ones removed. Moreover, the system mixes the exploration and exploitation phases. The main contribution of this paper is to formulate the learning process in QTuning within an active learning framework. The framework is based on a classical observation made in optimization: namely, the efficiency of random search in regret minimization. We improve our random search algorithm in including a machine learning classification approach and a set intersection problem. Finally, we discuss the experimental evaluation of the framework for the resolution of the satisfiability problem.

The aim of this paper is to show how to parallelize a compute intensive application in mathematics (Group Theory) for an institutional Desktop Grid platform coordinated by a meta-grid middleware named BonjourGrid. The paper is twofold: first of all, it shows how to parallelize a sequential program for a multicore CPU which participates in the computation and second it demonstrates the effort for launching multiple instances of the solutions for the mathematical problem with the BonjourGrid middleware. BonjourGrid is a fully decentralized Desktop Grid middleware. The main results of the paper are: a) an efficient multi-threaded version of a sequential program to compute Littlewood- Richardson coefficients, namely the Multi-LR program and b) a proof of concept, centered around the user needs, for the BonjourGrid middleware dedicated to coordinate multiple instances of programsfor Desktop Grids and with the help of Multi-LR. In this paper, the scientific work consists in starting from a model for the solution of a compute intensive problem in mathematics, to incorporate the concrete model into a middleware and running it on commodity PCs platform managed by an innovative meta Desktop Grid middleware.

In order to run Computational Fluid Dynamics (CFD) codes on large scale infrastructures, parallel computing has to be used because of the computational intensive nature of the problems. In this paper we investigate the ADAPT platform where we couple flow Partial Differential Equations and a Poisson equation. This leads to a linear system which we solve using direct methods. The implementation deals with the MUMPS parallel multi-frontal direct solver and mesh partitioning methods using METIS to improve the performance of the framework. We also investigate, in this paper, how the mesh partitioning methods are able to optimize the mesh cell distribution for the ADAPT solver. The experience gained in this paper facilitates the move to the 3D version of ADAPT and the move to a Service Oriented view of ADAPT as future work.

The present paper relates the involvement towards migration and adoption of cloud computing at Universite Sorbonne Paris Cite (USPC), a major french consortium of universities and higher education and research institutes. Migration to the cloud for a wide and diverse community of actors is nevertheless not straightforward. The ambitious interdisciplinary program ‘Imageries du Vivant’ (IDV or ‘Life Imaging’) at USPC, a key program dedicated to the development and use of life and biomedical imaging, constitutes our use case. It allows to sketch how cloud computing may change scientific practices and the landscape of computing, and to specify the steps to be followed for such purposes. The outcome of the paper is a methodology for accompanying adequate technological choices and acceptance by the users of the cultural changes when they migrate to cloud technologies. In short, we provide useful guidance for cloud adoption based on observations made for the IDV project.

In this paper we deal with the cloud brokering problem in the context of a multi-cloud infrastructure. The problem is by nature a multi-criterion optimization problem. The focus is put mainly (but not only) on the security/trust criterion which is rarely considered in the litterature. We use the well known Promethee method to solve the problem which is original in the context of cloud brokering. In other words, if we give a high priority to the secure deployment of a service, are we still able to satisfy all of the others required QoS constraints? Reciprocally, if we give a high priority to the RTT (Round-Trip Time) constraint to access the Cloud, are we still able to ensure a weak/medium/strong 'security level'? We decided to stay at a high level of abstraction for the problem formulation and to conduct experiments using 'real' data. We believe that the design of the solution and the simulation tool we introduce in the paper are practical, thanks to the Promethee approach that has been used for more than 25 years but never, to our knowledge, for solving Cloud optimization problems. We expect that this study will be a first step to better understand, in the future, potential constraints in terms of control over external cloud services in order to implement them in a simple manner. The contributions of the paper are the modeling of an optimization problem with security constraints, the problem solving with the Promethee method and an experimental study aiming to play with multiple constraints to measure the impact of each constraint on the solution. During this process, we also provide a sensitive analysis of the Consensus Assessments Initiative Questionnaire by the Cloud Security Alliance (CSA). The analysis deals with the variety, balance and disparity of the questionnaire answers.

We discuss parallel sorting algorithms and their implementations suitable for cluster architectures in order to optimize cluster resources. We focus on the time spent in computation and the load balancing properties when processors are running at different speeds, i.e. correlated by a multiplicative constant factor (our weak definition of heterogeneous platform). One scheme is under study: parallel sorting by sampling (either regular sampling technique introduced by Shi and Schaeffer [J. Parallel Distrib. Comput. 14 (4) (1992) 361] or the over-partitioning scheme introduced by Li and Sevcik [Parallel sorting by over-partitioning, in: Proceedings of the Sixth Annual Symposium on Parallel Algorithms and Architectures, ACM Press, New York, June 1994]). What is important in the paper is mainly the load balance factor and not necessary the execution time. It is clear that improved load balance leads to improved execution time. The results presented in the paper demonstrate that load balancing for the case of computers with heterogeneous processing capacity is more challenging than for the homogeneous case. The survey, through the sorting case study, allow us to identify some algorithmic issues and software challenges to master heterogeneous cluster platforms in order to better utilize them: data decomposition techniques, scheduling and load balancing methods.