Search

Cloud data centers, such as Amazon EC2, host myriad big data applications using Virtual Machines (VMs). As these applications are communication-intensive, optimizing network transfer between VMs is critical to the performance of these applications and network utilization of data centers. Previous studies have addressed this issue by scheduling network flows with coflow semantics or optimizing VM placement with traffic considerations. However, coflow scheduling and VM placement have been conducted orthogonally. In fact, these two mechanisms are mutually dependent, and optimizing these two complementary degrees of freedom independently turns out to be suboptimal. In this paper, we present VirtCO, a practical framework that jointly schedules coflows and places VMs ahead of VM launch to optimize the overall performance of data center applications. We model the joint coflow scheduling and VM placement optimization problem, and propose effective heuristics for solving it. We further implement VirtCO with OpenStack and deploy it in a testbed environment. Extensive evaluation of real-world traces shows that compared with state-of-the-art solutions, VirtCO greatly reduces the average coflow completion time by up to 36.5%. This new framework is also compatible with and readily deployable within existing data center architectures.

In the cloud environment, the virtualized network provides the connectivity to a massive of virtual machines through various virtual network devices. In such a complicated networking system, network faults are not occasional. It is urging for the system administrators to have the ability to investigate a fault and recover from it. However, the complexity of the virtualized network and the similarity among the symptom of faults makes the accurate diagnosis challenging. In this paper, we leverage the method of reinforcement learning to facilitate the fault diagnosis in the cloud environment, where it diagnoses the faults through an “exploration and exploitation” manner. Further, we investigate the key factors that influence the network performance and may cause the network faults. Based on this investigation, we present how to train the network diagnosis module with the Q-learning algorithm. Experimental results show that the diagnosis accuracy of our reinforcement learning based method is around 8% higher than traditional methods, and incurs very slight system overhead.

Large data centers are usually built to support increasing computational and data storage demand of growing global business and industry, which consume an enormous amount of energy, at a huge cost to both business and the environment. However, much of that energy is wasted to maintain excess service capacity during periods of low load. In this paper, we investigate the problem of "right-sizing" data center for energy-efficiency through virtualization which allows consolidation of workloads into smaller number of servers while dynamically powering off the idle ones. In view of the dynamic nature of data centers, we propose a stochastic model based on Queueing theory to capture the main characteristics. Solving this model, we notice that there exists a tradeoff between the energy consumption and performance. We hereby develop a BFGS based algorithm to optimize the tradeoff by searching for the optimal system parameter values for the data center operators to "right-size" the data centers. We implement our Stochastic Right-sizing Model (SRM) and deploy it in the real-world cloud data center. Experiments with two real-world workload traces show that SRM can significantly reduce the energy consumption while maintaining high performance. A Stochastic Right-sizing Model(SRM) based on Queueing theory is proposed.A BFGS based algorithm is proposed to achieve energy-efficiency.SRM is implemented with open-source cloud platform OpenStack.

With the fast development of cloud technologies, cloud-based operating systems are becoming increasingly popular, while several enterprise products, such as Chrome OS and SUNDE, are released continuously. According to the way of usage, the existing cloud-based operating systems can be divided into two categories: browser-driven operating system and virtualization-based operating system. However, traditional cloud-based operating systems have some disadvantages: As the functionality of browser is limited, it is difficult to transplant traditional desktop software to browser based cloud operating system; on the other hand, as for systems which are based on remote virtual machines (VMs) and Virtual Network Console (VNC), the system performance is constrained by the limited network bandwidth, which will also lead to performance degradation. To solve these problems, this paper designs AntOS, a full-featured cloud operating system. AntOS takes advantage of Docker and X11 technique, to reduce the load of network over 50% while increasing CPU utilization by 45%, which can effectively support the private cloud scenario in enterprises.

© 2008-2012 IEEE. Bandwidth allocation to virtual machines (VMs) has a significant impact on the performance of communication-intensive big data applications hosted in VMs. It is crucial to accurately determine how much bandwidth to be reserved for VMs and when to adjust it. Past approaches typically resort to predicting the long-term network demands of applications for bandwidth allocation. However, lacking of prediction accuracy, these methods lead to the unpredictable application performance. Recently, it is conceded that the network demands of applications can only be accurately derived right before each of their execution phases. Hence, it is challenging to timely allocate the bandwidth to VMs with limited information. In this paper, we design and implement AppBag, an Application-aware Bandwidth guarantee framework, which allocates the accurate bandwidth to VMs with one-step-ahead traffic information. We propose an algorithm to allocate the bandwidth to VMs and map them onto feasible hosts. To reduce the overhead when adjusting the allocation, an efficient Lazy Migration (LM) algorithm is proposed with bounded performance. We conduct extensive evaluations using real-world applications, showing that AppBag can handle the bandwidth requests at run-time, while reducing the execution time of applications by 47.3 percent and the global traffic by 36.7 percent, compared to the state-of-the-art methods.

Summary With the rapid development of information technology, enormous volumes of data are being generated by enterprises at all times. The management and storage of these large-scale data have always been challenging enterprises. As these data are usually shared among users in a collaborative manner, secure data access and access performance are 2 key concerns for data storage of enterprises. However, current solutions fail to meet the requirements of enterprises since they suffer from the following drawbacks: (1) they do not support fine-grained access control and cannot meet the strict secure data access requirements of enterprises, and (2) they suffer from the unpredictable access latency. Thus in this paper, we propose Frostor, an enterprise-oriented cloud storage system, which addresses the secure data access issue through a user account and IP-based fine-grained access control mechanism, and guarantees the access performance via a two-level performance optimization mechanism. We further implement Frostor and deploy it on the testbed environment in a real data center. Extensive evaluations have shown that Frostor implements fine-grained access control, while achieving a significant reduction (≥60%) on access latency.

Cloud computing offers infrastructure as a service to deliver large amount of computation and storage resources, in which fast provisioning of virtual machine(VM) instances has significant impacts on the overall system performance and elasticity. In this paper, we analyze the characteristics of image provisioning by studying the traces collected from the real-world cloud data centre. From the analysis results, we observe that the overloaded and dynamic requests for some popular images result in degradation and fluctuation of performance and availability of the system. Addressing this issue, we propose a stochastic model based on queueing theory, which captures the main factors in image provisioning to optimize the number and placement of image replication, so as to manage the VM images in a cost-effective manner. We implement our theoretical model based on open-source cloud platform and carry out trace driven evaluation to validate its effectiveness. The evaluation results show that our system is cost-effective and can achieve high and stable performance in VM provisioning while remaining high availability under different test scenarios.

Large data centers are usually built to support the enormous computation and storage capability of Cloud Computing which has attracted people's attention nowadays. However, such large scale data centers generally consume an enormous amount of energy, which not only increases the running cost but also simultaneously enhances their greenhouse gas emissions. Addressing this issue, Virtualization technology is introduced, through which multiple Virtual Machines(VMs) can be centralized to fewer servers while allowing the idle servers to be dynamically powered off in order to save the energy consumption. In the paper, we investigate the impact of Virtualization technology on the energy and performance in data center environment taking into consideration various factors such as server failures and the overhead introduced by the VM contention. Noticing that there exits a tradeoff between energy consumption and execution time, we propose a stochastic model of data centers using Queueing theory to optimize performance and energy consumption. From the data center operators' prospective, they are willing to do better business by saving the energy consumption while abiding by the SLAs. Therefore, we try to find an optimal Energy-Performance tradeoff policy for the data center operators to operate data centers. The simulation results show that our model can significantly reduce the energy consumption by up to 35.4 while sacrificing a little execution time.