Search

In recent years, deep neural networks (DNNs) have witnessed a booming of artificial intelligence Internet of Things applications with stringent demands across high accuracy and low latency. A widely adopted solution is to process such computation-intensive DNNs inference tasks with edge computing. Nevertheless, existing edge-based DNN processing methods still cannot achieve acceptable performance due to the intensive transmission data and unnecessary computation. To address the above limitations, we take the advantage of Multi-exit DNNs (ME-DNNs) that allows the tasks to exit early at different depths of the DNN during inference, based on the input complexity. However, naively deploying ME-DNNs in edge still fails to deliver fast and consistent inference in the wild environment. Specifically, 1) at the model-level, unsuitable exit settings will increase additional computational overhead and will lead to excessive queuing delay; 2) at the computation-level, it is hard to sustain high performance consistently in the dynamic edge computing environment. In this paper, we present a Low Latency Edge Intelligence Scheme based on Multi-Exit DNNs (LEIME) to tackle the aforementioned problem. At the model-level, we propose an exit setting algorithm to automatically build optimal ME-DNNs with lower time complexity; At the computation-level, we present a distributed offloading mechanism to fine-tune the task dispatching at runtime to sustain high performance in the dynamic environment, which has the property of close-to-optimal performance guarantee. Finally, we implement a prototype system and extensively evaluate it through testbed and large-scale simulation experiments. Experimental results demonstrate that LEIME significantly improves applications' performance, achieving 1.1–18.7 × speedup in different situations.

Cloud data centers, such as Amazon EC2, host myriad big data applications using Virtual Machines (VMs). As these applications are communication-intensive, optimizing network transfer between VMs is critical to the performance of these applications and network utilization of data centers. Previous studies have addressed this issue by scheduling network flows with coflow semantics or optimizing VM placement with traffic considerations. However, coflow scheduling and VM placement have been conducted orthogonally. In fact, these two mechanisms are mutually dependent, and optimizing these two complementary degrees of freedom independently turns out to be suboptimal. In this paper, we present VirtCO, a practical framework that jointly schedules coflows and places VMs ahead of VM launch to optimize the overall performance of data center applications. We model the joint coflow scheduling and VM placement optimization problem, and propose effective heuristics for solving it. We further implement VirtCO with OpenStack and deploy it in a testbed environment. Extensive evaluation of real-world traces shows that compared with state-of-the-art solutions, VirtCO greatly reduces the average coflow completion time by up to 36.5%. This new framework is also compatible with and readily deployable within existing data center architectures.

With the continuous enrichment of cloud services, an increasing number of applications are being deployed in data centers. These emerging applications are often communication-intensive and data-parallel, and their performance is closely related to the underlying network. With their distributed nature, the applications consist of tasks that involve a collection of parallel flows. Traditional techniques to optimize flow-level metrics are agnostic to task-level requirements, leading to poor application-level performance. In this paper, we address the heterogeneous task-level requirements of applications and propose task-aware flow scheduling. First, we model tasks' sensitivity to their completion time by utilities. Second, on the basis of Nash bargaining theory, we establish a flow scheduling model with heterogeneous utility characteristics, and analyze it using Lagrange multiplier method and KKT condition. Third, we propose two utility-aware bandwidth allocation algorithms with different practical constraints. Finally, we present Tasch, a system that enables tasks to maintain high utilities and guarantees the fairness of utilities. To demonstrate the feasibility of our system, we conduct comprehensive evaluations with real-world traffic trace. Communication stages complete up to 1.4× faster on average, task utilities increase up to 2.26×, and the fairness of tasks improves up to 8.66× using Tasch in comparison to per-flow mechanisms.

Remote Direct Memory Access (RDMA) suffers from unfairness issues and performance degradation when multiple applications share RDMA network resources. Hence, an efficient resource scheduling mechanism is urged to optimally allocates RDMA resources among applications. However, traditional Network Utility Maximization (NUM) based solutions are inadequate for RDMA due to three challenges: 1) The standard NUM-oriented algorithm cannot deal with coupling variables introduced by multiple dependent RDMA operations; 2) The stringent constraint of RDMA on-board resources complicates the standard NUM by bringing extra optimization dimensions; 3) Naively applying traditional algorithms for NUM suffers from scalability and convergence issues in solving a large-scale RDMA resource scheduling problem.

In the cloud environment, the virtualized network provides the connectivity to a massive of virtual machines through various virtual network devices. In such a complicated networking system, network faults are not occasional. It is urging for the system administrators to have the ability to investigate a fault and recover from it. However, the complexity of the virtualized network and the similarity among the symptom of faults makes the accurate diagnosis challenging. In this paper, we leverage the method of reinforcement learning to facilitate the fault diagnosis in the cloud environment, where it diagnoses the faults through an “exploration and exploitation” manner. Further, we investigate the key factors that influence the network performance and may cause the network faults. Based on this investigation, we present how to train the network diagnosis module with the Q-learning algorithm. Experimental results show that the diagnosis accuracy of our reinforcement learning based method is around 8% higher than traditional methods, and incurs very slight system overhead.

Large data centers are usually built to support increasing computational and data storage demand of growing global business and industry, which consume an enormous amount of energy, at a huge cost to both business and the environment. However, much of that energy is wasted to maintain excess service capacity during periods of low load. In this paper, we investigate the problem of "right-sizing" data center for energy-efficiency through virtualization which allows consolidation of workloads into smaller number of servers while dynamically powering off the idle ones. In view of the dynamic nature of data centers, we propose a stochastic model based on Queueing theory to capture the main characteristics. Solving this model, we notice that there exists a tradeoff between the energy consumption and performance. We hereby develop a BFGS based algorithm to optimize the tradeoff by searching for the optimal system parameter values for the data center operators to "right-size" the data centers. We implement our Stochastic Right-sizing Model (SRM) and deploy it in the real-world cloud data center. Experiments with two real-world workload traces show that SRM can significantly reduce the energy consumption while maintaining high performance. A Stochastic Right-sizing Model(SRM) based on Queueing theory is proposed.A BFGS based algorithm is proposed to achieve energy-efficiency.SRM is implemented with open-source cloud platform OpenStack.

Diagnosing faults in virtual networks is always a popular research area. Existing researches primarily focus on diagnosing faults in physical networks, while they could not identify the faults introduced by virtual networks. Besides, the high complexity of algorithms and the requirement for modifying hardware may limit their scope of use. To address these drawbacks, in this paper, we propose a novel approach to diagnose faults in virtual networks. The rational of our approach is that the faults can be identified when located in the packet traces, with the knowledge that the possible known faults that can happen in that location. To achieve this goal, we apply packet marking, fault injection and machine learning techniques to provide precise fault diagnosis. Experimental results show that our approach can efficiently identify 73% of the faults while for virtual network-specific faults, our approach can diagnose 86% of them. Our system can also support real-time or near real-time fault analysis.

© 2008-2012 IEEE. Bandwidth allocation to virtual machines (VMs) has a significant impact on the performance of communication-intensive big data applications hosted in VMs. It is crucial to accurately determine how much bandwidth to be reserved for VMs and when to adjust it. Past approaches typically resort to predicting the long-term network demands of applications for bandwidth allocation. However, lacking of prediction accuracy, these methods lead to the unpredictable application performance. Recently, it is conceded that the network demands of applications can only be accurately derived right before each of their execution phases. Hence, it is challenging to timely allocate the bandwidth to VMs with limited information. In this paper, we design and implement AppBag, an Application-aware Bandwidth guarantee framework, which allocates the accurate bandwidth to VMs with one-step-ahead traffic information. We propose an algorithm to allocate the bandwidth to VMs and map them onto feasible hosts. To reduce the overhead when adjusting the allocation, an efficient Lazy Migration (LM) algorithm is proposed with bounded performance. We conduct extensive evaluations using real-world applications, showing that AppBag can handle the bandwidth requests at run-time, while reducing the execution time of applications by 47.3 percent and the global traffic by 36.7 percent, compared to the state-of-the-art methods.

Cloud computing offers infrastructure as a service to deliver large amount of computation and storage resources, in which fast provisioning of virtual machine(VM) instances has significant impacts on the overall system performance and elasticity. In this paper, we analyze the characteristics of image provisioning by studying the traces collected from the real-world cloud data centre. From the analysis results, we observe that the overloaded and dynamic requests for some popular images result in degradation and fluctuation of performance and availability of the system. Addressing this issue, we propose a stochastic model based on queueing theory, which captures the main factors in image provisioning to optimize the number and placement of image replication, so as to manage the VM images in a cost-effective manner. We implement our theoretical model based on open-source cloud platform and carry out trace driven evaluation to validate its effectiveness. The evaluation results show that our system is cost-effective and can achieve high and stable performance in VM provisioning while remaining high availability under different test scenarios.

Large data centers are usually built to support the enormous computation and storage capability of Cloud Computing which has attracted people's attention nowadays. However, such large scale data centers generally consume an enormous amount of energy, which not only increases the running cost but also simultaneously enhances their greenhouse gas emissions. Addressing this issue, Virtualization technology is introduced, through which multiple Virtual Machines(VMs) can be centralized to fewer servers while allowing the idle servers to be dynamically powered off in order to save the energy consumption. In the paper, we investigate the impact of Virtualization technology on the energy and performance in data center environment taking into consideration various factors such as server failures and the overhead introduced by the VM contention. Noticing that there exits a tradeoff between energy consumption and execution time, we propose a stochastic model of data centers using Queueing theory to optimize performance and energy consumption. From the data center operators' prospective, they are willing to do better business by saving the energy consumption while abiding by the SLAs. Therefore, we try to find an optimal Energy-Performance tradeoff policy for the data center operators to operate data centers. The simulation results show that our model can significantly reduce the energy consumption by up to 35.4 while sacrificing a little execution time.