Search

In recent years, deep neural networks (DNNs) have witnessed a booming of artificial intelligence Internet of Things applications with stringent demands across high accuracy and low latency. A widely adopted solution is to process such computation-intensive DNNs inference tasks with edge computing. Nevertheless, existing edge-based DNN processing methods still cannot achieve acceptable performance due to the intensive transmission data and unnecessary computation. To address the above limitations, we take the advantage of Multi-exit DNNs (ME-DNNs) that allows the tasks to exit early at different depths of the DNN during inference, based on the input complexity. However, naively deploying ME-DNNs in edge still fails to deliver fast and consistent inference in the wild environment. Specifically, 1) at the model-level, unsuitable exit settings will increase additional computational overhead and will lead to excessive queuing delay; 2) at the computation-level, it is hard to sustain high performance consistently in the dynamic edge computing environment. In this paper, we present a Low Latency Edge Intelligence Scheme based on Multi-Exit DNNs (LEIME) to tackle the aforementioned problem. At the model-level, we propose an exit setting algorithm to automatically build optimal ME-DNNs with lower time complexity; At the computation-level, we present a distributed offloading mechanism to fine-tune the task dispatching at runtime to sustain high performance in the dynamic environment, which has the property of close-to-optimal performance guarantee. Finally, we implement a prototype system and extensively evaluate it through testbed and large-scale simulation experiments. Experimental results demonstrate that LEIME significantly improves applications' performance, achieving 1.1–18.7 × speedup in different situations.

Because of the low-latency and high-throughput benefits of RDMA, an increasing number of collaborative applications in datacenters are re-designed with RDMA to boost the performance. Among various low-level hardware primitives provided by RDMA, exposed as parameters of APIs, the application designers select and hardcode them to exploit all the performance benefits of RDMA. However, with the dynamic nature of datacenter application, the hardcoded and fixed parameter selection fails to take full advantages of RDMA capabilities, which can cause up to 35% throughput performance loss. To address this issue, we present a tunable RDMA parameter selection framework, which allows parameter tuning at runtime, adaptive to the dynamic application and server status. To attain the native RDMA performance, we use a lightweight decision tree to reduce the overhead of RDMA parameter selection. Finally, we implement the tunable RDMA parameter selection framework with native RDMA API to provide a more abstract API. To demonstrate the effectiveness of our method, we implement a key-value service based on the abstract API. Experiment results show that our implementation has only a very small overhead compared with the native RDMA, while the optimized key-value service achieves 112% more throughput than Pilaf and 66% more throughput than FaRM.

Remote Direct Memory Access (RDMA) suffers from unfairness issues and performance degradation when multiple applications share RDMA network resources. Hence, an efficient resource scheduling mechanism is urged to optimally allocates RDMA resources among applications. However, traditional Network Utility Maximization (NUM) based solutions are inadequate for RDMA due to three challenges: 1) The standard NUM-oriented algorithm cannot deal with coupling variables introduced by multiple dependent RDMA operations; 2) The stringent constraint of RDMA on-board resources complicates the standard NUM by bringing extra optimization dimensions; 3) Naively applying traditional algorithms for NUM suffers from scalability and convergence issues in solving a large-scale RDMA resource scheduling problem.

In the cloud environment, the virtualized network provides the connectivity to a massive of virtual machines through various virtual network devices. In such a complicated networking system, network faults are not occasional. It is urging for the system administrators to have the ability to investigate a fault and recover from it. However, the complexity of the virtualized network and the similarity among the symptom of faults makes the accurate diagnosis challenging. In this paper, we leverage the method of reinforcement learning to facilitate the fault diagnosis in the cloud environment, where it diagnoses the faults through an “exploration and exploitation” manner. Further, we investigate the key factors that influence the network performance and may cause the network faults. Based on this investigation, we present how to train the network diagnosis module with the Q-learning algorithm. Experimental results show that the diagnosis accuracy of our reinforcement learning based method is around 8% higher than traditional methods, and incurs very slight system overhead.

Modern multi-queue data centers often use the standard Explicit Congestion Notification (ECN) scheme to achieve high network performance. However, one substantial drawback of this approach is that micro-burst traffic can cause the instantaneous queue length to exceed the ECN’s threshold, resulting in numerous mismarkings. After enduring too many mismarkings, senders may overreact, leading to severe throughput loss. As a solution to this dilemma, we propose our own adaptationthe Micro-burst ECN (MBECN) scheme-to mitigate mismarking. MBECN finds a more appropriate threshold baseline for each queue to absorb micro-bursts, based on steady-state analysis and an ideal generalized processor sharing (GPS) model. By adopting a queue-occupation-based dynamically adjusting algorithm, MBECN effectively handles packet backlog without hurting latency. Through testbed experiments, we find that MBECN improves throughput by ~20% and reduces flow completion time (FCT) by ~40%. Using large scale simulations, we find that throughput can be improved by 1.5~2.4× with DCTCP and 1.26~1.35× with ECN*. We also measure network delay and find that latency only increases by 7.36%.

Packet loss is a common problem in data center networks. The factors causing packet loss are various. Among them, micro-burst is the most important reason. Some previous works have studied the causes and influence o f micro-burst in single queue data center. However, through simulations and experiments, we find that micro-burst could bring m ore serious performance degradation in multi-queue data centers. The micro-burst traffic could cause E CN marking ratio rising from 4% to 22%, and cause throughput loss by up to 40%. Through observing queue length, we find that the standard E CN, which adopts immutable threshold, is not suitable for micro-burst traffic because micro-burst could trigger spurious congestion signals frequently, especially in DCTCP. In this paper, we not only show how much influence the micro-burst brings, but also propose Queue-length Aware ECN (QA-ECN) scheme to mitigate micro-burst. Finally, the simulations and experiments show that QAECN could reduce ECN marking ratio to 2.5%. In addition, the throughput and flow completion time could be improved by up to 22.9% and 34.1%, respectively.

With the fast development of cloud technologies, cloud-based operating systems are becoming increasingly popular, while several enterprise products, such as Chrome OS and SUNDE, are released continuously. According to the way of usage, the existing cloud-based operating systems can be divided into two categories: browser-driven operating system and virtualization-based operating system. However, traditional cloud-based operating systems have some disadvantages: As the functionality of browser is limited, it is difficult to transplant traditional desktop software to browser based cloud operating system; on the other hand, as for systems which are based on remote virtual machines (VMs) and Virtual Network Console (VNC), the system performance is constrained by the limited network bandwidth, which will also lead to performance degradation. To solve these problems, this paper designs AntOS, a full-featured cloud operating system. AntOS takes advantage of Docker and X11 technique, to reduce the load of network over 50% while increasing CPU utilization by 45%, which can effectively support the private cloud scenario in enterprises.

Diagnosing faults in virtual networks is always a popular research area. Existing researches primarily focus on diagnosing faults in physical networks, while they could not identify the faults introduced by virtual networks. Besides, the high complexity of algorithms and the requirement for modifying hardware may limit their scope of use. To address these drawbacks, in this paper, we propose a novel approach to diagnose faults in virtual networks. The rational of our approach is that the faults can be identified when located in the packet traces, with the knowledge that the possible known faults that can happen in that location. To achieve this goal, we apply packet marking, fault injection and machine learning techniques to provide precise fault diagnosis. Experimental results show that our approach can efficiently identify 73% of the faults while for virtual network-specific faults, our approach can diagnose 86% of them. Our system can also support real-time or near real-time fault analysis.

Cloud computing offers infrastructure as a service to deliver large amount of computation and storage resources, in which fast provisioning of virtual machine(VM) instances has significant impacts on the overall system performance and elasticity. In this paper, we analyze the characteristics of image provisioning by studying the traces collected from the real-world cloud data centre. From the analysis results, we observe that the overloaded and dynamic requests for some popular images result in degradation and fluctuation of performance and availability of the system. Addressing this issue, we propose a stochastic model based on queueing theory, which captures the main factors in image provisioning to optimize the number and placement of image replication, so as to manage the VM images in a cost-effective manner. We implement our theoretical model based on open-source cloud platform and carry out trace driven evaluation to validate its effectiveness. The evaluation results show that our system is cost-effective and can achieve high and stable performance in VM provisioning while remaining high availability under different test scenarios.

Large data centers are usually built to support the enormous computation and storage capability of Cloud Computing which has attracted people's attention nowadays. However, such large scale data centers generally consume an enormous amount of energy, which not only increases the running cost but also simultaneously enhances their greenhouse gas emissions. Addressing this issue, Virtualization technology is introduced, through which multiple Virtual Machines(VMs) can be centralized to fewer servers while allowing the idle servers to be dynamically powered off in order to save the energy consumption. In the paper, we investigate the impact of Virtualization technology on the energy and performance in data center environment taking into consideration various factors such as server failures and the overhead introduced by the VM contention. Noticing that there exits a tradeoff between energy consumption and execution time, we propose a stochastic model of data centers using Queueing theory to optimize performance and energy consumption. From the data center operators' prospective, they are willing to do better business by saving the energy consumption while abiding by the SLAs. Therefore, we try to find an optimal Energy-Performance tradeoff policy for the data center operators to operate data centers. The simulation results show that our model can significantly reduce the energy consumption by up to 35.4 while sacrificing a little execution time.