Search

User-facing services are now evolving towards the microservice architecture where a service is built by connecting multiple microservice stages. Since the entire service is heavy, the microservice architecture shows the opportunity to only offload some microservice stages to the edge devices that are close to the end users. However, emerging techniques often result in the violation of Quality-of-Service (QoS) of microservice-based services in cloud-edge continuum, as they do not consider the communication overhead or the resource contention between microservices and external co-located tasks. We propose Nautilus, a runtime system that effectively deploys microservice-based user-facing services in cloud-edge continuum. Nautilus ensures the QoS of microservice-based user-facing services while minimizing the required computational resources, which is comprised of a communication-aware microservice mapper, a contention-aware resource manager and an IO-sensitive and load-aware microservice migration scheduler. The mapper divides the microservice graph into multiple partitions based on the communication overhead and maps the partitions to appropriate nodes. On each node, the resource manager determines the optimal resource allocation for its microservices based on reinforcement learning that may capture the complex contention behaviors. Once the microservices are suffered from external IO pressure, the IO-sensitive microservice scheduler migrates the critical one to idle nodes. Furthermore, when the load of microservices changes dynamically, the load-aware microservice scheduler migrates microservices from busy nodes to idle ones to ensure the QoS goal of the entire service. Our experimental results show that Nautilus can guarantee the required QoS target under external shared resources contention while the state-of-the-art suffers from QoS violations. Meanwhile, Nautilus reduces the computational resource usage by 23.9% and the network bandwidth usage by 53.4%, while achieving the required 99%-ile latency.

Battery service time is a critical constraint on the availability and functionality of mobile devices. To address the challenge, we propose CAPMAN, a management framework that jointly optimizes the cooling and active power management in a smartphone, a typical mobile device, equipped with a hybrid battery pack. We establish the framework with three major designs. First, we abstract the correlation among the batteries, devices and software into a finite state machine model, whose state transitions can be triggered by actions like system calls and user activities. Second, we propose a battery scheduling algorithm that determines the more suitable battery for cooling/active power use, with respect to the dynamic software behaviors and their impact on the hardware states, based on Markov decision process (MDP). Third, we design a facility for joint cooling and active power management by coordinating TECs and batteries. With the three major designs, CAPMAN realizes software defined management that schedules heterogeneous batteries and TEC cooling in a timely manner. We prototype CAPMAN on multiple popular smartphones and a PYNQ development board. CAPMAN can achieve 114% longer battery service time under skewed loads; compared to baselines, CAPMAN shows 55% performance gain and 53% less energy use on average

Data collection in big data is an effective way to aggregate information that the collector is interested in. However, there is no assurance for the data that the users provide. Since collector does not have the ability to check the authenticity of every piece of information, the trustworthiness of users participated in the collection become important. In this paper, we design an efficient approach to calculate users' trustworthiness in data collection for big data context. We divide the trustworthiness into familiarity trustworthiness and similarity trustworthiness, and study the influences of user actions on trustworthiness. To prevent malicious users from raising their trustworthiness and providing false information that may mislead final results, we also design a security queue to record users' historical trust information, so that we can detect malicious users with high accuracy. Simulation results show that our model can sensitively resist the malicious actions of users.

The parameter server (PS) paradigm has achieved great success in deploying large-scale distributed Deep Learning (DL) systems. However, these systems implicitly assume that the cluster is homogeneous and this assumption does not hold in many real-world cases. Although the previous efforts are paid to address heterogeneity, they mainly prioritize the contribution of fast workers and reduce the involvement of slow workers, resulting in the limitations of workload imbalance and computation inefficiency. We reveal that grouping workers into communities, an abstraction proposed by us, and handling parameter synchronization at the community level can conquer these limitations and accelerate the training convergence progress. The inspiration of community comes from our exploration of prior knowledge about the similarity between workers, which is often neglected by previous work. These observations motivate us to propose a new synchronization mechanism named Community-aware Synchronous Parallel (CASP), which uses the Asynchronous Advantage Actor-Critic (A3C)-based algorithm to intelligently determine community configuration and fully improve the synchronization performance. The whole idea has been implemented in a prototype system called ${\sf Petrel}$ Petrel that achieves a good balance between convergence efficiency and communication overhead. The evaluation under various benchmarks with multiple metrics and baseline comparison demonstrates the effectiveness of ${\sf Petrel}$ Petrel . Specifically, ${\sf Petrel}$ Petrel accelerates the training convergence speed by up to 1.87 × faster and reduces communication traffic by up to 26.85 percent, on average, over the non-community synchronization mechanisms.

With the rapid technological growth in the last decades, the number of devices and users has drastically increased. Software-defined networking (SDN) with machine learning (ML) has become an emergi...

The parameter server architecture has shown promising performance advantages when handling deep learning (DL) applications. One crucial issue in this regard is the presence of stragglers, which significantly retards DL training progress. Previous solutions for solving stragglers may not fully exploit the computation resource of the cluster as evidenced by our experiments, especially in the heterogeneous environment. This motivates us to design a heterogeneity-aware parameter server paradigm that addresses stragglers and accelerates DL training from the perspective of computation parallelism. We introduce a novel methodology named straggler projection to give a comprehensive inspection of stragglers and reveal practical guidelines to solve this problem in two aspects: (1) controlling each worker's training speed via elastic training parallelism control and (2) transferring blocked tasks from stragglers to pioneers to fully utilize the computation resource. Following these guidelines, we propose the abstraction of parallelism as an infrastructure and design the Elastic-Parallelism Synchronous Parallel (EPSP) algorithm to handle distributed training and parameter synchronization, supporting both enforced- and slack-synchronization schemes. The whole idea has been implemented into a prototype called ${\sf Falcon}$ Falcon which effectively accelerates the DL training speed with the presence of stragglers. Evaluation under various benchmarks with baseline comparison demonstrates the superiority of our system. Specifically, ${\sf Falcon}$ Falcon reduces the training convergence time, by up to 61.83, 55.19, 38.92, and 23.68 percent shorter than FlexRR, Sync-opt, ConSGD, and DynSGD, respectively.

In recent years, spatial-keyword queries have attracted much attention with the fast development of location-based services. However, current spatial-keyword techniques are disk-based, which cannot fulfill the requirements of high throughput and low response time. With the surging data size, people tend to process data in distributed in-memory environments to achieve low latency. In this paper, we present the distributed solution, i.e., Skia (Spatial-Keyword In-memory Analytics), to provide a scalable backend for spatial-textual analytics. Skia introduces a two-level index framework for big spatial-textual data including: (1) efficient and scalable global index, which prunes the candidate partitions a lot while achieving small space budget; and (2) four novel local indexes, that further support low latency services for exact and approximate spatial-keyword queries. Skia can support common spatial-keyword queries via traditional SQL programming interfaces. The experiments conducted on large-scale real datasets have demonstrated the promising performance of the proposed indexes and our distributed solution.

Online stores are capable analyzing user preference from click logs and transaction records, while retailers of physical stores still lack effective methods to understand user preference. Traditional ways are predominantly field surveys, which are not effective as they need labor-intensive survey thus limit to small populations. As mobile devices and social media are becoming more and more pervasive, user-generated heterogeneous information (e.g., check-in activities and textual reviews) from these platforms are providing rich information to in-depth understand user preference. In this paper, we present a recommendation model for physical stores by learning user's preference from user-generated heterogeneous information. Specifically, the proposed model consists of two phases: 1) offline modeling multi-relation among users, stores and aspects; 2) online graph-based recommendation. The offline modeling phase is designed to learn two kinds of relations: User-Store relation and Store-Aspect relation, while the online recommendation phase automatically produces top- $k$ k recommended stores based on the learnt relations with a graph-based model. To demonstrate the utility of our proposed model, we have performed a comprehensive experimental evaluation on two real-world datasets, which are collected by more than 120,000 users during 12 months. Experimental results show our method outperforms all baselines significantly in terms of recommendation effectiveness.

Specialized hardware accelerators for deep learning are widely introduced by many hardware vendors because of their high performance and efficiency. However, different vendors adopt different accelerator architectures, making it challenging for the compiler tool-chain to generate and optimize high-performance codes. Moreover, the current tool-chains provided by the vendors are either highly abstract, which makes it hard to optimize or contain too many hardware-related details, which makes it inconvenient to program. So, in this paper, we propose a middle layer compiler tool-chain for Cambricon MLU-100 to fill the gap between high-level runtime library and low operator-level SDK. Our tool-chain is based on the operator level SDK but abstracts away its redundant initialization and allocation statement. We also expose the interface of major optimization knobs compared to the existing runtime, thus enabling a considerable optimization space. We evaluate our work by several state-of-the-art neural networks and choose the line of code and optimization knobs as evaluation metrics. We also compare the performance against state-of-the-art tool-chain TensorRT applying simple optimization strategy and find that our work has great potential in optimization. Our work can guarantee the user a vast optimization space with only around $$ 20\% $$ amount of the codes that hides the redundant initialization and allocation statements from users.

While it is well understood that the Internet of things (IoT) offers the capability of integrating the physical world and the cyber world, it also presents many significant challenges with numerous heterogeneous things connected and interacted, such as how to efficiently annotate things with semantic labels (i.e., things categorization) for searching and recommendation. Traditional ways for things categorization are not effective due to several characteristics (e.g., thing's text profiles are usually short and noise, things are heterogeneous in terms of functionality and attributes) of IoT. In this paper, we develop a novel things categorization technique to automatically predict semantic labels for a given thing. Our proposed approach formulates things categorization as a multi-label classification problem and learns a binary support vector machine classifier for each label to support multi-label classification. We extract two types of features to train classification model: 1) explicit feature from thing's profiles and spatial-temporal pattern; 2) implicit feature from thing's latent relation strength. We utilize a latent variable model to uncover thing's latent relation strength from their interaction behaviours. We conduct a comprehensive experimental study based on three real datasets, and the results show fusing thing's latent relation strength can significantly boost things categorization.

Existing works adopt the Edge-Oriented Resource Allocation (EORA) scheme, in which edge nodes cache services and schedule user requests to distribute workloads over cloud and edge nodes, so as to achieve high-quality services and low latency. Unfortunately, EORA does not fully take into account the fact that service providers are sometimes independent from the edge operators with their own objectives. To deal with the conflict and cooperation between service providers and edge nodes, we devise a service-oriented resource allocation (SORA) scheme, where edge nodes and service providers adjust their resource allocations to provide requested services. We first prove that such resource allocation problem is NP-hard. We then propose a three-sided cyclic game (3CG) involving users, edge nodes, and service providers who make their individual decisions by choosing respectively high-quality services, high-value users, and cost-effective edge nodes for service deployment. Based on 3CG, we prove the existence and approximation ratio of pure-strategy Nash equilibriums (NEs). We also develop both centralized and distributed approximate algorithms for resource allocation. Finally, extensive experimental results validate the effectiveness and convergence of the proposed algorithms.

In the age of big data, companies tend to deploy their services in data centers rather than their own servers. The demands of big data analytics grow significantly, which leads to an extremely high electricity consumption at data centers. In this paper, we investigate the cost minimization problem of big data analytics on geo-distributed data centers connected to renewable energy sources with unpredictable capacity. To solve this problem, we propose a Reinforcement Learning (RL) based job scheduling algorithm by combining RL with neural network (NN). Moreover, two techniques are developed to enhance the performance of our proposal. Specifically, Random Pool Sampling (RPS) is proposed to retrain the NN via accumulated training data, and a novel Unidirectional Bridge Network (UBN) structure is designed for further enhancing the training speed by using the historical knowledge stored in the trained NN. Experiment results on real Google cluster traces and electricity price from Energy Information Administration show that our approach is able to reduce the data centers’ cost significantly compared with other benchmark algorithms.

Edge intelligence has already been widely regarded as a key enabling technology in a variety of domains. Along with the prosperity, increasing concern is raised on the security and privacy of intelligent applications. As these applications are usually deployed on shared and untrusted edge servers, malicious co-located attackers, or even untrustworthy infrastructure providers, may acquire highly security-sensitive data and code (i.e., the pre-trained model). Software Guard Extensions (SGX) provides an isolated Trust Execution Environment (TEE) for task security guarantee. However, we notice that DNN inference performance in SGX is severely affected by the limited enclave memory space due to the resultant frequent page swapping operations and the high enclave call overhead. To tackle this problem, we propose Lasagna, an SGX oriented DNN inference performance acceleration framework without compromising the task security. Lasagna consists of a local task scheduler and a global task balancer to optimize the system performance by exploring the layered-structure of DNN models. Our experiment results show that our layer-aware Lasagna effectively speeds up the well-known DNN inference in SGX by 1.31x-1.97x.

Federated learning (FL) is a distributed machine learning paradigm that allows clients to collaboratively train a model over their own local data. FL promises the privacy of clients and its security can be strengthened by cryptographic methods such as additively homomorphic encryption (HE). However, the efficiency of FL could seriously suffer from the statistical heterogeneity in both the data distribution discrepancy among clients and the global distribution skewness. We mathematically demonstrate the cause of performance degradation in FL and examine the performance of FL over various datasets. To tackle the statistical heterogeneity problem, we propose a pluggable system-level client selection method named Dubhe, which allows clients to proactively participate in training, meanwhile preserving their privacy with the assistance of HE. Experimental results show that Dubhe is comparable with the optimal greedy method on the classification accuracy, with negligible encryption and communication overhead., 10 pages

Nowadays, Solid State Drive (SSD) arrays have been widely used in commercial big data centers and high-performance storage services. Meanwhile, in the era of explosive data growth, data centers need to implement the array scaling schemes to meet the increasing storage capacity requirements. The existing state-of-the-art scaling methods, such as Round-Robin (RR) and FastScale, aim at ensuring a uniform data redistribution. However, most of them are designed for Hard Disk Drive (HDD) arrays, ignoring lifetime difference among extended and former-used disks, which leads to several additional penalties in SSD arrays. Furthermore, due to the sudden interdisk lifetime disparity, the extended SSD disks trigger frequently wear-leveling operations for controlling the wearing balance into the predefined threshold. These reactions result in inefficient scaling and I/O performance degradation. To address the above problem, we propose a Lazy W ear-L eveling (Lazy-WL) mechanism to reduce the conventional wear-leveling overhead during the scaling process. Its core idea is to reduce the unnecessary intensive wear-leveling migration significantly, via narrowing the difference of program/erase (P/E) cycles among new-added and former deployed disks smoothly and gradually. To demonstrate the effectiveness of this approach, we conduct several simulation via Disksim and real implementation via a Hadoop cluster. The evaluation results show that, compared to the typical inter and intra disk wear leveling methods, Lazy-WL could lower the triggered wear-leveling operations by up to 92.9% and achieve a maximal 85.2% response time reduction, which suggests that Lazy-WL performs a balanced I/O distribution, and maintains high performance of SSD array with high scaling efficiency.

Big data analytics in datacenters often involve scheduling of data-parallel jobs. Traditional scheduling techniques based on improving network resource utilization are subject to limited bandwidth in datacenter networks. To alleviate the shortage of bandwidth, some cluster frameworks employ techniques of traffic compression to reduce transmission consumption. However, they tackle scheduling in a coarse-grained manner at task level and do not perform well in terms of flow-level metrics due to high complexity. Fortunately, the abstraction of coflow pioneers a new perspective to facilitate scheduling efficiency. In this paper, we introduce a coflow compression mechanism to minimize the completion time in data-intensive applications. Due to the NP-hardness, we propose a heuristic algorithm called Fastest-Volume-Disposal-First (FVDF) to solve this problem. For online applicability, FVDF supports stage pipelining to accelerate scheduling and exploits recurrent neural networks (RNNs) to predict compression speed. Meanwhile, we build ${\sf Swallow}$ Swallow , an efficient scheduling system that implements our proposed algorithms. It minimizes coflow completion time (CCT) while guaranteeing resource conservation and starvation freedom. The results of both trace-driven simulations and real experiments show the superiority of our algorithm, over existing one. Specifically, ${\sf Swallow}$ Swallow speeds up CCT and job completion time (JCT) by up to $1.47 \times$ 1 . 47 × and $1.66 \times$ 1 . 66 × on average, respectively, over the SEBF in Varys, one of the most efficient coflow scheduling algorithms so far. Moreover, with coflow compression, ${\sf Swallow}$ Swallow reduces data traffic by up to 48.41 percent on average.

Emerging data analytic workloads such as graph processing, neural network and edge data preprocesing desire efficient memory read operations. Unfortunately, due to the necessity of dynamic refresh, modern DRAM systems have to stall access during refresh cycles. As DRAM device density continues to grow, refresh operations can be a crucial throughput bottleneck. To fully unleash memory access performance, we revisit conventional refresh mechanism and DRAM architecture. We propose DR refresh, a specific refresh mechanism that enable read and refresh operations to be done simultaneously. We devise DR DRAM, a specific memory hardware system that can efficiently deploy DR refresh. Unlike traditional refresh, DR explores device refresh that only refreshes a designated device at a time. Meanwhile, DR increases read efficiency by recovering the inaccessible data that resides on a device under refreshing. We also propose Hybrid Refresh Main Memory (HRMM) which can designate refresh schemes (DR or traditional refresh) in a specific memory space. We expect that our design can benefit many real-life tasks such as SPEC CPU2006, CNN, LLT and PageRank.

As an innovated and revolutionized technology, blockchain has been applied in many fields, such as cryptocurrency, food traceability, identity management, or even market prediction. To discover its great potential, both industry and academia have paid great attention to it and numerous researches have been conducted. Based on the literature and industry whitepapers, in this survey, we unroll and structure the blockchain related discoveries and scientific results in many aspects. Particularly, we classify blockchain technologies into four layers and carry out a comprehensive study on the consensus strategies, the network, and the applications of blockchain. Different blockchain applications are put into the corresponding categories based on the fields, especially in Internet of Things (IoT). When introducing each layer, we not only organize and summarize the related works, but also discuss the fundamental issues and future research directions. We hope this survey could shed some light on the research of blockchain and serve as a guide for further studies.

The video streaming technique and 4G LTE networks have been developing at a high speed, while the advancement of the battery technology is relatively slow. A lot of efforts have been made on understanding the power consumption in general 4G LTE networks, while little attention has been focused on reducing the power consumption of mobile devices for online video streaming in 4G LTE networks. In this paper, we attempt to develop efficient optimization techniques to address this problem. It is an important and also interesting problem, while there are many challenges needing to be alleviated (e.g., the uncertainty of users’ behavior modes). To attack the challenges, we suggest a self-adaptive method that can allow us to adjust various parameters dynamically and efficiently, achieving a relatively small energy consumption. We give the rigorous theoretical analysis for the proposed method, and conduct the empirical study to validate its effectiveness.

Energy consumption is one of the most significant aspects of large-scale storage systems where multilevel caches are widely used. In a typical hierarchical storage structure, upper-level storage serves as a cache for the lower level, forming a distributed multilevel cache system. In the past two decades, several classic LRU-based multilevel cache policies have been proposed to improve the overall I/O performance of storage systems. However, few power-aware multilevel cache policies focus on the storage devices in the bottom level, which consume more than 27% of the energy of the whole system [1]. To address this problem, we propose a novel power-aware multilevel cache (PAM) policy that can reduce the energy consumption of high-performance and I/O bandwidth storage devices. In our PAM policy, an appropriate number of cold dirty blocks in the upper level cache are identified and selected to flush directly to the storage devices, providing high probability extension of the lifetime of disks in standby mode. To demonstrate the effectiveness of our proposed policy, we conduct several simulations with real-world traces. Compared to existing popular cache schemes such as PA-LRU, PB-LRU, and Demote, PAM reduces power consumption by up to 15% under different I/O workloads, and improves energy efficiency by up to 50.5%.

The development of the cloud and data centers alike has caused users to increasingly share storage systems, and storage systems have adopted SSDs to improve performance. The number of users is growing sharply and causing resource competition, so the throughput allocation of a hybrid storage system has an important effect on users’ performance. A hybrid storage architect faces a challenge: sharing I/O throughput resources between SSDs and HDDs fairly with high resource utilization. Drawing on economic game-theory, many methods based on Leontief preference have been proposed to determine fair throughput allocation in a hybrid storage system with having the properties of Sharing Incentive (SI), Envy Freeness (EF) and Pareto Optimality (PO). However, users’ workload characteristics vary because of the adoption of SSDs, so there is heterogeneity among the storage media. The heterogeneity of storage media and the diversity of user workloads together create more constraints on fairness, which imposes restrictions to improve the resource utilization. To improve the resource utilization, some researchers have proposed relaxing the limitation of fairness properties by adjusting users’ allocations. However, when adjusting the allocation between users, some users’ allocations are reduced but others’ are improved, with the goal of increasing the resource utilization of the whole system, this will pose a problem because of the Leontief preference restriction: some users’ performance is degraded, while other users’ performance is improved. In this paper, we first propose one variant of the Cobb–Douglas preference based on users’ workloads and allocations, and then present a throughput allocation model based on this variant named TACD that can capture the effects of diminishing returns and substitutability in throughput allocation for hybrid storage. By making use of these effects, TACD can improve some users’ performance while providing the same performance for other users and achieving higher resource utilization in the whole system, and it can provide a fair allocation while enjoying SI, EF and PO properties. The variant of the Cobb–Douglas preference proposed in this paper has taken consideration of users’ workloads, so TACD can conduct the throughput allocation based on these workloads for meeting the performance requirements of users reasonably. Extensive experiments are conducted to prove the effectiveness of TACD. The simulation results show that TACD can fit the throughput allocation of hybrid storage system very well. The Linux results show that compared with the allocation method based on Leontief preference, TACD can provide a higher resource utilization and performance for users throughout the system, it can improve some users’ performance when maintaining comparable performance for other users, in contrast to the Leontief method based on weaker fairness under the same resource utilization, and its allocation result enjoys SI, EF and PO properties.

The emergence of edge computing has witnessed a fast-growing volume of data on edge devices belonging to different stakeholders which, however, cannot be shared among them due to the lack of the trust. By exploiting blockchain’s non-repudiation and non-tampering properties that enable trust, we develop a blockchain-based big data sharing framework to support various applications across resource-limited edges. In particular, we devise a number of novel resource-efficient techniques for the framework: (1) the PoC (Proof-of-Collaboration) based consensus mechanism with low computation complexity which is especially beneficial to the edge devices with low computation capacity, (2) the blockchain transaction filtering and offloading scheme that can significantly reduce the storage overhead, and (3) new types of blockchain transaction (i.e., Express Transaction) and block (i.e., Hollow Block) to enhance the communication efficiency. Extensive experiments are conducted and the results demonstrate the superior performance of our proposal.

Erasure codes (EC) have become a typical technology for distributed storage systems in place of data replication, providing similar data availability but lower storage cost. However, a great number of data computations and migrations during the EC recovery process bring high I/O and network latency penalties. Although several EC recovery methods have been designed to compromise the recovery penalty with high parallelism, the performance of these schemes was usually bounded by the straggler problems due to the various (I/O) performance among different nodes in the storage system. Moreover, the variation of the access popularity from the upper layer application causes the dynamic load fluctuation and asymmetry upon different nodes, which makes the scheduling more difficult during the recovery. To address the above problem, we propose a dynamic load-balanced scheduling algorithm for straggler recovery called EC-Scheduler. EC-Scheduler adjusts the recovery schedule dynamically with the awareness of continuous load fluctuation on the nodes, guaranteeing high parallelism and load balance ability simultaneously. To demonstrate the effectiveness of EC-Scheduler, we conduct several experiments in a cluster. The results show that, compared to typical recovery schemes such as Fast-PR and EC-Store, EC-Scheduler could achieve a 1.3X speed-up in the recovery process and 10X improvement in recovery load imbalance factor.

Acquiring the geographical distribution of neighbors can support more adaptive media access control (MAC) protocols and other safety applications in Vehicular ad hoc network (VANETs). However, it is very challenging for each vehicle to estimate its own neighbor distribution in a fully distributed setting. In this paper, we propose an online distributed neighbor distribution estimation scheme, called PeerProbe, in which vehicles collaborate with each other to probe their own neighborhood via simultaneous symbol-level wireless communication. An adaptive compressive sensing algorithm is developed to recover a neighbor distribution based on a small number of random probes with non-negligible noise. Moreover, the needed number of probes adapts to the sparseness of the distribution. We conduct extensive simulations and the results demonstrate that PeerProbe is lightweight and can accurately recover highly dynamic neighbor distributions in critical channel conditions.

While current deep learning frameworks are mainly optimized for dense-accessed models, they show low throughput and poor scalability in training sparse-accessed recommendation models. Our investigation shows that the poor performance is due to the parameter synchronization bottleneck. We therefore propose BiPS, a bi-tier parameter synchronization system that alleviates the parameter update and the sparse-accessed parameters communication bottleneck. BiPS includes a bi-tier parameter server that accelerates the traditional CPU-based parameter update process, a hotness-aware parameter placement and communication policy to balance the workloads between CPU and GPU and optimize the communication of sparse-accessed parameters. BiPS overlaps the worker computation with the synchronization stage to enable parameter updates in advance. We implement BiPS and incorporate it into mainstream DL frameworks including TensorFlow, MXNet, and PyTorch. The experimental results based on various deep learning frameworks show that BiPS greatly speeds up the training of recommenders (5 - 9$\times$) as the model scale increases, without degrading the accuracy.

In cloud storage systems, disk arrays are widely used because of their high reliability and low monetary cost. Due to the burst of I/O in sprinting computing scenarios (i.e. online retailer services on Black Friday or Cyber Monday), large scale cloud storage systems such as AWS S3 and GFS need to afford 10XI/O workloads. Therefore, rack level scaling for cloud disk arrays becomes urgent for sprinting services. Although several existing methods, such as Round-Robin(RR) and Scale-RS, are proposed to accelerate the scaling processes, the efficiencies of these approaches are limited. It is because that the cross-rack data migrations are ill-considered in their designs. To address the above problem, in this paper, we propose Rack-Scaling, a novel data redistribution method to accelerate rack level scaling process in cloud storage systems. The basic idea of Rack-Scaling is migrating appropriate data blocks within and among racks to achieve a uniform data distribution while minimizing the cross-rack migration, which costs more than intra-rack migration. We conduct simulations via Disksim and we also implement Rack-Scaling on Hadoop to demonstrate the effectiveness of Rack-Scaling. The results show that, compared to typical methods such as Round-Robin (RR), Semi-RR, Scale-RS and BDR, Rack-Scaling reduces the number of I/O operations and the data amount of cross-rack transmission by up to 90.4% and 99.9%, respectively, and speeds up the scaling by up to 8.77X.

User-facing services are now evolving towards the microservice architecture where a service is built by connecting multiple microservice stages. While an entire service is heavy, the microservice architecture shows the opportunity to only offload some microservice stages to the edge devices that are close to the end users. However, emerging techniques often result in the violation of Quality-of-Service (QoS) of microservice-based services in cloud-edge continuum, as they do not consider the communication overhead or the resource contention between microservices.We propose Nautilus, a runtime system that effectively deploys microservice-based user-facing services in cloud-edge continuum. It ensures the QoS of microservice-based user-facing services while minimizing the required computational resources. Nautilus is comprised of a communication-aware microservice mapper, a contention-aware resource manager and a load-aware microservice scheduler. The mapper divides the microservice graph into multiple partitions based on the communication overhead and maps the partitions to the nodes. On each node, the resource manager determines the optimal resource allocation for its microservices based on reinforcement learning that may capture the complex contention behaviors. The microservice scheduler monitors the QoS of the entire service, and migrates microservices from busy nodes to idle ones at runtime. Our experimental results show that Nautilus reduces the computational resource usage by 23.9% and the network bandwidth usage by 53.4%, while achieving the required 99%-ile latency.

The tenants of public cloud platforms share hard-ware resources on the same node, resulting in the potential for performance interference (or malicious attacks). A tenant is able to degrade the performance of its neighbors on the same node significantly through overuse of the shared memory bus, last level cache (LLC)/memory bandwidth, and power. To eliminate such unfairness we propose Alita, a runtime system consisting of an online interference identifier and adaptive interference eliminator. The interference identifier monitors hardware and system-level event statistics to identify resource polluters. The eliminator improves the performance of normal applications by throttling only the resource usage of polluters. Specifically, Alita adopts bus lock sparsification, bias LLC/bandwidth isolation, and selective power throttling to throttle the resource usage of polluters. Results for an experimental platform and in-production cloud platform with 30,000 nodes demonstrate that Alita significantly improves the performance of co-located virtual machines in the presence of resource polluters based on system-level knowledge.

The emerging trend of decomposing cloud applications into microservices has raised new questions about managing the performance/power trade-off of a datacenter at microsecondscale. We introduce ANT-Man, an Auto, Native and Transparent power Management framework that can exploit fine-grained microservice variability for system efficiency. To achieve this, ANT-Man abstracts away two major sources of latency overhead in traditional hierarchical power management frameworks. First, ANT-Man proposes an auto power budgeting scheme for reducing the power coordination latency at the datacenter level. It can proactively determine the power budget tailored to each individual microservice. Second, ANT-Man proposes a native and transparent power control scheme to overcome the power configuration latency for each microservice. It enables super-fast power budget enforcement with nanosecond-scale performance scaling. Extensive experiments on our prototyped system show that ANT-Man could slash power consumption by $ 7.8\sim 43.5\%$ and in the meantime reduce the $95^{\text{th}}$ tail latency by $ 9.7\sim 12.5\%$ compared to existing techniques.

As to address the impact of heterogeneity in distributed Deep Learning (DL) systems, most previous approaches focus on prioritizing the contribution of fast workers and reducing the involvement of slow workers, incurring the limitations of workload imbalance and computation inefficiency. We reveal that grouping workers into communities, an abstraction proposed by us, and handling parameter synchronization in community level can conquer these limitations and accelerate the training convergence progress. The inspiration of community comes from our exploration of prior knowledge about the similarity between workers, which is often neglected by previous work. These observations motivate us to propose a new synchronization mechanism named Community-aware Synchronous Parallel (CSP), which uses the Asynchronous Advantage Actor-Critic (A3C), a Reinforcement Learning (RL) based algorithm, to intelligently determine community configuration and fully improve the synchronization performance. The whole idea has been implemented in a system called Petrel that achieves a good balance between convergence efficiency and communication overhead. The evaluation under different benchmarks demonstrates our approach can effectively accelerate the training convergence speed and reduce synchro-nization traffic.

Deep learning is vulnerable to adversarial attacks, where carefully-crafted input perturbations could mislead a well-trained Deep Neural Network (DNN) to produce incorrect results. Adversarial attacks jeopardize the safety, security, and privacy of DNN-enabled systems. Today’s countermeasures to adversarial attacks either do not have the capability to detect adversarial samples at inference-time, or introduce prohibitively high overhead to be practical at inference-time.We propose Ptolemy, an algorithm-architecture co-designed system that detects adversarial attacks at inference time with low overhead and high accuracy. We exploit the synergies between DNN inference and imperative program execution: an input to a DNN uniquely activates a set of neurons that contribute significantly to the inference output, analogous to the sequence of basic blocks exercised by an input in a conventional program. Critically, we observe that adversarial samples tend to activate distinctive paths from those of benign inputs. Leveraging this insight, we propose an adversarial sample detection framework, which uses canary paths generated from offline profiling to detect adversarial samples at runtime. The Ptolemy compiler along with the co-designed hardware enable efficient execution by exploiting the unique algorithmic characteristics. Extensive evaluations show that Ptolemy achieves higher or similar adversarial sample detection accuracy than today’s mechanisms with a much lower (as low as 2%) runtime overhead.

Multi-tenant (or colocation) data centers are good solutions to support edge computing, since each enterprise or organization usually has limited servers at an edge site. When any data center tenant faces a burst of workload, renting resources from the other tenants in the same data center can provide the required resources while keeping the merits of edge computing, but its challenges of reliability and performance are daunting. In this paper, we propose OVERSEE, an outsourcing verification mechanism that enables resource sharing in multi-tenant data centers, fully exploiting the benefits of edge computing. OVERSEE addresses the above two challenges by making skillful use of Intel SGX suite. OVERSEE consists of two sub schemes, the Report-Proof mechanism and Sampling-Challenging mechanism. The Report-Proof mechanism guarantees a task outsourced by a tenant can be executed correctly, i.e., completely and without modification, in the operating environment provided by another tenant. The Sampling-Challenging mechanism can be used to verify that sufficient computing capacity is provided to achieve the required QoS according to the resource lease agreement between the tenants. The theoretical analysis shows the effectiveness of OVERSEE and the experimental results show that it brings minimal overhead.

Database platform-as-a-service (dbPaaS) is developing rapidly and a large number of databases have been migrated to run on the Clouds for the low cost and flexibility. Emerging Clouds rely on the tenants to provide the resource specification for their database workloads. However, they tend to over-estimate the resource requirement of their databases, resulting in the unnecessarily high cost and low Cloud utilization. A methodology that automatically suggests the “just-enough” resource specification that fulfills the performance requirement of every database workload is profitable. To this end, we propose URSA, a capacity planning and fair scheduling system that is comprised of an online capacity planner, a performance interference estimator, and a contention-aware scheduling engine. The capacity planner identifies the most cost-efficient resource specification for a database workload to achieve the required performance online. The interference estimator quantifies the pressure on the shared resource and the tolerance to the shared resource contention of each workload. The scheduling engine schedules the workloads across Cloud nodes carefully to eliminate unfair performance interference between the co-located workloads. Experimental results show that URSA reduces up to 25.9% of CPU usage, 53.4% of memory and reduces the performance unfairness between the co-located workloads by 47.6% usage compared to the prior works without hurting their performance.

While microservices that have stringent Quality-of-Service constraints are deployed in the Clouds, the long-term rented infrastructures that host the microservices are under-utilized except peak hours due to the diurnal load pattern. It is resource efficient for Cloud vendors and cost efficient for service maintainers to deploy the microservices in the long-term infrastructure at high load and in the serverless computing platform at low load. However, prior work fails to take advantage of the opportunity, because the contention between microservices on the serverless platform seriously affects their response latencies.Our investigation shows that the load of a microservice, the shared resource contentions on the serverless platform, and its sensitivities to the contention together affect the response latency of the microservice on the platform. To this end, we propose Amoeba, a runtime system that dynamically switches the deployment of a microservice. Amoeba is comprised of a contention-aware deployment controller, a hybrid execution engine, and a multi-resource contention monitor. The deployment controller predicts the tail latency of a microservice based on its load and the contention on the serverless platform, and determines the appropriate deployment of the microservice. The hybrid execution engine enables the quick switch of the two deploy modes. The contention monitor periodically quantifies the contention on multiple types of shared resources. Experimental results show that Amoeba is able to significantly reduce up to 72.9% of CPU usage and up to 84.9% of memory usage compared with the traditional pure IaaS-based deployment, while ensuring the required latency target.

Facial expression recognition has been well studied for its great importance in the areasof human&ndash, computer interaction and social sciences. With the evolution of deep learning, therehave been significant advances in this area that also surpass human-level accuracy. Althoughthese methods have achieved good accuracy, they are still suffering from two constraints (high computational power and memory), which are incredibly critical for small hardware-constrained devices. To alleviate this issue, we propose a new Convolutional Neural Network (CNN) architecture eXnet (Expression Net) based on parallel feature extraction which surpasses current methodsin accuracy and contains a much smaller number of parameters (eXnet: 4.57 million, VGG19:14.72 million), making it more efficient and lightweight for real-time systems. Several moderndata augmentation techniques are applied for generalization of eXnet, these techniques improvethe accuracy of the network by overcoming the problem of overfitting while containing the same size. We provide an extensive evaluation of our network against key methods on Facial ExpressionRecognition 2013 (FER-2013), Extended Cohn-Kanade Dataset (CK+), and Real-world Affective Faces Database (RAF-DB) benchmark datasets. We also perform ablation evaluation to show the importance of different components of our architecture. To evaluate the efficiency of eXnet on embedded systems,we&nbsp, deploy it on Raspberry Pi 4B. All these evaluations show the superiority of eXnet for emotionrecognition in the wild in terms of accuracy, the number of parameters, and size on disk.

Graph neural networks (GNN) represent an emerging line of deep learning models that operate on graph structures. It is becoming more and more popular due to its high accuracy achieved in many graph-related tasks. However, GNN is not as well understood in the system and architecture community as its counterparts such as multi-layer perceptrons and convolutional neural networks. This work tries to introduce the GNN to our community. In contrast to prior work that only presents characterizations of GCNs, our work covers a large portion of the varieties for GNN workloads based on a general GNN description framework. By constructing the models on top of two widely-used libraries, we characterize the GNN computation at inference stage concerning general-purpose and application-specific architectures and hope our work can foster more system and architecture research for GNNs., Comment: 4 pages, published in IEEE Computer Architecture Letters (CAL) 2020

Parallel computers now start to adopt Bandwidth-Asymmetric Memory architecture that consists of traditional DRAM memory and new High Bandwidth Memory (HBM) for high memory bandwidth. However, existing task schedulers suffer from low bandwidth usage and poor data locality problems in bandwidth-asymmetric memory architectures. To solve the two problems, we propose a Bandwidth and Locality Aware Task-stealing (BATS) system, which consists of an HBM-aware data allocator, a bandwidth-aware traffic balancer, and a hierarchical task-stealing scheduler. Leveraging compile-time code transformation and run-time data distribution, the data allocator enables HBM usage automatically without user interference. According to data access hotness, the traffic balancer migrates data to balance memory traffic across memory nodes proportional to their bandwidth. The hierarchical scheduler improves data locality at runtime without a priori program knowledge. Experiments on an Intel Knights Landing server that adopts bandwidth-asymmetric memory show that BATS reduces the execution time of memory-bound programs up to 83.5% compared with traditional task-stealing schedulers.

Load-balanced flow scheduling for big data centers in clouds, in which a large amount of data needs to be transferred frequently among thousands of interconnected servers, is a key and challenging issue. The OpenFlow is a promising solution to balance data flows in data center networks through its programmatic traffic controller. Existing OpenFlow based scheduling schemes, however, statically set up routes only at the initialization stage of data transmissions, which suffers from dynamical flow distribution and changing network states in data centers and often results in poor system performance. In this paper, we propose a novel dynamical load-balanced scheduling (DLBS) approach for maximizing the network throughput while balancing workload dynamically. We first formulate the DLBS problem, and then develop a set of efficient heuristic scheduling algorithms for the two typical OpenFlow network models, which balance data flows time slot by time slot. Experimental results demonstrate that our DLBS approach significantly outperforms other representative load-balanced scheduling algorithms Round Robin and LOBUS; and the higher imbalance degree data flows in data centers exhibit, the more improvement our DLBS approach will bring to the data centers.

In the age of big data, services in the pervasive edge environment are expected to offer end-users better Quality-of-Experience (QoE) than that in a normal edge environment. However, the combined impact of the storage, delivery, and sensors used in various types of edge devices in this environment is producing volumes of high-dimensional big data that are increasingly pervasive and redundant. Therefore, enhancing the QoE has become a major challenge in high-dimensional big data in the pervasive edge computing environment. In this paper, to achieve high QoE, we propose a QoE model for evaluating the qualities of services in the pervasive edge computing environment. The QoE is related to the accuracy of high-dimensional big data and the transmission rate of this accurate data. To realize high accuracy of high-dimensional big data and the transmission of accurate data through out the pervasive edge computing environment, in this study we focused on the following two aspects. First, we formulate the issue as a high-dimensional big data management problem and test different transmission rates to acquire the best QoE. Then, with respect to accuracy, we propose a Tensor-Fast Convolutional Neural Network (TF-CNN) algorithm based on deep learning, which is suitable for high-dimensional big data analysis in the pervasive edge computing environment. Our simulation results reveal that our proposed algorithm can achieve high QoE performance.

Carbon nanotube field effect transistor (CNFET), using the carbon nanotubes (CNTs) as the material for conducting, is a promising alternative of CMOS technology to overcome the “power wall” issue. Recently, a microprocessor solely based on CNFETs was fabricated and demonstrated, which is a big step forward to the industrial practice. However, CNFETs are inherently subject to much larger process variation or manufacturing defects; thereby it may cause significant design cost to build high performance processors. This is exacerbated in the large register file (RF) architectures widely used in single instruction multiple data (SIMD) architectures, e.g., general public utilities style processors, where the number of critical paths are multiplied by the SIMD width and thread count. In this paper, we seek cost-effective approaches to address the issues by judiciously exploiting the strong asymmetric spatial correlation in the variation unique to the CNFET fabrication process. This paper presents a microarchitectural model to characterize CNFET delay variation and malfunction, under which we show that the RF organizations coupled with the architectural schemes are critical to the performance and power consumption of the SIMD processor. Therefore, we propose several architectural techniques to mitigate the performance degradation and the impact of CNT metallization, leveraging the distinctive CNFET characteristics and the unique features in the SIMD processors. Experimental results verify the effectiveness of the proposed techniques and demonstrate the great opportunity offered by this new device technology.

The traditional dynamic random-access memory (DRAM) storage medium can be integrated on chips via modern emerging 3D-stacking technology to architect a DRAM shared cache in multicore systems. Compared with static random-access memory (SRAM), DRAM is larger but slower. In the existing research, a lot of work has been devoted to improving the workload performance using SRAM and stacked DRAM together in shared cache systems, ranging from SRAM structure improvement to optimizing cache tags and data access. However, little attention has been paid to designing a shared cache scheduling scheme for multiprogrammed workloads with different memory footprints in multicore systems. Motivated by this, we propose a hybrid shared cache scheduling scheme that allows a multicore system to utilize SRAM and 3D-stacked DRAM efficiently, thus achieving better workload performance. This scheduling scheme employs (1) a cache monitor, which is used to collect cache statistics; (2) a cache evaluator, which is used to evaluate the cache information during the process of programs being executed; and (3) a cache switcher, which is used to self-adaptively choose SRAM or DRAM shared cache modules. A cache data migration policy is naturally developed to guarantee that the scheduling scheme works correctly. Extensive experiments are conducted to evaluate the workload performance of our proposed scheme. The experimental results showed that our method can improve the multiprogrammed workload performance by up to 25% compared with state-of-the-art methods (including conventional and DRAM cache systems).

Modern large-scale computers have shifted to Multi-socket Multi-core (MSMC) architectures, where multiple CPU chips are integrated into a machine as sockets and multiple memory nodes are integrated into the shared main memory (NUMA). To improve the hardware utilization of MSMC computers, multiple programs are often executed concurrently. However, most work-stealing schedulers are designed for single-socket architectures and contention-free scenarios. Work-stealing programs suffer from very-high-frequency remote memory access and serious interference from co-located programs in MSMC architectures, which in turn significantly degrade their performance. To solve these two problems, we propose a Contention- and Locality-Aware Work-Stealing (CLAWS) scheduler. CLAWS first evenly distributes the data set of a program to all the memory nodes and allocates a task to the socket where the local memory node stores its data. Then, according to the real-time contention situation for each socket collected at runtime, CLAWS dynamically migrates data and re-allocates the corresponding tasks to balance the workload and reduce remote memory accesses. Experimental results show that CLAWS can improve the performance of memory-bound programs by 40.1 percent on average compared with traditional work-stealing schedulers. Meanwhile, CLAWS is also more energy efficient than traditional work-stealing schedulers.

Wi-Fi channel state information (CSI) provides adequate information for recognizing and analyzing human activities. Because of the short distance and low transmit power of Wi-Fi communications, people usually deploy multiple access points (APs) in a small area. Traditional Wi-Fi CSI-based human activity recognition methods adopt Wi-Fi CSI from a single AP, which is not very appropriate for a high-density Wi-Fi environment. In this article, we propose a learning method that analyzes the CSI of multiple APs in a small area to detect and recognize human activities. We introduce a deep learning model to process complex and large CSI from multiple APs. From extensive experiment results, our method performs better than other solutions in a given environment where multiple Wi-Fi APs exist.

Crowdsourcing allows requesters to allocate tasks to a group of workers on the Internet to make use of their collective intelligence. Quality control is a key design objective in incentive mechanisms for crowdsourcing as requesters aim at obtaining high-quality answers under a limited budget. However, when measuring workers’ long-term quality, existing mechanisms either fail to utilize workers’ historical information, or treat workers’ quality as stable and ignore its temporal characteristics, hence performing poorly in a long run. In this paper we propose MeLoDy , a long-term dynamic quality-aware incentive mechanism for crowdsourcing. MeLoDy models interaction between requesters and workers as reverse auctions that run continuously. In each run of MeLoDy , we design a truthful, individual rational, budget feasible and quality-aware algorithm for task allocation with polynomial-time computation complexity and $O(1)$ performance ratio. Moreover, taking into consideration the long-term characteristics of workers’ quality, we propose a novel framework in MeLoDy for quality inference and parameters learning based on Linear Dynamical Systems at the end of each run, which takes full advantage of workers’ historical information and predicts their quality accurately. Through extensive simulations, we demonstrate that MeLoDy outperforms existing work in terms of both quality estimation (reducing estimation error by $17.6\% \sim 24.2\%$ ) and social performance (increasing requester's utility by $18.2\% \sim 46.6\%$ ) in long-term scenarios.

While the video streaming technique and 4G LTE networks are developing rapidly, the advancement of the battery technology, however, is relatively slow. Existing works have paid much attention to understanding the power consumption in general 4G LTE networks, while few attention has been paid to reducing the power consumption for online video streaming in 4G LTE networks. Our previous work based on the real experimental platform showed that, the saving room in the network part is large, the transmission pattern and the number of RRC tails could be promising for optimizing the power consumption. In this paper, we attempt to develop efficient optimization strategies to save the energy cost of mobile devices for online video streaming over 4G LTE networks. This problem is clearly important and also interesting, while it is challenging, due to various reasons such as the uncertainty of users’ behavior modes. To alleviate the challenges, we suggest a self-adaptive method that can allow us to adjust various parameters dynamically and efficiently, so as to achieve a relatively small energy consumption. We give the rigorous theoretical analysis for the proposed method, and conduct extensive experiments to validate its effectiveness. The experimental results show that the proposed method consistently outperforms the classical method as well as other competitors adapted from existing methods.

Cognitive radio significantly mitigates the spectrum scarcity for various applications built on wireless communication. Current techniques on mobile cognitive ad hoc networks (MCADNs), however, cannot be directly applied to time-critical applications due to channel interference, node mobility as well as unexpected primary user activities. In multichannel multiflow MCADNs, it becomes even worse because multiple links potentially interfere with each other. In this paper, we propose a delay-minimized routing (DMR) protocol for multichannel multiflow MCADNs. First, we formulate the DMR problem with the objective of delay minimization. Next, we propose a delay prediction model based on a conflict probability. Finally, we design the minimized path delay as a routing metric, and propose a heuristic joint routing and channel assignment algorithm to solve the DMR problem. Our DMR can find out the path with a minimal end-to-end (e2e) delay for time-critical data transmission. NS2-based simulation results demonstrate that our DMR protocol significantly outperforms related proposals in terms of average e2e delay, throughput, and packet loss rate.

Video streaming, one of the most popular technologies for online video playback, has already been applied to 4G LTE networks. Previous work has been devoted to understanding the power consumption in general 4G LTE networks, while it is still unclear how the online video streaming makes impact on the power performance of mobile devices. Inspired by this, this paper investigates the relationship between the mobile device's power performance characteristics and the behaviours of video streaming in 4G LTE networks. There are many natural issues/questions that are clearly interesting and important, while it is non-trivial to answer these issues/questions exactly (e.g., where is the energy saving room? how much is it?). To address a series of issues like the above, we formulate our energy models together with an algorithm that can assist our analysis. Particularly, we design a systematic platform, and conduct a comprehensive and also deep analysis on the power consumption of video streaming in 4G LTE networks. Our experiments reveal us a series of valuable findings--the saving room in the network part is large (from 41.86 to 69.62%), the number of RRC tails and the transmission pattern could be promising for optimizing the power consumption, for example.

Mobile crowdsensing is a new paradigm, sharing sensing data collected by mobile devices such as smartphones and tablets. As mobile devices are usually connected by an opportunistic network for data transfer, it is hard to acknowledge the contribution of each mobile user in network forwarding and find a sustainable incentive mechanism. In this article, we propose a software defined opportunistic network scheme for mobile crowdsensing. We design a centralized control structure to manage the opportunistic network and mobile crowdsensing. By the centralized structure, we also design an incentive mechanism for data forwarding and collection in a software defined opportunistic network and solve the optimal decision of mobile devices and the sensing service provider. From the extensive simulation results, our incentive mechanism performs better than original solutions.