Search

As user-generated data diversifies, an increasing number of big data tasks now encompass unstructured data forms, including audio, imagery, and video content. The processing of these data often requires the support of GPU devices in the cluster. However, most of the existing cluster resource management frameworks lack effective decoupling of CPU resources and GPU resources, and the management granularity of GPU resources is too coarse, resulting in poor GPU sharing and low resource utilization. Given, for the inadequacy of the existing cluster resource management framework that cannot support batch stream integration and CPU-GPU resource scheduling, we design and implement a Hybrid Heterogeneous Resource Management (H-HRM) for CPU-GPU heterogeneous clusters. The resource queue binding mechanism on computing nodes provides flexible binding of CPU resources to GPU resources, which solves the problem that CPU resources and GPU resources in the cluster are difficult to decouple. According to the different uses of CPU or GPU as the main resource, a Hybrid Domain Resource Fairness (HDRF) model is proposed to realize the reasonable allocation of CPU resources and GPU resources. Through the queue stacking technology and the mechanism that multiple executors can run simultaneously on the queue, fine-grained sharing of GPU resources is realized and the utilization rate of GPU resources is improved. Finally, this paper realizes the docking of H-HRM and Spark programming framework and conducts real load tests, and after comparing with the performance of Mesos, it is proved that H-HRM can handle the mixed scenarios of batch jobs and stream processing jobs in the cluster, and the HDRF algorithm also greatly improves the utilization of the GPU. [ABSTRACT FROM AUTHOR]

With the rapid development of cloud computing technology and the broad application of the Internet, more and more user data are stored in the cloud storage system online. Data copy technology is an effective technical means to manage various data in cloud storage systems, which has the advantages of improving data access speed and enhancing data availability in cloud storage systems. Since a data copy management strategy has an important impact on the performance of cloud storage systems, how to design a data copy management strategy is of great significance to achieving high-quality cloud storage management. Firstly, this paper analyzes the HDFS random copy placement strategy and its shortcomings. Secondly, proposes a copy placement strategy based on evaluation value and load balancing. Thirdly, gives the evaluation function of rack and node by considering multiple evaluation indicators such as node disk space load capacity, node memory utilization, node CPU utilization, rack load rate and network distance between racks. Fourthly, finds suitable nodes through computer shelf evaluation values and node evaluation values when placing replicas. Finally, through simulation experiments, the experimental results of the HDFS random replica placement strategy and the evaluation results and load balancing-based replica placement strategy are compared and analyzed. [ABSTRACT FROM AUTHOR]

Cloud computing platform is a data-centric architecture, which utilize virtualization resource scheduling technology to integrate and distribute the processing power of distributed server cluster and other network devices distributed over the network, and uses service-oriented architecture to provide users with highly reliable, highly available, and efficient application data services. Although the cloud platform application based on various requirements has become a research hotspot in cloud computing technology, in some high Real-time requirements application scenarios, the research on how to guarantee the real time performance of cloud platform is few. Compared to traditional resource virtualization such as CPU and memory, Docker only provides a small portion of network resource virtualization. The four network modes provided by Docker officially have a single function, which makes it very difficult to apply to cloud platform scenarios with complex network bandwidth requirements and high real-time requirements. Based on Docker's traditional NAT communication method, this paper proposes a network virtualization solution based on SDN and Docker containers. Through comparison experiments, it is proved that the scheme can guarantee the Real-time performance of the container cloud platform. [ABSTRACT FROM AUTHOR]

Provable data possession (PDP) protocol is a mechanism that guarantees the integrity of user's cloud data, and many efficient protocols have been proposed. Many of them ignore data's privacy against the third-party auditor (TPA) and also suffer from intricate management of certificates, which heavily relies on the public key infrastructure (PKI). In order to overcome the two shortcomings, Li et al. recently proposed an "identity-based" (IB) PDP protocol with the privacy-preserving property (IEEE Syst J, https://doi.org/10.1109/JSYST.2020.2978146). However, we find out that (1) their protocol has great communication overhead, (2) a PKI-based signature scheme is used as a building block, which results in their protocol is not completely identity-based. Hence, in this paper, we try to improve the performance of this protocol. Concretely, by adopting flexible data-splitting and tag-aggregating techniques, we can greatly reduce its communication overhead. A concrete example shows that the total communication overhead can be reduced over 99%. Moreover, by replacing with an identity-based signature, we can twist this protocol into a complete IB-PDP protocol. [ABSTRACT FROM AUTHOR]